Nanoc»Internals

This document describes some of the internal workings of Nanoc. The audience of this document is potential contributors to Nanoc. You do not need to read or understand this document in order to use Nanoc.

This document is a work in progress. It’s highly incomplete, but will gradually be expanded to include more detail.

Outdatedness checking

Nanoc is a smart static-site generator in that it avoids recompiling items unless necessary.

Nanoc will recompile an item if it is deemed to be outdated. The class responsible for determining whether or not an item is outdated is the OutdatednessChecker. An item is considered as outdated if any of the following conditions hold:

  • The item’s content or attributes have changed
  • The site configuration has changed
  • The rule for this item has changed
  • One or more output files for this item do not exist
  • One or more output files for this item are newer than the item
  • A dependent item is considered as outdated

The last point is necessary, because an item might use the raw content, the compiled content, or an attribute of another item. If this is the case, Nanoc will record a dependency from the former item onto the latter item (the dependent item). This type of dependency is used purely to infer outdatedness of items, and is called a soft dependency.

Nanoc will record a dependency from one item onto another if any attribute, raw content or compiled content is accessed. For example:

<!-- Creates a dependency on the /about.* page -->
See the <%= @items['/about.*'][:title] %> page.

<!-- Creates a dependency on every item -->
See the <%= @items.select { |i| i[:title] =~ /About/ }.first %> page.

Further improvement work could include fine-grained dependency and change tracking, so that an item is only marked as outdated if the content and/or attributes it uses from a dependent item have changed; currently, any change in a dependent item will render its depending items as outdated, even if these dependent items only use content or attributes that have not changed.

Compilation order

The order in which item representations are compiled, depends on the dependencies on compiled content between individual item representations.

Dependencies on compiled content

An item representation can depend on the compiled content of another item representation. For example, blog archive pages typically tend to contain (snippets of) the content of individual blog posts. An item representation which contains the compiled content of another item representation cannot be compiled until the compiled content of the dependent item representation is available. If this compiled content is not available and Nanoc tries to compile the item representation, then an UnmetDependency error will be raised.

An UnmetDependency error is not a fatal error, but merely indicates Nanoc that the dependent item representation should be compiled first. The dependencies of an item representation are not known until Nanoc has finished compiling the item representation, and so UnmetDependency error are not only expected, but commonplace.

Even though an UnmetDependency error is a common condition, it is not a desirable error to have. Such an error means that compilation of this item representation has to be aborted and restarted at a later point in time, which effectively means that the work spent on compiling this item representation has gone to waste. An ongoing challenge for Nanoc is to improve the algorithms and heuristics to reduce the frequency of UnmetDependency errors.

A dependency from one item representation onto another item representation’s compiled content is called a hard dependency, as opposed to a soft dependency, which is used in outdatedness checking. An UnmetDependency error always indicates a hard dependency.

Compilation selection

Once Nanoc has selected the items representations that it deems are outdated (see the Outdatedness checking section), it will compile these item reps in the best possible order. The class responsible for determining this order is the ItemRepSelector.

Nanoc will attempt to compile every outdated item representation sequentially. If an item representation cannot be compiled due to an UnmetDependency error, Nanoc will schedule a new compilation attempt for this item representation after the dependent item representation has been compiled. If the dependent item representation was not selected for recompilation (which happens when the item representation is not deemed to be outdated), Nanoc will schedule the dependent item representation for recompilation.

A visual representation of the compilation selection process
The term “compile” or “recompile” does not always mean that the item will be recompiled from scratch; Nanoc keeps a cache of compiled content and will reuse cached compiled content if possible.

Further improvement work could include recompiling the dependent item representation right away, rather than scheduling it for compilation, or selecting items to recompile based on the dependency graph constructed in the previous compilation run (although that would require the dependency tracker to be able to distinguish between hard and soft dependencies).