Jirka Kosek jirka@kosek.cz 2011-04-20 2010-12-09 This document describes syntax, semantics and processing model of DocBook transclusion mechanism. Please be aware that this is early stage draft – everything described below might change or disappear completely. This proposal tries to resolve Requirements for transclusion in DocBook. DocBook TC welcomes any feedback on this draft, especially from users and developers of DocBook authoring and processing tools. Plese direct your comments to DocBook mailing list by sending email to docbook@lists.oasis-open.org. Actually, for now the document is written more like tutorial. If DocBook TC decides to incorporate this into DocBook, more formal and precise specification will follow. Transclusion in documents is described by ref element which references the content to transclude. There are two basic types of reference – inline and external. Inline references reference content which is defined in some other place using the definitions element. An external reference references some external content which might or might not be written using DocBook vocabulary.

An inline reference is denoted by the ref element with a mandatory name attribute which contains the name of referenced content. This name is case-sensitive and the document must contain a definition of the referenced content in the definitions element. The result of transclusion: The definition can contain arbitrary markup, not just text. All child nodes of the def element are transcluded by corresponding ref element. Result of transclusion: Definitions can be placed directly inside an info element or they can be stored in a separate file which can be referenced by multiple documents. Result of transclusion: Definitions from external file can be locally redefined. Result of transclusion: Definitions can be conditional. All effectivity attributes can be used to set conditions on definitions. Gershon: I'd like to see examples 14 and 15 be followed by a more complex exmaple that demonstrates the different result depending on processing order, so that we can also determine which order should be normative and correct per the DocBook spec. Result of transclusion: Effectivity attributes can be also used on references itself. Definitions are valid only in the subtree rooted at the parent of info element containing definitions. Result of transclusion: A reference can point to particular external definitions file not to all in-scope definitions. Result of transclusion: A transclusion processor implementation might provide a mechanism for overriding any definition. This mechanism is used first when finding a definition for a reference. The closest element containing info/definitions is found on the ancestor-or-self:: XPath axis. If definitions contains definitionfile attribute then content of referenced file is treated as if it was inserted before the respective definitions element. If there are multiple matching info/definitions/def elements then the last one is used. If there is no matching definition then we continue with . If no matching definition was find so far, an error is raised. Referenced definition file is searched for definition (def element with matching value in a name attribute). Should be definitionfile attribute supported here for chaining of definitions? Probably yes. If no matching definition is found, error is raised.

An external reference is denoted by the ref element with a mandatory fileref attribute which contains URI of referenced document. The reference is replaced with content of referenced document. The outermost transcluded element is augmented with xml:base attribute in order to retain relative references. Result of transclusion: It is possible to specify additional attributes xpointer, parse and encoding – their meaning is identical to corresponding attributes from XInclude. Please note that while process similar to XInclude Base URI Fixup is performed, Language Fixup is not performed.This effectively means that it is not necessary to specify xml:lang on each module. Result of transclusion:

Transcluded content can contain an xml:id attributes. If one fragment is transcluded more then once then the resulting document after transclusion will contain duplicate IDs. The same problem may arise if IDs are colliding in transcluded modules. To overcome this problem all IDs and references to them can be adjusted during the transclusion process. If there is xml:id attribute present on the ref element, then this ID will replace xml:id on the outermost element of any transcluded content. It is an error to transclude content which is not enclosed in a single element and specifying an xml:id attribute on ref element at the same time. How IDs are going to be adjusted during transclusion is controlled by the idfixup attribute on the ref element. It can have one of the following values. none No ID adjustment is done strip All IDs are stripped (except xml:id inherited from ref element) prefix All IDs are prefixed with a value specified in prefix attribute auto All IDs are prefixed with a value which is unique for each ref elementFor example XSLT based implementations can use generate-id() on ref element to generate such unique prefix. This is default behavior if attribute is not specified. Of course if IDs are adjusted then all corresponding references has to be also corrected. This is controlled by linkscope attribute. It can have one of the following values. user No IDREF adjustment is done local All IDREFs in transcluded content are prefixed by user specified prefix (when idfixup="prefix") or auto-generated prefix (when idfixup="auto"). Using this value with other idfixup values is an error.Maybe raising error is to strict approach. near All IDREFs in transcluded content are adjusted to point to the closest element which has a matching ID. A matching ID doesn't mean string equality between ID and IDREF values – it is sufficient if second part of ID and IDREF after removal of possibly added prefixes is matching. When searching for the closest element ancestor elements of an element with an IDREF attribute are gradually inspected and matching ID is searched between all their descendants. If there is no matching ID, then parent is inspected and so on until match is found or document root is reached. global All IDREFs in transcluded content are adjusted to point to the first element in document order which has a matching ID. A matching ID doesn't mean string equality between ID and IDREF values – it is sufficient if the second part of ID and IDREF after removal of possibly added prefixes are matching. By using various combinations of idfixup and linkscope attributes we can achieve different linking behavior. The following examples show the effect of using those two attributes. Examples are transcluding the following procedure which contains one internal link and one external (points to target outside of this module). Now lets assume that we want to transclude this module twice to show how we can deal with duplicate IDs problem. Result of transclusion: We haven't specified idfixup and linkscope attributes so the default behavior is triggered. All IDs in transcluded modules are automatically prefixed to prevent ID collisions. Then IDREFs are fixed so that links point to the nearest possible target. For example the link from step 2 to step 1 in procedure always points to the same instance of procedure. However buy link is pointing correctly to target in the main document. Result of transclusion: We used linkscope="global" on the second transclusion. Result is that link from step 2 in the second procedure now links to step 1 in the first procedure. Result of transclusion: We used linkscope="local" on the first transclusion. This means that no link from this transclusion can point outside of this transclusion. Because there was such link (buy link), thus the result of transclusion is broken because there is no corresponding target for IDREF d2e22---buy. Result of transclusion: If we care about the resulting IDs after transclusion we can manually assign some meaningful prefix before IDs in transcluded document. Result of transclusion: We have disabled ID fixup by idfixup="none". The resulting document thus contain duplicated IDs. Result of transclusion: We have stripped all IDs from the second transcluded procedure by idfixup="strip". Thus there are no duplicate IDs in the resulting document and links from second procedure always target first one. However IDs in the first procedure were automatically prefixed. Result of transclusion: We have stripped all IDs from the second transcluded procedure by idfixup="strip". Thus there are no duplicate IDs in the resulting document and links from second procedure always target first one. IDs in the first procedure were not automatically prefixed since we have specified idfixup="none". Should be there option to preserve original IDs (without prefix) in output when doing prefixes?

FIXME: This should provide some very generic mechanism for applying various transformations on referenced content before actual transclusion. For example DITA -> DocBook, V4.x -> V5.0 conversion and so on. FIXME: TC decided this is not in scope of transclusions. Sources should be prepared in advance.

The above DocBook transclusion proposal solves all use cases except UC-2. Transclusions are more usable and can solve various problems introduced by multiple inclusion of the same content. Some of those problems can be solved by using XInclude – but syntax is cumbersome and there is no good interoperability between various XInclude implementations. For the above reasons I think that transclusions should be added into core DocBook. I don't think that making them separate XML transclusion standard is viable approach. A transclusion processor has to know which attributes are of ID/IDREF type for each document type. History shown that generic standards relying on access to a schema are not successful.

Should we allow multiple file references in @definitionfile? Should we allow ref inside def? (Probably yes). Please note that this sample transclusion processor is not yet feature complete. It supports only subset of proposal. Some basic functionality of transclusions can be directly mapped into XInclude as show in the table below. However there are still some very important features which can't be replicated with pure XInclude, namely: ID/IDREF fixup – it was one of the most important requirements to provide solution to the duplicate IDs problem; profiling – it's not possible to specify profiling attributes on xi:include element as it is subject to special handling; changing ID during transclusion – it's not possible to specify new xml:id value for transcluded content on xi:include element as it is subject to special handling; redefinitions are not supported. Having more definitions in one file will lead to an invalid file as each definition has to be identified by an unique xml:id. Transclusion construct XInclude equivalent Note <ref name="foo"/> <xi:include xpointer="xpath(id(foo)/node())"/> XInclude implementations are not required to support missing href attribute. See <ref definitionfile="bar" name="foo"/> <xi:include href="bar" xpointer="xpath(id(foo)/node())"/> <ref fileref="foo"/> <xi:include href="foo"/> <definitions definitionfile="foo"/> <xi:include href="foo"/> DocBook schema and stylesheets has to be modified to ignore elements which are inserted only for further referencing. <def name="foo">…</def> <phrase xml:id="foo">…</phrase> Any generic DocBook element can be used instead of phrase. Element itself is not transcluded. Please note that xpath() XPointer scheme is not widely supported. Formerly xpointer() schema was popular although it's standardization was never finished. Later on xpath() schema appeared in the list of registered schemes . Today two special schemes exists – xpath1() and xpath2() – for respective versions of XPath language. This appendix summarizes points made in the following email . Any element may bear one of these two attributes: ref and copy. The value of these attributes is an URL, possibly ending with a fragment. When an element has a ref or a copy attribute, it must be completely empty.

ref is a compose transclusion directive. Examples:

]]> When ref attribute is used, there is no ID/IDREF fixup in the transcluded content.

copy is an instantiate a copy transclusion directive. Examples: ]]> When copy attribute is used, there is an automatic ID/IDREF fixup in the transcluded content, the one described above: Add a globally unique suffix to each ID defined in the copy. Update the IDREF/IDREFS which point to these IDs. Ignore the IDREF/IDREFS which point outside the copy.

Although very simple, this proposal covers many use-cases. Although compared to the original proposal several features are missing: It is not possible to include just a text node, or sequence of sibling nodes. Only one element (and its content) can be transcluded at one time. It is not possible to override definitions. It is not possible to configure way in which ID fixup is done. Verbosity for including simple inline content is similar to XInclude.