Linking Outside the Box

Cross referencing is a powerful tool when writing technical documentation. It lets you document something once in an authoritative manner, and then refer to that information from other points in the documentation as needed. There is less maintenance because information is not repeated, and the reader always has access to the most accurate information. With modern hypertext readers such as HTML and PDF browsers, the reader can jump immediately to the references.

Yet the basic linking mechanism built into XML does not support cross referencing to targets outside the current document. This has forced authors to put all the content within the scope of cross referencing into a single XML document. This is an unwieldy solution that runs counter to the goals of reusable content modules.

The W3C XLink Recommendation was supposed to address this deficit for XML, but it does not provide the methods to process such links, and almost no general purpose tools for XLink exist.

To satisfy this need for cross referencing, the DocBook community created the <olink> element. This elements lets the author identify an external document and a reference point within that document. The XSL stylesheet then processes that information into a link. This scheme enables several important new features in a DocBook publishing system:

Olink enables these features because it breaks free of the constraint of using ID/IDREF for cross references. Yet this system has been implemented without any changes to the XML standards.

The cross referencing system that is built into XML is based on ID and IDREF attribute types. An ID attribute establishes a target name for an element, making it a potential destination. Then an IDREF attribute is used in a cross reference element to point to a target ID.

XML requires that the ID reside in the same document as the IDREF. That way the document validation step can confirm that cross references are valid. This requirement means that any content you want to cross reference to must be in the same document. In DocBook, this is typically done by using larger document containers such as <book> or <set> elements to hold everything you want to cross reference among.

Cross referencing within a large document set using ID/IDREF creates many problems. It is unlikely you would put all the content in a single file, so writing and editing means working with system entity files or XIncludes that contain part of the content. But each such file cannot be validated if it has IDREF links to targets outside of itself. And when you want to process any of your content, you have to process all of it at once, even if you make a minor change in one component. These burdens in the editing and processing stages inhibit the use of cross references.

Cross referencing provides huge gains for managing a large documentation set:

Cross references become even more useful when the processing machinery can automatically generate their cross reference text from information in the target of the cross reference (e.g., the title and chapter number in the target document).

These benefits of cross referencing come at a cost: that of managing the cross references. The standard ID/IDREF mechanism manages cross references quite effectively by validating them when the document is validated. Cross references implemented without ID/IDREF must be managed carefully if you don't want to degrade your documents. If a reader follows a link and lands in the wrong place (or no place), then they are less likely to trust the information they are reading. The cross referencing machinery must help maintain the accuracy of cross references, and it must be easy and automatic. This is particularly true when creating and maintaining large documentation sets. There are never enough eyes to check every cross reference against its target for each product release cycle or publication.

A full-featured cross referencing system should be able to do the following:

The cross referencing system described above has been implemented in DocBook. There are four cross reference elements in DocBook, three of which cannot be used in such a system.

XLink was designed to establish links between XML documents. XLink received the status of a full W3C Recommendation on 27 June 2001. See XML Linking Language (XLink) Version 1.0 [http://www.w3.org/TR/xlink/]

XLink uses the idea of global attributes in a separate xlink namespace to make elements in a DTD into XLink elements. Thus DocBook's olink could be made into an Xlink element by adding new attributes to the DocBook DTD. The full XLink spec provides for multiple links and bidirectional links between local and remote resources. It also lets you maintain link information outside the documents themselves in separate link bases. It was expected that many applications would arise to make use of these extended link features.

But XLink also supports simple XLinks, which is the type that document authors are currently used to. Simple XLinks point from the current element to another element, which could be in the same document or another document. A simple XLink can have other XLink attributes such as role, title, and actuate to modify its behavior. But it is always a single link from here to there.

It is the XLink href attribute that establishes the link. It looks similar to an HTML href attribute. The first part is a URI that points to a document. It could be a URL such as those used in HTML documents, or it could be a more general URN (Uniform Resource Name) with some mechanism for resolving the name to an actual resource. A second optional part, known as a fragment identifier, follows a # character and points to some location within that document. XLink permits fragment identifiers to use the powerful syntax of XPointers to locate information within the document. But it also supports the simplest form, which is just the value of an ID attribute within the target document. For example:

<xref xlink:href="../OS-Userguide/book.xml#Mousing"/>

This assumes that a new xlink:href attribute has been added to the xref element in the DTD.

But how is this XLink to be interpreted when the current document is processed into HTML? Since this is an xref element, the stylesheet is expected to generate the hot link text, but what should it say? Also, exactly what does the user see when they click on the link? It is unlikely the author intended the reader to be dumped into the middle of an XML document to display raw XML markup like this:

<section id="Mousing">
  <title>Using a Mouse</title>
  <para>This is how you use a mouse</para>
</section>

Both the generated text and the destination XML must be styled appropriately for the presentation of the current document. That's easy, just apply a stylesheet, right?

Well, it turns out that it isn't all that easy. In a W3C Note XML Linking and Style [http://www.w3.org/TR/xml-link-style/], Norman Walsh outlines many of the complications that can arise, even with simple XLinks. Here is a sampling:

Several more complications could be added to the list:

The basic problem with XLink is how to resolve it to a styled result in your file. It appears you have to know their stylesheet and their profiling. You can't guess, you have to know with certainty what the results will be for your XLink to work.

If that is the case, then it becomes very difficult to link to the XML source document, which has several profiles in it and can be processed with several different stylesheets. Until that problem is solved, it is safer to cross reference to a rendered document with one profile and one style already applied. Cross references ultimately must land on a rendered version of the target if they are to be consumable.

But if we must link to a rendered document, why not just use <ulink>? Because ulink addresses are not flexible or robust enough. It is important to distinguish between the authoring process and the rendering process. You want your author to establish a logical link from one XML source document to another. You want a link that remains valid through several iterations of the product, and through several different renderings for print or online. A ulink that is valid in the current version may not be valid in the next. For ease of maintenance, the links should not require an editing pass through the document to update them for each new release. Rather, the processing machinery should resolve the abstract logical link to a rendered link for each release. An XLink or olink is the abstract logical link from XML to XML. An HTML href is a concrete working link.

At some point you render your document, that is, convert the XML to a final form for consumption. This conversion is done either ahead of time for packaging on the media, or on the fly in the server or browser. You want that logical link turned into a working link, using the actual rendered text, label, and URL of the rendered target. As was described above, it currently must resolve to a rendered version of the target document.

If cross references must finally resolve to a rendered document, then it is not necessary to process every target document just to resolve a link. If the appropriate information for linking was saved when the target document was rendered, then we can just use that information in the current document to form our links, and the target document would not have to be parsed. The datafile associated with olink in the DocBook stylesheets provides such a mechanism.

A separate DocBook stylesheet (targets.xsl) generates a structured distillation of information about all the potential cross reference targets in a document. The target data is stored in a separate file from the document. Once the target data is separated from the document itself, it becomes accessible for cross referencing applications. Of course, it must be kept up to date, and a Makefile or Ant task can help with that.

The target data file is an XML document that consists of a set of nested <div> elements. Each div element records the information for a single strucutural element in the document, such as book, chapter, section, etc. Other potential cross reference targets that are not part of the structural hierarchy are recorded in <obj> elements.

The implementation of olink in the DocBook XSL stylesheets has the following goals:

To meet these goals, it was necessary to add two new attributes to olink's attribute list. These new attributes appeared first in the DocBook XML 4.2 DTD.

To form an olink, the author provides just two attribute values. For example:

<olink 
   targetdoc="OS-Userguide" 
   targetptr="UseMouse"/>

The author can optionally provide link text in the olink content. For an empty olink, the stylesheet is expected to generate the text for the link.

To identify the target document, a targetdoc attribute is used. The targetdoc attribute is a simple identifier string (of type CDATA) that is resolved to an actual document by the stylesheet.

To identify the target element, a targetptr attribute is used. Its value must match that of an id attribute in the target document. Together, the two attributes make it easy to form an olink: you just have to identify the target document and the element within that document.

No other information need be supplied in the XML markup When the XSL stylesheet encounters an olink with these attributes, it uses XSL templates to resolve the link. As you will see, an optional xrefstyle attribute can be added to provide hints to the stylesheet on how to format a given olink instance.

Of course, the stylesheet cannot resolve such links on its own, so it must be supplied with sufficient information to do so. In the XSL implementation, the stylesheet is given a parameter (in a customization layer or at runtime) that points to a database file that provides that information. The advantage of supplying it at runtime is that it can be different for different builds of the document. A new release can refer to a new version of the target database. An HTML build can refer to an HTML rendering of the target documents.

The target database is a structured XML document with its own DTD. The data on potential cross references in a document is extracted using the special targets.xsl stylesheet. Then the data for the individual documents is merged into a single target database, and the resulting filename is passed to the DocBook stylesheet as a parameter.. The next section describes this data format.

A target database contains information for all the target documents the stylesheet may need during a given build. The overall structure looks like this:

<?xml version="1.0" encoding="utf-8"?> 
<!DOCTYPE targetset SYSTEM 
               "/tools/docbook-xsl-1.52.1/common/targetdatabase.dtd" [
<!ENTITY ugtargets SYSTEM "/doc/userguide/target.db"> 
<!ENTITY agtargets SYSTEM "/doc/adminguide/target.db">
<!ENTITY reftargets SYSTEM "/doc/man/target.db">
]>
<targetset> 
  <targetsetinfo> 
    Description of this target database document,
    which is for the examples in olink doc.
  </targetsetinfo>

  <!-- Site map for generating relative paths between documents -->
  <sitemap> 
    <dir name="documentation"> 
      <dir name="guides"> 
        <dir name="mailuser"> 
          <document targetdoc="MailUserGuide"  baseuri="userguide.html"> 
            &ugtargets; 
          </document>
        </dir>
        <dir name="mailadmin">
          <document targetdoc="MailAdminGuide">
            &agtargets;
          </document>
        </dir>
      </dir>
      <dir name="reference">
        <dir name="mailref">
          <document targetdoc="MailReference">
            &reftargets;
          </document>
        </dir>
      </dir>
    </dir>
  </sitemap>
</targetset>
     
      

		Set the database encoding to utf-8 for the database, regardless of what encoding your documents are written in. The individual data files are written out in utf-8 so a database can have mixed languages and not have mixed encodings.
		Declare a system entity for each document target data file.
		Root element for the database is targetset.
		The targetsetinfo element is optional, and contains a description of the collection.
		The sitemap element contains the framework for the hierarchy of HTML output directories.
		Directory that contains all the HTML output directories.
		Directory that contains only other directories, not documents.
		Directory that contains one or more document output.
		The document element has the document identifier in its targetdoc attribute.
		For documents processed without chunking, the output filename must be provided in the baseuri attribute since that name is not generated by the document itself.
		The system entity reference pulls in the target data for this document.

The data for each <document> is in a separate data file that is pulled in as a system entity reference. That data is structured in a manner similar to its target document. For every hierarchical element (book, chapter, section, appendix, etc.), there is a corresponding <div> element to record its target data. The nesting of div elements captures the parent-child relationships between elements. The value of each div element's id attribute would be captured, except that generated IDs are not included because they are not stable target names. In addition to the hierarchical elements, the collection includes all formal elements (tables, figures, etc.), and any block and inline elements that have ID attributes. These are stored as <obj> elements in the data set.

Here is an example of the target data for one document that contains only an article, a table, and a section:

<?xml version="1.0" ?>
<div element="article" href="#publish" number="" targetptr="publish">
  <ttl>Publishing DocBook Documents</ttl>
  <xreftext>Publishing DocBook Documents</xreftext>
  <obj element="table" href="xsl-processors" number="1" 
                                        targetptr="xsl-processors">
    <ttl>XSL Processors</ttl>
    <xreftext>Table 1</xreftext>
  </obj>
  <div element="section" href="#xsl-arch" number="" targetptr="xsl-arch">
    <ttl>DocBook XSL Architecture</ttl>
    <xreftext>the section called &#8220;DocBook XSL Architecture&#8221;
    </xreftext>
  </div>
</div>
      

In general, generated ID values are excluded from the data set because they are not reliable targets for cross referencing. Certain elements without ID may also be included, however. Those hierarchical elements without an ID attribute may provide some text or context for styling the link data in the cross reference. For example, a chapter might not have an ID attribute, but it still has a number and a title. When a cross reference is to a section that has an ID within that chapter, the link could be styled to say "see the section `Using a mouse' in Chapter 3: Peripherals".

The <div> and <obj> elements are generated using a special template included with DocBook XSL that extracts the target data from each document. The template is triggered with a stylesheet parameter named collect.xref.targets . See the DocBook XSL stylesheet documentation for more information on using the parameter.

The optional baseuri attribute is a prefix for hrefs generated for the target elements in this document. If an HTML document is output as a single HTML file, then the baseuri is that filename, and href attributes in the data file are of the form #elementid. The stylesheets combine the baseuri and href attribute values to form an olink. If the HTML document was chunked on output, then baseuri is usually blank. Then the href attributes in the data file are of the form filename.html#elementid.

The processing of olinks with the DocBook XSL stylesheets can be summarized as follows:

Each of these steps is described in detail below.

A target data file should be produced for each target document, and for each different rendering of a document. Such a rendering encompasses any profiling done from the original XML source file, as well as any transformations performed by the stylesheet. It is such renderings that are the target of rendered cross references from other documents.

In this scheme, the data file must closely match the rendering of the document. Effectively, the datafile should be produced as a side effect of styling the document, at the same time the document output is produced. But it should also be possible to extract the target data without producing the actual rendered output. The stylesheets support both modes of operation.

For a rendering to print, an additional step must be taken to extract the page numbers for each target after the document has been paginated. These page numbers are merged back into the data file based on the ID values associated with them. This feature is dependent on an extension to the FO processor.

During the development process, a target data file should be generated as often as the document is changed. The final processing run of a document before publication should produce a final data file that can be targeted by other documents.

Each document with olinks has a cross referencing scope, which is the collection of documents it is targeting for cross references. To process the olinks in a given document, the target data for all documents in that scope is merged into a single target database. The application can manage this in several ways.

If you are building a stable collection of documents that you cross reference among, you may just merge all of them into one database. Even if document A does not reference document B, it might after further editing or in a future release. If you can clearly define one scope and manage it as one database, then the job is easy.

If you are building documents that refer outside your own collection, then you need to be more careful. Each document may have its own scope, which is defined by the collection of olink targetdoc values in the document. The processing application may assemble a custom database on the fly from a collection of individual target data files. It is your job to make sure the target data is available for any of those target documents.

It is not necessary to move a lot of data around to form such a database. A database is a merger of several XML documents. This can be accomplished today using a wrapper around system entity references or XIncludes. On the other hand, some shops might use Makefiles or Ant tasks to load target data into a relational database such as MySQL, and then extract a custom target database document for each build.

Within a target database file, each document's target data is wrapped in a <document> element. See Section 3.2, “Target data format” for a complete description. Additional attributes of <document> are filled in when the database is assembled.

The database also may have a top-level sitemap child whose structure describes an output hierarchy of directories. Each dir element in the sitemap has a directory name attribute. Directory elements can be nested, as on a filesystem. At runtime, the applicaton passes in a parameter indicating the location in this directory hierarchy where the current document will be placed. This permits the stylesheet to compute a relative HREF from each processed document to each target document. If a target document is not in the output hierarchy, then its data container must have a baseuri attribute that provides the first part of its HREFs.

However the target database is assembled, it is passed at runtime to the DocBook stylesheet as a parameter. The application manages which database is used for each document. Put the pathname to the database document in the target.database.document parameter.

As the document is processed with the stylesheet, the olink template has a routine to look up the data it needs from the database. It uses the targetdoc attribute in the olink to locate the document container it needs in the database. It is an error if it isn't there.

Then it searches for the targetptr attribute among the div and obj descendants of that document container.

Once the proper div or obj element is found in the database, the olink template uses the data in that element to style the link. Depending on the attributes and content of a given olink, this may mean just generating the HREF, or it may include generating the cross reference text. In the case of a print document, the template may insert a page reference or add a document title.

A generated cross reference text string generally uses the already assembled string in the <xreftext> element that was added to the data file when the target document was processed. That string represents the style of cross references in the target document. Or an olink could use a customizable text template to style the raw data fields into a cross reference style that matches the current document. The generated text could use the same template as for <xref> elements, or it could use a template customized for olinks.

Currently olinks use the xref context for text templates to style links. Those templates are part of the common stylesheet templates that are shared between html and print output. If different styling of olinks is needed, a set of text templates using a new olink context rather than the xref context could be created.

An optional xrefstyle attribute can be added to a given olink element to provide hints to the stylesheet on how to format the link. The DTD does not specify standard values for the attribute, so stylesheets are free to implement whatever scheme they want. The DocBook XSL stylesheets provide three ways to use the xrefstyle attribute:

Each of these methods is described in more detail in the reference at the end of this article.

Olinks in the DocBook XSL stylesheets enable several new capbilities in DocBook.

The DocBook DTD supports large documents, all the way up to sets of books. If you want to form cross references among the books in a set using ID/IDREFs, then the set has to be in a single document. This is so unwieldy that most content managers break up the set into separate book system entity files, and only form the complete set at processing time.The individual books may be further broken down into chapter system entities.

System entities are not complete XML documents. If they contain a DOCTYPE declaration, then this will usually generate an error message when the system entity is pulled into the master document and processed.

The XInclude feature of XML permits each modular file to be a complete XML document. The master document uses <xinclude> elements, each of which uses an XPath expression to point to the root element of a modular file. This avoids the problem of the DOCTYPE declaration, which is left out of the inclusion.

Although each modular file may be a complete XML document, it still may not validate. If it has an xref or link element whose linkend attributes points to an ID in another file, then the modular document is not valid on its own. Both xref and link use the ID/IDREF mechanism, so the linkend must be in the current document.

If all the cross references to targets outside a modular component are converted to olink elements, then the file can be validated. That is because the targetdoc and targetptr attributes are not ID/IDREF, and so they won't be checked. The resolution of olinks takes place when the document is processed with a target database, not when the file is validated against the DTD.

Using olinks means a modular file can be processed on its own and produce reasonable output. Any links within itself would be resolved using ID/IDREF, and any links outside of itself would be written as olinks and resolved using the target database.

Of course, when a modular file is processed on its own, certain context information would be lost. The third chapter in a book would not know it was the third chapter when processed by itself, so its chapter number appear as "1". Likewise, all chapters would begin on page 1. But context information can be provided by the target database to correct these problems. The target database for the master document has all the information that's needed to establish the context for the current modular file. The stylesheet could be extended to supply the <div>'s number attribute. That number was generated the last time the target data file was generated for the whole document.

Although modular processing is convenient for development, it is not necessary when rendering the final document. To ensure complete sychronization among the modules, the master document can always be processed whole. This way you have reasonably accurate rendering in modular mode, and completely accurate rendering for final output. And it is the master document's target data file that would be used by other documents for linking.

Asynchronous processing means not having to process all your content at once to get cross references to work. You can use the target data from a previously processed document to form a link. The trick is to ensure that the target data won't change again after you process your document with the link to it.

If you are cross referencing among a collection of documents you are developing, then you have control over the target data. To generate a target database, you can freeze the documents and collect the target data. You can do this with a lock or a time stamp in a revision control system, or by simple agreement among developers. Each data collection becomes the reference point for the next cycle of development. At some point the collection is processed for a final time, the output is published, and the target database is archived for future reference.

An author can build their own output as they need to without having to build everything. Different departments or groups can work on their own documents, as long as they regularly synchronize the target data.

Going beyond parallel development is developing documents in sequence. Later documents can safely make references to documents that have been published and their target data files frozen. You can even cross reference to documents whose source you have never seen, as long as you have access to the final target data file.

Olinks managed with this scheme permit you to cross reference to open source documents that are authored in DocBook XML. Often open source documents are the most authoritative on a subject, or at least on the software they document. Most are published in isolation, even if they are dependent on other software, since they have no means of connecting to other documents. Many could at least make use of active links to reference pages for the operating system they run on.

Open source software can also be assembled like building blocks into larger applications that integrate the components into a system. The documentation for the application needs to cross reference to the component docs. Without such links, the application doc needs to duplicate the information, which introduces more development and maintenance work for the authors.

If the DocBook source files are available, and you are rendering those docs, then you can extract the target data yourself to add to your target database. Since open source docs are generally versioned, you can easily tell which version of content you are linking to. Since you are creating the rendering, you know the rendering version as well.

The distributor of the doc will often ship rendered versions along with or instead of the XML source. If the distributor builds a target data file for each rendering to include in the distribution, then that can be added to your target database. Even if the XML source is not included in the distribution, links can be made to the rendered docs.

If the target data is produced as a side effect of rendering the document (controlled by a command line parameter), then the data can be generated regularly. When the final rendering is done for release, the data file can be simply included.

With the delays in getting general support for styling XML directly in web browsers, many content providers want to convert XML to HTML on the fly in the web server. But if your XML is a DocBook book, then the user will likely experience significant delays while the whole book is processed. There will be even greater delays while the entire book is delivered as HTML. You can chunk the output to reduce delivery time, but you still must process the whole document each time to resolve ID/IDREF cross references.

These delays can be reduced by breaking up the large book files into smaller modular files. and using olink with a document target database to resolve cross references within the document. Also, the target database can provide the context for chapter numbering and such for modular files. It is much faster to parse the target data file than it is to parse the entire book to establish the context.

So an XML repository that consists of small modular XML files linked together with olink and a target database could be rendered to HTML on demand. Whenever a document module in the repository is updated, it also has its target data updated in the database. When another document that targets the changed document is requested and processed, it uses the updated data to render the links.

When managing a collection of documentation, it is often the case that only certain documents are translated into other languages. If your documents are cross referencing among themselves, then some attention must be given to this issue. You don't want to form a cross reference to a translated document that does not exist.

This problem is solved by properly assembling your target database for your translated documents. The target database supports document elements with duplicate values of the targetdoc attribute when there are also lang attributes to distinguish among them. That lets you to create a database with references to all language versions of a given document.

When you process your olinks, the stylesheet first tries to find a link in the same language as the document containing the olink (as specified by its lang attribute). If the link is not found in that language, then it tried other languages as specified in the stylesheet parameter named olink.lang.fallback.sequence. That is a space separated list of language codes that specifies the sequence of languages to try if the first fails.

This makes it easier to manage projects where not all documents are translated at once. As a translation becomes available, it can be added to the database. Then the next time a document with olinks is processed, that data is available for inclusion. But the fallback language is there until the translation is available.

This document has described the implementation of olink processing in the DocBook XSL stylesheets to support cross referencing between documents. The keys points of this scheme are:

For further information, see the chapter on olinking in DocBook XSL: the Complete Guide [http://www.sagehill.net/docbookxsl/index.html]

Biography