16 Linking, Segmentation, and Alignment

This chapter discusses a number of ways in which encoders may represent analyses of the structure of a text which are not necessarily linear or hierarchic. The module defined by this chapter provides for the following common requirements:

These facilities all use the same set of techniques based on the W3C XPointer framework (Grosso et al. (eds.) (2003)) This provides a variety of schemes; the most convenient of which, and that recommended by these Guidelines, makes use of the global xml:id attribute, as defined in section 1.3.1.1 Global Attributes, and introduced in the section of v. A Gentle Introduction to XML titled Identifiers and Indicators. When the linking module is included in a schema, the attribute class att.global is extended to include eight additional attributes to support the various kinds of linking listed above. Each of these attributes is introduced in the appropriate section below. In addition, for many of the topics discussed, a choice of methods of encoding is offered, ranging from simple but less general ones, which use attribute values only, to more elaborate and more general ones, which use specialized elements.

⚓︎16.1 Links

We say that one element points to others if the first has an attribute whose value is a reference to the others: such an element is called a pointer element, or simply a pointer. Among the pointers that have been introduced up to this point in these Guidelines are note, ref, and ptr. These elements all indicate an association between one place in the document (the location of the pointer itself) and one or more others (the elements whose identifiers are specified by the pointer's target attribute). The module described in this chapter introduces a variation on this basic kind of pointer, known as a link, which specifies both ‘ends’ of an association. In addition, we define a syntax for representing locations in a document by a variety of means not dependent on the use of xml:id attributes.

⚓︎16.1.1 Pointers and Links

In section 3.7 Simple Links and Cross-References we introduced the simplest pointer elements, ptr and ref. Here we introduce additionally the link element, which represents an association between two (or more) locations by specifying each location explicitly. Its own location is irrelevant to the intended linkage. All three elements use the attribute target, provided by the att.pointing class as a means of indicating the location or locations referenced or pointed to.

  • att.pointing provides a set of attributes used by all elements which point to other elements by means of one or more URI references.
    targetspecifies the destination of the reference by supplying one or more URI References
  • link (link) defines an association or hypertextual link among elements or passages, of some type not more precisely specifiable by other elements.

The ptr element may be called a ‘pure pointer’, because its primary function is simply to point. A pointer sets up a connection between an element (which, in the case of a pure pointer, is simply a location in a document), and one or more others, known collectively as its target. The ptr and ref elements point, conceptually, at a single target, even if that target may be discontinuous in the document. The link element specifies at least two targets and represents an association between them, independent of its own location.

These three elements also share a common set of attributes, derived from the att.pointing and att.typed classes:

  • att.pointing provides a set of attributes used by all elements which point to other elements by means of one or more URI references.
    evaluate(evaluate) specifies the intended meaning when the target of a pointer is itself a pointer.
  • att.typed provides attributes that can be used to classify or subclassify elements in any way.
    typecharacterizes the element in some sense, using any convenient classification scheme or typology.
    subtype(subtype) provides a sub-categorization of the element, if needed
Double connection among elements could also be expressed by a combination of pointer elements, for example, two ptr elements, or one ptr element and one note element. All that is required is that the value of the target (or other pointing) attribute of the one be the value of the xml:id attribute of the other. What the link element accomplishes is the handling of double connection by means of a single element. Thus, in the following encoding:
<ptr xml:id="sa-p1target="#sa-p2"/>
<ptr xml:id="sa-p2target="#sa-p1"/>
sa-p1 points to sa-p2, and sa-p2 points to sa-p1. This is logically equivalent to the more compact encoding:
<link target="#sa-p1 #sa-p2"/>

As noted elsewhere, the target attribute may take as value one or more URI reference. In the simplest case, each such reference will indicate an element in the current document (or in some other document), for example by supplying the value used for its global xml:id attribute. It may however carry as value any form of URI, such as a URL pointing to some other document or location on the Internet. Pointing or linking to external documents and pointing and linking where identifiers are not available is described below in section 16.2 Pointing Mechanisms.

⚓︎16.1.2 Using Pointers and Links

As an example of the use of mechanisms which establish connections among elements, consider the practice (common in 18th century English verse and elsewhere) of providing footnotes citing parallel passages from classical authors.
The figure shows the original page of Pope's Dunciad which is discussed in the text.
Such footnotes can of course simply be encoded using the note element (see section 3.9 Notes, Annotation, and Indexing) without a target attribute, placed adjacent to the passage to which the note refers:63
<l>(Diff'rent our parties, but with equal grace</l>
<l>The Goddess smiles on Whig and Tory race,</l>
<l>
 <note type="imitationplace="bottom">
  <bibl>Virg. Æn. 10.</bibl>
  <quote>
   <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
   <l>—— Rex Jupiter omnibus idem.</l>
  </quote>
 </note>'Tis the same rope at sev'ral ends they twist,
</l>
<l>To Dulness, Ridpath is as dear as Mist)</l>

This use of the note element can be called implicit pointing (or implicit linking). It relies on the juxtaposition of the note to the text being commented on for the connection to be understood. If it is felt that the mere juxtaposition of the note to the text does not make it sufficiently clear exactly what text segment is being commented on (for example, is it the immediately preceding line, or the immediately preceding two lines, or what?), or if it is decided to place the note at some distance from the text, then the pointing or the linking must be made explicit. We now consider various methods for doing that.

Firstly, a ptr element might be placed at an appropriate point within the text to link it with the annotation:
<l>(Diff'rent our parties, but with equal grace</l>
<l>The Goddess smiles on Whig and Tory race,
<ptr rend="unmarkedtarget="#note3.284"/>
</l>
<l>'Tis the same rope at sev'ral ends they twist,</l>
<l>To Dulness, Ridpath is as dear as Mist)</l>
<note xml:id="note3.284type="imitation"
 place="bottom">

 <bibl>Virg. Æn. 10.</bibl>
 <quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem.</l>
 </quote>
</note>
The note element has been given an arbitrary identifier (note3.284) to enable it to be specified as the target of the pointer element. Because there is nothing in the text to signal the existence of the annotation, the rend attribute has been given the value unmarked.
Secondly, the target attribute of the note element can be used to point at its associated text, provided that an xml:id attribute has been supplied for the associated text:
<l xml:id="L3.283">(Diff'rent our parties, but with equal grace</l>
<l xml:id="L3.284">The Goddess smiles on Whig and Tory race,</l>
<l xml:id="L3.285">'Tis the same rope at sev'ral ends they twist,</l>
<l xml:id="L3.286">To Dulness, Ridpath is as dear as Mist)</l>
<!-- ... -->
Given this encoding of the text itself, we can now link the various notes to it. In this case, the note itself contains a pointer to the place in the text which it is annotating; this could be encoded using a ref element, which bears a target attribute of its own and contains a (slightly misquoted) extract from the text marked as a quote element:
<note type="imitationplace="bottom"
 target="#L3.284">

 <ref rend="sctarget="#L3.284">Verse 283–84.
 <quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
 </ref>
 <bibl>Virg. Æn. 10.</bibl>
 <quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem. </l>
 </quote>
</note>
Combining these two approaches gives us the following associations:
  • a pointer within one line indicates the note
  • the note indicates the line
  • a pointer within the note indicates the line
Note that we do not have any way of pointing from the line itself to the note: the association is implied by containment of the pointer. We do not as yet have a true double link between text and note. To achieve that we will need to supply identifiers for the annotations as well as for the verse lines, and use a link element to associate the two. Note that the ptr element and the target attribute on the note may now be dispensed with:
<note xml:id="n3.284type="imitation"
 place="bottom">

 <ref rend="sctarget="#L3.284">Verse 283–84.
 <quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
 </ref>
 <bibl>Virg. Æn. 10.</bibl>
 <quote>
  <l>Tros Rutulusve fuat; nullo discrimine habebo.</l>
  <l>—— Rex Jupiter omnibus idem. </l>
 </quote>
</note>
<link target="#n3.284 #L3.284"/>
The target attribute of the link element here bears the identifier of the note followed by that of the verse line. We could also allocate an identifier to the reference within the note and encode the association between it and the verse line in the same way:
<note type="imitationplace="bottom">
 <ref rend="scxml:id="r3.284"
  target="#L3.284">
Verse 283–84.
 <quote>
   <l>——. With equal grace</l>
   <l>Our Goddess smiles on Whig and Tory race.</l>
  </quote>
 </ref>
<!-- ... -->
</note>
<!-- ... -->
<link target="#r3.284 #L3.284"/>
Indeed, the two links could be combined into one, as follows:
<link target="#n3.284 #r3.284 #L3.284"/>

⚓︎16.1.3 Groups of Links

Clearly, there are many reasons for which an encoder might wish to represent a link or association between different elements. For some of them, specific elements are provided in these Guidelines; some of these are discussed elsewhere in the present chapter. The link element is a general purpose element which may be used for any kind of association. The element linkGrp may be used to group links of a particular type together in a single part of the document; such a collection may be used to represent what is sometimes referred to in the literature of Hypertext as a web, a term introduced by the Brown University FRESS project in 1969, and not to be confused with the World Wide Web.

  • linkGrp (link group) defines a collection of associations or hypertextual links.

As a member of the class att.pointing.group, this element shares the following attributes with other members of that class:

  • att.pointing.group provides a set of attributes common to all elements which enclose groups of pointer elements.
    domainsoptionally specifies the identifiers of the elements within which all elements indicated by the contents of this element lie.
    targFunc(target function) describes the function of each of the values of the target attribute of the enclosed link, join, or alt tags.

It is also a member of the att.pointing and att.typed classes, and therefore also carries the attributes specified in section 16.1.1 Pointers and Links above, in particular the type attribute.

The linkGrp element provides a convenient way of establishing a default for the type attribute on a group of links of the same type: by default, the type attribute on a link element has the same value as that given for type on the enclosing linkGrp.

Typical software might hide a web entirely from the user, but use it as a source of information about links, which are displayed independently at their referenced locations. Alternatively, software might provide a direct view of the link collection, along with added functions for manipulating the collection, as by filtering, sorting, and so on. To continue our previous example, this text contains many other notes of a kind similar to the one shown above. Here are a few more of the lines to which annotations have to be attached, followed by the annotations themselves:
<l xml:id="L2.79">A place there is, betwixt earth, air and seas</l>
<l xml:id="L2.80">Where from Ambrosia, Jove retires for ease.</l>
<!-- ... -->
<l xml:id="L2.88">Sign'd with that Ichor which from Gods distills.</l>
<!-- ... -->
<note xml:id="n2.79place="bottom"
 anchored="false">

 <bibl>Ovid Met. 12.</bibl>
 <quote xml:lang="la">
  <l>Orbe locus media est, inter terrasq; fretumq;</l>
  <l>Cœlestesq; plagas —</l>
 </quote>
</note>
<note xml:id="n2.88place="bottom"
 anchored="false">
Alludes to <bibl>Homer, Iliad 5</bibl> ...
</note>
To avoid having to repeat the specification of type as imitation on each note, we may specify it once for all on a linkGrp element containing all links of this type.
<linkGrp type="imitation">
 <link target="#n2.79 #L2.79"/>
 <link target="#n2.88 #L2.88"/>
 <link target="#n3.284 #L3.284"/>
</linkGrp>
Additional information for applications that use linkGrp elements can be provided by means of special attributes. First, the domains attribute can be used to identify the text elements within which the individual targets of the links are to be found. Suppose that the text under discussion is organized into a body element, containing the text of the poem, and a back element containing the notes. Then the domains attribute can have as its value the identifiers of the body and the back, to enable an application to verify that the link targets are in fact contained by appropriate elements, or to limit its search space:

<!-- ... --><linkGrp type="imitation"
 domains="#dunciad #dunnotes">

 <link target="#n2.79 #L2.79"/>
 <link target="#n2.88 #L2.88"/>
<!-- ... -->
 <link target="#n3.284 #L3.284"/>
<!-- ... -->
</linkGrp>

Note that there must be a single parent element for each ‘domain’; if some notes are contained by a section with identifier dunnotes, and others by a section with identifier dunimits, an intermediate pointer must be provided (as described in section 16.1.4 Intermediate Pointers) within the linkGrp and its identifier used instead.

Next, the targFunc attribute can be used to provide further information about the role or function of the various targets specified for each link in the group. The value of the targFunc attribute is a list of names (formally, name tokens), one for each of the targets in the link; these names can be chosen freely by the encoder, but their significance should be documented in the encoding description in the header.64 In the current example, we might think of the note as containing the source of the imitation and the verse line as containing the goal of the imitation. Accordingly, we can specify the linkGrp in the preceding example thus:
<linkGrp type="imitation"
 domains="#dunciad #dunnotestargFunc="source goal">

 <link target="#n2.79 #L2.79"/>
 <link target="#n2.88 #L2.88"/>
<!-- ... -->
 <link target="#n3.284 #L3.284"/>
<!-- ... -->
</linkGrp>

⚓︎16.1.4 Intermediate Pointers

In the preceding examples, we have shown various ways of linking an annotation and a single verse line. However, the example cited in fact requires us to encode an association between the note and a pair of verse lines (lines 284 and 285); we call these two lines a span.

There are a number of possible ways of correcting this error: one could use the target attribute to indicate one end of the span and the special purpose targetEnd attribute on the note element to point to the other. Another possibility might be to create an element which represents the whole span itself and assign that an xml:id attribute, which can then be linked to the note and ref elements. This could be done using for example the lg element defined in section 3.13.1 Core Tags for Verse or the ‘virtual’ join element discussed in section 16.7 Aggregation.

A third possibility would be to use an ‘intermediate pointer’ as follows:
<ptr xml:id="L3.283-284"
 target="#L3.283 #L3.284"/>
When the target attribute of a ptr or ref element specifies more than one element, the indicated elements are intended to be combined or aggregated in some way to produce the object of the pointer. (Such aggregation is however the task of a processing application, and cannot be defined simply by the markup). The xml:id attribute of the ptr then provides an identifier which can be linked to the note and ref elements:
<link evaluate="all"
 target="#n3.284 #r3.284 #L3.283-284"/>

The all value of evaluate is used on the link element to specify that any pointer encountered as a target of that element is itself evaluated. If evaluate had the value none, the link target would be the pointer itself, rather than the objects it points to.

Where a linkGrp element is used to group a collection of link elements, any intermediate pointer elements used by those link elements should be included within the linkGrp.

⚓︎16.2 Pointing Mechanisms

This section introduces more formally the pointing mechanisms available in the TEI. In addition to those discussed so far, the TEI provides methods of pointing:

  • into documents other than the current document;
  • to a particular element in a document other than the current document using its xml:id;
  • to a particular element whether in the current document or not, using its position in the XML element tree;
  • at arbitrary content in any XML document using TEI-defined XPointer schemes.

All TEI attributes used to point at something else are declared as having the datatype teidata.pointer, which is defined as a URI reference65; the cases so far discussed are all simple examples of a URI reference. Another familiar example is the mechanism used in XHTML to create represent hypertext links by means of the XHTML href attribute. A URI reference can reference the whole of an XML resource such as a document or an XML element, or a sub-portion of such a resource, identified by means of an appropriate fragment identifier. Technically speaking, the ‘fragment identifier’ is that portion of a URI reference following the first unescaped ‘#’ character; in practice, it provides a means of accessing some part of the resource described by the URI which is less than the whole.

The first three of the following subsections provide only a brief overview and some examples of the W3C mechanisms recommended. More detailed information on the use of these mechanisms is readily available elsewhere.

⚓︎16.2.1 Pointing Elsewhere

Like the ubiquitous if misnamed XHTML pointing attribute href, the TEI pointing attributes can point to a document that is not the current document (the one that contains the pointing element) whether it is in the same local filesystem as the current document, or on a different system entirely. In either case, the pointing can be accomplished absolutely (using the entire address of the target document) or relatively (using an address relative to the current base URI in force). The ‘current base URI’ is defined according to Marsh and Tobin 2009. If there is none, the base URI is that of the current document. In common practice the current base URI in force is likely to be the value of the xml:base attribute of the closest ancestor that has one. However this may not be the case, since xml:base attributes are accumulated through the hierarchy by concatenation of path segments, beginning at the top of the hierarchy and proceeding down to the context node.

The following example demonstrates an absolute URI reference that points to a remote document:
The current base URI in force is as defined in the
W3C <ref target="https://www.w3.org/TR/xmlbase/">XML
Base</ref> recommendation.
This example points explicitly to a location on the Web, accessible via HTTP. Suppose however that we wish to access a document stored locally in a file. Again we will supply an absolute URI reference, but this time using a different protocol:
This Debian package is distributed under the terms
of the <ref target="file:///usr/share/common-licenses/GPL-2">GNU General Public License</ref>.
In the following example, we use a relative URI reference to point to a local document:
<figure rend="float fullpage">
 <graphic url="Images/compic.png"/>
 <figDesc>The figure shows the page from the <title>Orbis
     pictus</title> of Comenius which is discussed in the text.</figDesc>
</figure>
Since no xml:base is specified here, the location of the resource Images/compic.png is determined relative to the resource indicated by the current base URI, which is the current document.
In the following example, however, we first change the current base URI by setting a new value for xml:base. The resource required is then identified by means of a relative URI:
<div type="chap"
 xml:base="http://classics.mit.edu/">

 <head>On Ancient Persian Manners</head>
 <p>In the very first story of <ref target="Sadi/gulistan.2.i.html">
   <title>The Gulistan of
       Sa'di</title>
  </ref>,
   Sa'di relates moral advice worthy of Miss Minners ...</p>
<!-- ... -->
</div>

As noted above, the current base URI is found on the nearest ancestor. It is technically possible to use xml:base as a means to shorten URIs, but this usage is not recommended. Abbreviated pointers provide a more flexible and consistent method for creating shorthand links.

⚓︎16.2.2 Pointing Locally

A shorthand pointer, in which the URI consists only of # followed by the value of an xml:id acts as a pointer to the element in the current document with that xml:id, as in the following example.
<div type="sectionxml:id="sect106">
<!-- ... -->
</div>
<div type="sectionn="107xml:id="sect107">
 <head>Limitations on exclusive rights: Fair use</head>
 <p>Notwithstanding the provisions of
 <ref target="#sect106">section 106</ref>, the fair use of a
   copyrighted work, including such use by reproduction in copies
   or phonorecords or by any other means specified by that section,
   for purposes such as criticism, comment, news reporting,
   teaching (including multiple copies for classroom use),
   scholarship, or research, is not an infringement of copyright.
   In determining whether the use made of a work in any particular
   case is a fair use the factors to be considered shall
   include — 
 <list rend="bulleted">
   <item n="(1)">the purpose and character of the use, including
       whether such use is of a commercial nature or is for nonprofit
       educational purposes;</item>
   <item n="(2)">the nature of the copyrighted work;</item>
   <item n="(3)">the amount and substantiality of the portion
       used in relation to the copyrighted work as a whole;
       and</item>
   <item n="(4)">the effect of the use upon the potential market
       for or value of the copyrighted work.</item>
  </list>
   The fact that a work is unpublished shall not itself bar a
   finding of fair use if such finding is made upon consideration
   of all the above factors.</p>
</div>
This method of pointing, by referring to the xml:id of the target element as a bare name only (e.g., #sect106) is the simplest and often the best approach where it can be applied, i.e. where both the source element and target element are in the same XML document, and where the target element carries an identifier. It is the method used extensively in previous sections of this chapter and elsewhere in these Guidelines.

⚓︎16.2.3 Using Abbreviated Pointers

Even in the case of relative links on the local file system, ref or target attributes may become quite lengthy and make XML code difficult to read. To deal with this problem, the TEI provides a useful method of using abbreviated pointers and documenting a way to dereference them automatically.

Imagine a project which has a large collection of XML documents organized like this:

  • anthology
    • poetry
      • poem.xml
    • prose
      • novel.xml
  • references
    • people
      • personography.xml
If you want to link a name in the novel.xml file to a person in the personography.xml file, the link will look like this:
<name ref="../../references/people/personography.xml#fred">Fred</name>
If there are many names to tag in a single paragraph, the XML encoding will be congested, and such lengthy links are prone to typographical error. In addition, if the project organization is changed, every relative link will have to be found and altered.
One way to deal with this is to use what is often referred to as a "magic token". You could make such links using the key attribute:
<name key="fred">Fred</name>
and document the meaning of the key using (for instance) a taxonomy element in the TEI header, as described in 3.6.1 Referring Strings. However, such a link cannot be mechanically processed by an external system that does not know how to interpret it; a human will have to read the header explanation and write code explicitly to reconstruct the intended link.
A more robust alternative is to use a private URI scheme. This is a method of constructing a simple, key-like token which functions as a teidata.pointer, and can therefore be used as the value of any attribute which has that datatype, such as ref and target. Such a scheme consists of a prefix with a colon, and then a value. You might, for example, use the prefix psn (for "person"), and structure your name tags like this:
<name ref="psn:fred">Fred</name>
How is this different from a ‘magic token’? Essentially, it isn't, except that TEI provides a structured method of dereferencing it (turning it into a computable path, such as ../../references/people/personography.xml#fred) by means of a declaration inside encodingDesc in the TEI header, using the elements and attributes for prefix declaration:
  • listPrefixDef (list of prefix definitions) contains a list of definitions of prefixing schemes used in teidata.pointer values, showing how abbreviated URIs using each scheme may be expanded into full URIs.
  • prefixDef (prefix definition) defines a prefixing scheme used in teidata.pointer values, showing how abbreviated URIs using the scheme may be expanded into full URIs.
    identsupplies a name which functions as the prefix for an abbreviated pointing scheme such as a private URI scheme. The prefix constitutes the text preceding the first colon.
  • att.patternReplacement provides attributes for regular-expression matching and replacement.
    matchPatternspecifies a regular expression against which the values of other attributes can be matched.
    replacementPatternspecifies a ‘replacement pattern’, that is, the skeleton of a relative or absolute URI containing references to groups in the matchPattern which, once subpattern substitution has been performed, complete the URI.
This is how you might document a private URI scheme using the psn: prefix:
<listPrefixDef>
 <prefixDef ident="psn"
  matchPattern="([a-z]+)"
  replacementPattern="../../references/people/personography.xml#$1">

  <p> In the context of this project, private URIs with the prefix
     "psn" point to <gi>person</gi> elements in the project's
     personography.xml file.
  </p>
 </prefixDef>
</listPrefixDef>
This specifies that where a teidata.pointer value is constructed with a psn: prefix, a regular-expression replace operation can be performed on it to construct the full or relative URI to the target document or fragment. listPrefixDef is a child of encodingDesc, and it contains any number of prefixDef elements. Each prefixDef element provides a method of dereferencing or expanding an abbreviated pointer, based on a regular expression. The ident attribute specifies the prefix to which the expansion applies (without the colon). The matchPattern attribute contains a regular expression which is matched against the component of the pointer following the first colon, and the replacementPattern provides the string which will be used as a replacement. In this example, using psn:fred, the value fred would be matched by the matchPattern, and also captured (through the parentheses in the regular expression); it would then be replaced by the value ../../references/people/personography.xml#fred (with the the $1 in the replacementPattern being replaced by the captured value). The p element inside the prefixDef can be used to provide a human-readable explanation of the usage of this prefix.

Through this mechanism, any processor which encounters a teidata.pointer with a protocol unknown to it can check the listPrefixDef in the header to see if there is an available expansion for it, and if there is, it can automatically provide the expansion and generate a full or relative URI.

For any given prefix, it may be useful to supply more than one expansion. For instance, in addition to pointing at the person element in the personography file, it might also be useful to point to an external source which is available on the network, representing the same information in a different way. So there might be a second prefixDef like this:
<prefixDef ident="psn"
 matchPattern="([a-z]+)"
 replacementPattern="http://www.example.com/personography.html#$1">

 <p> Private URIs with the prefix "psn" can be converted to point
   to a fragment on the Personography page of the project Website.
 </p>
</prefixDef>
Any number of prefixDef elements may be provided for the same prefix. A processor may decide to process one or all of them; if it processes only one, it should choose the first one with the correct ident value, so the primary or most important prefixDef for any given prefix should appear first in its parent listPrefixDef.

When creating private URI schemes, it is recommended that you avoid using any existing registered prefix. A list of registered prefixes is maintained by IANA at https://www.iana.org/assignments/uri-schemes.html.

Note that this mechanism can also be used to dereference other abbreviated pointing systems which are based on prefixes, such as Tag URIs.

The matchPattern and replacementPattern attributes are also used in dereferencing canonical reference patterns, and further examples of the use of regular expressions are shown in 16.2.5 Canonical References.

⚓︎16.2.4 TEI XPointer Schemes

The pointing schemes described in this chapter are part of a number of such schemes envisaged by the W3C, which together constitute a framework for addressing data within XML documents, known as the XPointer Framework (Grosso et al 2003). This framework permits the definition of many other named addressing methods, each of which is known as an XPointer Scheme. The W3C has predefined a set of such schemes, and maintains a register for their expansion.

One important scheme, also defined by the W3C, and recommended by these Guidelines is the xpath() pointer scheme, which allows for any part of an XML structure to be selected using the syntax defined by the XPath specification. This is further discussed below, 16.2.4.2 xpath(). These Guidelines also define six other pointer schemes, which provide access to parts of an XML document such as points within data content or stretches of data content. These additional TEI pointer schemes are defined in sections 16.2.4.3 left() to 16.2.4.8 match() below.

⚓︎16.2.4.1 Introduction to TEI Pointers

Before discussing the TEI pointer schemes, we introduce slightly more formally the terminology used to define them. So far, we have discussed only ways of pointing at components of the XML information set node such as elements and attributes. However, there is often a need in text analysis to address additional types of location such as the ‘point’ locations between ‘nodes’, and ‘sequences’ that may arbitrarily cross the boundaries of nodes in a document. The content of an XML document is organized sequentially as well as hierarchically, and it makes sense to consider ranges of characters within a document independently of the nodes to which they belong. From the perspective of most of the pointer schemes discussed below, a TEI document is a tree structure superimposed upon a character stream. Nodes are entities available only in the tree, while points are available only in the stream. For this reason, the schemes below that rely upon character positions (string-index(), string-range(), and match()) cannot take nodes into account. Conversely, XPath (disregarding functions that return atomic values) is a method for locating nodes in the tree and treats those nodes as indivisible units, meaning it is unable to address parts of nodes in their document context.

The TEI pointer scheme thus distinguishes the following kinds of object:

Node
A node is an instance of one of the node kinds defined in the XQuery and XPath Data Model 3.1. It represents a single item in the XML information set for a document. For pointing purposes, the only nodes that are of interest are Text Nodes, Element Nodes, and Attribute Nodes.
Sequence
A Sequence follows the definition in the XPath 3.1 Data Model, with one alteration. A Sequence is an ordered collection of zero or more items, where an item is either a node or a partial text node.
Text Stream
A Text Stream is the concatenation of the text nodes in a document and behaves as though all tags had been removed. A text stream begins at a reference node and encompasses all of the text inside that node (if any) and all the text following it in document order. In XPath terms, this would encompass all of the text nodes beginning at a particular node, and following it on the following axis.
Point
A Point represents a dimensionless point between nodes or characters in a document. Every point is adjacent to either characters or elements, and never to another point. Points can only be referenced in relation to an element or text node in the document (i.e. something addressable by either an XPath or a fragment identifier). Points occur either immediately before or after an element, or at a numbered position inside a text stream. Position zero in the stream would be immediately before the first character. Note that points within attribute values cannot mark the beginning or end of a range extending beyond the attribute value, because points indicate a position within a document. Since attribute nodes are by definition un-ordered, they cannot be said to have a fixed position.

The TEI recommends the following seven pointer schemes:

xpath()
Addresses a node or node sequence using the XPath syntax. (16.2.4.2 xpath())
left() and right()
addresses the point before (left) or after (right) a node or node sequence (16.2.4.3 left() and 16.2.4.4 right())
string-index()
addresses a point inside a text node (16.2.4.5 string-index()
range()
addresses the range between two points (16.2.4.6 range())
string-range()
addresses a range of a specified length starting from a specified point (16.2.4.7 string-range())
match()
addresses a range which matches a specified string within a node (16.2.4.8 match())

The xpath() scheme refers to the existing XPath specification which is adopted with one modification: the default namespace for any XPath used as a parameter to this scheme is assumed to be the TEI namespace http://www.tei-c.org/ns/1.0.

The other six schemes overlap in functionality with a W3C draft specification known as the XPointer scheme draft, but are individually much simpler. At the time of this writing, there is no current or scheduled activity at the W3C towards revising this draft or issuing it as a recommendation.

A note on namespaces: The W3C defines an xmlns() scheme (see XPointer xmlns() Scheme) which when prepended to a resolvable pointer allows for the definition of namespace prefixes to be used in XPaths in subsequent pointers. TEI Pointer schemes assume that un-prefixed element names in TEI Pointer XPaths are in the TEI namespace, http://www.tei-c.org/ns/1.0. The use of xmlns() is thus optional, provided no new prefixes need to be defined. If the schemes described here are used to address non-TEI elements, then any new prefixes to be used in pointer XPaths may be defined using the xmlns() scheme.

⚓︎16.2.4.2 xpath()

Sequence xpath(XPATH)

The xpath() scheme locates zero or more nodes within an XML Information Set. The single argument XPATH is an XPath selection pattern, as defined in XSLT 3.0, that returns a node or sequence of nodes. XPaths returning atomic values (e.g. substring()) are illegal in the xpath() scheme because they represent extracted values rather than locations in the source document. Because the schemes below involve starting at a node and navigating from there, and because attribute nodes have no intrinsic order, XPath expressions that address attribute nodes should be avoided in schemes other than xpath().

The example below, and all subsequent examples in this section refer to the following TEI fragment:
<div xml:lang="latype="edition"
 xml:space="preserve">
<ab> <lb n="1xml:id="line1"/><supplied reason="lost">si</supplied> non <choice><reg>habui</reg><orig>abui</orig></choice> quidquam vaco  <lb n="2"/>si<gap reason="illegiblequantity="3"
 unit="character"/>
b<gap reason="illegiblequantity="3"
 unit="character"/>
    cohort<unclear>e</unclear> mi rescribas  <lb n="3"/><unclear>s</unclear>emp<unclear>er</unclear> in <choice><reg>mente</reg><orig>mentem</orig></choice>    <choice><reg>habe</reg><orig>abe</orig></choice> supra res  <lb n="4"/>scriptas<gap reason="lostextent="unknown"
 unit="character"/>
  <lb n="5"/>auge et opto u<unclear>t</unclear> bene valeas</ab>
</div>

A TEI Pointer that referenced the reg element in the choice in line 1 of the example might look like:
#xpath(//lb[@n='1']/following-sibling::choice[1]/reg). Note that XPath values must be assumed to start from the document root. They cannot be relative to the element bearing the attribute that uses the pointer because TEI Pointers are URIs. Care should be taken to ensure that XPaths used in TEI Pointers match only a single node, unless multiple matches are desired. The examples that follow are relatively simple because the document they refer to is short and does not contain many elements.

When an XPath is interpreted by a TEI processor, the information set of the referenced document is interpreted without any additional information supplied by any schema processing that may or may not be present. In particular this means that no whitespace normalization is applied to a document before the XPath is interpreted.

This pointer scheme allows easy, direct use of the most widely-implemented XML query method. It is probably the most robust pointing mechanism for the common situation of selecting an XML element or its contents where an xml:id is not present. The ability to use element names and attribute names and values makes xpath() pointers more robust than the other mechanisms discussed in this section even if the designated document changes. For durability in the presence of editing, use of xml:id is always recommended when possible.

⚓︎16.2.4.3 left()

Point left( IDREF | XPATH )

The left() scheme locates the point immediately preceding the node addressed by its argument, which is either an XPATH as defined above or an IDREF, the value of an xml:id occurring in the document addressed by the base URI in effect for the pointer.

Example: the pointer #left(//supplied[1]) indicates the point between the first lb and the first supplied in the example above.

Example: #left(//gap[1]) indicates the point immediately before the first gap element in line two and the string si.

Example: #left(line1) indicates the point immediately before the <lb n="1"> element.

⚓︎16.2.4.4 right()

Point right( IDREF | XPATH )

The right() scheme locates the point immediately following the node addressed by its argument.

Example: the pointer #right(//lb[@n='3']) indicates the point between the third lb and the <unclear>s</unclear> element in the example.

⚓︎16.2.4.5 string-index()

Point string-index( IDREF | XPATH, OFFSET )

The string-index() scheme locates a point based on character positions in a text stream relative to the node identified by the IDREF or XPATH parameter. The OFFSET parameter is a positive, negative, or zero integer which determines the position of the point. An offset of 0 represents the position immediately before the first character in either the first text node descendant of the node addressed in the first parameter or the first following text node, if the addressed element contains no text node descendants.

Example: #string-index(//lb[@n='2'],1) indicates the point between the ‘s’ and the ‘i’ in the word ‘si’ in line 2.

Note: The OFFSET parameter (and similarly the LENGTH parameter found below in the string-range() scheme) are measured in characters. What is considered a single character will depend on the Normalization Form in use (see UNICODE NORMALIZATION FORMS). A letter followed by a combining diacritic counts as two characters, but the same diacritic precombined with a letter would count as a single character. Compare, for example, é (U+0060 followed by U+0301) and é (>U+00E9). These are equivalent, and a conversion between Normalization Forms C and D will transform one into the other. This specification does not mandate a particular Normalization Form (see Precomposed and Combining Characters and Normalization), but users and implementers should be aware that it affects the character count and therefore the result of evaluating pointers that rely on character counting.

⚓︎16.2.4.6 range()

Sequence range( POINTER, POINTER[, POINTER, POINTER ...])

The range() scheme takes as parameters one or more pairs of POINTERs, which are each members of the set IDREF, XPATH, left(), right(), or string-index(). A range() locates a (possibly non-contiguous) sequence beginning at the first POINTER parameter and ending at the last. If a POINTER locates a node (i.e. is an XPATH or IDREF), then that node is a member of the addressed sequence. If a sequence addressed by a range pointer overlaps, but does not wholly contain, an element (i.e. it contains only the start but not the end tag or vice-versa), then that element is not part of the sequence.

Range()s may address sequences of non-contiguous nodes. For example, a range() might select text beginning before an app, encompassing the content of a single rdg and continuing after the app.

Example: #range(left(//lb[@n='3']),left(//lb[@n='4'])) indicates the whole of line 3 from the <lb n="3"/> to the point right before the following <lb n="4"/>.

Example: #range(right(//lb[@n='3']),string-index(//lb[@n='3'],15)) indicates the sequence <unclear>s</unclear>emp<unclear>er</unclear> in mente (an element, a text node, another element, and a partial text node).

Example: #range(string-index(//lb[@n='3'],7),string-index(//lb[@n='3'],10),string-index(//lb[@n='3'],15),string-index(//lb[@n='3'],21)) indicates the non-contiguous sequence ‘in mentem’.

⚓︎16.2.4.7 string-range()

Sequence string-range(IDREF | XPATH, OFFSET, LENGTH[, OFFSET, LENGTH ...])

The string-range() scheme locates a sequence based on character positions in a text stream relative to the node identified by the first parameter. The location of the beginning of the addressed sequence is determined precisely as for string-index(). The OFFSET parameter is defined as above in string-index(). The LENGTH parameter is a positive integer that denotes the length of the text stream captured by the sequence. As with range(), the addressed sequence may contain text nodes and elements. The string-range() scheme can accept multiple OFFSET, LENGTH pairs to address a non-contiguous sequence in much the same way that range() can accept multiple pairs of pointers.

Because string-range() addresses points in the text stream, tags are invisible to it. For example, if an empty tag like lb is encountered while processing a string-range(), it will be included in the resulting sequence, but the LENGTH count will not increment when it is captured.

Example: #string-range(//lb[@n='5'],0,27) indicates the whole of line 5 from the text immediately following the lb to the point right before the closing ab tag.

Example: #string-range(//lb[@n='3'],7,8) indicates the sequence ‘in mente’.

Example: #string-range(//lb[@n='3'],7,3,15,6) indicates the non-contiguous sequence ‘in mentem’.

⚓︎16.2.4.8 match()

Sequence match(IDREF | XPATH, 'REGEX' [, INDEX])

The match scheme locates a sequence based on matching the REGEX parameter against a text stream relative to the reference node identified by the first parameter. REGEX is a regular expression as defined by XQuery 1.0 and XPath 2.0 Functions and Operators (Second Edition), with some modifications:

  • Because the regular expression is delimited by apostrophe characters, any such characters (' or U+0027) occurring inside the expression must be escaped using the URI percent-encoding scheme %27.
  • Regular expressions in match() are assumed to operate in single-line mode. The end of the string to be matched against is either the end of the text contained by the element in the first parameter or the end of the document, if that parameter indicates an empty element. The meta-character ^ therefore matches the beginning of the text stream inside or following the reference node, and the meta-character $ matches the end of that stream.

The optional INDEX parameter is an integer greater than 0 which specifies which match should be chosen when there is more than one possibility. If omitted, the first match in the text stream will be used to resolve the match().

Like string-range(), match() may capture elements in the indicated sequence, even though they are ignored for purposes of evaluating the match.

Example: #match(//lb[@n='5'],'opto.*valeas') indicates the sequence opto u<unclear>t</unclear> bene valeas in line 5.

Example: #match(//lb[@n='3'],'semper') would indicate the word ‘semper’, but would not capture the unclear elements in <unclear>s</unclear>emp<unclear>er</unclear>, just their text children.

⚓︎16.2.5 Canonical References

By ‘canonical’ reference we mean any means of pointing into documents, specific to a community or corpus. For example, biblical scholars might understand ‘Matt 5:7’ to mean ‘the book called Matthew, chapter 5, verse 7.’ They might then wish to translate the string ‘Matt 5:7’ into a pointer into a TEI-encoded document, selecting the element which corresponds to the seventh div element within the fifth div element within the div element with the n attribute valued ‘Matt.’

Several elements in the TEI scheme (gloss, ptr, ref, and term) bear a special attribute, cRef, just for this purpose. Using the system described in this section, an encoder may specify references to canonical works in a discipline-familiar format, and expect software to derive a complete URI from it. The value of the cRef attribute is processed as described in this section, and the resulting URI reference is treated as if it were the value of the target attribute. The cRef and target attributes are mutually exclusive: only one or the other may be specified on any given occurrence of an element.

For the cRef attribute to function as required, a mechanism is needed to define the mapping between (for example) ‘the book called Matt and the part of the XML structure which corresponds with it. This is provided by the refsDecl element in the TEI header, which contains an algorithm for translating a canonical reference string (like Matt 5:7) into a URI such as #xpath(//div[@n='Matt']/div[5]/div[7]). The refsDecl element is described in section 2.3.6 The Reference System Declaration; the following example is discussed in more detail below in section 16.2.5.1 Worked Example. An alternative and less verbose method is described in section 16.2.5.4 Citation Structures.
<refsDecl xml:id="biblical">
 <cRefPattern matchPattern="(.+) (.+):(.+)"
  replacementPattern="#xpath(//div[@n='$1']/div[@n='$2']/div[@n='$3]')">

  <p>This pointer pattern extracts and references the <q>book,</q>
   <q>chapter,</q> and <q>verse</q> parts of a biblical reference.</p>
 </cRefPattern>
 <cRefPattern matchPattern="(.+) (.+)"
  replacementPattern="#xpath(//div[@n='$1']/div[$2])">

  <p>This pointer pattern extracts and references the <q>book</q> and
  <q>chapter</q> parts of a biblical reference.</p>
 </cRefPattern>
 <cRefPattern matchPattern="(.+)"
  replacementPattern="#xpath(//div[@n='$1'])">

  <p>This pointer pattern extracts and references just the <q>book</q>
     part of a biblical reference.</p>
 </cRefPattern>
</refsDecl>

When an application encounters a canonical reference as the value of cRef attribute, it might follow this sequence of specific steps to transform it into a URI reference:

  1. Ascertain the correct refsDecl following the rules summarized in section 15.3.3 Summary.
  2. For each cRefPattern element encountered in the appropriate refsDecl, in the order encountered:
    1. match the value of the cRef attribute to the regular expression found as the value of the matchPattern attribute
    2. if the value of the cRef attribute matches:
      1. take the value of the replacementPattern attribute and substitute the back references ($1, $2, etc.) with the corresponding matched substrings
      2. the result is taken as if it were a relative or absolute URI reference specified on the target attribute; i.e., it should be used as is or combined with the current xml:base attribute value as usual
      3. no further processing of this value of the cRef attribute against the refsDecl should take place
    3. if, however, the value of the cRef attribute does not match the regular expression specified in the value of the matchPattern attribute, proceed to the next cRefPattern
  3. If all the cRefPattern elements are examined in turn and none matches, the pointer fails.

The regular expression language used as the value of the matchPattern attribute is that used for the pattern facet of the World Wide Web Consortium's XML Schema Language in an Appendix to XML Schema Part 2.66 The value of the replacementPattern attribute is simply a string, except that occurrences of ‘$1’ through ‘$9’ are replaced by the corresponding substring match. Note that since a maximum of nine substring matches are permitted, the string ‘$18’ means ‘the value of the first matched substring followed by the character ‘8’ as opposed to ‘the eighteenth matched substring’. If there is a need for an actual string including a dollar sign followed by a digit that is not supposed to be replaced, the dollar sign should be written as $$. Implementations must convert $$ to $ during processing.

⚓︎16.2.5.1 Worked Example

Let us presume that with the example refsDecl above, an application comes across a cRef value of Matt 5:7. The application would first apply the regular expression (.+) (.+):(.+) to ‘Matt 5:7’. This regular expression would successfully match. The first matched substring would be ‘Matt’, the second ‘5’, and the third ‘7’. The application would then apply these substrings to the pattern #xpath(//div[@n='$1']/div[$2]/div[$3]), producing #xpath(//div[@n='Matt']/div[5]/div[7]).

If, however, the input string had been ‘Matt 5’, the first regular expression would not have matched. The application would have then tried the second, (.+) (.+), producing a successful match, and the matched substrings ‘Matt’ and ‘5’. It would then have substituted those matched substrings into the pattern #xpath(//div[@n='$1']/div[$2]) to produce a fragment identifier indicating the referenced element.

If the input string had been ‘Matt’, neither the first nor the second regular expressions would have successfully matched. The application would have then tried the third, (.+), producing the matched substring ‘Matt’, and the URI Reference #xpath(//div[@n='Matt']).

a cRefPattern should not reference more matched substrings. For example:
<cRefPattern matchPattern="(.+) (.+):(.+)"
 replacementPattern="//div[@n='$1']/div[$2]/div[$3]/p[$4]"/>
is faulty, since only three matched substrings would have been produced, but a fourth ($4) was referenced.
⚓︎16.2.5.2 Complete and Partial URI Examples
In the above example, the value of cRef was used to generate a Fragment Identifier. An absolute URI could be generated directly, as in the following example.
<refsDecl xml:id="USC">
 <cRefPattern matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Cc](h(\.|ap(ter|\.)?)?)?\s*([1-9][0-9]*)"
  replacementPattern="http://uscode.house.gov/download/pls/$1C$5.txt">

  <p>Matches most standard references to particular
     chapters of the United States Code, e.g.
  <val>11USCC7</val>, <val>17 U.S.C. Chapter 3</val>, or
  <val>14 USC Ch. 5</val>. Note that a leading zero is
     required for the title (must be two digits), but is not
     permitted for the chapter number.</p>
 </cRefPattern>
 <cRefPattern matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Pp](re(lim(inary)?)?)?\s*[Mm](at(erial)?)?"
  replacementPattern="http://uscode.house.gov/download/pls/$1T.txt">

  <p>Matches references to the preliminary material for a
     given title, e.g. <val>11USCP</val>, <val>17 U.S.C.
       Prelim Mat</val>, or <val>14 USC pm</val>.</p>
 </cRefPattern>
 <cRefPattern matchPattern="([0-9][0-9])\s*U\.?S\.?C\.?\s*[Aa](ppend(ix)?)?"
  replacementPattern="http://uscode.house.gov/download/pls/$1A.txt">

  <p>Matches references to the appendix of a given tile,
     e.g. <val>05USCA</val>, <val>11 U.S.C. Appendix</val>,
     or <val>18 USC Append</val>.</p>
 </cRefPattern>
</refsDecl>
<!-- ... -->
<p>The example in section 10 is taken
from <ref cRef="17 USC Ch 1">Subject Matter and Scope of
   Copyright</ref>.</p>

See 16.2.3 Using Abbreviated Pointers for another related use of the matchPattern and replacementPattern attributes.

⚓︎16.2.5.3 Miscellaneous Usages

Canonical reference pointers are intended for use by TEI encoders. However, this specification might be useful to the development of a process for recognizing canonical references in non-TEI documents (such as plain text documents), possibly as part of their conversion to TEI.

⚓︎16.2.5.4 Citation Structures

Citation structures provide a more thorough and concise mechanism for describing canonical references and the ways those references map on to parts of a TEI document. A citeStructure element describes a single step in a reference, such as ‘Matt’, and may nest to handle multi-part references.

The equivalent structure to the set of cRefPatterns in 16.2.5.1 Worked Example would be:
<refsDecl xml:id="biblical2">
 <citeStructure unit="book"
  match="//body/divuse="@n">

  <citeStructure unit="chaptermatch="div"
   use="@ndelim=" ">

   <citeStructure unit="versematch="div"
    use="@ndelim=":"/>

  </citeStructure>
 </citeStructure>
</refsDecl>

An application wishing to resolve a canonical reference such as Matt 5:7 might follow this procedure:

  1. Ascertain the correct refsDecl following the rules summarized in section 15.3.3 Summary.
  2. Begin with the outer <citeStructure>. If it has a delim attribute and the reference begins with the value of delim then take the portion of the reference after the value of delim as input for the next child <citeStructure>.
  3. For each nested <citeStructure>, if the input reference string contains the value of the delim, then split the string on the value of the delim attribute. If the input string does not contain delim, then stop. Take the portion of the input string after the value of delim and use it as the input string for the child <citeStructure>. After processing the outer <citeStructure>, the output will be ('Matt 5:7'), after the second, ('Matt', '5:7'). The end result will be a sequence like ('Matt','5','7').
  4. For each item in the resulting sequence, resolve the matching node by evaluating the XPath in match with the predicate found in use, using the context of the previously matched node, if any. Start with the outer citeStructure and move to the next child <citeStructure> for each step in the sequence. For example, for the first <citeStructure>, we could construct an XPath //div[@n='Matt']. The full XPath after the reference Matt 5:7 is resolved will be //div[@n='Matt']/div[@n='5']/div[@n='7'].

One advantage citeStructure has is that it can be used to generate canonical references, using the declared citation structure to query the text structure. This means it is possible to automatically produce a list of resolvable citations for a TEI document. It also enables the automatic breaking of documents into smaller chunks for presentation and automated generation of tables of contents.

Citation structures may in addition specify how informational properties are to be extracted from the document sections they identify, using the citeData element. For example, if a TEI document is divided into chapters with a div per chapter and those chapters have titles, contained in head elements, then we might declare a citation structure for the document thus:
<citeStructure unit="chapter"
 match="/TEI/text/body/divuse="position()delim="ch. ">

 <citeData property="http://purl.org/dc/terms/title"
  use="head"/>

</citeStructure>

This specifies that chapter references are given in the form ‘ch. n’, where ‘n’ is the position of the div in the body of the document, and that we may obtain the title of the chapter (identified by the Dublin Core property title) from the chapter heading. This would, for example, enable the automated generation of a chapter listing for the document.

⚓︎16.3 Blocks, Segments, and Anchors

In this section, we discuss three general purposes elements which may be used to mark and categorize both a span of text and a point within one. These elements have several uses, most notably to provide elements which can be given identifiers for use when aligning or linking to parts of a document, as discussed elsewhere in this chapter. They also provide a convenient way of extending the semantics of the TEI markup scheme in a theory-neutral manner, by providing for two neutral or ‘anonymous’ elements to which the encoder can add any meaning not supplied by other TEI defined elements.

  • anchor (anchor point) attaches an identifier to a point within a text, whether or not it corresponds with a textual element.
  • ab (anonymous block) contains any component-level unit of text, acting as a container for phrase or inter level elements analogous to, but without the same constraints as, a paragraph.
  • seg (arbitrary segment) represents any segmentation of text below the ‘chunk’ level.

The elements anchor, ab, and seg are members of the class att.typed, from which they inherit the following attributes:

  • att.typed provides attributes that can be used to classify or subclassify elements in any way.
    typecharacterizes the element in some sense, using any convenient classification scheme or typology.
    subtype(subtype) provides a sub-categorization of the element, if needed

The elements ab, and seg are members of the class att.fragmentable, from which they inherit the following attribute:

  • att.fragmentable provides attributes for representing fragmentation of a structural element, typically as a consequence of some overlapping hierarchy.
    partspecifies whether or not its parent element is fragmented in some way, typically by some other overlapping structure: for example a speech which is divided between two or more verse stanzas, a paragraph which is split across a page division, a verse line which is divided between two speakers.

The seg element is also a member of the class att.segLike from which it inherits the following attribute:

  • att.segLike provides attributes for elements used for arbitrary segmentation.
    function(function) characterizes the function of the segment.

The anchor element may be thought of as an empty seg, or as an artifice enabling an identifier to be attached to any position in a text. Like the milestone element discussed in section 3.11 Reference Systems, it is useful where multiple views of a document are to be combined, for example, when a logical view based on paragraphs or verse lines is to be mapped on to a physical view based on manuscript lines. Like those elements, it is a member of the class model.global and can therefore appear anywhere within a document when the module defined by this chapter is included in a schema. Unlike the other elements in its class, the anchor element is primarily intended to mark an arbitrary point used for alignment, or as the target of a spanning element such as those discussed in section 11.3.1.4 Additions and Deletions, rather than as a means of marking segment boundaries for some arbitrary segmentation of a text.

For example, suppose that we wish to mark the end of the fifth word following each occurrence of some term in a particular text, perhaps to assist with some collocational analysis. This can most easily be done with the help of the anchor element, as follows:
English language. Except for not very<anchor xml:id="eng1"/>
English at all at the time<anchor xml:id="eng2"/>
English was still full of flaws<anchor xml:id="eng3"/>
English. This was revised by young
<anchor xml:id="eng4"/>
In section 16.5.1 Correspondence we discuss ways in which these anchor points might be used to represent an alignment such as one might get in a keyword-in-context concordance.

The seg element may be used at the encoder's discretion to mark almost any segment of the text of interest for processing. One use of the element is to mark text features for which no appropriate markup is otherwise defined, i.e. as a simple extension mechanism. Another use is to provide an identifier for some segment which is to be pointed at by some other element, i.e. to provide a target, or a part of a target, for a ptr or other similar element.

Several examples of uses for the seg element are provided elsewhere in these Guidelines. For example:

In the following simple example, the seg element simply delimits the extent of a stutter, a textual feature for which no element is provided in these Guidelines.
<q>Don't say <q>
  <seg type="stutter">I-I-I</seg>'m afraid,</q> Melvin, just say <q>I'm
   afraid.</q>
</q>
The seg element is particularly useful for the markup of linguistically significant constituents such as the phrases that may be the output of an automatic parsing system. This example also demonstrates the use of the xml:id attribute to carry an identifier which other parts of a document may use to point to, or align with:
<seg xml:id="bl0034type="sentence">
 <seg xml:id="bl0034.1type="phrase">Literate and illiterate speech</seg>
 <seg xml:id="bl0034.2type="phrase">in a language like English</seg>
 <seg xml:id="bl0034.3type="phrase">are plainly different.</seg>
</seg>
As the above example shows, seg elements may be nested directly within one another, to any degree of analysis considered appropriate. This is taken a little further in the following example, where the type and subtype attributes have been used to further categorize each word of the sentence (the xml:id attributes have been removed to reduce the complexity of the example):
<seg type="sentencesubtype="declarative">
 <seg type="phrasesubtype="noun">
  <seg type="wordsubtype="adjective">Literate</seg>
  <seg type="wordsubtype="conjunction">and</seg>
  <seg type="wordsubtype="adjective">illiterate</seg>
  <seg type="wordsubtype="noun">speech</seg>
 </seg>
 <seg type="phrasesubtype="preposition">
  <seg type="wordsubtype="preposition">in</seg>
  <seg type="wordsubtype="article">a</seg>
  <seg type="wordsubtype="noun">language</seg>
  <seg type="wordsubtype="preposition">like</seg>
  <seg type="wordsubtype="noun">English</seg>
 </seg>
 <seg type="phrasesubtype="verb">
  <seg type="wordsubtype="verb">are</seg>
  <seg type="wordsubtype="adverb">plainly</seg>
  <seg type="wordsubtype="adjective">different</seg>
 </seg>
 <seg type="punct">.</seg>
</seg>
(The example values shown are chosen for simplicity of comprehension, rather than verisimilitude). It should also be noted that specialized segment elements are defined in section 17.1 Linguistic Segment Categories to facilitate this particular kind of analysis. These allow for the explicit markup of units called s-units, clauses, phrases, words, morphemes, and characters, which may be felt preferable to the more generic approach typified by use of the seg element. Using these, the first phrase above might be encoded simply as
<phr type="noun">
 <w type="adjective">Literate</w>
 <w type="conjunction">and</w>
 <w type="adjective">illiterate</w>
 <w type="noun">speech</w>
</phr>
Note the way in which the type attribute of these specialized elements now carries the value carried by the subtype attribute of the more general seg element. For an analysis not using these traditional linguistic categories however, the seg element provides a simple but powerful mechanism.
In language corpora and similar material, the seg element may be used to provide an end-to-end segmentation as an alternative to the more specific s element proposed in chapter 17.1 Linguistic Segment Categories for the markup of orthographic sentences, or s-units. However, it may be more useful to use the s element for this purpose, since this means that the seg element can then be used to mark both features within s-units and segments composed of s-units, as in the following example:67
<seg xml:id="s1s3type="narrative_unit">
 <s xml:id="s1">Sigmund, the <seg type="patronymic">son of Volsung</seg>,
   was a king in Frankish country.</s>
 <s xml:id="s2">Sinfiotli was the eldest of his sons.</s>
 <s xml:id="s3"> ... </s>
</seg>
Like other elements, the seg tag must be properly enclosed within other elements. Thus, a single seg element can be used to group together words in different sentences only if the sentences are not themselves tagged. The first of the following two encodings is legal, but the second is not.
Give me <seg type="phrase">a dozen. Or two or three.</seg>
<!-- Illegal! -->
<s>Give me <seg type="phrase">a dozen.</s>
<s>Or two or three.</s></seg>
The part attribute may be used as one simple method of overcoming this restriction:
<s>Give me <seg type="phrasepart="I">a dozen.</seg>
</s>
<s>
 <seg part="F">Or two or three.</seg>
</s>
Another solution is to use the join element discussed in section 16.7 Aggregation; this requires that each of the seg elements be given an identifier. For further discussion of this generic encoding problem, see also chapter 20 Non-hierarchical Structures.

The seg element has the same content as a paragraph in prose: it can therefore be used to group together consecutive sequences of model.inter class elements, such as lists, quotations, notes, stage directions, etc. as well as to contain sequences of phrase-level elements. It cannot however be used to group together sequences of paragraphs or similar text units such as verse lines; for this purpose, the encoder should use intermediate pointers, as described in section 16.1.4 Intermediate Pointers or the methods described in section 16.7 Aggregation. It is particularly important that the encoder provide a clear description of the principles by which a text has been segmented, and the way in which that segmentation is represented. This should include a description of the method used and the significance of any categorization codes. The description should be provided as a series of paragraphs within the segmentation element of the encoding description in the TEI header, as described in section 2.3.3 The Editorial Practices Declaration.

The seg element may also be used to encode simultaneous or mutually exclusive variants of a text when the more special purpose elements for simple editorial changes, abbreviation and expansion, addition and deletion, or for a critical apparatus are not appropriate. In these circumstances, one seg is encoded for each possible variant, and the set of them is enclosed in a choice element.

For example, if one were writing dual-platform instructions for installation of software, it might be useful to use seg to record platform-specific pieces of mutually exclusive text.
…pressing <choice>
 <seg type="platformsubtype="Mac">option</seg>
 <seg type="platformsubtype="PC">alt</seg>
</choice>-f will …

Elsewhere in this chapter we provide a number of examples where the seg element is used simply to provide an element to which an identifier may be attached, for example so that another segment may be linked or related to it in some way.

The ab (anonymous block) element functions like the seg element, but is used for portions of the text which occur at the chunk level (See 1.3.2.1 Informal Element Classifications ). It is therefore a member of the model.pLike class. Like seg, but unlike p, ab may nest within other ab elements. So it may be used to represent texts with irregular chunk-level structures.

Probably the most common use for ab is as a container for text that does not naturally belong in a paragraph. The text of an epitaph, for example, may not be paragraph-like, but still requires a container:
<ab style="text-align: center;">
 <lb/>SACRED
<lb/>to the memory of
<lb/>
 <abbr>M<hi rend="superscript">rs</hi>
  <am>.</am>
 </abbr> ALICE HUTTON
<lb/>born at Che<g ref="#long-s">s</g>terfield in
<lb/>Great Britain,
<lb/>
 <abbr>Jan<hi rend="superscript">y</hi>
  <am>.</am>
 </abbr>
 <abbr>25<hi rend="superscript">th</hi>
  <am>.</am>
 </abbr> 1722:
<lb/>and died in Carolina
<lb/>
 <abbr>Sept<hi rend="superscript">r</hi>
  <am>.</am>
 </abbr>
 <abbr>26<hi rend="superscript">th</hi>
  <am>.</am>
 </abbr> 1777.
<lb/>This <g ref="#long-s">s</g>tone
<lb/>is placed by her only Child
<lb/>ANN BOOTH POLLOK
</ab>
Encoders may also prefer to treat other types of short documents, such as receipts, in this way.
The ab element may also be used, for example, to tag the canonical verse divisions of Biblical texts:
<div1 n="Gentype="book">
 <head>The First Book of Moses, Called</head>
 <head type="main">Genesis</head>
 <div2 n="1type="chapter">
  <ab n="1">In the beginning God created the heaven and the
     earth.</ab>
  <ab n="2">And the earth was without form, and void; and darkness
  <hi>was</hi> upon the face of the deep. And the Spirit of God
     moved upon the face of the waters.</ab>
  <ab n="3">And God said, Let there be light: and there was
     light.</ab>
 </div2>
</div1>
In other cases, where the text clearly indicates paragraph divisions containing one or more verses, the p element may be used to tag the paragraphs, and the seg element used to subdivide them. The ab element is provided as an alternative to the p element; it may not be used within paragraphs. The seg element, by contrast, may appear only within and not between paragraphs (or anonymous block elements).
<div1 n="Gentype="book">
 <head>Das Erste Buch Mose.</head>
 <div2 n="1type="chapter">
  <p>
   <seg n="1">Am Anfang schuff Gott Himel vnd Erden.</seg>
   <seg n="2">Vnd die Erde war wüst vnd leer / vnd es war
       finster auff der Tieffe / Vnd der Geist Gottes schwebet auff
       dem Wasser.</seg>
  </p>
  <p>
   <seg n="3">Vnd Gott sprach / Es werde Liecht / Vnd es ward
       Liecht.</seg>
  </p>
 </div2>
</div1>
The ab element is also useful for marking dramatic speeches when it is not clear whether the speech is to be regarded as prose or verse. If, for example, an encoder does not wish to express an opinion as to whether the opening lines of Shakespeare's The Tempest are to be regarded as prose or as verse, they might be tagged as follows:
<div1 n="Itype="act">
 <div2 n="1type="scene">
  <head rend="italic">Actus primus, Scena prima.</head>
  <stage rend="italictype="setting"> A tempestuous noise of
     Thunder and Lightning heard:
     Enter a Ship-master, and a Boteswaine.</stage>
  <sp>
   <speaker>Master.</speaker>
   <ab>Bote-swaine.</ab>
  </sp>
  <sp>
   <speaker>Botes.</speaker>
   <ab>Heere Master: What cheere?</ab>
  </sp>
  <sp>
   <speaker>Mast.</speaker>
   <ab>Good: Speake to th' Mariners: fall too't, yarely,
       or we run our selues a ground, bestirre, bestirre.
   <stage type="move">Exit.</stage>
   </ab>
  </sp>
  <stage type="move">Enter Mariners.</stage>
  <sp>
   <speaker>Botes.</speaker>
   <ab>Heigh my hearts, cheerely, cheerely my harts: yare, yare:
       Take in the toppe-sale: Tend to th' Masters whistle: Blow
       till thou burst thy winde, if roome e-nough.</ab>
  </sp>
 </div2>
</div1>
See further 3.13.2 Core Tags for Drama and 7.2.5 Speech Contents.

⚓︎16.4 Synchronization

In the previous section we discussed two particular kinds of alignment: alignment of parallel texts in different languages; and alignment of texts and portions of an image. In this section we address another specialized form of alignment: synchronization. The need to mark the relative positions of text components with respect to time arises most naturally and frequently in transcribed spoken texts, but it may arise in any text in which quoted speech occurs, or events are described within a time frame. The methods described here are also generalizable for other kinds of alignment (for example, alignment of text elements with respect to space).

⚓︎16.4.1 Aligning Synchronous Events

Provided that explicit elements are available to represent the parts or places to be synchronized, then the global linking attribute synch may be used to encode such synchronization, once it has been identified.

  • att.global.linking provides a set of attributes for hypertextual linking.
    synch(synchronous) points to elements that are synchronous with the current element.

This is another of the attributes made globally available by the mechanism described in the introduction to this chapter. Alternatively, the link and linkGrp elements may be used to make explicit the fact that the synchronous elements are aligned.

To illustrate the use of these mechanisms for marking synchrony, consider the following representation of a spoken text:

B: The first time in twenty five years, we've cooked Christmas
   (unclear) for a blooming great load of people.
A: So you're [1] (unclear) [2]
B: [1] It will be [2] nice in a way, but, [3] be strange. [4]
A: [3] Yeah [4], yeah, cos it, it's [5] the [6]
B: [5] not [6]

This representation uses numbers in brackets to mark the points at which speakers overlap each other. For example, the [1] in A's first speech is to be understood as coinciding with the [1] in B's second speech.68

To encode this we use the spoken texts module, described in chapter 8 Transcriptions of Speech, together with the module described in the present chapter. First, we transcribe this text, marking the synchronous points with anchor elements, and providing a synch attribute on one of each of the pairs of synchronous anchors. As noted in the example given above (section 16.5.2 Alignment of Parallel Texts), correspondence, and hence synchrony, is a symmetric relation; therefore the attribute need only be specified on one of the pairs of synchronous anchors.
<div xml:id="BNC-d1type="convers">
 <u xml:id="u2bwho="#b"> The first time in twenty five years,
   we've cooked Christmas <unclear> for a blooming great
     load of people.</unclear>
 </u>
 <u xml:id="u3awho="#a">So you're
 <anchor synch="#t1bxml:id="t1a"/>
  <unclear>
   <anchor synch="#t2bxml:id="t2a"/>
  </unclear>
 </u>
 <u xml:id="u3bwho="#b">
  <anchor xml:id="t1b"/>It will be <anchor xml:id="t2b"/>
   nice in a way, but, <anchor xml:id="t3b"/>
   be strange.<anchor xml:id="t4b"/>
 </u>
 <u xml:id="u4awho="#a">
  <anchor synch="#t3bxml:id="t3a"/>Yeah
 <anchor synch="#t4bxml:id="t4a"/>, yeah, cos it, its
 <anchor synch="#t5bxml:id="t5a"/>the
 <anchor synch="#t6bxml:id="t6a"/>
 </u>
 <u xml:id="u4bwho="#b">
  <anchor xml:id="t5b"/>not<anchor xml:id="t6b"/>
 </u>
<!-- ... -->
</div>
We can encode this same example using link and linkGrp elements to make the temporal alignment explicit. A back element has been used to enclose the linkGrp element, but the links may be located anywhere the encoder finds convenient:
<back>
 <linkGrp xml:id="lg1"
  domains="#BNC-d1 #BNC-d1targFunc="speaker.a speaker.b"
  type="synchronous_alignment">

  <link xml:id="L1target="#t1a #t1b"/>
  <link xml:id="L2target="#t2a #t2b"/>
  <link xml:id="L3target="#t3a #t3b"/>
  <link xml:id="l4target="#t4a #t4b"/>
  <link xml:id="l5target="#t5a #t5b"/>
  <link xml:id="l6target="#t6a #t6b"/>
 </linkGrp>
</back>
The xml:id attributes are provided for the link and linkGrp elements here for reasons discussed in the next section, 16.4.2 Placing Synchronous Events in Time.
As with other forms of alignment, synchronization may be expressed between stretches of speech as well as between points. When complete utterances are synchronous, for example, if one person says What? and another No! at the same time, that can be represented without anchor elements as follows.
<u synch="#u02xml:id="u01who="#a">What?</u>
<u xml:id="u02who="#b">No!</u>
A simple way of expressing overlap (where one speaker starts speaking before another has finished) is thus to use the seg element to encode the overlapping portions of speech. For example,
<u who="#a"> So you're <unclear synch="#u-b1"/>
</u>
<u who="#b">
 <seg xml:id="u-b1"> It will be </seg> nice in a way, but,
<seg synch="#u-a3"> be strange. </seg>
</u>
<u who="#a">
 <seg xml:id="u-a3"> Yeah </seg>, yeah, cos it,
its <seg synch="#u-b2"> the </seg>
</u>
<u xml:id="u-b2who="#b"> not </u>
Note in this encoding how synchronization has been effected between an empty unclear element and the content of a seg element, and between the content of a u element and that of another seg, using the synch attribute. Alternatively, a linkGrp could be used in the same way as above.

⚓︎16.4.2 Placing Synchronous Events in Time

A synchronous alignment specifies which points in a spoken text occur at the same time, and the order in which they occur, but does not say at what time those points actually occur. If that information is available to the encoder it can be represented by means of the when and timeline elements, whose description and attributes are the following:

  • when indicates a point in time either relative to other elements in the same timeline tag, or absolutely.
    absolutesupplies an absolute value for the time.
    intervalspecifies a time interval either as a number or as one of the keywords defined by the datatype teidata.interval
    unitspecifies the unit of time in which the interval value is expressed, if this is not inherited from the parent timeline. Suggested values include: 1] d (days); 2] h (hours); 3] min (minutes); 4] s (seconds); 5] ms (milliseconds)
    sinceidentifies the reference point for determining the time of the current when element, which is obtained by adding the interval to the time of the reference point.
  • timeline (timeline) provides a set of ordered points in time which can be linked to elements of a spoken text to create a temporal alignment of that text.
    origindesignates the origin of the timeline, i.e. the time at which it begins.
    intervalspecifies a time interval either as a positive integral value or using one of a set of predefined codes.
    unitspecifies the unit of time corresponding to the interval value of the timeline or of its constituent points in time. Suggested values include: 1] d (days); 2] h (hours); 3] min (minutes); 4] s (seconds); 5] ms (milliseconds)

Each when element indicates a point in time, either directly by means of the absolute attribute, whose value is a string which specifies a particular time, or indirectly by means of the since attribute, which points to another when. If the since is used, then the interval and unit attributes should also be used to indicate the amount of time that has elapsed since the time specified by the element pointed to by the since attribute; the value -1 can be given to indicate that the interval is unknown.

If the when elements are uniformly spaced in time, then the interval and unit values need be given once in the timeline, and not repeated in any of the when elements. If the intervals vary, but the units are all the same, then the unit attribute alone can be given in the timeline element, and the interval attribute given in the when element.

The origin attribute in the timeline element points to a when element which specifies the reference or origin for the timings within the timeline; this must, of course, specify its position in time absolutely. If the origin of a timeline is unknown, then this attribute may be omitted.

The following timeline might be used to accompany the marked up conversation shown in the preceding section:
<timeline xml:id="tL1origin="#w0"
 unit="ms">

 <when xml:id="w0absolute="11:30:00"/>
 <when xml:id="w1interval="unknown"
  since="#w0"/>

 <when xml:id="w2interval="100"
  since="#w1"/>

 <when xml:id="w3interval="200"
  since="#w2"/>

 <when xml:id="w4interval="150"
  since="#w3"/>

 <when xml:id="w5interval="250"
  since="#w4"/>

 <when xml:id="w6interval="100"
  since="#w5"/>

</timeline>
The information in this timeline could now be linked to the information in the linkGrp which provides the temporal alignment (synchronization) for the text, as follows:
<linkGrp type="temporal_specification"
 domains="#lg1 #tL1targFunc="synch.points when">

 <link target="#L1 #w1"/>
 <link target="#L2 #w2"/>
 <link target="#L3 #w3"/>
 <link target="#l4 #w4"/>
 <link target="#l5 #w5"/>
 <link target="#l6 #w6"/>
</linkGrp>
To avoid the need for two distinct link groups (one marking the synchronization of anchors with each other, and the other marking their alignment with points on the time line) it would be better to link the when elements with the synchronous points directly:
<linkGrp type="temporal_specification"
 domains="#BNC-d1 #BNC-d1 #tL1targFunc="speaker.a speaker.b when">

 <link target="#t1a #t1b #w1"/>
 <link target="#t2a #t2b #w2"/>
 <link target="#t3a #t3b #w3"/>
 <link target="#t4a #t4b #w4"/>
 <link target="#t5a #t5b #w5"/>
 <link target="#t6a #t6b #w6"/>
</linkGrp>
Finally, suppose that a digitized audio recording is also available, and an XML file that assigns identifiers to the various temporal spans of sound is available. For example, the following Synchronized Multimedia Integration Language (SMIL, pronounced "smile") fragment:
<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au1begin="05.2s"/>

<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au2begin="05.7s"/>

<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au3begin="05.9s"/>

<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au4begin="06.3s"/>

<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au5begin="06.9s"/>

<audio xmlns="http://www.w3.org/2001/SMIL20/Language" src="rtsp://soundstage.pi.cnr.it:554/home/az/bncSound/xmas4lots.mp3"
 xml:id="au6begin="07.4s"/>
URIs pointing to the <audio> elements could also be included as a fourth component in each of the above link elements, thus providing a synchronized audio track to complement the transcribed text.

For further discussion of this and related aspects of encoding transcribed speech, refer to chapter 8 Transcriptions of Speech.

⚓︎16.5 Correspondence and Alignment

In this section we introduce the notions of correspondence, expressed by the corresp attribute, and of alignment, which is a special kind of correspondence involving an ordered set of correspondences. Both cases may be represented using the link and linkGrp elements introduced in section 16.1 Links. We also discuss the special case of alignment in time or synchronization, for which special purpose elements are proposed in section 16.4 Synchronization.

⚓︎16.5.1 Correspondence

A common requirement in text analysis is to represent correspondences between two or more parts of a single document, or between places in different documents. Provided that explicit elements are available to represent the parts or places to be linked, then the global linking attribute corresp may be used to encode such correspondence, once it has been identified.

  • att.global.linking provides a set of attributes for hypertextual linking.
    corresp(corresponds) points to elements that correspond to the current element in some way.

This is one of the attributes made available by the mechanism described in the introduction to this chapter (16 Linking, Segmentation, and Alignment). Correspondence can also be expressed by means of the link element introduced in section 16.1 Links.

Where the correspondence is between spans, the seg element should be used, if no other element is available. Where the correspondence is between points, the anchor element should be used, if no other element is available.

The use of the corresp attribute with spans of content is illustrated by the following example:
<title xml:id="SHIRLEY">Shirley</title>, which made
its Friday night debut only a month ago, was
not listed on <name xml:id="NBC">NBC</name>'s new schedule,
although <seg corresp="#NBCxml:id="NETWORK">the network</seg>
says <seg corresp="#SHIRLEYxml:id="SHOW">the show</seg>
still is being considered.
Here the anaphoric phrases the network and the show have been associated directly with the elements to which they refer by means of corresp attributes. This mechanism is simple to apply, but has the drawback that it is not possible to specify more exactly what kind of correspondence is intended. Where this attribute is used, therefore, encoders are encouraged to specify their intent in the associated encoding description in the TEI header.
Essentially, what the corresp attribute does is to specify that elements bearing this attribute and those to which the attribute points are doubly linked. In the example above, the use of the corresp attribute indicates that the seg element containing ‘the show’ and the title element containing ‘Shirley’ correspond to each other: the correspondence relationship is not ‘from’ one to the other, but ‘between’ the two objects. It is thus different from the target attribute, and provides functionality more similar to that of the link and linkGrp elements defined in section 16.1 Links, although it lacks the ability to indicate more precisely what kind of correspondence is intended as in the following retagging of the preceding example.
<title xml:id="shirley">Shirley</title>, which made
its Friday night debut only a month ago, was not
listed on <name xml:id="nbc">NBC</name>'s new schedule,
although <seg xml:id="network">the network</seg> says
<seg xml:id="show">the show</seg> still is being considered.

<linkGrp type="anaphoric_link"
 targFunc="antecedent anaphor">

 <link target="#shirley #show"/>
 <link target="#nbc #network"/>
</linkGrp>
In the following example, we use the same mechanism to express a correspondence amongst the anchors introduced following the fifth word after English in a text:
English language. Except for not very<anchor xml:id="en1"/>
<!-- ... -->
English at all at the time
<anchor xml:id="en2"/>
<!-- ... -->
English was still full of flaws
<anchor xml:id="en3"/>
<!-- ... -->
English. This was revised by young
<anchor xml:id="en4"/>
<!-- ... -->
<linkGrp type="five-word_collocates">
 <link type="collocates_of_ENGLISH"
  target="#en1 #en2 #en3 #en4"/>

<!-- ... -->
</linkGrp>

⚓︎16.5.2 Alignment of Parallel Texts

One very important application area for the alignment of parallel texts is multilingual corpora. Consider, for example, the need to align ‘translation pairs’ of sentences drawn from a corpus such as the Canadian Hansard, in which each sentence is given in both English and French. Concerning this problem, Gale and Church write:

Most English sentences match exactly one French sentence, but it is possible for an English sentence to match two or more French sentences. The first two English sentences [in the example below] illustrate a particularly hard case where two English sentences align to two French sentences. No smaller alignments are possible because the clause ‘...sales...were higher...’ in the first English sentence corresponds to (part of) the second French sentence. The next two alignments ... illustrate the more typical case where one English sentence aligns with exactly one French sentence. The final alignment matches two English sentences to a single French sentence. These alignments [which were produced by a computer program] agreed with the results produced by a human judge.69

The alignment produced by Gale and Church's program can be expressed in four different ways. The encoder must first decide whether to represent the alignment in terms of points within each text (using the anchor element) or in terms of whole stretches of text, using the seg element. To some extent the choice will depend on the process by which the software works out where alignment occurs, and the intention of the encoder. Secondly, the encoder may elect to represent the actual encoding using either corresp attributes attached to the individual anchor or seg elements, or using a free-standing linkGrp element.

We present first a solution using anchor elements bearing only corresp attributes:
<div xml:lang="entype="subsection">
 <p>
  <anchor corresp="#fa1xml:id="ea1"/>According to our survey, 1988
   sales of mineral water and soft drinks were much higher than in 1987,
   reflecting the growing popularity of these products. Cola drink
   manufacturers in particular achieved above-average growth rates.
 <anchor corresp="#fa2xml:id="ea2"/>The higher turnover was largely
   due to an increase in the sales volume.
 <anchor corresp="#fa3xml:id="ea3"/>Employment and investment levels also climbed.
 <anchor corresp="#fa4xml:id="ea4"/>Following a two-year transitional period,
   the new Foodstuffs Ordinance for Mineral Water came into effect on
   April 1, 1988. Specifically, it contains more stringent requirements
   regarding quality consistency and purity guarantees.</p>
</div>
<div xml:lang="frtype="subsection">
 <p>
  <anchor corresp="#ea1xml:id="fa1"/>Quant aux eaux minérales
   et aux limonades, elles rencontrent toujours plus d'adeptes. En effet,
   notre sondage fait ressortir des ventes nettement supérieures
   à celles de 1987, pour les boissons à base de cola
   notamment. <anchor corresp="#ea2xml:id="fa2"/>La progression des
   chiffres d'affaires résulte en grande partie de l'accroissement
   du volume des ventes. <anchor corresp="#ea3xml:id="fa3"/>L'emploi et
   les investissements ont également augmenté.
 <anchor corresp="#ea4xml:id="fa4"/>La nouvelle ordonnance fédérale
   sur les denrées alimentaires concernant entre autres les eaux
   minérales, entrée en vigueur le 1er avril 1988 après
   une période transitoire de deux ans, exige surtout une plus
   grande constance dans la qualité et une garantie de la
   pureté.</p>
</div>

There is no requirement that the corresp attribute be specified in both English and French texts, since (as noted above) this attribute is defined as representing a mutual association. However, it may simplify processing to do so, and also avoids giving the impression that the English is translating the French, or vice versa. More seriously, this encoding does not make explicit that it is in fact the entire stretch of text between the anchors which is being aligned, not simply the points themselves. If for example one text contained material omitted from the other, this approach would not be appropriate.

We now present the same passage using the alternative linkGrp mechanism and marking explicitly the segments which have been aligned:
<div xml:id="div-exml:lang="en"
 type="subsection">

 <p>
  <seg xml:id="e_1">According to our survey, 1988 sales of mineral
     water and soft drinks were much higher than in 1987,
     reflecting the growing popularity of these products. Cola
     drink manufacturers in particular achieved above-average
     growth rates.</seg>
  <seg xml:id="e_2">The higher turnover was largely due to an
     increase in the sales volume.</seg>
  <seg xml:id="e_3">Employment and investment levels also climbed.</seg>
  <seg xml:id="e_4">Following a two-year transitional period, the new
     Foodstuffs Ordinance for Mineral Water came into effect on
     April 1, 1988. Specifically, it contains more stringent
     requirements regarding quality consistency and purity
     guarantees.</seg>
 </p>
</div>
<div xml:id="div-fxml:lang="fr"
 type="subsection">

 <p>
  <seg xml:id="f_1">Quant aux eaux minérales et aux limonades,
     elles rencontrent toujours plus d'adeptes. En effet, notre
     sondage fait ressortir des ventes nettement
     supérieures à celles de 1987, pour les
     boissons à base de cola notamment.</seg>
  <seg xml:id="f_2">La progression des chiffres d'affaires
     résulte en grande partie de l'accroissement du volume
     des ventes.</seg>
  <seg xml:id="f_3">L'emploi et les investissements ont
     également augmenté.</seg>
  <seg xml:id="f_4">La nouvelle ordonnance fédérale sur
     les denrées alimentaires concernant entre autres les
     eaux minérales, entrée en vigueur le 1er avril
     1988 après une période transitoire de deux
     ans, exige surtout une plus grande constance dans la
     qualité et une garantie de la pureté.</seg>
 </p>
</div>
<linkGrp type="alignment"
 domains="#div-e #div-f">

 <link target="#e_1 #f_1"/>
 <link target="#e_2 #f_2"/>
 <link target="#e_3 #f_3"/>
 <link target="#e_4 #f_4"/>
</linkGrp>
Note that use of the ab element allows us to mark up the orthographic sentences in both languages independently of the alignment: the first translation pair in this example might be marked up as follows:
<div xml:id="englishxml:lang="en"
 type="subsection">

 <ab xml:id="english1">
  <s>According to our survey, 1988 sales of mineral water and soft
     drinks were much higher than in 1987, reflecting the growing popularity
     of these products.</s>
  <s>Cola drink manufacturers in particular achieved above-average
     growth rates.</s>
 </ab>
</div>
<div xml:id="frenchxml:lang="fr"
 type="subsection">

 <ab xml:id="french1">
  <s xml:id="fs1">Quant aux eaux minérales et aux limonades, elles
     rencontrent toujours plus d'adeptes.</s>
  <s xml:id="fs2">En effet, notre sondage fait ressortir des ventes nettement
     supérieures à celles de 1987, pour les boissons à
     base de cola notamment.</s>
 </ab>
</div>

⚓︎16.5.3 A Three-way Alignment

The preceding encoding of the alignment of parallel passages from two texts requires that those texts and the alignment all be part of the same document. If the texts are in separate documents, then complete URIs, whether absolute or relative (section 16 Linking, Segmentation, and Alignment), will be required. These external pointers may appear anywhere within the document, but if they are created solely for use in encoding links, they may for convenience be grouped within the linkGrp (or other grouping element that uses them for linking).

To demonstrate this facility, we consider how we might encode the alignments in an extract from Comenius' Orbis Sensualium Pictus, in the English translation of Charles Hoole (1659).

Each topic covered in this work has three parts: a picture, a prose text in Latin describing the topic, and a carefully-aligned translation of the Latin into English, German, or some other vernacular. Key terms in the two texts are typographically distinct, and are linked to the picture by numbers, which appear in the two texts and within the picture as well.

First, we consider the text portions. The English and Latin portions have been encoded as distinct div elements. Identifiers have been attached to each typographic line, but no other encoding added, to simplify the example.
<div xml:id="e98xml:lang="en"
 type="lesson">

 <head>The Study</head>
 <p>
  <seg xml:id="e9801">The Study</seg>
  <seg xml:id="e9802">is a place</seg>
  <seg xml:id="e9803">where a Student,</seg>
  <seg xml:id="e9804">a part from men,</seg>
  <seg xml:id="e9805">sitteth alone,</seg>
  <seg xml:id="e9806">addicted to his Studies,</seg>
  <seg xml:id="e9807">whilst he readeth</seg>
  <seg xml:id="e9808">Books,</seg>
 </p>
</div>
<div xml:id="l98xml:lang="la"
 type="lesson">

 <head>Muséum</head>
 <p>
  <seg xml:id="l9801">Museum</seg>
  <seg xml:id="l9802">est locus</seg>
  <seg xml:id="l9803">ubi Studiosus,</seg>
  <seg xml:id="l9804">secretus ab hominibus,</seg>
  <seg xml:id="l9805">solus sedet,</seg>
  <seg xml:id="l9806">Studiis deditus,</seg>
  <seg xml:id="l9807">dum lectitat</seg>
  <seg xml:id="l9808">Libros,</seg>
 </p>
</div>
Next we consider the non-textual parts of the page. Encoding this requires providing two distinct components: firstly a digitized rendering of the page itself, and secondly a representation of the areas within that image which are to be aligned. In section 11.1 Digital Facsimiles we present a simple way of doing this using the TEI-defined markup for alignment of facsimiles. In the present chapter we demonstrate a more powerful means of aligning arbitrary polygons and points, which uses the XML notation SVG (see SVG). This provides appropriate facilities for both these requirements:
<svg xmlns="http://www.w3.org/2000/svg">
 <image xlink:href="p1764.pngwidth="597"
  height="897id="p981"/>

 <rect id="p982x="75y="75width="25"
  height="10"/>

 <rect id="p983x="55y="42width="25"
  height="10"/>

</svg>
This example of SVG defines two rectangles at the locations with the specified x and y coordinates. A view is defined on these, enabling them to be mapped by an SVG processor to the image found at the URL specified (p1764.png). It also defines unique identifiers for the whole image, and the two views of it, which we will use within our alignment, as shown next (for further discussion of the handling of images and graphics, see section 14.4 Specific Elements for Graphic Images; for further discussion of using non-TEI XML vocabularies such as SVG within a TEI document, see section 22.8.2 Combining TEI and Non-TEI Modules).

As printed, the Comenius text exhibits three kinds of alignment.

  1. The English and Latin portions are printed in two parallel columns, with corresponding phrases, (represented above by seg elements), more or less next to each other.
  2. Particular words or phrases are marked as terms in the two languages by a change of rendition: the English text, which otherwise uses black letter type throughout, has the words The Study, a Student, Studies, and Books in a roman font; in the Latin text, which is printed in roman, the corresponding words (Museum, Studiosus, Studiis, and Libros) are all in italic.
  3. Numbered labels appear within the text portions, linking keywords to each other and to sections of the picture. These labels, which have been left out of the above encoding, are attached to the first, third, and last segments in each language quoted below, and also appear (rather indistinctly) within the picture itself. Thus, the images of the study, the student, and his books are each aligned with the correct term for them in the two languages.
The first kind of alignment might be represented by using the corresp attribute on the seg element. The second kind might be represented by using the gloss and term mechanism described in section 3.4.1 Terms and Glosses. The third kind of alignment might be represented using pointers embedded within the texts, for example:

<!--... --><seg xml:id="xe9803">where a <ref n="2target="#xp982">Student</ref>,</seg>
<seg xml:id="xl9803">ubi <ref n="2target="#xp982">Studiosus</ref>,</seg>
<!--... -->
We choose however to use the link element, since this provides a more efficient way of representing the three-way alignment between English, Latin, and picture without redundancy.
<linkGrp type="alignment">
 <link target="#xe9801 #xl9801 #xp981"/>
 <link target="#xe9802 #xl9802"/>
 <link target="#xe9803 #xl9803 #xp982"/>
 <link target="#xe9804 #xl9804"/>
 <link target="#xe9805 #xl9805"/>
 <link target="#xe9806 #xl9806"/>
 <link target="#xe9807 #xl9807"/>
 <link target="#xe9808 #xl9808 #xp983"/>
</linkGrp>

This map, of course, only aligns whole segments and image portions, since these are the only parts of our encoding which bear identifiers and can therefore be pointed to. To add to it the alignment between the typographically distinct words mentioned above, new elements must be defined, either within the text itself or externally by using stand off techniques. Encoding these word pairs as term and gloss, although intuitively obvious, requires a non-trivial decision as to whether the Latin text is glossing the English, or vice versa. Tagging all the marked words as term avoids the difficult decision, but might be thought by some encoders to convey the wrong information about the words in question. Simply tagging them as additional embedded seg elements with identifiers that can be aligned like the others is also a possibility.

These solutions all require the addition of further markup to the text. This may pose no problems, or it may be infeasible, for example because the text is held on a read-only medium. If it is not feasible to add more markup to the original text, some form of stand-off markup will be needed. Any item within the text that can be pointed to using the various pointer schemes discussed in this chapter may be used, not simply those which rely on the existence of an xml:id attribute. Suppose our example had been more lightly tagged, as follows:
<div xml:id="E98xml:lang="en"
 type="lesson">

 <head>The Study</head>
 <ab>The Study</ab>
 <ab>is a place</ab>
 <ab>where a Student,</ab>
</div>
<div xml:id="L98xml:lang="la"
 type="lesson">

 <head>Muséum</head>
 <ab>Museum</ab>
 <ab>est locus</ab>
 <ab>ubi Studiosus,</ab>
</div>
To express the same alignment mentioned above, we could use an XPath expression to identify the required ab elements:
<linkGrp type="alignment">
 <link target="#xpath(//div[@xml:id='L98']/ab[1]) #xpath(//div[@xml:id='E98']/ab[1])"/>
 <link target="#xpath(//div[@xml:id='L98']/ab[2]) #xpath(//div[@xml:id='E98']/ab[2])"/>
</linkGrp>
In the absence of any markup around individual substrings of the element content, the string-range pointer scheme discussed in 16.2.4.7 string-range() may also be helpful: for example, to indicate that the words Studies and Studiis correspond, we might express the link between them as follows:
<link target="#string-range(e9806,16,7) #string-range(l9806,0,7)"/>

⚓︎16.6 Identical Elements and Virtual Copies

This section introduces the notion of a virtual element, that is, an element which is not explicitly present in a text, but the presence of which an application can infer from the encoding supplied. In this section, we are concerned with virtual elements made by simply cloning existing elements. In the next section (16.7 Aggregation), we discuss virtual elements made by aggregating existing elements.

Provided that explicit elements are available to represent the parts or places to be linked, then the global linking attributes sameAs and copyOf may be used to encode this kind of equivalence:

  • att.global.linking provides a set of attributes for hypertextual linking.
    sameAspoints to an element that is the same as the current element.
    copyOfpoints to an element of which the current element is a copy.
It is useful to be able to represent the fact that one element of text is identical to others, for analytical purposes, or (especially if the elements have lengthy content) to obviate the need to repeat the content. For example, consider the repetition of the date element in the following material:
<p>In small clumsy letters he wrote:
<q rend="centered italic">
  <date xml:id="d840404">April 4th,
     1984</date>.</q>
</p>
<p>He sat back. A sense of complete helplessness had
descended upon him. ...</p>
<p>His small but childish handwriting straggled up
and down the page, shedding first its capital letters
and finally even its full stops:
<q rend="italic">
  <date>April 4th, 1984</date>.
   Last night to the flicks. ... </q>
</p>
Suppose now that we wish to encode the fact that the second date element above has identical content to the first. The sameAs attribute is provided for this purpose. Using it, we could recode the last line of the above example as follows:
<date sameAs="#d840404">April 4th,
1984</date>
Last night to the flicks ...

The sameAs attribute may be used to document the fact that two elements have identical content. It may be regarded as a special kind of link. It should only be attached to an element with identical content to that which it targets, or to one the content of which clearly designates it as a repetition, such as the word repeat or bis in the representation of the chorus of a song, the second time it is to be sung. The relation specified by the sameAs attribute is symmetric: if a chorus is repeated three times and each repetition bears a sameAs attribute indicating the first occurrence of the element concerned, it is implied that each chorus is identical, and there is no need for the first occurrence to specify any of its copies.

The copyOf attribute is used in a similar way to indicate that the content of the element bearing it is identical to that of another. The difference is that the content is not itself repeated. The effect of this attribute is thus to create a virtual copy of the element indicated. Using this attribute, the repeated date in the first example above could be recoded as follows:
<date rend="italiccopyOf="#d840404"/>

An application program should replace whatever is the actual content of an element bearing a copyOf attribute with the content of the element specified by it. If the content of the element specified includes other elements, these will become embedded within the element bearing the attribute. Care must be taken to ensure that the document is valid both before and after this embedding takes place. If, for example, the element bearing a copyOf attribute requires a mandatory sub-component, then this component must be present (though possibly empty), even though it will be replaced by the content of the targetted element.

The following example demonstrates how the copyOf attribute may be used in conjunction with the seg element to highlight the differences between almost identical repetitions:
<sp>
 <speaker>Mikado</speaker>
 <l>My <seg xml:id="Mik-L1s">object all sublime</seg>
 </l>
 <l>I shall <seg xml:id="Mik-L2s">achieve in time</seg></l>
 <l xml:id="Mik-L3">To let <seg xml:id="L3s">the punishment fit the crime</seg>,</l>
 <l xml:id="Mik-l4">
  <seg copyOf="#Mik-L3s"/>;</l>
 <l xml:id="Mik-l5">And make each pris'ner pent</l>
 <l xml:id="Mik-l6">Unwillingly represent</l>
 <l xml:id="Mik-l7">A source <seg xml:id="Mik-l7s">of innocent merriment</seg>,</l>
 <l xml:id="Mik-l8">
  <seg copyOf="#Mik-l7s"/>!</l>
</sp>
<sp>
 <speaker>Chorus</speaker>
 <l>His <seg copyOf="#Mik-L1s"/>
 </l>
 <l>He will <seg copyOf="#Mik-L2s"/>
 </l>
 <l copyOf="#Mik-L3"/>
 <l copyOf="#Mik-l4"/>
 <l copyOf="#Mik-l5"/>
 <l copyOf="#Mik-l6"/>
 <l copyOf="#Mik-l7"/>
 <l copyOf="#Mik-l8"/>
</sp>

For further examples of the use of this attribute, see 16.8 Alternation and 19.3 Another Tree Notation.

⚓︎16.7 Aggregation

Because of the strict hierarchical organization of elements, or for other reasons, it may not always be possible or desirable to include all the parts of a possibly fragmented text segment within a single element. In section 16.1.4 Intermediate Pointers we introduced the notion of an intermediate pointer as a way of pointing to discontinuous segments of this kind. In this section we first describe another way of linking the parts of a discontinuous whole, using a set of linking attributes, which are made available for any tag by following the procedure described at the beginning of this chapter. We then describe how the link element may be used to aggregate such segments, and finally introduce the join element, which is a special-purpose linking element specifically for representing the aggregation of parts, and the joinGrp for grouping join elements.

The linking attributes for aggregation are next and prev; each of these attributes has a single identifier as its value:

  • att.global.linking provides a set of attributes for hypertextual linking.
    nextpoints to the next element of a virtual aggregate of which the current element is part.
    prev(previous) points to the previous element of a virtual aggregate of which the current element is part.

It is recommended that the elements indicated by these attributes be of the same type as the element bearing them.

The join element is also a member of the class of att.pointing elements, and so may carry any of the attributes of that class; for the list, see section 16.1 Links.

Here is the material on which we base our first illustration of the use of these mechanisms. Our problem is to represent the s-units identified below as qs3 and qs4 as a single (but discontinuous) whole:
<q>
 <s xml:id="qs2">Monsieur Paul, after he has taken equal
   parts of goose breast and the finest pork, and
   broken a certain number of egg yolks into them,
   and ground them <emph>very</emph>, very fine,
   cooks all with seasoning for some three hours.</s>
 <s xml:id="qs3">
  <emph>But</emph>,</s>
</q>
<s xml:id="ps2">she pushed her face nearer, and looked with
ferocious gloating at the pâté
inside me, her eyes like X rays,</s>
<q>
 <s xml:id="qs4">he never stops stirring it!</s>
 <s xml:id="qs5">Figure to yourself the work of it —</s>
 <s xml:id="qs6">stir, stir, never stopping!</s>
</q>
Using the prev and next attributes, we can link the s-units with identifiers qs3 and qs4, either singly or doubly as follows:
  <s xml:id="qs3" next="#qs4"><emph>But</emph>,</s>
  <s xml:id="qs4">he never stops stirring it!</s>
  <s xml:id="qs3"><emph>But</emph>,</s>
  <s xml:id="qs4" prev="#qs3">he never stops stirring it!</s>
  <s xml:id="qs3" next="#qs4"><emph>But</emph>,</s>
  <s xml:id="qs4" prev="#qs3">he never stops stirring it!</s>
Double linking of the two s-units, as illustrated by the last of these encodings, is equivalent to specifying a link element:
<link type="jointarget="#qs3 #qs4"/>

Such a link element must carry a type attribute with a value of join to specify that the link is to be understood as joining its targets into a single aggregate.

The join element is equivalent to a link element of type join. Unlike the link element, the join element can additionally specify information about the virtual element which it represents, by means of its result attribute. And finally, unlike the link element, the position of a join element within a text is significant: it must be supplied at a position where the element indicated by its result attribute would be contextually legal.
  • join (join) identifies a possibly fragmented segment of text, by pointing at the possibly discontiguous elements which compose it.
    resultspecifies the name of an element which this aggregation may be understood to represent.
  • joinGrp (join group) groups a collection of join elements and possibly pointers.
    resultsupplies the default value for the result on each join included within the group.
To conclude the above example, we now use a join element to represent the virtual sentence formed by the aggregation of s1 and s2:
<join target="#qs3 #qs4result="s"/>
As a further example, consider the following list of authors' names. The object of the join element here is to provide another list, composed of those authors from the larger list who happen to come from Heidelberg:
<list>
 <head>Authors</head>
 <item xml:id="a_uf">Figge, Udo </item>
 <item xml:id="a_ch">Heibach, Christiane </item>
 <item xml:id="a_gh">Heyer, Gerhard </item>
 <item xml:id="a_bp">Philipp, Bettina </item>
 <item xml:id="a_ms">Samiec, Monika </item>
 <item xml:id="a_ss">Schierholz, Stefan </item>
</list>
<join target="#a_ch #a_bp #a_ss"
 result="list">

 <desc>Authors from Heidelberg</desc>
</join>

The following example shows how join can be used to reconstruct a text cited in fragments presented out of order. The poem being remembered (an unusual translation of a well-known poem by Basho) runs ‘When the old pond / gets a new frog, / it's a new pond.’

<sp>
 <speaker>Hughie</speaker>
 <p>How does it go?
 <q>
   <l xml:id="frog-x1">da-da-da</l>
   <l xml:id="frog-L2">gets a new frog</l>
   <l>...</l>
  </q>
 </p>
</sp>
<sp>
 <speaker>Louie</speaker>
 <p>
  <q>
   <l xml:id="frog-L1">When the old pond</l>
   <l>...</l>
  </q>
 </p>
</sp>
<sp>
 <speaker>Dewey</speaker>
 <p>
  <q>...
  <l xml:id="frog-L3">It's a new pond.</l>
  </q>
 </p>
 <join target="#frog-L1 #frog-L2 #frog-L3"
  result="lgscope="root"/>

</sp>
⚓︎
As with other forms of link, a grouping element joinGrp is available for use when a number of join elements of the same kind co-occur. This avoids the need to specify the result attribute for each join if they are all of the same type, and also allows us to restrict the domain within which their target elements are to be found, in the same way as for linkGrp elements (see 16.1.3 Groups of Links). Like a join, a joinGrp may appear only where the elements represented by its contents are legal. Thus if we had created many join tags of the sort just described, we could group them together, and require that their components are all contained by an element with the identifier MFKFhungry as follows:
<joinGrp domains="#mfkfhungry #mfkfhungry"
 result="s">

 <join target="#qs3 #qs4"/>
 <join target="#qs5 #qs6"/>
</joinGrp>

The join element is useful as a means of representing non-hierarchic structures (as further discussed in chapter 20 Non-hierarchical Structures). It may also be used as a convenient way of representing a variety of analytic units, like the span and interp elements discussed in chapter 17 Simple Analytic Mechanisms. As an example, consider the following famous Zen koan:

Zui-Gan called out to himself every day, ‘Master.’

Then he answered himself, ‘Yes, sir.’

And then he added, ‘Become sober.’

Again he answered, ‘Yes, sir.’

‘And after that,’ he continued, ‘do not be deceived by others.’

‘Yes, sir; yes, sir,’ he replied.

Suppose now that we wish to represent an interpretation of the above passage in which we distinguish between the various ‘voices’ adopted by Zui-Gan. In the following encoding, the who attribute has been used for this purpose; its value on each occasion supplies a pointer to the ‘voice’ to which each speech is attributed. (For convenience in this example, we use simply the first occurrence of the names used for each voice as the target for these pointers.) Note also that we add xml:id attributes to each distinct speech fragment, which we can then use to link the material spoken by each voice:
<text xml:id="zuitxt">
 <body>
  <p>
   <name xml:id="zuigan">Zui-Gan</name> called out to himself every day,
  <q next="#zuiq2xml:id="zuiq1"
    who="#zuigan">

    <name xml:id="master">Master</name>.</q>
  </p>
  <p>Then he answered himself,
  <q next="#zuiq4xml:id="zuiq2"
    who="#zuigan">
Yes, sir.</q>
  </p>
  <p>And then he added,
  <q next="#zuiq5xml:id="zuiq3"
    who="#master">
Become sober.</q>
  </p>
  <p>Again he answered,
  <q next="#zuiq7xml:id="zuiq4"
    who="#zuigan">
Yes, sir.</q>
  </p>
  <p>
   <q next="#zuiq6xml:id="zuiq5"
    who="#master">
And after that,</q>
     he continued,
  <q xml:id="zuiq6who="#master">do not be deceived by others.</q>
  </p>
  <p>
   <q xml:id="zuiq7who="#zuigan">Yes, sir; yes, sir,</q>
     he replied.</p>
 </body>
</text>
However, by using the join element, we can directly represent the complete speech attributed to each voice:
<joinGrp result="q">
 <join target="#zuiq1 #zuiq2 #zuiq4 #zuiq7">
  <desc>what Zui-Gan said</desc>
 </join>
 <join target="#zuiq3 #zuiq5 #zuiq6">
  <desc>what Master said</desc>
 </join>
</joinGrp>

Note the use of the desc child element within the two joins making up the q element here. These enable us to document the speakers of the two virtual q elements represented by the join elements; this is necessary because the there is no way of specifying the attributes to be associated with a virtual element, in particular there is no way to specify a who value for them.

Suppose now that xml:id attributes, for whatever reasons, are not available. Then ptr elements may be created using any of the methods described in section 16.2.4 TEI XPointer Schemes. The xml:id attributes of these elements may now be specified by the target attribute on the join elements.
<text>
 <body>
<!-- five div1 elements -->
  <div1>
   <p>Zui-Gan called out to himself every day, <q>Master.</q>
   </p>
   <p>Then he answered himself, <q>Yes, sir.</q>
   </p>
   <p>And then he added, <q>Become sober.</q>
   </p>
   <p>Again he answered, <q>Yes, sir.</q>
   </p>
   <p>
    <q>And after that,</q> he continued, <q>do not be deceived by others.</q>
   </p>
   <p>
    <q>Yes, sir; yes, sir,</q> he replied.</p>
   <ab type="aggregation">
    <ptr xml:id="rzuiq1"
     target="#xpath(//div1[6]/p[1]/q[1])"/>

    <ptr xml:id="rzuiq2"
     target="#xpath(//div1[6]/p[2]/q[1])"/>

    <ptr xml:id="rzuiq3"
     target="#xpath(//div1[6]/p[3]/q[1])"/>

    <ptr xml:id="rzuiq4"
     target="#xpath(//div1[6]/p[4]/q[1])"/>

    <ptr xml:id="rzuiq5"
     target="#xpath(//div1[6]/p[5]/q[1])"/>

    <ptr xml:id="rzuiq6"
     target="#xpath(//div1[6]/p[5]/q[2])"/>

    <ptr xml:id="rzuiq7"
     target="#xpath(//div1[6]/p[6]/q[1])"/>

    <joinGrp evaluate="oneresult="q">
     <join target="#rzuiq1 #rzuiq2 #rzuiq4 #rzuiq7">
      <desc>what Zui-Gan said</desc>
     </join>
     <join target="#rzuiq3 #rzuiq5 #rzuiq6">
      <desc>what Master said</desc>
     </join>
    </joinGrp>
   </ab>
  </div1>
 </body>
</text>

The extended pointer with identifier rzuiq2, for example, may be read as ‘the first q in the first p, within the sixth div1 element of the current document.’

⚓︎16.8 Alternation

This section proposes elements for the representation of alternation. We say that two or more elements are in exclusive alternation if any of those elements could be present in a text, but one and only one of them is; in addition, we say that those elements are mutually exclusive. We say that the elements are in inclusive alternation if at least one (and possibly more) of them is present. The elements that are in alternation may also be called alternants.

The need to mark exclusive alternation arises frequently in text encoding. A common situation is one in which it can be determined that exactly one of several different words appears in a given location, but it cannot be determined which one. One way to mark such an exclusive alternation is to use the linking attribute exclude. Having marked an exclusive alternation, it can sometimes later be determined which of the alternants actually appears in the given location. To preserve the fact that an alternation was posited, one can add the linking attribute select to a tag which hierarchically encompasses the alternants, which points to the one which actually appears. To assign responsibility and degree of certainty to the choice, one can use the certainty tag described in chapter 21 Certainty, Precision, and Responsibility. Also see that chapter for further discussion of certainty in general.

The exclude and select attributes may be used with any element assuming that they have been declared following the procedure discussed in the introduction to this chapter.

  • att.global.linking provides a set of attributes for hypertextual linking.
    excludepoints to elements that are in exclusive alternation with the current element.
    selectselects one or more alternants; if one alternant is selected, the ambiguity or uncertainty is marked as resolved. If more than one alternant is selected, the degree of ambiguity or uncertainty is marked as reduced by the number of alternants not selected.

A more general way to mark alternation, encompassing both exclusive and inclusive alternation, is to use the linking element alt. The description and attributes of this tag and of the associated grouping tag altGrp are as follows. These elements are also members of the att.pointing class and therefore have all the attributes associated with that class.

  • alt (alternation) identifies an alternation or a set of choices among elements or passages.
    weightsIf mode is excl, each weight states the probability that the corresponding alternative occurs. If mode is incl each weight states the probability that the corresponding alternative occurs given that at least one of the other alternatives occurs.
  • altGrp (alternation group) groups a collection of alt elements and possibly pointers.
To take a simple hypothetical example, suppose in transcribing a spoken text, we encounter an utterance that we can understand either as We had fun at the beach today. or as We had sun at the beach today. We can represent the exclusive alternation of these two possibilities by means of the exclude attribute as follows.
<div type="interview">
 <u exclude="#we.sun1xml:id="we.fun1">We had fun at the beach today.</u>
 <u exclude="#we.fun1xml:id="we.sun1">We had sun at the beach today.</u>
</div>
If it is then determined that the speaker said fun, not sun, the encoder could amend the text by deleting the alternant containing sun and the exclude attribute on the remaining alternant. Alternatively, the encoder could preserve the fact that there was uncertainty in the original transcription by retaining the alternants, and assigning the we.fun value to the select attribute value on the div element that encompasses the alternants, as in:
<div select="#we.fun2type="interview">
 <u exclude="#we.sun2xml:id="we.fun2">We had fun at the beach
   today.</u>
 <u exclude="#we.fun2xml:id="we.sun2">We had sun at the beach today.</u>
</div>
The above alternation (including the select attribute) could be recoded by assigning the exclude attributes to tags that enclose just the words or even the characters that are mutually exclusive, as in:70
<div type="interview">
 <u select="#fun3">We had
 <seg exclude="#sun3xml:id="fun3"
   type="word">
fun</seg>
  <seg exclude="#fun3xml:id="sun3"
   type="word">
sun</seg>
   at the beach today.</u>
</div>
<div type="interview">
 <u>We had
 <seg select="#id-ftype="word">
   <seg exclude="#id-sxml:id="id-f"
    type="character">
f</seg>
   <seg exclude="#id-fxml:id="id-s"
    type="character">
s</seg>
     un</seg>
   at the beach today.</u>
</div>
Now suppose that the transcriber is uncertain whether the first word in the utterance is We or Lee, but is certain that if it is Lee, then the other uncertain word is definitely fun and not sun. The three utterances that are in mutual exclusion can be encoded as follows.
<div type="interview">
<!-- ... -->
 <u exclude="#we.sun4 #lee.fun4"
  xml:id="we.fun4">
We had fun at the beach today.</u>
 <u exclude="#we.fun4 #lee.fun4"
  xml:id="we.sun4">
We had sun at the beach today.</u>
 <u exclude="#we.fun4 #we.sun4"
  xml:id="lee.fun4">
Lee had fun at the beach today.</u>
<!-- ... -->
</div>
The preceding example can also be encoded with exclude attributes on the word segments We, Lee, fun, and sun:
<u>
 <seg exclude="#leexml:id="wetype="word">We</seg>
 <seg exclude="#we #sunxml:id="lee"
  type="word">
Lee</seg>
had
<seg exclude="#sunxml:id="fun"
  type="word">
fun</seg>
 <seg exclude="#fun #leexml:id="sun"
  type="word">
sun</seg>
at the beach today.
</u>
The value of the select attribute is defined as a list of identifiers; hence it can also be used to narrow down the range of alternants, as in:
<div select="#we.fun5 #lee.fun5"
 type="interview">

 <u exclude="#we.sun5 #lee.fun5"
  xml:id="we.fun5">
We had fun at the beach today.</u>
 <u exclude="#we.fun5 #lee.fun5"
  xml:id="we.sun5">
We had sun at the beach today.</u>
 <u exclude="#we.fun5 #we.sun5"
  xml:id="lee.fun5">
Lee had fun at the beach today.</u>
</div>
This is interpreted to mean that either the first or the third u element tag appears, and is thus equivalent to just the alternation of those two tags:
<div type="interview">
 <u exclude="#lee.fun6xml:id="we.fun6">We had fun at the beach
   today.</u>
 <u exclude="#we.fun6xml:id="lee.fun6">Lee had fun at the beach today.</u>
</div>
The exclude attribute can also be used in case there is uncertainty about the tag that appears in a certain position. For example, the occurrence of the word May in the s-unit Let's go to May can be interpreted, in the absence of other information, either as a person's name or as a date. The uncertainty can be rendered as follows, using the exclude attribute.
<s>Let's go to
<name exclude="#maynxml:id="mayd">May</name>
 <date copyOf="#maydexclude="#mayd"
  xml:id="mayn"/>
.</s>

Note the use of the copyOf attribute discussed in section 16.6 Identical Elements and Virtual Copies; this avoids having to repeat the content of the element whose correct tagging is in doubt.

The copyOf and the exclude attributes also provide for a simple way of indicating uncertainty about exactly where a particular element occurs in a document.71 For example suppose that a particular div2 element appears either as the third and last of the div2 elements within the first div1 element in the body of a document, or as the first div2 of the second div1. One solution would be to record the div2 in its entirety in the first of these positions, and a virtual copy of it in the second, and mark them as excluding each other as follows:
<body>
 <div1 xml:id="C1">
  <div2 xml:id="C1S3exclude="#C2S1"/>
 </div1>
 <div1 xml:id="C2">
  <div2 xml:id="C2S1copyOf="#C1S3"
   exclude="#C1S3"/>

 </div1>
</body>
In this case, the select attribute, if used, would appear on the body element.
Mutual exclusion can also be expressed using a link; the first example in this section can be recoded by removing the exclude attributes from the u elements, and adding a link element as follows:72
<div type="interview">
 <u xml:id="we.had.fun">We had fun at the beach today.</u>
 <u xml:id="we.had.sun">We had sun at the beach today.</u>
 <link type="exclusiveAlternation"
  target="#we.had.fun #we.had.sun"/>

</div>
Now we define the specialized linking element alt, making it a member of the class att.pointing, and assigning it a mode attribute, which can have either of the values excl (for exclusive) or incl (for inclusive). Then the following equivalence holds:
<alt target="#a #bmode="excl"/>
=
<link target="#a #b"
 type="exclusive_alternation"/>
The preceding link element may therefore be recoded as the following alt element.
<alt target="#we.had.fun #we.had.sun"
 mode="excl"/>

Another attribute that is defined specifically for the alt element is weights, which is to be used if one wishes to assign probabilistic weights to the targets (alternants). Its value is a list of numbers, corresponding to the targets, expressing the probability that each target appears. If the alternants are mutually exclusive, then the weights must sum to 1.0.

Suppose in the preceding example that it is equiprobable whether fun or sun appears. Then the alt element that represents the alternation may be stated as follows:
<alt target="#we.fun #we.had.sun"
 mode="exclweights="0.5 0.5"/>
The assignment of a weight of 1.0 to one target (and weights of 0 to all the others) is equivalent to selecting that target. Thus the following encoding is equivalent to the second example at the beginning of this section.
<u xml:id="we.fun">We had fun at the beach today.</u>
<u xml:id="we.sun">We had sun at the beach today.</u>
<alt target="#we.fun #we.sunmode="excl"
 weights="1 0"/>
The sum of the weights for <alt mode="incl"> tags ranges from 0% to (100 × k)%, where k is the number of targets. If the sum is 0%, then the alternation is equivalent to exclusive alternation; if the sum is (100 x k)%, then all of the alternants must appear, and the situation is better encoded without an alt tag.

If it is desired, alt elements may be grouped together in an altGrp element, and attribute values shared by the individual alt elements may be identified on the altGrp element. The targFunc attribute defaults to the value first.alternant next.alternant.

To illustrate, consider again the example of a transcribed utterance, in which it is uncertain whether the first word is We or Lee, whether the third word is fun or sun, but that if the first word is Lee, then the third word is fun. Now suppose we have the following additional information: if we occurs, then the probability that fun occurs is 50% and that sun occurs is 50%; if fun occurs, then the probability that we occurs is 40% and that Lee occurs is 60%. This situation can be encoded as follows.
<u>
 <seg exclude="#lee2xml:id="we2"
  type="word">
We</seg>
 <seg exclude="#we2xml:id="lee2"
  type="word">
Lee</seg>
had
<seg exclude="#sun2xml:id="fun2"
  type="word">
fun</seg>
 <seg exclude="#fun2xml:id="sun2"
  type="word">
sun</seg>
at the beach today.
</u>
<altGrp>
 <alt target="#we2 #lee2"/>
 <alt target="#fun2 #sun2"/>
 <alt target="#we2 #fun2mode="incl"
  weights="0.5 0.5"/>

 <alt target="#lee2 #fun2mode="incl"
  weights="1.0 0.6"/>

</altGrp>
As noted above, when the mode attribute on an alt has the value incl, then each weight states the probability that the corresponding alternative occurs, given that at least one of the other alternatives occurs.

From the information in this encoding, we can determine that the probability is about 28.5% that the utterance is ‘We had fun at the beach today’, 28.5% that it is We had sun at the beach today, and 43% that it is Lee had fun at the beach today.

Another very similar example is the following regarding the text of a Broadway song. In three different versions of the song, the same line reads ‘Her skin is tender as a leather glove’, ‘Her skin is tender as a baseball glove’, and ‘Her skin is tender as Dimaggio's glove.’73

If we wish to express this textual variation using the alt element, we can record our relative confidence in the readings Dimaggio's (with probability 50%), a leather (25%), and a baseball (25%).

Let us extend the example with a further (imaginary) variation, supposing for the sake of the argument that the next line is variously given as and she bats from right to left (with probability 50%) or now ain't that too damn bad (with probability 50%). Using the alt element, we can express the conviction that if the first choice for the second line is correct, then the probability that the first line contains Dimaggio's is 90%, and each of the others 5%; whereas if the second choice for the second line is correct, then the probability that the first line contains Dimaggio's is 10%, and each of the others is 45%. This can be encoded, with an altGrp element containing a combination of exclusive and inclusive alt elements, as follows.
<div xml:id="bmtype="song">
 <l>Her skin is tender as
 <seg xml:id="dm">Dimaggio's</seg>
  <seg xml:id="lt">a leather</seg>
  <seg xml:id="bb">a baseball</seg>
   glove,</l>
 <l xml:id="rl">and she bats from right to left.</l>
 <l xml:id="db">now ain't that too damn bad.</l>
</div>
<altGrp>
 <alt target="#dm #lt #bbmode="excl"
  weights="0.5 0.25 0.25"/>

 <alt target="#rl #dbmode="excl"
  weights="0.50 0.50"/>

</altGrp>
<altGrp mode="incl">
 <alt target="#dm #rlweights="0.90 0.90"/>
 <alt target="#lt #rlweights="0.5 0.5"/>
 <alt target="#bb #rlweights="0.5 0.5"/>
 <alt target="#dm #dbweights="0.10 0.10"/>
 <alt target="#lt #dbweights="0.45 0.90"/>
 <alt target="#bb #dbweights="0.45 0.90"/>
</altGrp>

⚓︎16.9 Stand-off Markup

⚓︎16.9.1 Introduction

Most of the mechanisms defined in this chapter rely to a greater or lesser extent on the fact that tags in a marked-up document can both assert a property for a span of text which they enclose, and assert the existence of an association between themselves and some other span of text elsewhere. In stand-off markup, there is a clear separation of these two behaviours: the markup does not directly contain any part of the text, but instead includes it by reference. One specific mechanism recommended by these Guidelines for this purpose is the standard XInclude mechanism defined by the W3C; another is to use pointers as demonstrated elsewhere in this chapter.

There are many reasons for using stand-off markup: the source text might be read-only so that additional markup cannot be added, or a single text may need to be marked up according to several hierarchically incompatible schemes, or a single scheme may need to accommodate multiple hierarchical ambiguities, so that a single markup tree is not the most faithful representation of the source material.

This section describes a generic mechanism for expressing all kinds of markup externally as stand-off tags, for use whenever it is appropriate; and a place in the TEI structure (standOff) to contain certain common kinds of stand-off markup.

Throughout this section the following terms will be systematically used in specific senses.

source document
a document to which the stand-off markup refers (a source document can be either XML or plain text); there may be more than one source document.
internal markup
markup that is already present in an XML source document
stand-off markup
markup that is either outside of the source document and points in to it to the data it describes, or is pointed at by the data that refers to it; or alternatively is in another part of the source document and points elsewhere within the document to the data it describes, or is pointed at by data elsewhere that refers to it.
external document
a document that contains stand-off markup that points to a different, source document
internalize
the action of creating a new XML document with external markup and data integrated with the source document data, and possibly some source document markup as well
externalize
a process applied to markup from a pre-existing XML document, which splits it into two documents, an XML (external) document containing some of the markup of the original document, and another (source) XML document containing whatever text content and markup has not been extracted into the stand-off document; if all markup has been externalized from a document, the new source may be a plain text document

The three major requirements satisfied by this scheme for stand-off markup are:

  1. any valid TEI markup can be either internal or external,
  2. external markup can be internalized by applying it to the document content by either substituting the existing markup or adding to it, to form a valid TEI document, and
  3. the external markup itself specifies whether an internalized document is to be created by substituting the existing internal markup or by adding to it.

⚓︎16.9.2 Overview of XInclude

Stand-off markup which relies on the inclusion of virtual content is adequately supported by the W3C XInclude recommendation, which is also recommended for use by these Guidelines.74 XInclude defines a namespace (http://www.w3.org/2001/XInclude), which in these Guidelines will be associated with the prefix xi:, and exactly two elements, <xi:include> and <xi:fallback>. XInclude relies on the XPointer framework discussed elsewhere in this chapter to point to the actual fragments of text to be internalized. Although XInclude only requires support for the element() scheme of XPointer, these Guidelines permit the use of any of the pointing schemes discussed in section 16.2 Pointing Mechanisms.

XInclude is a W3C recommendation which specifies a syntax for the inclusion within an XML document of data fragments placed in different resources. Included resources can be either plain text or XML. XInclude instructions within an XML document are meant to be replaced by a resource targetted by a URI, possibly augmented by an XPointer that identifies the exact subresource to be included.

The <xi:include> element uses the href attribute to specify the location of the resource to be included; its value is an URI containing, if necessary, an XPointer. Additionally, it uses the parse attribute (whose only valid values are text and xml) to specify whether the included content is plain text or an XML fragment, and the encoding attribute to provide a hint, when the included fragment is text, of the character encoding of the fragment. An optional <xi:fallback> element is also permitted within an <xi:include>; it specifies alternative content to be used when the external resource cannot be fetched for some reason. Its use is not however recommended for stand-off markup.

⚓︎16.9.3 Stand-off Markup in TEI

The operations of internalizing and externalizing markup are very useful and practically important. XInclude processing as defined by the W3C is internalization of one or more source documents' content into a stand-off document. TEI use of XInclude for stand-off markup enables use of XInclude-conformant software to perform this useful operation. However, internalization is not clearly defined for all stand-off files, because the structure of the internal and external markup trees may overlap. In particular, when an external markup document selects a range that overlaps partial elements in the source document, it is not clear how the semantics of internalization (inclusion) should work, since partial elements are not XML objects.75 XInclude defines a semantics for this case that involves only complete elements.

When a range selection partially overlaps a number of elements in a source document, XInclude specifies that the partially overlapping elements should be included as well as all completely overlapping elements and characters (partially overlapping characters are not possible). The effect of this is that elements that straddle the start or end of a selected range will be included as wrappers for those of their children that are completely or partially selected by the range. For example, given the following source document:
<body>
 <p xml:id="par1">home, <emph>home</emph> on Brokeback Mountain.</p>
 <p xml:id="par2">That was the <emph>song</emph> that I sang</p>
</body>
and the following external document:
  <body>
     <div><include href="example1.xml" xmlns="http://www.w3.org/2001/XInclude"
xpointer="range(xpath(id('par1')//emph),xpath(id('par2')//emph))"/>
     </div>
 </body>   
the resulting document after XInclude processing of this external document would be:
   <body>
   <div>
     <p xml:id="par1">home, <emph>home</emph> on Brokeback Mountain.</p>
     <p xml:id="par2">That was the <emph>song</emph> that I sang</p>
   </div>
   </body>
The result of the inclusion is two paragraph elements, while the original range designated in the source document overlapped two paragraph fragments. The semantics of XInclude require the creation of well-formed XML results even though the pointing mechanisms it uses do not necessarily respect the hierarchical structure of XML documents, as in this case. While this is a good way to ensure that internalization is always possible, it has implications for the use of XInclude as a notation for the description of overlapping markup structures.

When overlapping hierarchies need to be represented for a single document, each hierarchy must be represented by a separate set of XInclude tags pointing to a common source document. This sort of structure corresponds to common practice in work with linguistic text corpora. In such corpora, each potentially overlapping hierarchy of elements for the text is represented as a separate stream of stand-off markup. Generally the source text contains markup for the smallest significant units of analysis in the corpus, such as words or morphemes, this information and its markup representing a layer of common information that is shared by all the various hierarchies. As a way of organizing the representation of complex data, this technique generally allows a large number of xml:id attributes to be attached to the shared elements, providing robust anchors for links and facilitating adjustments to the source document without breaking external documents that reference it.

Any tag can be externalized by removing its content and replacing it with an <xi:include> element that contains an XPointer pointing to the desired content.

For instance the following portion of a TEI document:
<text>
 <body>
  <head>1755</head>
  <l>To make a prairie it takes a clover and one bee,</l>
  <l>One clover, and a bee,</l>
  <l>And revery.</l>
  <l>The revery alone will do,</l>
  <l>If bees are few.</l>
 </body>
</text>
can be externalized by placing the actual text in a separate document, and providing exactly the same markup with the <xi:include> elements:
Source.xml
<content>To make a prairie it takes a clover and one bee,\n
One clover, and a bee,\n
And revery.\n
The revery alone will do,\n
If bees are few.\n
</content>

External.xml
<text xmlns:xi="http://www.w3.org/2001/XInclude">
 <body>
  <head>1755</head>
   <l>
    <xi:include href="Source.xml" parse="xml"
 xpointer="string-range(element(/1),  0, 48)"/>
   </l>
   <l>
    <xi:include href="Source.xml" parse="xml"
 xpointer="string-range(element(/1), 49, 71)"/>
   </l>
   <l>
    <xi:include href="Source.xml" parse="xml"
 xpointer="string-range(element(/1), 72, 83)"/>
   </l>
   <l>
    <xi:include href="Source.xml" parse="xml"
 xpointer="string-range(element(/1), 84,109)"/>
   </l>
   <l>
    <xi:include href="Source.xml" parse="xml"
 xpointer="string-range(element(/1),110,126)"/>
   </l>
 </body>
</text>

Please note that this specification requires that the XInclude namespace declaration is present in all cases. The <xi:fallback> element contains text or XML fragments to be placed in the document if the inclusion fails for any reason (for instance due to inaccessibility of an external resource). The <xi:fallback> element is optional; if it is not present an XInclude processor must signal a fatal error when a resource is not found. This is the preferred behaviour for use with stand-off markup. These Guidelines recommend against the use of <xi:fallback> for stand-off markup.

⚓︎16.9.4 Well-formedness and Validity of Stand-off Markup

The whole source fragment identified by an XInclude element, as well as any markup therein contained is inserted in the position specified, and an XInclude processor is required to ensure that the resulting internalized document is well-formed. This has obvious implications when the external document contains XML markup. A plain text source document will always create a well-formed internalized document.

While a TEI customization may permit <xi:include> elements in various places in a TEI document instance, in general these Guidelines suggest that validity be verified after the resolution of all the <xi:include> elements.

⚓︎16.9.5 Including Text or XML Fragments

When the source text is plain text the overall form of the XPointer pointing to it is of minimal importance. The form of the XPointer matters considerably, on the other hand, when the source document is XML.

In this case, it is rather important to distinguish whether we intend to substitute the source XML with the new one, or just to add new markup to it. The XPointers used in the references can express both cases.

A simple way is to make sure to select only textual data in the XPointer. For instance, given the following document:
Source.xhtml
<xhtml:html>
 <xhtml:body>
  <xhtml:div>To make a prairie it takes a <xhtml:a href="clover.gif">clover</xhtml:a>
     and one <xhtml:a href="bee.gif">bee</xhtml:a>,</xhtml:div>
  <xhtml:div>One <xhtml:a href="clover.gif">clover</xhtml:a>, and
     a <xhtml:a href="bee.gif">bee</xhtml:a>,</xhtml:div>
  <xhtml:div>And revery.</xhtml:div>
  <xhtml:div>The revery alone will do,</xhtml:div>
  <xhtml:div>If bees are few.</xhtml:div>
 </xhtml:body>
</xhtml:html>
the expression range(element(/1/2/1.0),element(/1/2/11.1)) will select the whole poem, text content and div elements and hypertext links (NB: in XPointer whitespace-only text nodes count).

On the contrary, the expressions xpointer(//text()/range-to(.)) and xpointer(string-range(//text(),"To")/range-to(//text(),"few.") will only select the text of the poem, with no markup inside.

Thus, the following could be a valid stand-off document for the Source.xhtml document:
External2.xml

<text xmlns:xi="http://www.w3.org/2001/XInclude">
 <body>
  <head>1755</head>
  <l>
   <xi:include href="Source.xhtml"
 xpointer='xpointer(string-range(//div[1]/text(),"To")/range-to(//div[1]/text(),"bee,")'/>
  </l>
  <l>
   <xi:include href="Source.xhtml"
 xpointer='xpointer(string-range(//div[2]/text(),"One")/range-to(//div[2]/text(),"bee,")'/>
  </l>
  <l>
   <xi:include href="Source.xhtml"
 xpointer='xpointer(string-range(//div[3]/text(),"And")/range-to(//div[3]/text(),".")'/>
  </l>
  <l>
   <xi:include href="Source.xhtml"
 xpointer='xpointer(string-range(//div[4]/text(),"The")/range-to(//div[4]/text(),",")'/>
  </l>
  <l>
   <xi:include href="Source.xhtml"
 xpointer='xpointer(string-range(//div[5]/text(),"If")/range-to(//div[5]/text(),".")'/>
  </l>
 </body>
</text>

⚓︎16.10 The <standOff> Container

The standOff element is intended to hold content that does not fit well in the text (e.g. because it is not transcribed from the source), nor in the teiHeader (e.g. because it is not metadata about the source or transcription). Examples include contextual information about named entities (typically encoded using listBibl, listOrg, listNym, listPerson, or listPlace), annotations indicating the morphosyntactic features of a text, and annotations commenting on or associating parts of a text with additional information.

  • standOff Functions as a container element for linked data, contextual information, and stand-off annotations embedded in a TEI document.
As a member of model.resource, standOff may occur as a child of TEI (or teiCorpus). If the metadata that describes the standOff is largely the same as the metadata that describes the associated resource (e.g., the transcribed text in text), then the standOff and the encoded associated resource may appear as children of the same TEI element. The example below has a transcription with <placename> elements in the text linked to a list of place elements in the standOff section.
<TEI xmlns="http://www.tei-c.org/ns/1.0">
 <teiHeader>
<!-- ... -->
 </teiHeader>
 <standOff>
  <listPlace>
<!-- ... -->
   <place xml:id="Cilicia">
    <placeName>Cilicia</placeName>
    <idno type="URI">https://pleiades.stoa.org/places/658440</idno>
   </place>
   <place xml:id="Creta">
    <placeName xml:lang="la">Creta</placeName>
    <placeName xml:lang="en">Crete</placeName>
    <idno type="URI">https://pleiades.stoa.org/places/589748</idno>
   </place>
<!-- ... -->
   <place xml:id="Rhodus">
    <placeName xml:lang="la">Rhodus</placeName>
    <placeName xml:lang="en">Rhodes</placeName>
    <idno type="URI">https://pleiades.stoa.org/places/590031</idno>
   </place>
   <place xml:id="Syria">
    <placeName>Syria</placeName>
    <idno type="URI">https://pleiades.stoa.org/places/1306</idno>
   </place>
<!-- ... -->
  </listPlace>
 </standOff>
 <text>
  <body>
   <div type="edition"
    xml:id="edition-text">

    <div type="textpartn="1"
     xml:id="part1">

     <head>Bellum Alexandrinum</head>
     <p n="1xml:id="p1">
      <seg n="1xml:id="seg-1.1">Bello Alexandrino conflato <persName ref="#Caesar">Caesar</persName>
       <app>
        <lem>
         <placeName ref="#Rhodus">Rhodo</placeName>
        </lem>
        <rdg wit="#Sana="#orthographical">Ordo</rdg>
       </app> atque ex <placeName ref="#Syria">Syria</placeName>
       <placeName ref="#Cilicia">Cilicia</placeName>que omnem classem arcessit; <app>
        <lem>
         <placeName ref="#Creta">Creta</placeName>
        </lem>
        <rdg wit="#Sana="#orthographical">certa</rdg>
       </app> sagittarios, equites ab rege <orgName ref="#Nabataei">Nabataeorum</orgName>
       <persName ref="#Malchus">Malcho</persName> euocat; tormenta undique conquiri et
             frumentum mitti, auxilia adduci iubet.</seg>
<!-- ... -->
     </p>
<!-- ... -->
    </div>
<!-- ... -->
   </div>
  </body>
 </text>
</TEI>

⚓︎16.11 Annotations

The annotation element's structure is based on the Web Annotation Data Model (WADM). A Web Annotation may have one or more targets, which are URIs, and zero or more bodies, which may be either URIs or embedded text. A Web Annotation may also contain metadata about the annotation, such as the creator, creation and modification dates, and license information. The annotation element implements a subset of WADM, using TEI elements and attributes to encode the same information, with a focus on annotating TEI documents. Targets are represented using the target attribute on annotation. URI bodies are represented using ref or ptr and embedded text bodies using note. Lifecycle and license information may be given using respStmt, revisionDesc, and licence.

TEI annotations are, in general, intended to capture the output of processes that annotate TEI texts without altering the text and markup in the text. They allow this kind of output to be represented directly in TEI, and thus to be processed using the same toolchains as the texts they annotate. A named entity recognition workflow might use annotation, for example to associate names in the text with person elements instead of attempting to rewrite the TEI text using inline persName. Projects may wish to use this mechanism to layer information onto a TEI text in case where using inline elements might result in complicated markup.

The example below illustrates how stand-off annotations can be used to connect words in a text with place elements in a list. The words ‘Gallia’ and ‘Galliae’ in the edition are connected by an annotation in the standOff section which points to them using string-range() pointers and references the definition of the place (also in the standOff section). If the set of annotations were created in a process separate from the creation of the transcription and then imported into the transcription document, then wrapping them in a TEI element with its own teiHeader providing metadata for the annotations might be advisable.
<TEI xmlns="http://www.tei-c.org/ns/1.0">
 <teiHeader>
  <fileDesc>
   <titleStmt>
    <title>De Bello Gallico</title>
   </titleStmt>
<!-- ... -->
  </fileDesc>
 </teiHeader>
 <text>
  <body>
   <div type="edition">
    <div type="textpartsubtype="chapter"
     n="1xml:id="ch1">

     <p n="1xml:id="c1p1">
      <seg n="1xml:id="c1p1s1">Gallia est omnis divisa in partes tres, quarum unam
             incolunt Belgae, aliam Aquitani, tertiam qui ipsorum lingua Celtae, nostra
             Galli appellantur.</seg>
<!-- ... -->
      <seg n="6xml:id="c1p1s6">Belgae ab extremis Galliae finibus oriuntur,
             pertinent ad inferiorem partem fluminis Rheni, spectant in septentrionem et
             orientem solem.</seg>
      <seg n="7xml:id="c1p1s7">Aquitania a Garumna flumine ad Pyrenaeos montes et
             eam partem Oceani quae est ad Hispaniam pertinet; spectat inter occasum solis
             et septentriones.]</seg>
<!-- ... -->
     </p>
<!-- ... -->
    </div>
<!-- Ch. 2 etc. ... -->
   </div>
  </body>
 </text>
 <standOff>
  <listPlace>
   <place xml:id="Gallia">
    <placeName xml:lang="la">Gallia</placeName>
    <placeName xml:lang="en">Gaul</placeName>
    <idno type="URI">https://pleiades.stoa.org/places/993</idno>
   </place>
<!-- ... -->
  </listPlace>
 </standOff>
 <TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
<!-- Metadata applying just to the set of annotations -->
  </teiHeader>
  <standOff>
   <listAnnotation>
    <annotation xml:id="ann01"
     motivation="linkingtarget="#Gallia">

     <respStmt xml:id="ed">
      <resp>creator</resp>
      <persName>Fred Editor</persName>
     </respStmt>
     <revisionDesc>
      <change status="created"
       when="2020-05-21T13:59:00Zwho="#ed"/>

      <change status="modified"
       when="2020-05-21T19:48:00Zwho="#ed"/>

     </revisionDesc>
     <licence target="http://creativecommons.org/licenses/by/3.0/"/>
<!-- Gallia in seg 1 -->
     <ptr target="#string-range(c1p1s1,0,6)"/>
<!-- Galliae in seg 6 -->
     <ptr target="#string-range(c1p1s6,19,7)"/>
    </annotation>
<!-- ... -->
   </listAnnotation>
  </standOff>
 </TEI>
</TEI>

⚓︎16.12 Connecting Analytic and Textual Markup

In chapters 17 Simple Analytic Mechanisms and 18 Feature Structures and elsewhere, provision is made for analytic and interpretive markup to be represented outside of textual markup, either in the same document or in a different document. The elements in these separate domains can be connected, either with the pointing attributes ana (for analysis) and inst (for instance), or by means of link and linkGrp elements. Numerous examples are given in these chapters.

Another more specific form of annotation is available through the TEI ruby element and its children, described in detail in 3.4.2 Ruby Annotations.

⚓︎16.13 Module for Linking, Segmentation, and Alignment

The module described in this chapter makes available the following components:

Module linking: Linking, segmentation, and alignment

The selection and combination of modules to form a TEI schema is described in 1.2 Defining a TEI Schema.

Notes
62
We use the term alignment as a special case for the more general notion of correspondence. Using A as a short form for ‘an element with its attribute xml:id set to the value A, and suppose elements A1, A2, and A3 occur in that order and form one group, while elements B1, B2, and B3 occur in that order and form another group. Then a relation in which A1 corresponds to B1, A2 corresponds to B2, and A3 corresponds to B3 is an alignment. On the other hand, a relation in which A1 corresponds to B2, B1 to C2, and C1 to A2 is not an alignment.
63
The type attribute on the note is used to classify the notes using the typology established in the Advertisement to the work: ‘The Imitations of the Ancients are added, to gratify those who either never read, or may have forgotten them; together with some of the Parodies, and Allusions to the most excellent of the Moderns.’ In the source text, the text of the poem shares the page with two sets of notes, one headed ‘Remarks’ and the other ‘Imitations’.
64
Since no special element is provided for this purpose in the present version of these Guidelines, the information should be supplied as a series of paragraphs at the end of the encodingDesc element described in section 2.3 The Encoding Description.
65
The URI (Universal Resource Indicator) is defined in RFC 3986
66
As always seems to be the case, no two regular expression languages are precisely the same. For those used to Perl regular expressions, be warned that while in Perl the pattern tei matches any string that contains tei, in the W3C language it only matches the string ‘tei’.
67
See section 17.3 Spans and Interpretations, where the text from which this fragment is taken is analyzed.
68
This sample is taken from a conversation collected and transcribed for the British National Corpus.
69
See Gale and Church (1993), from which the example in the text is taken.
70
See section 17.1 Linguistic Segment Categories for discussion of the w and c tags that can be used in the following examples instead of the <seg type="word"> and <seg type="character"> tags.
71
An alternative way of representing this problem is discussed in chapter 21 Certainty, Precision, and Responsibility.
72
In this example, we have placed the link next to the elements that represent the alternants. It could also have been placed elsewhere in the document, perhaps within a linkGrp.
73
The variant readings are found in the commercial sheet music, the performance score, and the Broadway cast recording.
74
The version on which this text is based is the W3C Recommendation dated 20 December 2004..
75
This corresponds to the observation that overlapping XML tags reflecting a textual version of such an inclusion would not even be well-formed XML. This kind of overlap in textual phenomena of interest is in fact the major reason that stand-off markup is needed.

[English] [Deutsch] [Español] [Italiano] [Français] [日本語] [한국어] [中文]




TEI Guidelines P5 Version 4.7.0. Last updated on 16th November 2023, revision e5dd73ed0. This page generated on 2023-11-16T20:03:55Z.