This is a list inside a paragraph:
Poodles Persian Cats Weaver Finches Gechos
This following is an XHTML table, inside a Table Wrapper %lt;table-wrap> wrapper.
Additional caption material
| A cell! | Another | Still a third> |
| 2 A cell! | An xref: |
2 Still a third> |
The IDREFs need to point to something
No Matter How Wise You Get, Wet Birds Don't Fly at Night
Post Hoc Propter Ergo Hoc
We thank you all.
Just a reference or two for testing:
Overlapping structures in XML are not symptoms of a misunderstanding of the intrinsic characteristics of a text document nor evidence of extreme scholarly requirements far beyond those needed by the most common XML‐based applications. On the contrary, overlaps have started to appear in a large number of incredibly popular applications hidden under the guise of syntactical tricks to the basic hierarchy of the XML data format. Unfortunately, syntactical tricks have the drawback that the affected structures require complicated workarounds to support even the simplest query or usage. In this article, we present Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), an approach to overlapping markup that simplifies and streamlines the management of multiple hierarchies on the same content, and provides an approach to sophisticated queries and usages over such structures without the need of ad‐hoc applications, simply by using Semantic Web tools and languages. We compare how relevant tasks (e.g., the identification of the contribution of an author in a word processor document) are of some substantial complexity when using the original data format and become more or less trivial when using EARMARK. We finally evaluate positively the memory and disk requirements of EARMARK documents in comparison to Open Office and Microsoft Word XML‐based formats.
+The overwhelming consensus among XML practitioners is that documents are trees, the hierarchy is the fundamental data structure, and violations of the hierarchy are errors or unnecessary complications. Therefore, overlapping markup has received ambivalent, almost schizoid considerations in the field of markup languages. Traditionally, overlaps were the hallmarks of bad HTML coders and nave HTML page editors, taking advantage of the unjustified benevolence in web browsers that would display basically any HTML regardless of proper nesting. At the same time, far from the awareness of the general public, overlaps have been a fringe, almost esoteric, discipline of scholars in the humanities, competently used for arcane specifications of linguistic annotations and literary analysis.
+Although the first type of overlap was judged with scorn and the second with awe, they both fundamentally represent a situation that is more common than was thought, and the scholars were only more aware, and not more justified, about the need to represent overlaps.
+Generally, overlap is needed when multiple independent items refer to the same segment, either when considering textual markup documents or multimedia structures (Salembier & Benitez, + 2007). Regarding documents with markup, we need overlap whenever multiple markup elements need to be applied over the same content, and these elements happen to be independent of each other. In some (rather frequent) situations, this independence means that the content referred to by some elements is partially, but not completely, the same as the content referred to by other elements.
+This situation is more frequent than it may appear: Not only do bad HTML code and arcane linguistic annotations use overlap, but many more mainstream and mundane examples exist. For instance, change tracking in an office document is often at odds with the underlying structure of the text, microformats (Allsopp, + 2007) and Resource Description Framework–in–attributes (RDFa; Adida, Birbeck, McCarron, & Pemberton, + 2008) annotations may need to refer to concepts that span across multiple XML elements, complex data structures (e.g., biological data) force graphs into trees and hide multiple parentage as internal references, and so on.
+Differently from SGML, which is able to handle some overlapping scenarios through the CONCUR notation (Goldfarb, + 1990), XML grammatically imposes and requires a strict hierarchy of containment generating a single mathematical tree of the document where no overlap is allowed. This requirement has been turned into an intrinsic characteristic of the documents XML was meant to represent rather than a syntactical and conceptual constraint into which these documents need to fit. Thus, whenever authors needed to cope with independent markup elements, they managed either by navely ignoring the hierarchical limitation (and therefore creating invalid documents) or by creating careful workarounds within the syntactical constraint, or even by inventing completely new markup languages that allow some types of overlap. But while new multihierarchical markup languages such as TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003) and LMNL (Tennison & Piez, + 2002) have a small number of adepts and applications, and while bad HTML coders and bad HTML page editors are disappearing from the market, the careful workarounds within the XML syntax (TEI Consortium, + 2005), such as segmentation, milestones, or standoff markup, are to this day frequently used and ubiquitous.
+All workarounds share the same approach of hiding structural information about a secondary hierarchy under the guise of something else: split individual elements, empty boundary elements, indirect references, and so on. The result is that the secondary structural information is hidden or its importance is lessened to not break or obfuscate the main hierarchy expressed in the visible XML structure. But this comes at a price: Structures specified through workarounds are more difficult to find, identify, and act upon than the are structures in the main XML hierarchy. Thus, trivial searches that should amount to a short XPath in a more direct situation end up being multiple‐lines long, pretty basic visualizations require incredibly complex XSLT stylesheets, specific choices of the main markup hierarchy actually prevent some features of the secondary markup to even exist, and so on. So, although workarounds exist and can be used, hierarchies expressed through them are “second‐class citizens” that cannot fully exploit the sophisticated tools that the XML language provides.
+In this article, we show how Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), our proposal for managing overlapping markup, does not generate first‐ and second‐class hierarchies, and allows existing, sophisticated tools to be used on all markup—even in the presence of overlaps. Rather than creating a completely new language requiring completely new tools and competencies, EARMARK uses Semantic Web technologies and Semantic Web tools to obtain many of the results obtainable with usual XML tools.
+EARMARK defines markup vocabularies by means of OWL ontologies (W3C OWL Working Group, + 2009). Since each individual markup item is an independent assertion over some content or some other assertions, overlaps of content are not a problem, nor are all the issues connected to physical embedding and containments, such as contiguity and document order. Furthermore, by using standard Semantic Web technologies, fairly sophisticated functionalities can be provided over EARMARK documents.
+Through EARMARK, operations that were previously very hard or impossible exactly because of the interferences of the multiple hierarchies or of the workarounds they employed now become fundamentally trivial since no syntactical tricks are employed and the different hierarchies do not interfere with each other. Thus, for instance, identifying the individual contributions in a multi‐authored MS Word or Open Office document is quite hard on their original XML formats, and becomes trivial when the same documents are converted into EARMARK.
+This article is an extended version of previous works on EARMARK (Di Iorio, Peroni, & Vitali,
+ 2010; Peroni & Vitali,
+ 2009). In those works, we focused on identifying workarounds for overlapping data existing in real XML documents and translating them into EARMARK assertions. We also sketched the EARMARK ontology and presented a simple implementation of EARMARK‐aware tools. This article follows and extends them and also provides some novel contributions:
+ The systematic analysis of the EARMARK model, with particular attention to data typing and overlapping structures The discussion of further applications for the ontological EARMARK approach. In particular, we show how EARMARK can be used to improve the content filtering and reversions mechanisms of wikis. The brief description of a process, called ROCCO, for generating EARMARK documents from existing XML documents (even ones that use workarounds for overlapping structures) An evaluation of EARMARK efficiency when dealing with multiple hierarchies in comparison with the XML structures used by popular XML‐based formats such as Office Open and MS Word.
+
+
The article is structured as follows: First, we provide a brief overview of existing approaches to overlap using workarounds in XML or ad hoc markup metalanguages, and then give a few examples of situations where overlaps are used today and sometimes in rather mainstream situations. Next, we present the EARMARK model and its rules. Then, we provide some use cases that are meant to demonstrate the superiority of the EARMARK approach to a traditional XML format, especially when overlaps come into question, and show the generation of EARMARK documents, converting legacy documents. An initial evaluation of the efficiency of EARMARK compared to popular XML based data formats such as Open Document (ODT) and Office Open XML (OOXML) is presented, followed by our onclusions.
+The need for multiple overlapping structures over documents using markup syntaxes such as XML and SGML is an age‐old issue, and a large amount of literature exists on the techniques, languages, and tools that allow users to create multiple entangled hierarchies over the same content. A good review can be found in DeRose ( + 2004).
+Some research has proposed using plain hierarchical markup (i.e., XML) and employing specially tailored elements or attributes to express the semantics of overlapping in an implicit way. The TEI Guidelines (TEI Consortium,
+ 2005) presented a number of different techniques that use SGML/XML constructs to force multiple hierarchies into a single one, including:
+ milestones (i.e., verlapping structures are expressed through empty elements to mark the boundaries of the “content”), fragmentation (i.e., overlapping structures are split into individual, nonoverlapping elements that may even be linked through id–idref pairs), and standoff markup (i.e., overlapping structures are placed elsewhere and indirectly refer to their would‐be locations through pointers, locators, and/or id–idref pairs).
+
+
Given the large number of techniques to deal with overlapping structures in XML, in Marinelli, Vitali, and Zacchiroli ( + 2008), we presented a number of algorithms to convert XML documents with overlapping structures from and to the most common approaches, as well as a prototype implementation.
+Riggs ( + 2002) introduced a slightly different technique for fragmentation within XML structures. In this proposal, floating elements (i.e., those elements that do not fall in a proper or meaningful hierarchical order) are created using the name of the element followed by an index referring to its semantically related parent element. For example, the floating element <name.person[2]>John</name.person [2] means that <name>John</name> is semantically a child of the second occurrence of the element person, even though the floating element is not structurally contained by its logical parent.
+Other research even has proposed to get rid of the theory of trees at the base of XML/SGML altogether and use different underlying models and newly invented XML‐like languages that allow the expression of overlaps through some kind of syntactical flourishing. For instance, a general ordered‐descendant directed acyclic graph (GODDAG; Sperberg‐McQueen & Huitfeldt, + 2004) is a family of graph‐theoretical data structures to handle overlapping markup. A GODDAG's nodes represent markup elements and text. Arcs are used to explicitly represent containment and father–child relations. Since multiple arcs can be directed to the same node, overlapping structures can be straightforwardly represented in GODDAG. Full GODDAGs cannot be linearized in any form using embedded markup, but restricted GODDAGs, a subset thereof, can be and have been linearized into TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003), a multihierarchical markup language that also allows full GODDAGs through appropriate nonembedding workarounds such as standoff markup.
+LMNL (Tennison & Piez, + 2002) is a general data model based on the idea of layered text fragments and ranges, where multiple types of overlap can be modeled using concepts drawn from the mathematical theory of intervals. Multiple serializations of LMNL exist, such as CLIX and LMNL‐syntax.
+XConcur (Schonefeld & Witt, + 2006) is a similar solution based on the representation of multiple hierarchies within the same document through layers. Strictly related to its predecessor CONCUR as it was included in the SGML, XConcur was developed in conjunction with the validation language XConcur‐CL to handle relationships and constraints between multiple hierarchies.
+The variant graph approach (Schmidt & Colomb, + 2009) also is based on graph theory. Developed to deal with textual variations that generate multiple versions of the same document with multiple overlapping hierarchies, this theory proposes a new data model to represent literary documents and a graph linearization (based on lists) that scales well even with a large number of versions. The same authors recently presented an extension of their theory that also allows users to merge multiple variants into one document (Schmidt, + 2009). In Portier and Calabretto ( + 2009), a detailed survey about overlapping approaches was presented, and also discussed the MultiX2 data model, which uses W3C standard languages such as XInclude to link and fetch text fragments within overlapping structures, and a prototype editor for the creation of multistructured documents.
+Tummarello, Morbidoni, and Pierazzo ( + 2005) proposed using RDF as a standoff notation for overlapping structures of XML documents. Since this proposal has many affinities with the one we are presenting in this article, we later discuss its characteristics and compare it with ours.
+Overlapping structures have been considered often as appropriate only in highly specific contexts and basically for scholars: The solutions that have been proposed in the literature were complex since they were considered grounded in the intrinsic complexity of the topics themselves. Yet, overlapping structures can be found in many more fields than these, and even mainstream applications generate and use markup with overlapping structures. While the complexity of overlapping is hidden to the final user, applications that consume such data may very well find it rather difficult to handle such information. We next discuss three very different contexts where overlapping already exists and fairly relevant information is encoded in multiple independent structures, leaving to special code the task of managing the complexity.
+Word processors such as Microsoft Word and Open Office provide users with powerful tools for tracking changes, allowing each individual modification by individual authors to be identified, highlighted, and acted upon (e.g., by accepting or discarding them). The intuitiveness of the relevant interfaces actually hides the complexity of the data format and of the algorithms necessary to handle such information.
+For instance, the standard ODT format (JTC1/SC34 WG 6, + 2006) used by Open Office, when saving change‐tracking information, relies on two specific constructs for insertions and deletions that may overlap with the structural markup. Adding a few words within a paragraph is not in itself complex, as it does not require the breaking of the fundamental structural hierarchy; conversely, changes that affect the structure itself (e.g., the split of one paragraph in two by the insertion of a return character, or the joining of two paragraphs by the elimination of the intermediate return character) require that annotations are associated to the end of a paragraph and the beginning of the next, in an unavoidably overlapping pattern. ODT uses milestones and standoff markup for insertions and deletions, respectively, and also relies on standoff markup for annotations about the authorship and date of the change.
+For instance, the insertion of a return character and a few characters in a paragraph creates a structure as follows:
+
+
The empty elements <text:change‐start/> and <text:change‐end/> are milestones marking the beginning and the end, respectively, of the range that constituted the insertion while the element <text:insertion>, before the beginning of the document content, is standoff markup for the metadata about the change (author and date information).
+Similarly, a deletion creates a structure as follows:
+
+
The element <text:change/> represents a milestone of the location where the deletion took place in the content, and the corresponding standoff markup annotation <text:deletion> contains not only the metadata about the change but also the text that was deleted.
+The OOXML format (JTC1/SC34 WG 4, + 2008) (the XML‐based format used by Microsoft Office 2007 and standardized by ISO in 2008), on the other hand, uses a form of segmentation to store change‐tracking information across all previous elements involved.
+
+
This heavily simplified version of an OOXML document shows two separate changes: (a) the insertion of a return character and (b) the insertion of a word. These modifications are not considered as a single change; therefore, the segments are not connected to each other but simply created as needed to fit the underlying structure.
+In fact, change tracking in OOXML is a fairly complex proposition. Although providing more complete coverage of special cases and situations than does ODT, dealing with its intricacies is not for the casual programmer. Even a simple XSLT stylesheet to show inserted text in a different color and hide deleted text may run several hundred lines of code.
+ 1
+
+
Microformats (Allsopp, + 2007) add semantic markup to web documents by using common structures of the HTML language itself—in particular, the class attribute.
+The HTML code is annotated using microformats to provide new semantic, machine‐processable assertions. In the following example, a plain HTML table is enriched with metadata about events
+ 2
+ HCalendar,
+ HCard,
+
+
The table was enriched by additional data declaring it to be an event (a conference), and data about the event itself (URL, summary, location) and about four relevant individuals (with their names and roles within the conference) were associated where necessary to the actual content of the table.
+So far, so good, and no overlap to speak about. Things change dramatically, though, when the overall structure of the main hierarchy (the HTML table) is at odds with the intrinsic hierarchy of the microformat data, such as if the people are organized in columns rather than rows. For instance:
+
+
Unfortunately, vcards are a hierarchy themselves, and if the hierarchy of vcards is organized differently from the hierarchy of the HTML table, as in the latter case, it is just impossible to define the four vcards for the four people organizing the conference. Thus, in plain HTML, the choice of one of two possible presentation models for the main hierarchy of content makes trivial or completely impossible the existence of the second hierarchy.
+A possible and partial solution to express vcard hierarchies in the latter example is RDFa (Adida et al., + 2008), a W3C recommendation. It describes a mechanism to embed RDF statements into HTML documents by using some HTML attributes (href, rel, rev, content) in combination with other ad hoc attributes (property, about, typeof) proposed in the recommendation itself.
+
+
Since all attributes live in the context of elements, the price to pay is that to assert everything we want to assert, we often need to add some structurally unnecessary elements to the current markup hierarchy of a document, needed only to add the RDF statements (e.g., the span elements emphasized earlier). Even if that does not represent a significant problem for strict Semantic Web theorists, document architects and markup experts see this as a kludge and an inelegant compromise.
+The strength of wikis lies in their allowing users to modify content at any time. The mechanisms of change‐tracking and rollback that are characteristics of all wikis, in fact, promote users' contributions and make “malicious attacks” pointless in the long run since previous versions can be easily restored.
+A number of tools exist that automatically discover “wiki vandalisms” and provide users with powerful interfaces to surf changes, identify differences between subsequent versions, and revert content. For instance, Huggle
+ 4
+
+
Even client‐side tools—meant to be installed as browser extensions or bookmarklets—exist to extend the rollback mechanisms of Wikipedia, giving users more flexibility and control over (vandalistic) changes. For instance, Lupin
+ 5
+
+
+
These tools are successful in highlighting vandalism and in identifying versions created by malicious users. However, although it is possible to revert the page to any previous version, all changes (even acceptable ones) that were subsequent to the malicious version cannot be automatically inherited by the restored page.
+For instance consider Versions V1, V2, and V3 of a wiki page, where Version V1 contains a baseline (i.e., acceptable) content, and Version V2 is identified as a partial vandalism and is agreed to be removed, but Version V3 contains (possibly, in a completely different section than the target of the malicious attack) relevant and useful content that was added before the vandalistic Version V2 was declared as such. The task of removing the modifications of Version V2 while maintaining (whatever is possible of) Version V3 is a difficult, error‐prone, and time‐consuming task if done manually, yet there is no tool we are aware of that automatically filters contributions from multiple versions and merges them into a new one (or, equivalently, removes only selected intermediate versions).
+However, it is possible to theoretically characterize the interdependencies between subsequent changes to a document. In fact, literature has existed for a long time on exactly these themes (e.g., Durand, + 1994, + 2008). Although a detailed discussion of abstract models of interconnected changes is out of scope for this article (Details and authoritative references can be found in the aforementioned works.), what is relevant in this discussion is that they happen to assume a hierarchical form that is frequently at odds with the hierarchical structure of the content of the document, and as such, most issues derive from the data structures in which content is stored and from the model for manipulating these structures. For instance, the fact that in the wiki perspective each version is an independent unit that shares no content (even unchanged content) with the other versions prevents considering multiple versions as overlapping structures coexisting on the same document. If we were able to make these hierarchies explicit, we would be able to create models and tools to manipulate these documents in a more powerful way and to exploit the existing interconnections between the overlapping hierarchies.
+The presence of hidden overlapping structures—transparent to users, but very difficult to handle by applications—is the common denominator for the scenarios described in the previous section. More than the overlap itself, which cannot be ignored because it does exist and carries important meanings, the problem we face lies in the way applications store such overlapping structures. In the XML world, in fact, the only way to do so is through the use of (complex) workarounds that force the multiple hierarchies into one hierarchy of an XML document. That makes it very tricky to perform sophisticated analysis and searches.
+This section discusses a different approach to metamarkup, EARMARK (Di Iorio, Peroni, & Vitali, + 2009; Di Iorio et al., + 2010; Peroni & Vitali, + 2009) based on ontologies and Semantic Web technologies. The basic idea is to model EARMARK documents as collections of addressable text fragments, and to associate such text content with OWL assertions that describe structural features as well as semantic properties of (parts of) that content. As a result, EARMARK allows not only documents with single hierarchies (as with XML) but also multiple overlapping hierarchies where the textual content within the markup items belongs to some hierarchies, but not to others. Moreover, EARMARK makes it possible to add semantic annotations to the content though assertions that may overlap with existing ones.
+One of the advantages of using EARMARK is the capability to access and query documents by using well‐known and widely supported tools for the Semantic Web. In fact, EARMARK assertions are simply RDF assertions while EARMARK documents are modeled through OWL ontologies. The consequence is that query languages (e.g., SPARQL; Garlik & Seaborne,
+ 2010) and actual existing tools such as Jena
+ 7
+
+
+
In the rest of this section, we give a brief overview of the EARMARK model and then describe how EARMARK can be used to deal with the issues presented earlier. The model itself is defined through an OWL document,
+ 9
+
+
A UML‐like representation of the EARMARK ontology. [Color figure can be viewed in the online issue, which is available at
+
The ghost classes describe three disjoint base concepts—docuverses, ranges, and markup items—through three different and disjoint OWL classes.
+ 10
+ All our OWL samples are presented using the Manchester Syntax (Horridge & Patel‐Schneider,
+ 2009), which is one of the standard linearization syntaxes of OWL. The prefixes rdfs and xsd refer to RDF Schema and XML Schema namespaces, respectively, while the empty prefix refers to the EARMARK ontology URI plus “#.” Moreover, we use the prefix c to indicate entities taken from an imported ontology made for the SWAN project (Ciccarese et al.,
+ 2008); available at
+
The textual content of an EARMARK document is conceptually separated from its annotations, and is referred to through the Docuverse class.
+ 11
+ This class (and its name) is based on the concept introduced by Ted Nelson (
+ 1980) in his Xanadu Project to refer to the collection of text fragments that can be interconnected to each other and transcluded into new documents.
+
Any individual of the Docuverse class—commonly called a docuverse (lowercase to distinguish it from the class)—specifies its actual content with the property hasContent.
+We then define the class Range for any text lying between two locations of a docuverse. A range (i.e., an individual of the class Range) is defined by a starting and an ending location (any literal) of a specific docuverse through the properties begins, ends, and refersTo, respectively.
+
+
There is no restriction on locations used for the begins and ends properties. That is very useful because it allows us to define ranges that follow or reverse the text order of the docuverse to which they refer. For instance, the string “desserts” can be considered both in document order, with the begins location lower than the ends location, or in the opposite order, forming “stressed.”
+ 12
+
+
The class MarkupItem is the superclass defining artifacts to be interpreted as markup (e.g., elements and attributes).
+
+
A markupitem individual is a collection (c:Set, c:Bag, or c:List, where the latter is a subclass of the second one, and all of them are subclasses of c:Collection) of individuals belonging to the classes MarkupItem and Range. Through these collections, it is possible to define a markup item as a set, a bag, or a list of other markup items, using the properties element (for sets) and item and itemContent (for bags and lists). Thus, it becomes possible to define elements containing nested elements or text, or attributes containing values, as well as overlapping and complex structures. Note also that handling collections directly in OWL allows us to reason about content models for markup items, which would not be possible if we had used the corresponding constructs in RDF.
+ 13
+
+
A markupitem also might have a name, specified in the functional property hasGeneralIdentifier (recalling the SGML term to refer to the name of elements; Goldfarb, + 1990), and a namespace, specified using the functional property hasNamespace. Note that we can have anonymous markup items—as is possible in LMNL (Tennison & Piez, + 2002) and in GODDAG (Sperberg‐McQueen & Huitfeldt, + 2004)—by simply asserting that the item belongs to the class of all those markupitems that do not have a general identifier (i.e., hasGeneralIdentifier exactly 0).
+The ghost classes discussed so far give us an abstract picture of the EARMARK framework. We need to specialize our model, defining a concrete description of our classes. These new shell subclasses apply specific restrictions to the ghost classes.
+First, the class Docuverse is restricted to be either a StringDocuverse (i.e., the content is specified by a string) or a URIDocuverse (i.e., the actual content is located at the URI specified).
+
+
Depending on particular scenarios or on the kind of docuverse we are dealing with (plain‐text, XML, LaTeX, a picture, etc.), we need to use different kinds of ranges. Therefore, the class Range has three different subclasses:
+ PointerRange defines a range by counting characters. In that case, the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream, remembering that the value 0 refers to the location immediately before the first character, the value 1 refers to the location after the first character and before the second one, and so on. By using the hasKey OWL property, we also assert that two pointer ranges having equal docuverse and begin and end locations are the same range. XPathRange defines a range considering the whole docuverse or its particular context specifiable through an XPath expression (Berglund et al.,
+ 2007) as value of the property hasXPathContext. Note that by using these ranges, we implicitly admit that the docuverse it refers to must be an XML structure. Moreover, the properties' begins and ends have to be applied on the string value obtained by juxtaposing all the text nodes identified by the XPath. By using the hasKey OWL property, we also assert that two xpath ranges having equal docuverse, XPath context, and begin and end locations are the same range. XPathPointerRange is an XPathRange in which the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream as described for the class PointerRange.
+
+
+
MarkupItem is specialized in three disjointed subclasses—Element, Attribute, and Comment—that allow a more precise characterization of markup items.
+
+
The presence of overlap in EARMARK is worth discussing in more detail. Different types of overlap exist, according to the subset of items involved, and different strategies are needed to detect them. In particular, there is a clear distinction between overlapping ranges and overlapping markup items.
+By definition, overlapping ranges are two ranges that refer to the same docuverse, so that at least one of the locations of the first range is contained in the interval described by the locations of the second range (excluding its terminal points). Totally overlapping ranges have the locations of the first range completely contained in the interval of the second range, or vice versa, while partially overlapping ranges have either exactly one location inside the interval and the other outside or identical terminal points in reversed roles.
+Thus, if we consider the following excerpt:
+
+
we can infer, through a reasoner such as Pellet, that these two ranges overlap by using the following rules:
+
+
where P is one of:
+ lessThan(b1,e1) ˆ greaterThan(b2,b1) ˆ lessThan(b2,e1) lessThan(b1,e1) ˆ greaterThan(e2,b1) ˆ lessThan(e2,e1) lessThan(e1,b1) ˆ greaterThan(b2,e1) ˆ lessThan(b2,b1) lessThan(e1,b1) ˆ greaterThan(e2,e1) ˆ lessThan(e2,b1).
+
+
The case of overlapping markup items is slightly more complicated. We define that two markup items A and B overlap when at least one of the following sentences holds:
+ [Overlap by range]: A contains a range that overlaps with another range contained by B. [Overlap by content hierarchy]: A and B contain at least a range in common. [Overlap by markup hierarchy]: A and B contain at least a markup item in common.
+
+
The three possible scenarios for such item overlap are summarized in Figure
+ 2.
+ 14
+ The EARMARK documents describing these three overlapping scenarios and all the other ones presented in the following sections are available at
+
Three EARMARK examples of overlapping between elements p. [Color figure can be viewed in the online issue, which is available at
+
The EARMARK ontology, in fact, is completed by another ontology
+ 15
+
+
If we ignore for a moment the semantic implications of using EARMARK and concentrate on its syntactical aspects only, it is easy to observe that EARMARK is nothing but yet another standoff notation, where the markup specifications point to, rather than contain, the relevant substructure and text fragments.
+Standoff notations, also known in literature as out‐of‐line notations (TEI Consortium,
+ 2005), are hardly new, but never really caught on for a number of reasons, most having to do with their perceived fragility under the circumstances of desynchronized modification to the text. In Georg, Schonefeld, Trippel, and Witt (
+ 2010) and Bański (
+ 2010), we can find a pair of recent and substantially complete analyses of their merits and demerits. In particular, according to Georg, Schonefeld, Trippel, and Witt (
+ 2010), “standoff annotation has … quite a few disadvantages:
+ very difficult to read for humans The information, although included, is difficult to access using generic methods. Limited software support as standard parsing or editing software cannot be employed. Standard document grammars can be used only for the level which contains both markup and textual data. New layers require a separate interpretation. Layers, although separate, often depend on each other.”
+ 16
+ To individually address the issues, we edited the original bullets into a numbered list.
+
+
And yet, although EARMARK is in practice a standoff notation, it provides a number of workarounds to most of the aforementioned issues.
+First, since EARMARK is based on OWL and can be linearized in any of the large number of OWL caricaturization syntaxes, it follows that (a) readability, (b) access, and (c) software support for it are exactly those existing for well‐known, widespread, and important W3C standards such as RDF and OWL. Being able to employ common RDF and OWL tools such as Jena and SPARQL for EARMARK documents was in fact a major motivation for it.
+Issue 4 should be examined beyond the mere validation against document grammars and toward a general evaluation of the level of compliancy of the markup to some formally specified expectations. EARMARK documents, while being subject to no document grammar in the stricter XML sense, allow the specification of any number of constraints, expressed either directly in OWL or SWRL (Horrocks et al., + 2004), or even in SPARQL, that trigger or generate validity evaluations. In Di Iorio, Peroni, and Vitali (in press), we tried to show that a large number of requirements, from hierarchical well‐formedness in the XML sense, to validation requirements in terms of XML DTDs, to adherence to design patterns, can be expressed satisfactorily using these technologies.
+Issue 5 regards the difficulty of standoff notations to provide interlayer analysis on XML structures: Separate interpretation of markup layers is easy, but identification and validation of overlapping situations are more complex: Standoff markup is mainly composed of pointers to content and does not have any direct way to determine overlap locations without some kind of pointer arithmetics to compute them. Validation of contexts allowing overlaps as describable using rabbit/duck grammars (Sperberg‐McQueen, + 2006) also is not trivial. In this regard, EARMARK provides yet again a solution that does not require special tools: Although OWL does not allow direct pointer arithmetics, SWRL on the contrary does, as shown earlier where we described a batch of (SWRL‐implementable) rules that do, in fact, determine overlapping locations on EARMARK documents with good efficiency.
+Finally, Issue 6 refers to the fact that evolution of separate markup annotation layers needs to synchronously take place, lest one of them becomes misaligned with the new state of the document. This is, in summary, the fragility of pointers, which can be considered the fundamental weakness of standoff, as well as of any notation that has markup separate from its content: If a modification occurs to the underlying (probably text‐based) source, all standoff pointers that could not be updated at the same time of the change become outdated and possibly wrong. All standoff notations fall prey to this weakness, and there is no way to completely get rid of it.
+What is possible is to identify exactly the conditions under which such weakness acts, and see if there is a way to reduce the mere frequency of such events. In fact, for a standoff pointer to become outdated, several conditions must take place at the same time:
+ The standoff notation must be used as a storage format, rather than just as a processing format; the source document must make sense even without the additional standoff markup (i.e., the standoff notation contains no information that is necessary for at least some types of document modifications); the source document must be editable (and, in fact, must be edited) on its own; the standoff pointers must rely on positions that change when the source is edited (e.g., character‐based locations); editing must be done in contexts and with tools that cannot or do not update the standoff pointers; and there must be no computable way to determine the modifications of the document (e.g., via a diff between the old and new versions).
+
+
Of course, no standoff notation can rule out that these conditions occur on their documents, but note that all six of them must occur for standoff pointers to become outdated. EARMARK is not safe from these occurrences either, but at least for the use cases here described, one or more of these conditions simply do not apply: EARMARK is mostly used as a processing format, with no need to save it on disk (Conversion from the source formats such as MS Word is described later and does not require special storage.), the data format described is either in a very specific format (e.g., MS Word or ODT) that in fact already does handle internally its data changes and requires the overlapping data exactly for this purpose, or is in fact the result of a diff action on successive versions of a document (as in the case of the wiki pages). Finally, EARMARK allows references to relatively stable fragment ids of the documents (by using XPath ranges without specifying explicitly begin and end locations) rather than the extremely fragile character locations, further reducing the chances of outdated pointers. For this reason, without being able to completely rule out the possibility of standoff pointers going wrong, we tend to consider it as a significantly little risk, at least for the use case described here.
+EARMARK is strongly based on OWL 2 DL (W3C OWL Working Group, + 2009) to express multiple markup layers with possible overlapping ranges over the same content. OWL 2 DL is not the only possible choice for expressing standoff notations via Semantic Web technologies. In fact, RDF is another valid and effective model for dealing with the same issue, as shown in Tummarello et al. ( + 2005), by means of the open‐source application programming interface (API) RDF Textual Encoding Framework (RDFTef). This API was created to demonstrate a plausible way for handling overlapping markup within documents and identifying textual content of a document as a set of independent RDF resources that can be linked mutually and with other parent resources.
+Besides giving the possibility to define multiple structural markup hierarchies over the same text content, the use of RDF as the language for encoding markup allows to specify semantic data on textual content as well. But the real main advantage in using RDF is the possibility of using particular built‐in resources appositely defined in the RDF syntax specification (Beckett, + 2004) for describing and dealing with different kinds of containers, either ordered (rdf:Seq) or unordered (rdf:Bag). Thus, RDF resources can be used to represent every printable element in the text—words, punctuation, characters, typographical symbols, and so on—while RDF containers also can be used to combine such fragments and containers.
+Although RDF is not sufficient to define a formal vocabulary for structural markup, does a given resource represent an element, an attribute, a comment, or a text node? In which way is a resource of a certain type related to others? The specification of an RDFS (Brickley & Guha,
+ 2004) or of an OWL layer can successfully address these issues. Hybrid solutions obtained by mixing different models, even when they are built one upon another, may seem elegant, but not necessarily the best choice. In fact, there exist well‐known interoperability limits between OWL 2 DL and RDF that prevent the correct use of Semantic Web tools and technologies. In particular:
+ Any markup document made using RDF containers (e.g., to describe what markup items contain and in which order) and OWL ontologies (e.g., to define classes of markup entities and their semantics) results in a set of axioms that end up outside of OWL DL and well within OWL Full, which limits the applicability of the most frequently used Semantic Web tools that are usually built upon the (computationally tractable) description logic underlying OWL 2 DL. The individual analysis of each language may be not applicable when we have to check particular properties that lie between RDF and OWL layers. For example, verifying the validity of a markup document against a particular schema, which is one of the most common activities with markup, needs to be made to work with both markup item structures (that would be defined in RDF) and logical constraints about classes of markup items (e.g., elements only, attributes only, the element “p,” all the element of a particular namespace, etc., all of them definable in OWL).
+
+
Being able to express everything we need directly in OWL quite straightforwardly addresses both issues. The well‐known absence of containers and sequences in OWL can be overcome by modeling classes in specific ways using specific design patterns such as those in Ciccarese et al. ( + 2008) and in Drummond et al. ( + 2006).
+There are multiple applications for the EARMARK approach. The most interesting for this article is its capability of dealing with overlapping structures in an elegant and straightforward manner. Under EARMARK, such structures do not need to be specified through complex workarounds as with XML, but they are explicit and can be easily described and accessed. Sophisticated searches and content manipulations become very simple when using this ontological model.
+The goal of this section is to demonstrate the soundness and applicability of EARMARK by discussing how the use cases presented earlier are addressed. Note that throughout the section we investigate multiple EARMARK data structures and documents, focusing on the feasibility and potentiality of such an ontological representation.
+The discussion about change tracking in office document formats showed that both ODT (OpenOffice format) and OOXML (Microsoft Word format) use complex data structures to store overlaps generated by change‐tracking functionalities. These structures make it very difficult to search and manipulate the content when using XML languages and tools. Even very simple edits generate a rather tangled set of overlapping elements.
+Let us recall the example mentioned earlier where the user “John Smith” splits a single paragraph into two. The ODT representation is:
+
+
The OOXML representation (shown earlier) is even more complex. In fact, these formats exploit in large scale (tangled) fragmentation (OOXML) or milestones and stand‐off markup (ODT) to deal with overlaps.
+EARMARK, on the other hand, stores overlapping data in a direct and streamlined manner that does not require tools to rebuild information from the twists of a tree‐based XML structure. The information already is available and expressed through consistent RDF and OWL statements. Figure + 3 graphically shows the corresponding EARMARK document.
+Encoding in EARMARK the ODT change‐tracking example. [Color figure can be viewed in the online issue, which is available at
+
The original paragraph content and the new string “also” are now encoded as two docuverses over which the ranges r1, r2, and r3 are defined. The original paragraph is then composed of the (content of) ranges r1 and r2 while the paragraphs resulting after the (text and carriage return) insertion now comprise range r1 and ranges r2 and r3, respectively. Metadata about the author and the modification date are encoded as further RDF statements.
+
+
The advantages of streamlining overlaps becomes apparent if we consider tasks a little beyond the mere display. For instance, the query for “the textual content of all paragraphs inserted by John Smith” ends up rather entangled if we used XPath on the ODT structure. The process for finding that textual content needs to browse the current version of the document and look for all the text:change‐start/text:change‐end pairs that refer to an insertion made by John Smith involving the creation of a new paragraph (i.e., text:change‐start is in a first paragraph while its pair, text:change‐end, is in the following one) that are either currently present in the document body or hidden behind a subsequent deletion made by someone else. Once the paragraphs are identified, we need to retrieve the content that originally was contained there (i.e., the text fragments that still are within those boundaries or that may have been deleted in subsequent versions). The following XPath represenst an implementation of this process:
+
+
The XML structure of an MS Word file, using segmentation rather than milestones, does simplify the query a bit, but still presents some radical complexities. The process starts by choosing all those w:p elements that were inserted by John Smith as well as all their previous and contiguous w:p elements that were deleted before or inserted after the first ones. In OOXML, each sequence of contiguous w:p elements implicitly represents one paragraph. Therefore, we can now take all the text fragments contained in each w:p sequence that were inserted before or deleted after the paragraph defined by the sequence itself. The following is the resulting XPath for an OOXML document.
+
+
The complexity of both XPath queries is due to the intrinsic complexity of the data structure on which the query has to work. Although the interface of OpenOffice or MS Word may provide tools to directly deal with these queries using specific strategies on the internal data structures, applications working directly on the XML structure have very little help in disentangling the mess of the data formats.
+On the other hand, since EARMARK documents are actually OWL files, it is possible to access and query them with plain Semantic Web tools. Powerful searches then can be performed without using niche‐specific tools or complex and long XPath expressions simply with mainstream technologies such as SPARQL (Garlik & Seaborne, + 2010).
+The corresponding SPARQL query for (“the textual content of all paragraphs inserted by John Smith”) therefore can be written as follows:
+
+
But EARMARK is useful for even more than querying: EARMARK also decreases the costs, in terms of efforts and lines of code, for manipulating documents.
+Let us consider the task of generating an intermediate version (i.e., neither the first nor the last one of a version chain) from a document that includes change‐tracking information about the whole document history.
+The process of rebuilding these versions by working on the XML structure without specific APIs is both complex and inefficient. For example, a basic XSLT that returns an XML document defining the desired version requires to at least:
+ define templates for all the elements actively involved in the change tracking (e.g., for ODT, text:changed‐region, text:change‐start, text:change‐end, and text:change and similarly for OOXML) to understand, by looking at their creation date, whether they must be considered or ignored when building the requested version. In particular, we must exclude insertions following and deletions preceding the version we are building; define templates for paragraphs to handle cases where the paragraph is the result of an insertion or a deletion of other paragraphs to identify whether it should be considered for the result and, in such case, finding out its real text content and remembering that in the following versions, such content may have spread out among other paragraphs; define templates for handling insertions/deletions for structures such as images, sections, lists, and tables; and define an identity template for the other elements to visit the entire document.
+
+
Even the most basic and incomplete implementation of such XSLT requires hundreds of lines of complex and convoluted code and a large number of ad hoc decisions based on the specificities of whether we start from ODT or OOXML. Note also that a Java‐based implementation (or in any other procedural language) of the same process would be equally or even more complex.
+The same result can be achieved on EARMARK documents with a few lines of Java code:
+
+
This approach uses the EARMARK Java API
+ 17
+
+
This method heavily uses Semantic Web technologies on the structures provided by EARMARK whose characteristics are always explicit and clear. In fact, since all versions coexist within the EARMARK document and each version can be encoded explicitly as a tree within the overall graph, this operation is straightforward and fast.
+EARMARK also can be exploited to improve semantic annotations. As noted earlier, there are in fact strong limitations in the same process of annotating web documents with semantic structures that overlap the structural ones. The same example—of vcards that cannot be created on the top of tables organized per rows—will be used in this section.
+We solve this by converting the web document with annotations into an EARMARK document, allowing both semantic and structural annotations to coexist. Through EARMARK, we can explicitly express both markup structures and vcard assertions. Figure + 4 shows how the vcard example can be modeled (Once again, we show a graphical representation for the sake of clarity.)
+The abstract model of the EARMARK document solving the microformats issue. [Color figure can be viewed in the online issue, which is available at
+
The textual content of the original table cells is now encoded in two different docuverses: one for the header (with roles) and one for the body (with names of committee members). Ranges r1, r2, …, r8 are then created to distinguish each role and name. Two independent and coexisting hierarchies are then built on top of the same set of ranges: the HTML table that includes one cell for each range (in blue) and the vcards about each person (in green) that include only the relevant ranges and overlap the previous one. Note also that the vcards are defined in such a way that does not interfere with the structural features of the table. The full linearization in OWL of this example can be found at
+
EARMARK can be used to improve wiki reversion mechanisms and overcome the limitations discussed earlier: The automatic filtering and merging of contributions from multiple versions of the same page are still a manual process, but it can be fully automatized if the overlapping structures buried in the whole history of the page become explicit.
+The role of EARMARK is to make those structures explicit and available for more sophisticated content manipulation. To understand the extent EARMARK structures can be derived from wikis and how they can be exploited by the final users, we use as our example the wiki platform MediaWiki
+ 18
+
+
MediaWiki offers sophisticated functionalities for creating diffs of wiki content. Users can compare any two revisions in the page history and highlight changes in a friendly interface that shows modifications with a word‐level granularity. Diff pages contain metadata about each compared version (when the version was created, who the author was, or which IP address an anonymous author was connected from, etc.) and a two‐column table showing the changes side by side. Changes are detected a posteriori by comparing two arbitrary versions, which are not even required to be temporally contiguous.
+The output of the MediaWiki diff engine has regularities that can be exploited to automatically build the overlapping structures of the diff and to express them in EARMARK. Let us consider a fictitious example summarized in Table + 1, where an initial text is revised three times by different authors.
+To display the differences between V1 ad V2, Mediawiki creates a page whose HTML code is as follows:
+ 19
+ For the sake of clarity, we removed all markup irrelevant to our discussion.
+
This is an HTML table of two rows, the first showing metadata (date and author of the modification), and the second showing the actual modifications. The first cell of the second row contains all the unmodified text and a del element for each inline fragment that was deleted. The second cell contains all the unmodified text and an ins element for each inline fragment that was inserted. Thus, these cells share exactly the same unmodified part(s) of the two compared versions.
+When the structure itself is modified rather than merely the text, the source code of the MediaWiki diff is slightly different. Thus, the diff between V3 and V4 (which splits a paragraph in two) is as follows:
+
+
The diff output is not complete or sophisticated, and of course, it is a completely different task to replan such an algorithm (but for a first idea of natural changes in diffing XML documents, see Di Iorio, Marchetti, Schirinzi, & Vitali, + 2009). Thus, limitations of that algorithm are inevitably shared by any EARMARK representation. Yet, this output is sufficiently rich to allow us to extract the overlapping information we need. For instance, the insertion of a nonbreaking space or a carriage return generates rows according to specific rules that can be easily detected to capture the actual change by the author.
+Figure + 5 shows the aforementioned example rebuilt in EARMARK. All versions are encoded in the same document by creating overlapping assertions over the docuverses. Metadata and RDF statements are layered on top of those assertions and create a rich knowledge‐base about the history of the documents and, in particular, about the history of each fragment.
+The wiki sample versions encoded in a single EARMARK document. [Color figure can be viewed in the online issue, which is available at
+
Due to the complexity of the example, we labeled arrows with numbers that indicate the position of each range within each markup item. Consider, for instance, Version V4: It is composed of two DIV elements, the first one containing the concatenation of “Bob was farming carrots” + “,” + “tomatoes” + “and” + “green” + “beans” + “.” and the second one containing the string “They are all tasteful.”
+Implementing a wiki content‐filtering mechanism on top of such a structure is rather simple. For instance, the removal of all the contributions of “Angelo Di Iorio” that leaves untouched all the content written (previously and subsequently) by “Silvio Peroni” and “Fabio Vitali” can be performed straightforwardly. Three steps are enough to apply such an intermediate content reversion:
+ the identification of the fragments written by “Angelo Di Iorio,” which is a straightforward SPARQL query on the embedded statements; the creation of a new version where references to those fragments are removed and references to fragments no longer in the document are correctly fixed; the translation of that document into an actual MediaWiki page through the serialization process described in Peroni and Vitali (
+ 2009).
+
+
Of course, an automatic process may generate ambiguities or even errors in the resulting content (i.e., some parts may become dangling, wrong, or unclear after removing text fragments elsewhere); grammar discrepancies also might be generated by the same approach. Linguistic and semantic problems, however, become a problem once the technical issues of managing independent, yet successive, edits are solved. What is important is that all the information about overlaps and dependencies among fragments is available in EARMARK and can easily be searched, filtered, and manipulated. Besides, foreseeing a manual intervention for checking and polishing automatically filtered content is perfectly in line with the wiki philosophy, so that the wiki community itself can wisely use the reversion tools to revise the content and adjust any intervening minor nuisances or imperfections. Such checks would still be far simpler and faster than would the manual process of partially reverting versions as we have today.
+Since we do not expect documents to be natively written in EARMARK or manually created by users, we need a way to extract EARMARK data structures from existing XML‐based resources, which is trivial when the XML is simple and clearly hierarchical and slightly more complex when the XML contains workarounds to force an intrinsically overlapping situation into a single hierarchy.
+We designed a reliable process to transform XML files into EARMARK documents that fully captures overlapping structures even when the overlaps are hidden in one the many well‐known workarounds. This approach takes as input an XML file and produces the corresponding EARMARK document in five steps: Read, Overhaul, Convert, Classify, and Organize (hence, the name ROCCO).
+Since ROCCO is not the main topic of this article, we very briefly discuss the issue of converting XML into EARMARK, explaining how each step works. The ROCCO algorithm performs five steps, described next.
+The first two steps consist of loading the XML source file and, if needed, adding information useful for further processing. In EARMARK, there is a clear distinction between the textual content of a document and the structures built on top of it: The content is stored as plain text—within docuverses—and all structures are externalized and expressed through OWL and RDF assertions.
+While OpenOffice stores all overlapping structures in the main document file, some other editors (e.g., MS Word) store overlaps in many different ways, even in a separate file. The overhaul step extracts such data and adds them to the main content document by exploiting format‐specific procedures, implemented via XSLT in most cases.
+The subsequent step consists of converting the XML source file into an early EARMARK document that expresses exactly the same information and hierarchies. No interpretation or disentanglement of workarounds is performed at this step.
+Since the input is XML, this translation can be performed directly via a generic XSLT stylesheet. It basically consists of a recursive algorithm that parses the source file and generates the corresponding instances in the EARMARK ontology. Such a translation is straightforward and not difficult.
+The “Classify” step extends the EARMARK document built so far with information about the workarounds used to encode overlaps. That information will be exploited in the subsequent steps to make those overlaps explicit.
+The basic idea is to exploit OWL reasoners to detect workarounds in an early EARMARK document D by:
+ defining an ontology O that models all the workarounds used by applications, such as milestones, stand‐off markup, etc.; these workarounds are specific to the data format used in the source document; specifying the EARMARK document D as an ABox for the ontology O; defining SWRL rules that capture the role of each element in D and check relationships between elements; running an OWL reasoner, such as Pellet, on D+O to create new OWL instances and properties that identify which workarounds are present.
+
+
The actual detection of workarounds is delegated to an external reasoner. Refining detection strategies and even adding new strategies for new formats all can be done via OWL and SPARQL. Indeed, tricky issues need to be addressed—mostly depending on the idiosyncrasies of the original formats—but no procedural code is required.
+The final step consists of building yet another EARMARK document that expresses the overlaps and metadata in an explicit way, based on the information collected by the previous steps. This phase consists of mapping operations from the native format into the EARMARK structure. Such conversion relies on the identification of metadata to classify the operations and to externalize relevant metadata in separate RDF statements.
+One of the most frequent criticisms when proposing a different approach to solving a well‐known problem in information and communication technology is that the new solution may simplify the difficulties of the specific problem, but brings with it hidden costs in terms of size of the data structure, computation efforts, or conversion restrictions that compensate the advantages. In our case, one of the anonymous reviewers of our article (Di Iorio, Peroni, & Vitali, + 2009) wondered whether a difference in file size could weigh in on the convenience of adopting EARMARK as opposed to working with the original files.
+As such, a discussion of cost functions of EARMARK versus other formats is in order. Yet, a systematic discussion of the relative costs (e.g., in byte size) of some original XML‐based data structures versus their EARMARK equivalent is an open‐ended undertaking that heavily depends on the original XML data structure and the specific features present in the document, and is badly defined anyway: While XML is a linearization format immediately expressible in actual bytes, OWL (or more precisely, RDF, the language in which OWL ontologies are expressed) is an abstract structure that allows a large number of linearization formats (including XML itself) with corresponding huge differences in the final byte counts.
+For these reasons, to provide at least an initial test of meaningful concepts, we selected two XML‐based data formats (OOXML and ODT) and, specifically, a set of documents where overlapping tricks were present (i.e., where change‐tracking was active). To bypass the size discussion, we decided not to test byte lengths (which are not meaningful and easily skewed, e.g., by reducing the string length of the element names or of the class names) but the number of nodes for XML documents and of triples for OWL documents. This comparison again is not particularly appropriate (Triples are naturally numerous in OWL ontologies, and it is customary to deal with hundreds of thousands and even millions of assertions in Semantic Web applications.) but closer to meaningfulness than is the mere byte count.
+Our comparison was carried on a small set of documents in ODT and OOXML that included change‐tracking information. As discussed in the previous sections, change‐tracking facilities generate rather complex overlaps even for basic operations on small text fragments, which in turn are expressed as a potentially huge number of standoffs and milestone markup within the XML hierarchy. The same documents were individually converted into EARMARK. We then charted how simple edits under change‐tracking affect the number of nodes in XML formats and of statements in OWL files.
+ 20
+ The full details about each version and each format also are available at
+
We created seven different versions, named after the “Seven Dwarfs” for recognizability, by applying very common edits (e.g., the insertion of few words, the deletion of some sentences, the split of a paragraph, etc.) on a small document, creating multiple overlaps. Figure + 6 shows the results of our comparison.
+A graph summarizing the results of the first experiment. [Color figure can be viewed in the online issue, which is available at
+
The overall trend is interesting and comforting: While in simple documents with no overlap the node count of XML is lower than is the assertion count of EARMARK triples, the presence of overlaps makes EARMARK and XML formats comparable. The growth of EARMARK statements is in fact very close to the growth of XML nodes when the number of overlaps increases. EARMARK is even more efficient than is XML for more complex documents.
+The measure for each format was done by counting only those nodes and statements instrumental to encode content and (overlapping) structures: We did not take into account either the presentational information for ODT and OOXML (Each file, for instance, includes a very long list of style definitions that are not relevant for the purposes of our analysis.) or namespace declarations (OOXML files, for instance, list all relevant namespaces for the Office toolkit.) or ignorable white spaces (that are only added to indent content and improve readability).
+Interestingly, EARMARK and ODT show a very similar increase in size while OOXML is much more verbose and grows faster. The content of the first version, for instance, is encoded using four nodes in ODT, 13 statements in EARMARK, and 54 nodes in OOXML; the last one contains 241 ODT nodes, 233 EARMARK statements, and 452 OOXML nodes. To return to our original inquiry, it is clear that the weight of EARMARK documents is very good compared to the other ones.
+Also note the regularity in the growth of EARMARK statements. Regardless of the actual modifications applied to the document, in fact, EARMARK adds about 40 statements for each edit. Both OOXML and ODT, on the contrary, show a more irregular “pace.” The reason for this is that EARMARK externalizes all assertions, so that all modifications (either to leaf nodes or to intermediate nodes in the original XML) are “flattened” onto the docuverses and do not depend on the complexity of the structure within which the edit took place.
+Figure + 7 shows the results of a similar comparison on a different set of documents and edits. We collected seven versions named after the days of the week and created by seven different authors when editing a very simple document. The overall trend does not change, and shows that EARMARK and ODT again have a comparable behavior, far better than that of OOXML.
+A graph summarizing the results of the second experiment. [Color figure can be viewed in the online issue, which is available at
+
In conclusion, although preliminary, this study shows clear trends of a very conservative behavior of EARMARK with respect to document size.
+Overlaps, far from being an obscure requirement for sophisticated functionalities of arcane markup languages, are a very frequent undertaking even in major data formats and in rather frequent situations. Yet, since the XML language does not allow them, consciously or not, designers of data formats have adopted a huge and entangled array of tricks, special cases, and workarounds that, although solving the actual problem of storing overlapping structures, open new and complicated ones when approaching even basic chores on documents containing them, such as queries.
+The EARMARK approach drastically reduces the efforts needed to perform such chores on overlapping structures since it does not allow the corresponding multiple trees to actually entangle and complicate the job. EARMARK is radically different from both special markup metalanguages that allow overlaps and the introduction of workarounds within the traditional tree‐oriented XML language because it treats multiple trees over the same content as first‐class citizens of the language, yet uses well‐known and standard W3C technologies and languages to perform all tasks. EARMARK documents, at the end, are OWL ontologies. Thus, any Semantic Web technology (e.g., SPARQL) can be used straightforwardly to perform operations on their content.
+Improving queries is not the only application of EARMARK. Validation is another interesting field that we are investigating. In fact, the same ontological framework can be used to prove properties concerning a document, such as validity against a schema, compliance to co‐constraint specifications, or adherence to structural patterns. Moreover, inspired by Marcoux and Rizkallah ( + 2009), in which they described an approach for defining natural‐language semantics for XML‐based languages, we also are developing an ontology‐based approach for encoding markup semantics—that is, the formal definition of meanings of markup elements, besides the syntactical structure of a markup document—within EARMARK documents.
+Digital Humanities (DH) and Digital Library (DL) projects are complex systems that require specialized programming skills. Many encoders cannot take their work to the next level by transforming their collections of structured XML texts into a web searchable and browsable database. Often teams of text encoders are able to encode their texts with a high degree of sophistication, but unless they have funds to hire a programmer, their collections far too often remain on local disk storage away from public access. <
+
Overlapping structures in XML are not symptoms of a misunderstanding of the intrinsic characteristics of a text document nor evidence of extreme scholarly requirements far beyond those needed by the most common XML‐based applications. On the contrary, overlaps have started to appear in a large number of incredibly popular applications hidden under the guise of syntactical tricks to the basic hierarchy of the XML data format. Unfortunately, syntactical tricks have the drawback that the affected structures require complicated workarounds to support even the simplest query or usage. In this article, we present Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), an approach to overlapping markup that simplifies and streamlines the management of multiple hierarchies on the same content, and provides an approach to sophisticated queries and usages over such structures without the need of ad‐hoc applications, simply by using Semantic Web tools and languages. We compare how relevant tasks (e.g., the identification of the contribution of an author in a word processor document) are of some substantial complexity when using the original data format and become more or less trivial when using EARMARK. We finally evaluate positively the memory and disk requirements of EARMARK documents in comparison to Open Office and Microsoft Word XML‐based formats.
+The overwhelming consensus among XML practitioners is that documents are trees, the hierarchy is the fundamental data structure, and violations of the hierarchy are errors or unnecessary complications. Therefore, overlapping markup has received ambivalent, almost schizoid considerations in the field of markup languages. Traditionally, overlaps were the hallmarks of bad HTML coders and nave HTML page editors, taking advantage of the unjustified benevolence in web browsers that would display basically any HTML regardless of proper nesting. At the same time, far from the awareness of the general public, overlaps have been a fringe, almost esoteric, discipline of scholars in the humanities, competently used for arcane specifications of linguistic annotations and literary analysis.
+Although the first type of overlap was judged with scorn and the second with awe, they both fundamentally represent a situation that is more common than was thought, and the scholars were only more aware, and not more justified, about the need to represent overlaps.
+Generally, overlap is needed when multiple independent items refer to the same segment, either when considering textual markup documents or multimedia structures (Salembier & Benitez, + 2007). Regarding documents with markup, we need overlap whenever multiple markup elements need to be applied over the same content, and these elements happen to be independent of each other. In some (rather frequent) situations, this independence means that the content referred to by some elements is partially, but not completely, the same as the content referred to by other elements.
+This situation is more frequent than it may appear: Not only do bad HTML code and arcane linguistic annotations use overlap, but many more mainstream and mundane examples exist. For instance, change tracking in an office document is often at odds with the underlying structure of the text, microformats (Allsopp, + 2007) and Resource Description Framework–in–attributes (RDFa; Adida, Birbeck, McCarron, & Pemberton, + 2008) annotations may need to refer to concepts that span across multiple XML elements, complex data structures (e.g., biological data) force graphs into trees and hide multiple parentage as internal references, and so on.
+Differently from SGML, which is able to handle some overlapping scenarios through the CONCUR notation (Goldfarb, + 1990), XML grammatically imposes and requires a strict hierarchy of containment generating a single mathematical tree of the document where no overlap is allowed. This requirement has been turned into an intrinsic characteristic of the documents XML was meant to represent rather than a syntactical and conceptual constraint into which these documents need to fit. Thus, whenever authors needed to cope with independent markup elements, they managed either by navely ignoring the hierarchical limitation (and therefore creating invalid documents) or by creating careful workarounds within the syntactical constraint, or even by inventing completely new markup languages that allow some types of overlap. But while new multihierarchical markup languages such as TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003) and LMNL (Tennison & Piez, + 2002) have a small number of adepts and applications, and while bad HTML coders and bad HTML page editors are disappearing from the market, the careful workarounds within the XML syntax (TEI Consortium, + 2005), such as segmentation, milestones, or standoff markup, are to this day frequently used and ubiquitous.
+All workarounds share the same approach of hiding structural information about a secondary hierarchy under the guise of something else: split individual elements, empty boundary elements, indirect references, and so on. The result is that the secondary structural information is hidden or its importance is lessened to not break or obfuscate the main hierarchy expressed in the visible XML structure. But this comes at a price: Structures specified through workarounds are more difficult to find, identify, and act upon than the are structures in the main XML hierarchy. Thus, trivial searches that should amount to a short XPath in a more direct situation end up being multiple‐lines long, pretty basic visualizations require incredibly complex XSLT stylesheets, specific choices of the main markup hierarchy actually prevent some features of the secondary markup to even exist, and so on. So, although workarounds exist and can be used, hierarchies expressed through them are “second‐class citizens” that cannot fully exploit the sophisticated tools that the XML language provides.
+In this article, we show how Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), our proposal for managing overlapping markup, does not generate first‐ and second‐class hierarchies, and allows existing, sophisticated tools to be used on all markup—even in the presence of overlaps. Rather than creating a completely new language requiring completely new tools and competencies, EARMARK uses Semantic Web technologies and Semantic Web tools to obtain many of the results obtainable with usual XML tools.
+EARMARK defines markup vocabularies by means of OWL ontologies (W3C OWL Working Group, + 2009). Since each individual markup item is an independent assertion over some content or some other assertions, overlaps of content are not a problem, nor are all the issues connected to physical embedding and containments, such as contiguity and document order. Furthermore, by using standard Semantic Web technologies, fairly sophisticated functionalities can be provided over EARMARK documents.
+Through EARMARK, operations that were previously very hard or impossible exactly because of the interferences of the multiple hierarchies or of the workarounds they employed now become fundamentally trivial since no syntactical tricks are employed and the different hierarchies do not interfere with each other. Thus, for instance, identifying the individual contributions in a multi‐authored MS Word or Open Office document is quite hard on their original XML formats, and becomes trivial when the same documents are converted into EARMARK.
+This article is an extended version of previous works on EARMARK (Di Iorio, Peroni, & Vitali,
+ 2010; Peroni & Vitali,
+ 2009). In those works, we focused on identifying workarounds for overlapping data existing in real XML documents and translating them into EARMARK assertions. We also sketched the EARMARK ontology and presented a simple implementation of EARMARK‐aware tools. This article follows and extends them and also provides some novel contributions:
+ The systematic analysis of the EARMARK model, with particular attention to data typing and overlapping structures The discussion of further applications for the ontological EARMARK approach. In particular, we show how EARMARK can be used to improve the content filtering and reversions mechanisms of wikis. The brief description of a process, called ROCCO, for generating EARMARK documents from existing XML documents (even ones that use workarounds for overlapping structures) An evaluation of EARMARK efficiency when dealing with multiple hierarchies in comparison with the XML structures used by popular XML‐based formats such as Office Open and MS Word.
+
+
The article is structured as follows: First, we provide a brief overview of existing approaches to overlap using workarounds in XML or ad hoc markup metalanguages, and then give a few examples of situations where overlaps are used today and sometimes in rather mainstream situations. Next, we present the EARMARK model and its rules. Then, we provide some use cases that are meant to demonstrate the superiority of the EARMARK approach to a traditional XML format, especially when overlaps come into question, and show the generation of EARMARK documents, converting legacy documents. An initial evaluation of the efficiency of EARMARK compared to popular XML based data formats such as Open Document (ODT) and Office Open XML (OOXML) is presented, followed by our onclusions.
+The need for multiple overlapping structures over documents using markup syntaxes such as XML and SGML is an age‐old issue, and a large amount of literature exists on the techniques, languages, and tools that allow users to create multiple entangled hierarchies over the same content. A good review can be found in DeRose ( + 2004).
+Some research has proposed using plain hierarchical markup (i.e., XML) and employing specially tailored elements or attributes to express the semantics of overlapping in an implicit way. The TEI Guidelines (TEI Consortium,
+ 2005) presented a number of different techniques that use SGML/XML constructs to force multiple hierarchies into a single one, including:
+ milestones (i.e., verlapping structures are expressed through empty elements to mark the boundaries of the “content”), fragmentation (i.e., overlapping structures are split into individual, nonoverlapping elements that may even be linked through id–idref pairs), and standoff markup (i.e., overlapping structures are placed elsewhere and indirectly refer to their would‐be locations through pointers, locators, and/or id–idref pairs).
+
+
Given the large number of techniques to deal with overlapping structures in XML, in Marinelli, Vitali, and Zacchiroli ( + 2008), we presented a number of algorithms to convert XML documents with overlapping structures from and to the most common approaches, as well as a prototype implementation.
+Riggs ( + 2002) introduced a slightly different technique for fragmentation within XML structures. In this proposal, floating elements (i.e., those elements that do not fall in a proper or meaningful hierarchical order) are created using the name of the element followed by an index referring to its semantically related parent element. For example, the floating element <name.person[2]>John</name.person [2] means that <name>John</name> is semantically a child of the second occurrence of the element person, even though the floating element is not structurally contained by its logical parent.
+Other research even has proposed to get rid of the theory of trees at the base of XML/SGML altogether and use different underlying models and newly invented XML‐like languages that allow the expression of overlaps through some kind of syntactical flourishing. For instance, a general ordered‐descendant directed acyclic graph (GODDAG; Sperberg‐McQueen & Huitfeldt, + 2004) is a family of graph‐theoretical data structures to handle overlapping markup. A GODDAG's nodes represent markup elements and text. Arcs are used to explicitly represent containment and father–child relations. Since multiple arcs can be directed to the same node, overlapping structures can be straightforwardly represented in GODDAG. Full GODDAGs cannot be linearized in any form using embedded markup, but restricted GODDAGs, a subset thereof, can be and have been linearized into TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003), a multihierarchical markup language that also allows full GODDAGs through appropriate nonembedding workarounds such as standoff markup.
+LMNL (Tennison & Piez, + 2002) is a general data model based on the idea of layered text fragments and ranges, where multiple types of overlap can be modeled using concepts drawn from the mathematical theory of intervals. Multiple serializations of LMNL exist, such as CLIX and LMNL‐syntax.
+XConcur (Schonefeld & Witt, + 2006) is a similar solution based on the representation of multiple hierarchies within the same document through layers. Strictly related to its predecessor CONCUR as it was included in the SGML, XConcur was developed in conjunction with the validation language XConcur‐CL to handle relationships and constraints between multiple hierarchies.
+The variant graph approach (Schmidt & Colomb, + 2009) also is based on graph theory. Developed to deal with textual variations that generate multiple versions of the same document with multiple overlapping hierarchies, this theory proposes a new data model to represent literary documents and a graph linearization (based on lists) that scales well even with a large number of versions. The same authors recently presented an extension of their theory that also allows users to merge multiple variants into one document (Schmidt, + 2009). In Portier and Calabretto ( + 2009), a detailed survey about overlapping approaches was presented, and also discussed the MultiX2 data model, which uses W3C standard languages such as XInclude to link and fetch text fragments within overlapping structures, and a prototype editor for the creation of multistructured documents.
+Tummarello, Morbidoni, and Pierazzo ( + 2005) proposed using RDF as a standoff notation for overlapping structures of XML documents. Since this proposal has many affinities with the one we are presenting in this article, we later discuss its characteristics and compare it with ours.
+Overlapping structures have been considered often as appropriate only in highly specific contexts and basically for scholars: The solutions that have been proposed in the literature were complex since they were considered grounded in the intrinsic complexity of the topics themselves. Yet, overlapping structures can be found in many more fields than these, and even mainstream applications generate and use markup with overlapping structures. While the complexity of overlapping is hidden to the final user, applications that consume such data may very well find it rather difficult to handle such information. We next discuss three very different contexts where overlapping already exists and fairly relevant information is encoded in multiple independent structures, leaving to special code the task of managing the complexity.
+Word processors such as Microsoft Word and Open Office provide users with powerful tools for tracking changes, allowing each individual modification by individual authors to be identified, highlighted, and acted upon (e.g., by accepting or discarding them). The intuitiveness of the relevant interfaces actually hides the complexity of the data format and of the algorithms necessary to handle such information.
+For instance, the standard ODT format (JTC1/SC34 WG 6, + 2006) used by Open Office, when saving change‐tracking information, relies on two specific constructs for insertions and deletions that may overlap with the structural markup. Adding a few words within a paragraph is not in itself complex, as it does not require the breaking of the fundamental structural hierarchy; conversely, changes that affect the structure itself (e.g., the split of one paragraph in two by the insertion of a return character, or the joining of two paragraphs by the elimination of the intermediate return character) require that annotations are associated to the end of a paragraph and the beginning of the next, in an unavoidably overlapping pattern. ODT uses milestones and standoff markup for insertions and deletions, respectively, and also relies on standoff markup for annotations about the authorship and date of the change.
+For instance, the insertion of a return character and a few characters in a paragraph creates a structure as follows:
+
+
The empty elements <text:change‐start/> and <text:change‐end/> are milestones marking the beginning and the end, respectively, of the range that constituted the insertion while the element <text:insertion>, before the beginning of the document content, is standoff markup for the metadata about the change (author and date information).
+Similarly, a deletion creates a structure as follows:
+
+
The element <text:change/> represents a milestone of the location where the deletion took place in the content, and the corresponding standoff markup annotation <text:deletion> contains not only the metadata about the change but also the text that was deleted.
+The OOXML format (JTC1/SC34 WG 4, + 2008) (the XML‐based format used by Microsoft Office 2007 and standardized by ISO in 2008), on the other hand, uses a form of segmentation to store change‐tracking information across all previous elements involved.
+
+
This heavily simplified version of an OOXML document shows two separate changes: (a) the insertion of a return character and (b) the insertion of a word. These modifications are not considered as a single change; therefore, the segments are not connected to each other but simply created as needed to fit the underlying structure.
+In fact, change tracking in OOXML is a fairly complex proposition. Although providing more complete coverage of special cases and situations than does ODT, dealing with its intricacies is not for the casual programmer. Even a simple XSLT stylesheet to show inserted text in a different color and hide deleted text may run several hundred lines of code.
+ 1
+
+
Microformats (Allsopp, + 2007) add semantic markup to web documents by using common structures of the HTML language itself—in particular, the class attribute.
+The HTML code is annotated using microformats to provide new semantic, machine‐processable assertions. In the following example, a plain HTML table is enriched with metadata about events
+ 2
+ HCalendar,
+ HCard,
+
+
The table was enriched by additional data declaring it to be an event (a conference), and data about the event itself (URL, summary, location) and about four relevant individuals (with their names and roles within the conference) were associated where necessary to the actual content of the table.
+So far, so good, and no overlap to speak about. Things change dramatically, though, when the overall structure of the main hierarchy (the HTML table) is at odds with the intrinsic hierarchy of the microformat data, such as if the people are organized in columns rather than rows. For instance:
+
+
Unfortunately, vcards are a hierarchy themselves, and if the hierarchy of vcards is organized differently from the hierarchy of the HTML table, as in the latter case, it is just impossible to define the four vcards for the four people organizing the conference. Thus, in plain HTML, the choice of one of two possible presentation models for the main hierarchy of content makes trivial or completely impossible the existence of the second hierarchy.
+A possible and partial solution to express vcard hierarchies in the latter example is RDFa (Adida et al., + 2008), a W3C recommendation. It describes a mechanism to embed RDF statements into HTML documents by using some HTML attributes (href, rel, rev, content) in combination with other ad hoc attributes (property, about, typeof) proposed in the recommendation itself.
+
+
Since all attributes live in the context of elements, the price to pay is that to assert everything we want to assert, we often need to add some structurally unnecessary elements to the current markup hierarchy of a document, needed only to add the RDF statements (e.g., the span elements emphasized earlier). Even if that does not represent a significant problem for strict Semantic Web theorists, document architects and markup experts see this as a kludge and an inelegant compromise.
+The strength of wikis lies in their allowing users to modify content at any time. The mechanisms of change‐tracking and rollback that are characteristics of all wikis, in fact, promote users' contributions and make “malicious attacks” pointless in the long run since previous versions can be easily restored.
+A number of tools exist that automatically discover “wiki vandalisms” and provide users with powerful interfaces to surf changes, identify differences between subsequent versions, and revert content. For instance, Huggle
+ 4
+
+
Even client‐side tools—meant to be installed as browser extensions or bookmarklets—exist to extend the rollback mechanisms of Wikipedia, giving users more flexibility and control over (vandalistic) changes. For instance, Lupin
+ 5
+
+
+
These tools are successful in highlighting vandalism and in identifying versions created by malicious users. However, although it is possible to revert the page to any previous version, all changes (even acceptable ones) that were subsequent to the malicious version cannot be automatically inherited by the restored page.
+For instance consider Versions V1, V2, and V3 of a wiki page, where Version V1 contains a baseline (i.e., acceptable) content, and Version V2 is identified as a partial vandalism and is agreed to be removed, but Version V3 contains (possibly, in a completely different section than the target of the malicious attack) relevant and useful content that was added before the vandalistic Version V2 was declared as such. The task of removing the modifications of Version V2 while maintaining (whatever is possible of) Version V3 is a difficult, error‐prone, and time‐consuming task if done manually, yet there is no tool we are aware of that automatically filters contributions from multiple versions and merges them into a new one (or, equivalently, removes only selected intermediate versions).
+However, it is possible to theoretically characterize the interdependencies between subsequent changes to a document. In fact, literature has existed for a long time on exactly these themes (e.g., Durand, + 1994, + 2008). Although a detailed discussion of abstract models of interconnected changes is out of scope for this article (Details and authoritative references can be found in the aforementioned works.), what is relevant in this discussion is that they happen to assume a hierarchical form that is frequently at odds with the hierarchical structure of the content of the document, and as such, most issues derive from the data structures in which content is stored and from the model for manipulating these structures. For instance, the fact that in the wiki perspective each version is an independent unit that shares no content (even unchanged content) with the other versions prevents considering multiple versions as overlapping structures coexisting on the same document. If we were able to make these hierarchies explicit, we would be able to create models and tools to manipulate these documents in a more powerful way and to exploit the existing interconnections between the overlapping hierarchies.
+The presence of hidden overlapping structures—transparent to users, but very difficult to handle by applications—is the common denominator for the scenarios described in the previous section. More than the overlap itself, which cannot be ignored because it does exist and carries important meanings, the problem we face lies in the way applications store such overlapping structures. In the XML world, in fact, the only way to do so is through the use of (complex) workarounds that force the multiple hierarchies into one hierarchy of an XML document. That makes it very tricky to perform sophisticated analysis and searches.
+This section discusses a different approach to metamarkup, EARMARK (Di Iorio, Peroni, & Vitali, + 2009; Di Iorio et al., + 2010; Peroni & Vitali, + 2009) based on ontologies and Semantic Web technologies. The basic idea is to model EARMARK documents as collections of addressable text fragments, and to associate such text content with OWL assertions that describe structural features as well as semantic properties of (parts of) that content. As a result, EARMARK allows not only documents with single hierarchies (as with XML) but also multiple overlapping hierarchies where the textual content within the markup items belongs to some hierarchies, but not to others. Moreover, EARMARK makes it possible to add semantic annotations to the content though assertions that may overlap with existing ones.
+One of the advantages of using EARMARK is the capability to access and query documents by using well‐known and widely supported tools for the Semantic Web. In fact, EARMARK assertions are simply RDF assertions while EARMARK documents are modeled through OWL ontologies. The consequence is that query languages (e.g., SPARQL; Garlik & Seaborne,
+ 2010) and actual existing tools such as Jena
+ 7
+
+
+
In the rest of this section, we give a brief overview of the EARMARK model and then describe how EARMARK can be used to deal with the issues presented earlier. The model itself is defined through an OWL document,
+ 9
+
+
A UML‐like representation of the EARMARK ontology. [Color figure can be viewed in the online issue, which is available at
+
The ghost classes describe three disjoint base concepts—docuverses, ranges, and markup items—through three different and disjoint OWL classes.
+ 10
+ All our OWL samples are presented using the Manchester Syntax (Horridge & Patel‐Schneider,
+ 2009), which is one of the standard linearization syntaxes of OWL. The prefixes rdfs and xsd refer to RDF Schema and XML Schema namespaces, respectively, while the empty prefix refers to the EARMARK ontology URI plus “#.” Moreover, we use the prefix c to indicate entities taken from an imported ontology made for the SWAN project (Ciccarese et al.,
+ 2008); available at
+
The textual content of an EARMARK document is conceptually separated from its annotations, and is referred to through the Docuverse class.
+ 11
+ This class (and its name) is based on the concept introduced by Ted Nelson (
+ 1980) in his Xanadu Project to refer to the collection of text fragments that can be interconnected to each other and transcluded into new documents.
+
Any individual of the Docuverse class—commonly called a docuverse (lowercase to distinguish it from the class)—specifies its actual content with the property hasContent.
+We then define the class Range for any text lying between two locations of a docuverse. A range (i.e., an individual of the class Range) is defined by a starting and an ending location (any literal) of a specific docuverse through the properties begins, ends, and refersTo, respectively.
+
+
There is no restriction on locations used for the begins and ends properties. That is very useful because it allows us to define ranges that follow or reverse the text order of the docuverse to which they refer. For instance, the string “desserts” can be considered both in document order, with the begins location lower than the ends location, or in the opposite order, forming “stressed.”
+ 12
+
+
The class MarkupItem is the superclass defining artifacts to be interpreted as markup (e.g., elements and attributes).
+
+
A markupitem individual is a collection (c:Set, c:Bag, or c:List, where the latter is a subclass of the second one, and all of them are subclasses of c:Collection) of individuals belonging to the classes MarkupItem and Range. Through these collections, it is possible to define a markup item as a set, a bag, or a list of other markup items, using the properties element (for sets) and item and itemContent (for bags and lists). Thus, it becomes possible to define elements containing nested elements or text, or attributes containing values, as well as overlapping and complex structures. Note also that handling collections directly in OWL allows us to reason about content models for markup items, which would not be possible if we had used the corresponding constructs in RDF.
+ 13
+
+
A markupitem also might have a name, specified in the functional property hasGeneralIdentifier (recalling the SGML term to refer to the name of elements; Goldfarb, + 1990), and a namespace, specified using the functional property hasNamespace. Note that we can have anonymous markup items—as is possible in LMNL (Tennison & Piez, + 2002) and in GODDAG (Sperberg‐McQueen & Huitfeldt, + 2004)—by simply asserting that the item belongs to the class of all those markupitems that do not have a general identifier (i.e., hasGeneralIdentifier exactly 0).
+The ghost classes discussed so far give us an abstract picture of the EARMARK framework. We need to specialize our model, defining a concrete description of our classes. These new shell subclasses apply specific restrictions to the ghost classes.
+First, the class Docuverse is restricted to be either a StringDocuverse (i.e., the content is specified by a string) or a URIDocuverse (i.e., the actual content is located at the URI specified).
+
+
Depending on particular scenarios or on the kind of docuverse we are dealing with (plain‐text, XML, LaTeX, a picture, etc.), we need to use different kinds of ranges. Therefore, the class Range has three different subclasses:
+ PointerRange defines a range by counting characters. In that case, the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream, remembering that the value 0 refers to the location immediately before the first character, the value 1 refers to the location after the first character and before the second one, and so on. By using the hasKey OWL property, we also assert that two pointer ranges having equal docuverse and begin and end locations are the same range. XPathRange defines a range considering the whole docuverse or its particular context specifiable through an XPath expression (Berglund et al.,
+ 2007) as value of the property hasXPathContext. Note that by using these ranges, we implicitly admit that the docuverse it refers to must be an XML structure. Moreover, the properties' begins and ends have to be applied on the string value obtained by juxtaposing all the text nodes identified by the XPath. By using the hasKey OWL property, we also assert that two xpath ranges having equal docuverse, XPath context, and begin and end locations are the same range. XPathPointerRange is an XPathRange in which the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream as described for the class PointerRange.
+
+
+
MarkupItem is specialized in three disjointed subclasses—Element, Attribute, and Comment—that allow a more precise characterization of markup items.
+
+
The presence of overlap in EARMARK is worth discussing in more detail. Different types of overlap exist, according to the subset of items involved, and different strategies are needed to detect them. In particular, there is a clear distinction between overlapping ranges and overlapping markup items.
+By definition, overlapping ranges are two ranges that refer to the same docuverse, so that at least one of the locations of the first range is contained in the interval described by the locations of the second range (excluding its terminal points). Totally overlapping ranges have the locations of the first range completely contained in the interval of the second range, or vice versa, while partially overlapping ranges have either exactly one location inside the interval and the other outside or identical terminal points in reversed roles.
+Thus, if we consider the following excerpt:
+
+
we can infer, through a reasoner such as Pellet, that these two ranges overlap by using the following rules:
+
+
where P is one of:
+ lessThan(b1,e1) ˆ greaterThan(b2,b1) ˆ lessThan(b2,e1) lessThan(b1,e1) ˆ greaterThan(e2,b1) ˆ lessThan(e2,e1) lessThan(e1,b1) ˆ greaterThan(b2,e1) ˆ lessThan(b2,b1) lessThan(e1,b1) ˆ greaterThan(e2,e1) ˆ lessThan(e2,b1).
+
+
The case of overlapping markup items is slightly more complicated. We define that two markup items A and B overlap when at least one of the following sentences holds:
+ [Overlap by range]: A contains a range that overlaps with another range contained by B. [Overlap by content hierarchy]: A and B contain at least a range in common. [Overlap by markup hierarchy]: A and B contain at least a markup item in common.
+
+
The three possible scenarios for such item overlap are summarized in Figure
+ 2.
+ 14
+ The EARMARK documents describing these three overlapping scenarios and all the other ones presented in the following sections are available at
+
Three EARMARK examples of overlapping between elements p. [Color figure can be viewed in the online issue, which is available at
+
The EARMARK ontology, in fact, is completed by another ontology
+ 15
+
+
If we ignore for a moment the semantic implications of using EARMARK and concentrate on its syntactical aspects only, it is easy to observe that EARMARK is nothing but yet another standoff notation, where the markup specifications point to, rather than contain, the relevant substructure and text fragments.
+Standoff notations, also known in literature as out‐of‐line notations (TEI Consortium,
+ 2005), are hardly new, but never really caught on for a number of reasons, most having to do with their perceived fragility under the circumstances of desynchronized modification to the text. In Georg, Schonefeld, Trippel, and Witt (
+ 2010) and Bański (
+ 2010), we can find a pair of recent and substantially complete analyses of their merits and demerits. In particular, according to Georg, Schonefeld, Trippel, and Witt (
+ 2010), “standoff annotation has … quite a few disadvantages:
+ very difficult to read for humans The information, although included, is difficult to access using generic methods. Limited software support as standard parsing or editing software cannot be employed. Standard document grammars can be used only for the level which contains both markup and textual data. New layers require a separate interpretation. Layers, although separate, often depend on each other.”
+ 16
+ To individually address the issues, we edited the original bullets into a numbered list.
+
+
And yet, although EARMARK is in practice a standoff notation, it provides a number of workarounds to most of the aforementioned issues.
+First, since EARMARK is based on OWL and can be linearized in any of the large number of OWL caricaturization syntaxes, it follows that (a) readability, (b) access, and (c) software support for it are exactly those existing for well‐known, widespread, and important W3C standards such as RDF and OWL. Being able to employ common RDF and OWL tools such as Jena and SPARQL for EARMARK documents was in fact a major motivation for it.
+Issue 4 should be examined beyond the mere validation against document grammars and toward a general evaluation of the level of compliancy of the markup to some formally specified expectations. EARMARK documents, while being subject to no document grammar in the stricter XML sense, allow the specification of any number of constraints, expressed either directly in OWL or SWRL (Horrocks et al., + 2004), or even in SPARQL, that trigger or generate validity evaluations. In Di Iorio, Peroni, and Vitali (in press), we tried to show that a large number of requirements, from hierarchical well‐formedness in the XML sense, to validation requirements in terms of XML DTDs, to adherence to design patterns, can be expressed satisfactorily using these technologies.
+Issue 5 regards the difficulty of standoff notations to provide interlayer analysis on XML structures: Separate interpretation of markup layers is easy, but identification and validation of overlapping situations are more complex: Standoff markup is mainly composed of pointers to content and does not have any direct way to determine overlap locations without some kind of pointer arithmetics to compute them. Validation of contexts allowing overlaps as describable using rabbit/duck grammars (Sperberg‐McQueen, + 2006) also is not trivial. In this regard, EARMARK provides yet again a solution that does not require special tools: Although OWL does not allow direct pointer arithmetics, SWRL on the contrary does, as shown earlier where we described a batch of (SWRL‐implementable) rules that do, in fact, determine overlapping locations on EARMARK documents with good efficiency.
+Finally, Issue 6 refers to the fact that evolution of separate markup annotation layers needs to synchronously take place, lest one of them becomes misaligned with the new state of the document. This is, in summary, the fragility of pointers, which can be considered the fundamental weakness of standoff, as well as of any notation that has markup separate from its content: If a modification occurs to the underlying (probably text‐based) source, all standoff pointers that could not be updated at the same time of the change become outdated and possibly wrong. All standoff notations fall prey to this weakness, and there is no way to completely get rid of it.
+What is possible is to identify exactly the conditions under which such weakness acts, and see if there is a way to reduce the mere frequency of such events. In fact, for a standoff pointer to become outdated, several conditions must take place at the same time:
+ The standoff notation must be used as a storage format, rather than just as a processing format; the source document must make sense even without the additional standoff markup (i.e., the standoff notation contains no information that is necessary for at least some types of document modifications); the source document must be editable (and, in fact, must be edited) on its own; the standoff pointers must rely on positions that change when the source is edited (e.g., character‐based locations); editing must be done in contexts and with tools that cannot or do not update the standoff pointers; and there must be no computable way to determine the modifications of the document (e.g., via a diff between the old and new versions).
+
+
Of course, no standoff notation can rule out that these conditions occur on their documents, but note that all six of them must occur for standoff pointers to become outdated. EARMARK is not safe from these occurrences either, but at least for the use cases here described, one or more of these conditions simply do not apply: EARMARK is mostly used as a processing format, with no need to save it on disk (Conversion from the source formats such as MS Word is described later and does not require special storage.), the data format described is either in a very specific format (e.g., MS Word or ODT) that in fact already does handle internally its data changes and requires the overlapping data exactly for this purpose, or is in fact the result of a diff action on successive versions of a document (as in the case of the wiki pages). Finally, EARMARK allows references to relatively stable fragment ids of the documents (by using XPath ranges without specifying explicitly begin and end locations) rather than the extremely fragile character locations, further reducing the chances of outdated pointers. For this reason, without being able to completely rule out the possibility of standoff pointers going wrong, we tend to consider it as a significantly little risk, at least for the use case described here.
+EARMARK is strongly based on OWL 2 DL (W3C OWL Working Group, + 2009) to express multiple markup layers with possible overlapping ranges over the same content. OWL 2 DL is not the only possible choice for expressing standoff notations via Semantic Web technologies. In fact, RDF is another valid and effective model for dealing with the same issue, as shown in Tummarello et al. ( + 2005), by means of the open‐source application programming interface (API) RDF Textual Encoding Framework (RDFTef). This API was created to demonstrate a plausible way for handling overlapping markup within documents and identifying textual content of a document as a set of independent RDF resources that can be linked mutually and with other parent resources.
+Besides giving the possibility to define multiple structural markup hierarchies over the same text content, the use of RDF as the language for encoding markup allows to specify semantic data on textual content as well. But the real main advantage in using RDF is the possibility of using particular built‐in resources appositely defined in the RDF syntax specification (Beckett, + 2004) for describing and dealing with different kinds of containers, either ordered (rdf:Seq) or unordered (rdf:Bag). Thus, RDF resources can be used to represent every printable element in the text—words, punctuation, characters, typographical symbols, and so on—while RDF containers also can be used to combine such fragments and containers.
+Although RDF is not sufficient to define a formal vocabulary for structural markup, does a given resource represent an element, an attribute, a comment, or a text node? In which way is a resource of a certain type related to others? The specification of an RDFS (Brickley & Guha,
+ 2004) or of an OWL layer can successfully address these issues. Hybrid solutions obtained by mixing different models, even when they are built one upon another, may seem elegant, but not necessarily the best choice. In fact, there exist well‐known interoperability limits between OWL 2 DL and RDF that prevent the correct use of Semantic Web tools and technologies. In particular:
+ Any markup document made using RDF containers (e.g., to describe what markup items contain and in which order) and OWL ontologies (e.g., to define classes of markup entities and their semantics) results in a set of axioms that end up outside of OWL DL and well within OWL Full, which limits the applicability of the most frequently used Semantic Web tools that are usually built upon the (computationally tractable) description logic underlying OWL 2 DL. The individual analysis of each language may be not applicable when we have to check particular properties that lie between RDF and OWL layers. For example, verifying the validity of a markup document against a particular schema, which is one of the most common activities with markup, needs to be made to work with both markup item structures (that would be defined in RDF) and logical constraints about classes of markup items (e.g., elements only, attributes only, the element “p,” all the element of a particular namespace, etc., all of them definable in OWL).
+
+
Being able to express everything we need directly in OWL quite straightforwardly addresses both issues. The well‐known absence of containers and sequences in OWL can be overcome by modeling classes in specific ways using specific design patterns such as those in Ciccarese et al. ( + 2008) and in Drummond et al. ( + 2006).
+There are multiple applications for the EARMARK approach. The most interesting for this article is its capability of dealing with overlapping structures in an elegant and straightforward manner. Under EARMARK, such structures do not need to be specified through complex workarounds as with XML, but they are explicit and can be easily described and accessed. Sophisticated searches and content manipulations become very simple when using this ontological model.
+The goal of this section is to demonstrate the soundness and applicability of EARMARK by discussing how the use cases presented earlier are addressed. Note that throughout the section we investigate multiple EARMARK data structures and documents, focusing on the feasibility and potentiality of such an ontological representation.
+The discussion about change tracking in office document formats showed that both ODT (OpenOffice format) and OOXML (Microsoft Word format) use complex data structures to store overlaps generated by change‐tracking functionalities. These structures make it very difficult to search and manipulate the content when using XML languages and tools. Even very simple edits generate a rather tangled set of overlapping elements.
+Let us recall the example mentioned earlier where the user “John Smith” splits a single paragraph into two. The ODT representation is:
+
+
The OOXML representation (shown earlier) is even more complex. In fact, these formats exploit in large scale (tangled) fragmentation (OOXML) or milestones and stand‐off markup (ODT) to deal with overlaps.
+EARMARK, on the other hand, stores overlapping data in a direct and streamlined manner that does not require tools to rebuild information from the twists of a tree‐based XML structure. The information already is available and expressed through consistent RDF and OWL statements. Figure + 3 graphically shows the corresponding EARMARK document.
+Encoding in EARMARK the ODT change‐tracking example. [Color figure can be viewed in the online issue, which is available at
+
The original paragraph content and the new string “also” are now encoded as two docuverses over which the ranges r1, r2, and r3 are defined. The original paragraph is then composed of the (content of) ranges r1 and r2 while the paragraphs resulting after the (text and carriage return) insertion now comprise range r1 and ranges r2 and r3, respectively. Metadata about the author and the modification date are encoded as further RDF statements.
+
+
The advantages of streamlining overlaps becomes apparent if we consider tasks a little beyond the mere display. For instance, the query for “the textual content of all paragraphs inserted by John Smith” ends up rather entangled if we used XPath on the ODT structure. The process for finding that textual content needs to browse the current version of the document and look for all the text:change‐start/text:change‐end pairs that refer to an insertion made by John Smith involving the creation of a new paragraph (i.e., text:change‐start is in a first paragraph while its pair, text:change‐end, is in the following one) that are either currently present in the document body or hidden behind a subsequent deletion made by someone else. Once the paragraphs are identified, we need to retrieve the content that originally was contained there (i.e., the text fragments that still are within those boundaries or that may have been deleted in subsequent versions). The following XPath represenst an implementation of this process:
+
+
The XML structure of an MS Word file, using segmentation rather than milestones, does simplify the query a bit, but still presents some radical complexities. The process starts by choosing all those w:p elements that were inserted by John Smith as well as all their previous and contiguous w:p elements that were deleted before or inserted after the first ones. In OOXML, each sequence of contiguous w:p elements implicitly represents one paragraph. Therefore, we can now take all the text fragments contained in each w:p sequence that were inserted before or deleted after the paragraph defined by the sequence itself. The following is the resulting XPath for an OOXML document.
+
+
The complexity of both XPath queries is due to the intrinsic complexity of the data structure on which the query has to work. Although the interface of OpenOffice or MS Word may provide tools to directly deal with these queries using specific strategies on the internal data structures, applications working directly on the XML structure have very little help in disentangling the mess of the data formats.
+On the other hand, since EARMARK documents are actually OWL files, it is possible to access and query them with plain Semantic Web tools. Powerful searches then can be performed without using niche‐specific tools or complex and long XPath expressions simply with mainstream technologies such as SPARQL (Garlik & Seaborne, + 2010).
+The corresponding SPARQL query for (“the textual content of all paragraphs inserted by John Smith”) therefore can be written as follows:
+
+
But EARMARK is useful for even more than querying: EARMARK also decreases the costs, in terms of efforts and lines of code, for manipulating documents.
+Let us consider the task of generating an intermediate version (i.e., neither the first nor the last one of a version chain) from a document that includes change‐tracking information about the whole document history.
+The process of rebuilding these versions by working on the XML structure without specific APIs is both complex and inefficient. For example, a basic XSLT that returns an XML document defining the desired version requires to at least:
+ define templates for all the elements actively involved in the change tracking (e.g., for ODT, text:changed‐region, text:change‐start, text:change‐end, and text:change and similarly for OOXML) to understand, by looking at their creation date, whether they must be considered or ignored when building the requested version. In particular, we must exclude insertions following and deletions preceding the version we are building; define templates for paragraphs to handle cases where the paragraph is the result of an insertion or a deletion of other paragraphs to identify whether it should be considered for the result and, in such case, finding out its real text content and remembering that in the following versions, such content may have spread out among other paragraphs; define templates for handling insertions/deletions for structures such as images, sections, lists, and tables; and define an identity template for the other elements to visit the entire document.
+
+
Even the most basic and incomplete implementation of such XSLT requires hundreds of lines of complex and convoluted code and a large number of ad hoc decisions based on the specificities of whether we start from ODT or OOXML. Note also that a Java‐based implementation (or in any other procedural language) of the same process would be equally or even more complex.
+The same result can be achieved on EARMARK documents with a few lines of Java code:
+
+
This approach uses the EARMARK Java API
+ 17
+
+
This method heavily uses Semantic Web technologies on the structures provided by EARMARK whose characteristics are always explicit and clear. In fact, since all versions coexist within the EARMARK document and each version can be encoded explicitly as a tree within the overall graph, this operation is straightforward and fast.
+EARMARK also can be exploited to improve semantic annotations. As noted earlier, there are in fact strong limitations in the same process of annotating web documents with semantic structures that overlap the structural ones. The same example—of vcards that cannot be created on the top of tables organized per rows—will be used in this section.
+We solve this by converting the web document with annotations into an EARMARK document, allowing both semantic and structural annotations to coexist. Through EARMARK, we can explicitly express both markup structures and vcard assertions. Figure + 4 shows how the vcard example can be modeled (Once again, we show a graphical representation for the sake of clarity.)
+The abstract model of the EARMARK document solving the microformats issue. [Color figure can be viewed in the online issue, which is available at
+
The textual content of the original table cells is now encoded in two different docuverses: one for the header (with roles) and one for the body (with names of committee members). Ranges r1, r2, …, r8 are then created to distinguish each role and name. Two independent and coexisting hierarchies are then built on top of the same set of ranges: the HTML table that includes one cell for each range (in blue) and the vcards about each person (in green) that include only the relevant ranges and overlap the previous one. Note also that the vcards are defined in such a way that does not interfere with the structural features of the table. The full linearization in OWL of this example can be found at
+
EARMARK can be used to improve wiki reversion mechanisms and overcome the limitations discussed earlier: The automatic filtering and merging of contributions from multiple versions of the same page are still a manual process, but it can be fully automatized if the overlapping structures buried in the whole history of the page become explicit.
+The role of EARMARK is to make those structures explicit and available for more sophisticated content manipulation. To understand the extent EARMARK structures can be derived from wikis and how they can be exploited by the final users, we use as our example the wiki platform MediaWiki
+ 18
+
+
MediaWiki offers sophisticated functionalities for creating diffs of wiki content. Users can compare any two revisions in the page history and highlight changes in a friendly interface that shows modifications with a word‐level granularity. Diff pages contain metadata about each compared version (when the version was created, who the author was, or which IP address an anonymous author was connected from, etc.) and a two‐column table showing the changes side by side. Changes are detected a posteriori by comparing two arbitrary versions, which are not even required to be temporally contiguous.
+The output of the MediaWiki diff engine has regularities that can be exploited to automatically build the overlapping structures of the diff and to express them in EARMARK. Let us consider a fictitious example summarized in Table + 1, where an initial text is revised three times by different authors.
+To display the differences between V1 ad V2, Mediawiki creates a page whose HTML code is as follows:
+ 19
+ For the sake of clarity, we removed all markup irrelevant to our discussion.
+
This is an HTML table of two rows, the first showing metadata (date and author of the modification), and the second showing the actual modifications. The first cell of the second row contains all the unmodified text and a del element for each inline fragment that was deleted. The second cell contains all the unmodified text and an ins element for each inline fragment that was inserted. Thus, these cells share exactly the same unmodified part(s) of the two compared versions.
+When the structure itself is modified rather than merely the text, the source code of the MediaWiki diff is slightly different. Thus, the diff between V3 and V4 (which splits a paragraph in two) is as follows:
+
+
The diff output is not complete or sophisticated, and of course, it is a completely different task to replan such an algorithm (but for a first idea of natural changes in diffing XML documents, see Di Iorio, Marchetti, Schirinzi, & Vitali, + 2009). Thus, limitations of that algorithm are inevitably shared by any EARMARK representation. Yet, this output is sufficiently rich to allow us to extract the overlapping information we need. For instance, the insertion of a nonbreaking space or a carriage return generates rows according to specific rules that can be easily detected to capture the actual change by the author.
+Figure + 5 shows the aforementioned example rebuilt in EARMARK. All versions are encoded in the same document by creating overlapping assertions over the docuverses. Metadata and RDF statements are layered on top of those assertions and create a rich knowledge‐base about the history of the documents and, in particular, about the history of each fragment.
+The wiki sample versions encoded in a single EARMARK document. [Color figure can be viewed in the online issue, which is available at
+
Due to the complexity of the example, we labeled arrows with numbers that indicate the position of each range within each markup item. Consider, for instance, Version V4: It is composed of two DIV elements, the first one containing the concatenation of “Bob was farming carrots” + “,” + “tomatoes” + “and” + “green” + “beans” + “.” and the second one containing the string “They are all tasteful.”
+Implementing a wiki content‐filtering mechanism on top of such a structure is rather simple. For instance, the removal of all the contributions of “Angelo Di Iorio” that leaves untouched all the content written (previously and subsequently) by “Silvio Peroni” and “Fabio Vitali” can be performed straightforwardly. Three steps are enough to apply such an intermediate content reversion:
+ the identification of the fragments written by “Angelo Di Iorio,” which is a straightforward SPARQL query on the embedded statements; the creation of a new version where references to those fragments are removed and references to fragments no longer in the document are correctly fixed; the translation of that document into an actual MediaWiki page through the serialization process described in Peroni and Vitali (
+ 2009).
+
+
Of course, an automatic process may generate ambiguities or even errors in the resulting content (i.e., some parts may become dangling, wrong, or unclear after removing text fragments elsewhere); grammar discrepancies also might be generated by the same approach. Linguistic and semantic problems, however, become a problem once the technical issues of managing independent, yet successive, edits are solved. What is important is that all the information about overlaps and dependencies among fragments is available in EARMARK and can easily be searched, filtered, and manipulated. Besides, foreseeing a manual intervention for checking and polishing automatically filtered content is perfectly in line with the wiki philosophy, so that the wiki community itself can wisely use the reversion tools to revise the content and adjust any intervening minor nuisances or imperfections. Such checks would still be far simpler and faster than would the manual process of partially reverting versions as we have today.
+Since we do not expect documents to be natively written in EARMARK or manually created by users, we need a way to extract EARMARK data structures from existing XML‐based resources, which is trivial when the XML is simple and clearly hierarchical and slightly more complex when the XML contains workarounds to force an intrinsically overlapping situation into a single hierarchy.
+We designed a reliable process to transform XML files into EARMARK documents that fully captures overlapping structures even when the overlaps are hidden in one the many well‐known workarounds. This approach takes as input an XML file and produces the corresponding EARMARK document in five steps: Read, Overhaul, Convert, Classify, and Organize (hence, the name ROCCO).
+Since ROCCO is not the main topic of this article, we very briefly discuss the issue of converting XML into EARMARK, explaining how each step works. The ROCCO algorithm performs five steps, described next.
+The first two steps consist of loading the XML source file and, if needed, adding information useful for further processing. In EARMARK, there is a clear distinction between the textual content of a document and the structures built on top of it: The content is stored as plain text—within docuverses—and all structures are externalized and expressed through OWL and RDF assertions.
+While OpenOffice stores all overlapping structures in the main document file, some other editors (e.g., MS Word) store overlaps in many different ways, even in a separate file. The overhaul step extracts such data and adds them to the main content document by exploiting format‐specific procedures, implemented via XSLT in most cases.
+The subsequent step consists of converting the XML source file into an early EARMARK document that expresses exactly the same information and hierarchies. No interpretation or disentanglement of workarounds is performed at this step.
+Since the input is XML, this translation can be performed directly via a generic XSLT stylesheet. It basically consists of a recursive algorithm that parses the source file and generates the corresponding instances in the EARMARK ontology. Such a translation is straightforward and not difficult.
+The “Classify” step extends the EARMARK document built so far with information about the workarounds used to encode overlaps. That information will be exploited in the subsequent steps to make those overlaps explicit.
+The basic idea is to exploit OWL reasoners to detect workarounds in an early EARMARK document D by:
+ defining an ontology O that models all the workarounds used by applications, such as milestones, stand‐off markup, etc.; these workarounds are specific to the data format used in the source document; specifying the EARMARK document D as an ABox for the ontology O; defining SWRL rules that capture the role of each element in D and check relationships between elements; running an OWL reasoner, such as Pellet, on D+O to create new OWL instances and properties that identify which workarounds are present.
+
+
The actual detection of workarounds is delegated to an external reasoner. Refining detection strategies and even adding new strategies for new formats all can be done via OWL and SPARQL. Indeed, tricky issues need to be addressed—mostly depending on the idiosyncrasies of the original formats—but no procedural code is required.
+The final step consists of building yet another EARMARK document that expresses the overlaps and metadata in an explicit way, based on the information collected by the previous steps. This phase consists of mapping operations from the native format into the EARMARK structure. Such conversion relies on the identification of metadata to classify the operations and to externalize relevant metadata in separate RDF statements.
+One of the most frequent criticisms when proposing a different approach to solving a well‐known problem in information and communication technology is that the new solution may simplify the difficulties of the specific problem, but brings with it hidden costs in terms of size of the data structure, computation efforts, or conversion restrictions that compensate the advantages. In our case, one of the anonymous reviewers of our article (Di Iorio, Peroni, & Vitali, + 2009) wondered whether a difference in file size could weigh in on the convenience of adopting EARMARK as opposed to working with the original files.
+As such, a discussion of cost functions of EARMARK versus other formats is in order. Yet, a systematic discussion of the relative costs (e.g., in byte size) of some original XML‐based data structures versus their EARMARK equivalent is an open‐ended undertaking that heavily depends on the original XML data structure and the specific features present in the document, and is badly defined anyway: While XML is a linearization format immediately expressible in actual bytes, OWL (or more precisely, RDF, the language in which OWL ontologies are expressed) is an abstract structure that allows a large number of linearization formats (including XML itself) with corresponding huge differences in the final byte counts.
+For these reasons, to provide at least an initial test of meaningful concepts, we selected two XML‐based data formats (OOXML and ODT) and, specifically, a set of documents where overlapping tricks were present (i.e., where change‐tracking was active). To bypass the size discussion, we decided not to test byte lengths (which are not meaningful and easily skewed, e.g., by reducing the string length of the element names or of the class names) but the number of nodes for XML documents and of triples for OWL documents. This comparison again is not particularly appropriate (Triples are naturally numerous in OWL ontologies, and it is customary to deal with hundreds of thousands and even millions of assertions in Semantic Web applications.) but closer to meaningfulness than is the mere byte count.
+Our comparison was carried on a small set of documents in ODT and OOXML that included change‐tracking information. As discussed in the previous sections, change‐tracking facilities generate rather complex overlaps even for basic operations on small text fragments, which in turn are expressed as a potentially huge number of standoffs and milestone markup within the XML hierarchy. The same documents were individually converted into EARMARK. We then charted how simple edits under change‐tracking affect the number of nodes in XML formats and of statements in OWL files.
+ 20
+ The full details about each version and each format also are available at
+
We created seven different versions, named after the “Seven Dwarfs” for recognizability, by applying very common edits (e.g., the insertion of few words, the deletion of some sentences, the split of a paragraph, etc.) on a small document, creating multiple overlaps. Figure + 6 shows the results of our comparison.
+A graph summarizing the results of the first experiment. [Color figure can be viewed in the online issue, which is available at
+
The overall trend is interesting and comforting: While in simple documents with no overlap the node count of XML is lower than is the assertion count of EARMARK triples, the presence of overlaps makes EARMARK and XML formats comparable. The growth of EARMARK statements is in fact very close to the growth of XML nodes when the number of overlaps increases. EARMARK is even more efficient than is XML for more complex documents.
+The measure for each format was done by counting only those nodes and statements instrumental to encode content and (overlapping) structures: We did not take into account either the presentational information for ODT and OOXML (Each file, for instance, includes a very long list of style definitions that are not relevant for the purposes of our analysis.) or namespace declarations (OOXML files, for instance, list all relevant namespaces for the Office toolkit.) or ignorable white spaces (that are only added to indent content and improve readability).
+Interestingly, EARMARK and ODT show a very similar increase in size while OOXML is much more verbose and grows faster. The content of the first version, for instance, is encoded using four nodes in ODT, 13 statements in EARMARK, and 54 nodes in OOXML; the last one contains 241 ODT nodes, 233 EARMARK statements, and 452 OOXML nodes. To return to our original inquiry, it is clear that the weight of EARMARK documents is very good compared to the other ones.
+Also note the regularity in the growth of EARMARK statements. Regardless of the actual modifications applied to the document, in fact, EARMARK adds about 40 statements for each edit. Both OOXML and ODT, on the contrary, show a more irregular “pace.” The reason for this is that EARMARK externalizes all assertions, so that all modifications (either to leaf nodes or to intermediate nodes in the original XML) are “flattened” onto the docuverses and do not depend on the complexity of the structure within which the edit took place.
+Figure + 7 shows the results of a similar comparison on a different set of documents and edits. We collected seven versions named after the days of the week and created by seven different authors when editing a very simple document. The overall trend does not change, and shows that EARMARK and ODT again have a comparable behavior, far better than that of OOXML.
+A graph summarizing the results of the second experiment. [Color figure can be viewed in the online issue, which is available at
+
In conclusion, although preliminary, this study shows clear trends of a very conservative behavior of EARMARK with respect to document size.
+Overlaps, far from being an obscure requirement for sophisticated functionalities of arcane markup languages, are a very frequent undertaking even in major data formats and in rather frequent situations. Yet, since the XML language does not allow them, consciously or not, designers of data formats have adopted a huge and entangled array of tricks, special cases, and workarounds that, although solving the actual problem of storing overlapping structures, open new and complicated ones when approaching even basic chores on documents containing them, such as queries.
+The EARMARK approach drastically reduces the efforts needed to perform such chores on overlapping structures since it does not allow the corresponding multiple trees to actually entangle and complicate the job. EARMARK is radically different from both special markup metalanguages that allow overlaps and the introduction of workarounds within the traditional tree‐oriented XML language because it treats multiple trees over the same content as first‐class citizens of the language, yet uses well‐known and standard W3C technologies and languages to perform all tasks. EARMARK documents, at the end, are OWL ontologies. Thus, any Semantic Web technology (e.g., SPARQL) can be used straightforwardly to perform operations on their content.
+Improving queries is not the only application of EARMARK. Validation is another interesting field that we are investigating. In fact, the same ontological framework can be used to prove properties concerning a document, such as validity against a schema, compliance to co‐constraint specifications, or adherence to structural patterns. Moreover, inspired by Marcoux and Rizkallah ( + 2009), in which they described an approach for defining natural‐language semantics for XML‐based languages, we also are developing an ontology‐based approach for encoding markup semantics—that is, the formal definition of meanings of markup elements, besides the syntactical structure of a markup document—within EARMARK documents.
+Overlapping structures in XML are not symptoms of a misunderstanding of the intrinsic characteristics of a text document nor evidence of extreme scholarly requirements far beyond those needed by the most common XML‐based applications. On the contrary, overlaps have started to appear in a large number of incredibly popular applications hidden under the guise of syntactical tricks to the basic hierarchy of the XML data format. Unfortunately, syntactical tricks have the drawback that the affected structures require complicated workarounds to support even the simplest query or usage. In this article, we present Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), an approach to overlapping markup that simplifies and streamlines the management of multiple hierarchies on the same content, and provides an approach to sophisticated queries and usages over such structures without the need of ad‐hoc applications, simply by using Semantic Web tools and languages. We compare how relevant tasks (e.g., the identification of the contribution of an author in a word processor document) are of some substantial complexity when using the original data format and become more or less trivial when using EARMARK. We finally evaluate positively the memory and disk requirements of EARMARK documents in comparison to Open Office and Microsoft Word XML‐based formats.
+The overwhelming consensus among XML practitioners is that documents are trees, the hierarchy is the fundamental data structure, and violations of the hierarchy are errors or unnecessary complications. Therefore, overlapping markup has received ambivalent, almost schizoid considerations in the field of markup languages. Traditionally, overlaps were the hallmarks of bad HTML coders and nave HTML page editors, taking advantage of the unjustified benevolence in web browsers that would display basically any HTML regardless of proper nesting. At the same time, far from the awareness of the general public, overlaps have been a fringe, almost esoteric, discipline of scholars in the humanities, competently used for arcane specifications of linguistic annotations and literary analysis.
+Although the first type of overlap was judged with scorn and the second with awe, they both fundamentally represent a situation that is more common than was thought, and the scholars were only more aware, and not more justified, about the need to represent overlaps.
+Generally, overlap is needed when multiple independent items refer to the same segment, either when considering textual markup documents or multimedia structures (Salembier & Benitez, + 2007). Regarding documents with markup, we need overlap whenever multiple markup elements need to be applied over the same content, and these elements happen to be independent of each other. In some (rather frequent) situations, this independence means that the content referred to by some elements is partially, but not completely, the same as the content referred to by other elements.
+This situation is more frequent than it may appear: Not only do bad HTML code and arcane linguistic annotations use overlap, but many more mainstream and mundane examples exist. For instance, change tracking in an office document is often at odds with the underlying structure of the text, microformats (Allsopp, + 2007) and Resource Description Framework–in–attributes (RDFa; Adida, Birbeck, McCarron, & Pemberton, + 2008) annotations may need to refer to concepts that span across multiple XML elements, complex data structures (e.g., biological data) force graphs into trees and hide multiple parentage as internal references, and so on.
+Differently from SGML, which is able to handle some overlapping scenarios through the CONCUR notation (Goldfarb, + 1990), XML grammatically imposes and requires a strict hierarchy of containment generating a single mathematical tree of the document where no overlap is allowed. This requirement has been turned into an intrinsic characteristic of the documents XML was meant to represent rather than a syntactical and conceptual constraint into which these documents need to fit. Thus, whenever authors needed to cope with independent markup elements, they managed either by navely ignoring the hierarchical limitation (and therefore creating invalid documents) or by creating careful workarounds within the syntactical constraint, or even by inventing completely new markup languages that allow some types of overlap. But while new multihierarchical markup languages such as TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003) and LMNL (Tennison & Piez, + 2002) have a small number of adepts and applications, and while bad HTML coders and bad HTML page editors are disappearing from the market, the careful workarounds within the XML syntax (TEI Consortium, + 2005), such as segmentation, milestones, or standoff markup, are to this day frequently used and ubiquitous.
+All workarounds share the same approach of hiding structural information about a secondary hierarchy under the guise of something else: split individual elements, empty boundary elements, indirect references, and so on. The result is that the secondary structural information is hidden or its importance is lessened to not break or obfuscate the main hierarchy expressed in the visible XML structure. But this comes at a price: Structures specified through workarounds are more difficult to find, identify, and act upon than the are structures in the main XML hierarchy. Thus, trivial searches that should amount to a short XPath in a more direct situation end up being multiple‐lines long, pretty basic visualizations require incredibly complex XSLT stylesheets, specific choices of the main markup hierarchy actually prevent some features of the secondary markup to even exist, and so on. So, although workarounds exist and can be used, hierarchies expressed through them are “second‐class citizens” that cannot fully exploit the sophisticated tools that the XML language provides.
+In this article, we show how Extremely Annotational Resource Description Framework (RDF) Markup (EARMARK), our proposal for managing overlapping markup, does not generate first‐ and second‐class hierarchies, and allows existing, sophisticated tools to be used on all markup—even in the presence of overlaps. Rather than creating a completely new language requiring completely new tools and competencies, EARMARK uses Semantic Web technologies and Semantic Web tools to obtain many of the results obtainable with usual XML tools.
+EARMARK defines markup vocabularies by means of OWL ontologies (W3C OWL Working Group, + 2009). Since each individual markup item is an independent assertion over some content or some other assertions, overlaps of content are not a problem, nor are all the issues connected to physical embedding and containments, such as contiguity and document order. Furthermore, by using standard Semantic Web technologies, fairly sophisticated functionalities can be provided over EARMARK documents.
+Through EARMARK, operations that were previously very hard or impossible exactly because of the interferences of the multiple hierarchies or of the workarounds they employed now become fundamentally trivial since no syntactical tricks are employed and the different hierarchies do not interfere with each other. Thus, for instance, identifying the individual contributions in a multi‐authored MS Word or Open Office document is quite hard on their original XML formats, and becomes trivial when the same documents are converted into EARMARK.
+This article is an extended version of previous works on EARMARK (Di Iorio, Peroni, & Vitali,
+ 2010; Peroni & Vitali,
+ 2009). In those works, we focused on identifying workarounds for overlapping data existing in real XML documents and translating them into EARMARK assertions. We also sketched the EARMARK ontology and presented a simple implementation of EARMARK‐aware tools. This article follows and extends them and also provides some novel contributions:
+ The systematic analysis of the EARMARK model, with particular attention to data typing and overlapping structures The discussion of further applications for the ontological EARMARK approach. In particular, we show how EARMARK can be used to improve the content filtering and reversions mechanisms of wikis. The brief description of a process, called ROCCO, for generating EARMARK documents from existing XML documents (even ones that use workarounds for overlapping structures) An evaluation of EARMARK efficiency when dealing with multiple hierarchies in comparison with the XML structures used by popular XML‐based formats such as Office Open and MS Word.
+
+
The article is structured as follows: First, we provide a brief overview of existing approaches to overlap using workarounds in XML or ad hoc markup metalanguages, and then give a few examples of situations where overlaps are used today and sometimes in rather mainstream situations. Next, we present the EARMARK model and its rules. Then, we provide some use cases that are meant to demonstrate the superiority of the EARMARK approach to a traditional XML format, especially when overlaps come into question, and show the generation of EARMARK documents, converting legacy documents. An initial evaluation of the efficiency of EARMARK compared to popular XML based data formats such as Open Document (ODT) and Office Open XML (OOXML) is presented, followed by our onclusions.
+The need for multiple overlapping structures over documents using markup syntaxes such as XML and SGML is an age‐old issue, and a large amount of literature exists on the techniques, languages, and tools that allow users to create multiple entangled hierarchies over the same content. A good review can be found in DeRose ( + 2004).
+Some research has proposed using plain hierarchical markup (i.e., XML) and employing specially tailored elements or attributes to express the semantics of overlapping in an implicit way. The TEI Guidelines (TEI Consortium,
+ 2005) presented a number of different techniques that use SGML/XML constructs to force multiple hierarchies into a single one, including:
+ milestones (i.e., verlapping structures are expressed through empty elements to mark the boundaries of the “content”), fragmentation (i.e., overlapping structures are split into individual, nonoverlapping elements that may even be linked through id–idref pairs), and standoff markup (i.e., overlapping structures are placed elsewhere and indirectly refer to their would‐be locations through pointers, locators, and/or id–idref pairs).
+
+
Given the large number of techniques to deal with overlapping structures in XML, in Marinelli, Vitali, and Zacchiroli ( + 2008), we presented a number of algorithms to convert XML documents with overlapping structures from and to the most common approaches, as well as a prototype implementation.
+Riggs ( + 2002) introduced a slightly different technique for fragmentation within XML structures. In this proposal, floating elements (i.e., those elements that do not fall in a proper or meaningful hierarchical order) are created using the name of the element followed by an index referring to its semantically related parent element. For example, the floating element <name.person[2]>John</name.person [2] means that <name>John</name> is semantically a child of the second occurrence of the element person, even though the floating element is not structurally contained by its logical parent.
+Other research even has proposed to get rid of the theory of trees at the base of XML/SGML altogether and use different underlying models and newly invented XML‐like languages that allow the expression of overlaps through some kind of syntactical flourishing. For instance, a general ordered‐descendant directed acyclic graph (GODDAG; Sperberg‐McQueen & Huitfeldt, + 2004) is a family of graph‐theoretical data structures to handle overlapping markup. A GODDAG's nodes represent markup elements and text. Arcs are used to explicitly represent containment and father–child relations. Since multiple arcs can be directed to the same node, overlapping structures can be straightforwardly represented in GODDAG. Full GODDAGs cannot be linearized in any form using embedded markup, but restricted GODDAGs, a subset thereof, can be and have been linearized into TexMecs (Huitfeldt & Sperberg‐McQueen, + 2003), a multihierarchical markup language that also allows full GODDAGs through appropriate nonembedding workarounds such as standoff markup.
+LMNL (Tennison & Piez, + 2002) is a general data model based on the idea of layered text fragments and ranges, where multiple types of overlap can be modeled using concepts drawn from the mathematical theory of intervals. Multiple serializations of LMNL exist, such as CLIX and LMNL‐syntax.
+XConcur (Schonefeld & Witt, + 2006) is a similar solution based on the representation of multiple hierarchies within the same document through layers. Strictly related to its predecessor CONCUR as it was included in the SGML, XConcur was developed in conjunction with the validation language XConcur‐CL to handle relationships and constraints between multiple hierarchies.
+The variant graph approach (Schmidt & Colomb, + 2009) also is based on graph theory. Developed to deal with textual variations that generate multiple versions of the same document with multiple overlapping hierarchies, this theory proposes a new data model to represent literary documents and a graph linearization (based on lists) that scales well even with a large number of versions. The same authors recently presented an extension of their theory that also allows users to merge multiple variants into one document (Schmidt, + 2009). In Portier and Calabretto ( + 2009), a detailed survey about overlapping approaches was presented, and also discussed the MultiX2 data model, which uses W3C standard languages such as XInclude to link and fetch text fragments within overlapping structures, and a prototype editor for the creation of multistructured documents.
+Tummarello, Morbidoni, and Pierazzo ( + 2005) proposed using RDF as a standoff notation for overlapping structures of XML documents. Since this proposal has many affinities with the one we are presenting in this article, we later discuss its characteristics and compare it with ours.
+Overlapping structures have been considered often as appropriate only in highly specific contexts and basically for scholars: The solutions that have been proposed in the literature were complex since they were considered grounded in the intrinsic complexity of the topics themselves. Yet, overlapping structures can be found in many more fields than these, and even mainstream applications generate and use markup with overlapping structures. While the complexity of overlapping is hidden to the final user, applications that consume such data may very well find it rather difficult to handle such information. We next discuss three very different contexts where overlapping already exists and fairly relevant information is encoded in multiple independent structures, leaving to special code the task of managing the complexity.
+Word processors such as Microsoft Word and Open Office provide users with powerful tools for tracking changes, allowing each individual modification by individual authors to be identified, highlighted, and acted upon (e.g., by accepting or discarding them). The intuitiveness of the relevant interfaces actually hides the complexity of the data format and of the algorithms necessary to handle such information.
+For instance, the standard ODT format (JTC1/SC34 WG 6, + 2006) used by Open Office, when saving change‐tracking information, relies on two specific constructs for insertions and deletions that may overlap with the structural markup. Adding a few words within a paragraph is not in itself complex, as it does not require the breaking of the fundamental structural hierarchy; conversely, changes that affect the structure itself (e.g., the split of one paragraph in two by the insertion of a return character, or the joining of two paragraphs by the elimination of the intermediate return character) require that annotations are associated to the end of a paragraph and the beginning of the next, in an unavoidably overlapping pattern. ODT uses milestones and standoff markup for insertions and deletions, respectively, and also relies on standoff markup for annotations about the authorship and date of the change.
+For instance, the insertion of a return character and a few characters in a paragraph creates a structure as follows:
+
+
The empty elements <text:change‐start/> and <text:change‐end/> are milestones marking the beginning and the end, respectively, of the range that constituted the insertion while the element <text:insertion>, before the beginning of the document content, is standoff markup for the metadata about the change (author and date information).
+Similarly, a deletion creates a structure as follows:
+
+
The element <text:change/> represents a milestone of the location where the deletion took place in the content, and the corresponding standoff markup annotation <text:deletion> contains not only the metadata about the change but also the text that was deleted.
+The OOXML format (JTC1/SC34 WG 4, + 2008) (the XML‐based format used by Microsoft Office 2007 and standardized by ISO in 2008), on the other hand, uses a form of segmentation to store change‐tracking information across all previous elements involved.
+
+
This heavily simplified version of an OOXML document shows two separate changes: (a) the insertion of a return character and (b) the insertion of a word. These modifications are not considered as a single change; therefore, the segments are not connected to each other but simply created as needed to fit the underlying structure.
+In fact, change tracking in OOXML is a fairly complex proposition. Although providing more complete coverage of special cases and situations than does ODT, dealing with its intricacies is not for the casual programmer. Even a simple XSLT stylesheet to show inserted text in a different color and hide deleted text may run several hundred lines of code.
+ 1
+
+
Microformats (Allsopp, + 2007) add semantic markup to web documents by using common structures of the HTML language itself—in particular, the class attribute.
+The HTML code is annotated using microformats to provide new semantic, machine‐processable assertions. In the following example, a plain HTML table is enriched with metadata about events
+ 2
+ HCalendar,
+ HCard,
+
+
The table was enriched by additional data declaring it to be an event (a conference), and data about the event itself (URL, summary, location) and about four relevant individuals (with their names and roles within the conference) were associated where necessary to the actual content of the table.
+So far, so good, and no overlap to speak about. Things change dramatically, though, when the overall structure of the main hierarchy (the HTML table) is at odds with the intrinsic hierarchy of the microformat data, such as if the people are organized in columns rather than rows. For instance:
+
+
Unfortunately, vcards are a hierarchy themselves, and if the hierarchy of vcards is organized differently from the hierarchy of the HTML table, as in the latter case, it is just impossible to define the four vcards for the four people organizing the conference. Thus, in plain HTML, the choice of one of two possible presentation models for the main hierarchy of content makes trivial or completely impossible the existence of the second hierarchy.
+A possible and partial solution to express vcard hierarchies in the latter example is RDFa (Adida et al., + 2008), a W3C recommendation. It describes a mechanism to embed RDF statements into HTML documents by using some HTML attributes (href, rel, rev, content) in combination with other ad hoc attributes (property, about, typeof) proposed in the recommendation itself.
+
+
Since all attributes live in the context of elements, the price to pay is that to assert everything we want to assert, we often need to add some structurally unnecessary elements to the current markup hierarchy of a document, needed only to add the RDF statements (e.g., the span elements emphasized earlier). Even if that does not represent a significant problem for strict Semantic Web theorists, document architects and markup experts see this as a kludge and an inelegant compromise.
+The strength of wikis lies in their allowing users to modify content at any time. The mechanisms of change‐tracking and rollback that are characteristics of all wikis, in fact, promote users' contributions and make “malicious attacks” pointless in the long run since previous versions can be easily restored.
+A number of tools exist that automatically discover “wiki vandalisms” and provide users with powerful interfaces to surf changes, identify differences between subsequent versions, and revert content. For instance, Huggle
+ 4
+
+
Even client‐side tools—meant to be installed as browser extensions or bookmarklets—exist to extend the rollback mechanisms of Wikipedia, giving users more flexibility and control over (vandalistic) changes. For instance, Lupin
+ 5
+
+
+
These tools are successful in highlighting vandalism and in identifying versions created by malicious users. However, although it is possible to revert the page to any previous version, all changes (even acceptable ones) that were subsequent to the malicious version cannot be automatically inherited by the restored page.
+For instance consider Versions V1, V2, and V3 of a wiki page, where Version V1 contains a baseline (i.e., acceptable) content, and Version V2 is identified as a partial vandalism and is agreed to be removed, but Version V3 contains (possibly, in a completely different section than the target of the malicious attack) relevant and useful content that was added before the vandalistic Version V2 was declared as such. The task of removing the modifications of Version V2 while maintaining (whatever is possible of) Version V3 is a difficult, error‐prone, and time‐consuming task if done manually, yet there is no tool we are aware of that automatically filters contributions from multiple versions and merges them into a new one (or, equivalently, removes only selected intermediate versions).
+However, it is possible to theoretically characterize the interdependencies between subsequent changes to a document. In fact, literature has existed for a long time on exactly these themes (e.g., Durand, + 1994, + 2008). Although a detailed discussion of abstract models of interconnected changes is out of scope for this article (Details and authoritative references can be found in the aforementioned works.), what is relevant in this discussion is that they happen to assume a hierarchical form that is frequently at odds with the hierarchical structure of the content of the document, and as such, most issues derive from the data structures in which content is stored and from the model for manipulating these structures. For instance, the fact that in the wiki perspective each version is an independent unit that shares no content (even unchanged content) with the other versions prevents considering multiple versions as overlapping structures coexisting on the same document. If we were able to make these hierarchies explicit, we would be able to create models and tools to manipulate these documents in a more powerful way and to exploit the existing interconnections between the overlapping hierarchies.
+The presence of hidden overlapping structures—transparent to users, but very difficult to handle by applications—is the common denominator for the scenarios described in the previous section. More than the overlap itself, which cannot be ignored because it does exist and carries important meanings, the problem we face lies in the way applications store such overlapping structures. In the XML world, in fact, the only way to do so is through the use of (complex) workarounds that force the multiple hierarchies into one hierarchy of an XML document. That makes it very tricky to perform sophisticated analysis and searches.
+This section discusses a different approach to metamarkup, EARMARK (Di Iorio, Peroni, & Vitali, + 2009; Di Iorio et al., + 2010; Peroni & Vitali, + 2009) based on ontologies and Semantic Web technologies. The basic idea is to model EARMARK documents as collections of addressable text fragments, and to associate such text content with OWL assertions that describe structural features as well as semantic properties of (parts of) that content. As a result, EARMARK allows not only documents with single hierarchies (as with XML) but also multiple overlapping hierarchies where the textual content within the markup items belongs to some hierarchies, but not to others. Moreover, EARMARK makes it possible to add semantic annotations to the content though assertions that may overlap with existing ones.
+One of the advantages of using EARMARK is the capability to access and query documents by using well‐known and widely supported tools for the Semantic Web. In fact, EARMARK assertions are simply RDF assertions while EARMARK documents are modeled through OWL ontologies. The consequence is that query languages (e.g., SPARQL; Garlik & Seaborne,
+ 2010) and actual existing tools such as Jena
+ 7
+
+
+
In the rest of this section, we give a brief overview of the EARMARK model and then describe how EARMARK can be used to deal with the issues presented earlier. The model itself is defined through an OWL document,
+ 9
+
+
A UML‐like representation of the EARMARK ontology. [Color figure can be viewed in the online issue, which is available at
+
The ghost classes describe three disjoint base concepts—docuverses, ranges, and markup items—through three different and disjoint OWL classes.
+ 10
+ All our OWL samples are presented using the Manchester Syntax (Horridge & Patel‐Schneider,
+ 2009), which is one of the standard linearization syntaxes of OWL. The prefixes rdfs and xsd refer to RDF Schema and XML Schema namespaces, respectively, while the empty prefix refers to the EARMARK ontology URI plus “#.” Moreover, we use the prefix c to indicate entities taken from an imported ontology made for the SWAN project (Ciccarese et al.,
+ 2008); available at
+
The textual content of an EARMARK document is conceptually separated from its annotations, and is referred to through the Docuverse class.
+ 11
+ This class (and its name) is based on the concept introduced by Ted Nelson (
+ 1980) in his Xanadu Project to refer to the collection of text fragments that can be interconnected to each other and transcluded into new documents.
+
Any individual of the Docuverse class—commonly called a docuverse (lowercase to distinguish it from the class)—specifies its actual content with the property hasContent.
+We then define the class Range for any text lying between two locations of a docuverse. A range (i.e., an individual of the class Range) is defined by a starting and an ending location (any literal) of a specific docuverse through the properties begins, ends, and refersTo, respectively.
+
+
There is no restriction on locations used for the begins and ends properties. That is very useful because it allows us to define ranges that follow or reverse the text order of the docuverse to which they refer. For instance, the string “desserts” can be considered both in document order, with the begins location lower than the ends location, or in the opposite order, forming “stressed.”
+ 12
+
+
The class MarkupItem is the superclass defining artifacts to be interpreted as markup (e.g., elements and attributes).
+
+
A markupitem individual is a collection (c:Set, c:Bag, or c:List, where the latter is a subclass of the second one, and all of them are subclasses of c:Collection) of individuals belonging to the classes MarkupItem and Range. Through these collections, it is possible to define a markup item as a set, a bag, or a list of other markup items, using the properties element (for sets) and item and itemContent (for bags and lists). Thus, it becomes possible to define elements containing nested elements or text, or attributes containing values, as well as overlapping and complex structures. Note also that handling collections directly in OWL allows us to reason about content models for markup items, which would not be possible if we had used the corresponding constructs in RDF.
+ 13
+
+
A markupitem also might have a name, specified in the functional property hasGeneralIdentifier (recalling the SGML term to refer to the name of elements; Goldfarb, + 1990), and a namespace, specified using the functional property hasNamespace. Note that we can have anonymous markup items—as is possible in LMNL (Tennison & Piez, + 2002) and in GODDAG (Sperberg‐McQueen & Huitfeldt, + 2004)—by simply asserting that the item belongs to the class of all those markupitems that do not have a general identifier (i.e., hasGeneralIdentifier exactly 0).
+The ghost classes discussed so far give us an abstract picture of the EARMARK framework. We need to specialize our model, defining a concrete description of our classes. These new shell subclasses apply specific restrictions to the ghost classes.
+First, the class Docuverse is restricted to be either a StringDocuverse (i.e., the content is specified by a string) or a URIDocuverse (i.e., the actual content is located at the URI specified).
+
+
Depending on particular scenarios or on the kind of docuverse we are dealing with (plain‐text, XML, LaTeX, a picture, etc.), we need to use different kinds of ranges. Therefore, the class Range has three different subclasses:
+ PointerRange defines a range by counting characters. In that case, the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream, remembering that the value 0 refers to the location immediately before the first character, the value 1 refers to the location after the first character and before the second one, and so on. By using the hasKey OWL property, we also assert that two pointer ranges having equal docuverse and begin and end locations are the same range. XPathRange defines a range considering the whole docuverse or its particular context specifiable through an XPath expression (Berglund et al.,
+ 2007) as value of the property hasXPathContext. Note that by using these ranges, we implicitly admit that the docuverse it refers to must be an XML structure. Moreover, the properties' begins and ends have to be applied on the string value obtained by juxtaposing all the text nodes identified by the XPath. By using the hasKey OWL property, we also assert that two xpath ranges having equal docuverse, XPath context, and begin and end locations are the same range. XPathPointerRange is an XPathRange in which the value of the properties' begins and ends must be a nonnegative integer that identifies unambiguous positions in the character stream as described for the class PointerRange.
+
+
+
MarkupItem is specialized in three disjointed subclasses—Element, Attribute, and Comment—that allow a more precise characterization of markup items.
+
+
The presence of overlap in EARMARK is worth discussing in more detail. Different types of overlap exist, according to the subset of items involved, and different strategies are needed to detect them. In particular, there is a clear distinction between overlapping ranges and overlapping markup items.
+By definition, overlapping ranges are two ranges that refer to the same docuverse, so that at least one of the locations of the first range is contained in the interval described by the locations of the second range (excluding its terminal points). Totally overlapping ranges have the locations of the first range completely contained in the interval of the second range, or vice versa, while partially overlapping ranges have either exactly one location inside the interval and the other outside or identical terminal points in reversed roles.
+Thus, if we consider the following excerpt:
+
+
we can infer, through a reasoner such as Pellet, that these two ranges overlap by using the following rules:
+
+
where P is one of:
+ lessThan(b1,e1) ˆ greaterThan(b2,b1) ˆ lessThan(b2,e1) lessThan(b1,e1) ˆ greaterThan(e2,b1) ˆ lessThan(e2,e1) lessThan(e1,b1) ˆ greaterThan(b2,e1) ˆ lessThan(b2,b1) lessThan(e1,b1) ˆ greaterThan(e2,e1) ˆ lessThan(e2,b1).
+
+
The case of overlapping markup items is slightly more complicated. We define that two markup items A and B overlap when at least one of the following sentences holds:
+ [Overlap by range]: A contains a range that overlaps with another range contained by B. [Overlap by content hierarchy]: A and B contain at least a range in common. [Overlap by markup hierarchy]: A and B contain at least a markup item in common.
+
+
The three possible scenarios for such item overlap are summarized in Figure
+ 2.
+ 14
+ The EARMARK documents describing these three overlapping scenarios and all the other ones presented in the following sections are available at
+
Three EARMARK examples of overlapping between elements p. [Color figure can be viewed in the online issue, which is available at
+
The EARMARK ontology, in fact, is completed by another ontology
+ 15
+
+
If we ignore for a moment the semantic implications of using EARMARK and concentrate on its syntactical aspects only, it is easy to observe that EARMARK is nothing but yet another standoff notation, where the markup specifications point to, rather than contain, the relevant substructure and text fragments.
+Standoff notations, also known in literature as out‐of‐line notations (TEI Consortium,
+ 2005), are hardly new, but never really caught on for a number of reasons, most having to do with their perceived fragility under the circumstances of desynchronized modification to the text. In Georg, Schonefeld, Trippel, and Witt (
+ 2010) and Bański (
+ 2010), we can find a pair of recent and substantially complete analyses of their merits and demerits. In particular, according to Georg, Schonefeld, Trippel, and Witt (
+ 2010), “standoff annotation has … quite a few disadvantages:
+ very difficult to read for humans The information, although included, is difficult to access using generic methods. Limited software support as standard parsing or editing software cannot be employed. Standard document grammars can be used only for the level which contains both markup and textual data. New layers require a separate interpretation. Layers, although separate, often depend on each other.”
+ 16
+ To individually address the issues, we edited the original bullets into a numbered list.
+
+
And yet, although EARMARK is in practice a standoff notation, it provides a number of workarounds to most of the aforementioned issues.
+First, since EARMARK is based on OWL and can be linearized in any of the large number of OWL caricaturization syntaxes, it follows that (a) readability, (b) access, and (c) software support for it are exactly those existing for well‐known, widespread, and important W3C standards such as RDF and OWL. Being able to employ common RDF and OWL tools such as Jena and SPARQL for EARMARK documents was in fact a major motivation for it.
+Issue 4 should be examined beyond the mere validation against document grammars and toward a general evaluation of the level of compliancy of the markup to some formally specified expectations. EARMARK documents, while being subject to no document grammar in the stricter XML sense, allow the specification of any number of constraints, expressed either directly in OWL or SWRL (Horrocks et al., + 2004), or even in SPARQL, that trigger or generate validity evaluations. In Di Iorio, Peroni, and Vitali (in press), we tried to show that a large number of requirements, from hierarchical well‐formedness in the XML sense, to validation requirements in terms of XML DTDs, to adherence to design patterns, can be expressed satisfactorily using these technologies.
+Issue 5 regards the difficulty of standoff notations to provide interlayer analysis on XML structures: Separate interpretation of markup layers is easy, but identification and validation of overlapping situations are more complex: Standoff markup is mainly composed of pointers to content and does not have any direct way to determine overlap locations without some kind of pointer arithmetics to compute them. Validation of contexts allowing overlaps as describable using rabbit/duck grammars (Sperberg‐McQueen, + 2006) also is not trivial. In this regard, EARMARK provides yet again a solution that does not require special tools: Although OWL does not allow direct pointer arithmetics, SWRL on the contrary does, as shown earlier where we described a batch of (SWRL‐implementable) rules that do, in fact, determine overlapping locations on EARMARK documents with good efficiency.
+Finally, Issue 6 refers to the fact that evolution of separate markup annotation layers needs to synchronously take place, lest one of them becomes misaligned with the new state of the document. This is, in summary, the fragility of pointers, which can be considered the fundamental weakness of standoff, as well as of any notation that has markup separate from its content: If a modification occurs to the underlying (probably text‐based) source, all standoff pointers that could not be updated at the same time of the change become outdated and possibly wrong. All standoff notations fall prey to this weakness, and there is no way to completely get rid of it.
+What is possible is to identify exactly the conditions under which such weakness acts, and see if there is a way to reduce the mere frequency of such events. In fact, for a standoff pointer to become outdated, several conditions must take place at the same time:
+ The standoff notation must be used as a storage format, rather than just as a processing format; the source document must make sense even without the additional standoff markup (i.e., the standoff notation contains no information that is necessary for at least some types of document modifications); the source document must be editable (and, in fact, must be edited) on its own; the standoff pointers must rely on positions that change when the source is edited (e.g., character‐based locations); editing must be done in contexts and with tools that cannot or do not update the standoff pointers; and there must be no computable way to determine the modifications of the document (e.g., via a diff between the old and new versions).
+
+
Of course, no standoff notation can rule out that these conditions occur on their documents, but note that all six of them must occur for standoff pointers to become outdated. EARMARK is not safe from these occurrences either, but at least for the use cases here described, one or more of these conditions simply do not apply: EARMARK is mostly used as a processing format, with no need to save it on disk (Conversion from the source formats such as MS Word is described later and does not require special storage.), the data format described is either in a very specific format (e.g., MS Word or ODT) that in fact already does handle internally its data changes and requires the overlapping data exactly for this purpose, or is in fact the result of a diff action on successive versions of a document (as in the case of the wiki pages). Finally, EARMARK allows references to relatively stable fragment ids of the documents (by using XPath ranges without specifying explicitly begin and end locations) rather than the extremely fragile character locations, further reducing the chances of outdated pointers. For this reason, without being able to completely rule out the possibility of standoff pointers going wrong, we tend to consider it as a significantly little risk, at least for the use case described here.
+EARMARK is strongly based on OWL 2 DL (W3C OWL Working Group, + 2009) to express multiple markup layers with possible overlapping ranges over the same content. OWL 2 DL is not the only possible choice for expressing standoff notations via Semantic Web technologies. In fact, RDF is another valid and effective model for dealing with the same issue, as shown in Tummarello et al. ( + 2005), by means of the open‐source application programming interface (API) RDF Textual Encoding Framework (RDFTef). This API was created to demonstrate a plausible way for handling overlapping markup within documents and identifying textual content of a document as a set of independent RDF resources that can be linked mutually and with other parent resources.
+Besides giving the possibility to define multiple structural markup hierarchies over the same text content, the use of RDF as the language for encoding markup allows to specify semantic data on textual content as well. But the real main advantage in using RDF is the possibility of using particular built‐in resources appositely defined in the RDF syntax specification (Beckett, + 2004) for describing and dealing with different kinds of containers, either ordered (rdf:Seq) or unordered (rdf:Bag). Thus, RDF resources can be used to represent every printable element in the text—words, punctuation, characters, typographical symbols, and so on—while RDF containers also can be used to combine such fragments and containers.
+Although RDF is not sufficient to define a formal vocabulary for structural markup, does a given resource represent an element, an attribute, a comment, or a text node? In which way is a resource of a certain type related to others? The specification of an RDFS (Brickley & Guha,
+ 2004) or of an OWL layer can successfully address these issues. Hybrid solutions obtained by mixing different models, even when they are built one upon another, may seem elegant, but not necessarily the best choice. In fact, there exist well‐known interoperability limits between OWL 2 DL and RDF that prevent the correct use of Semantic Web tools and technologies. In particular:
+ Any markup document made using RDF containers (e.g., to describe what markup items contain and in which order) and OWL ontologies (e.g., to define classes of markup entities and their semantics) results in a set of axioms that end up outside of OWL DL and well within OWL Full, which limits the applicability of the most frequently used Semantic Web tools that are usually built upon the (computationally tractable) description logic underlying OWL 2 DL. The individual analysis of each language may be not applicable when we have to check particular properties that lie between RDF and OWL layers. For example, verifying the validity of a markup document against a particular schema, which is one of the most common activities with markup, needs to be made to work with both markup item structures (that would be defined in RDF) and logical constraints about classes of markup items (e.g., elements only, attributes only, the element “p,” all the element of a particular namespace, etc., all of them definable in OWL).
+
+
Being able to express everything we need directly in OWL quite straightforwardly addresses both issues. The well‐known absence of containers and sequences in OWL can be overcome by modeling classes in specific ways using specific design patterns such as those in Ciccarese et al. ( + 2008) and in Drummond et al. ( + 2006).
+There are multiple applications for the EARMARK approach. The most interesting for this article is its capability of dealing with overlapping structures in an elegant and straightforward manner. Under EARMARK, such structures do not need to be specified through complex workarounds as with XML, but they are explicit and can be easily described and accessed. Sophisticated searches and content manipulations become very simple when using this ontological model.
+The goal of this section is to demonstrate the soundness and applicability of EARMARK by discussing how the use cases presented earlier are addressed. Note that throughout the section we investigate multiple EARMARK data structures and documents, focusing on the feasibility and potentiality of such an ontological representation.
+The discussion about change tracking in office document formats showed that both ODT (OpenOffice format) and OOXML (Microsoft Word format) use complex data structures to store overlaps generated by change‐tracking functionalities. These structures make it very difficult to search and manipulate the content when using XML languages and tools. Even very simple edits generate a rather tangled set of overlapping elements.
+Let us recall the example mentioned earlier where the user “John Smith” splits a single paragraph into two. The ODT representation is:
+
+
The OOXML representation (shown earlier) is even more complex. In fact, these formats exploit in large scale (tangled) fragmentation (OOXML) or milestones and stand‐off markup (ODT) to deal with overlaps.
+EARMARK, on the other hand, stores overlapping data in a direct and streamlined manner that does not require tools to rebuild information from the twists of a tree‐based XML structure. The information already is available and expressed through consistent RDF and OWL statements. Figure + 3 graphically shows the corresponding EARMARK document.
+Encoding in EARMARK the ODT change‐tracking example. [Color figure can be viewed in the online issue, which is available at
+
The original paragraph content and the new string “also” are now encoded as two docuverses over which the ranges r1, r2, and r3 are defined. The original paragraph is then composed of the (content of) ranges r1 and r2 while the paragraphs resulting after the (text and carriage return) insertion now comprise range r1 and ranges r2 and r3, respectively. Metadata about the author and the modification date are encoded as further RDF statements.
+
+
The advantages of streamlining overlaps becomes apparent if we consider tasks a little beyond the mere display. For instance, the query for “the textual content of all paragraphs inserted by John Smith” ends up rather entangled if we used XPath on the ODT structure. The process for finding that textual content needs to browse the current version of the document and look for all the text:change‐start/text:change‐end pairs that refer to an insertion made by John Smith involving the creation of a new paragraph (i.e., text:change‐start is in a first paragraph while its pair, text:change‐end, is in the following one) that are either currently present in the document body or hidden behind a subsequent deletion made by someone else. Once the paragraphs are identified, we need to retrieve the content that originally was contained there (i.e., the text fragments that still are within those boundaries or that may have been deleted in subsequent versions). The following XPath represenst an implementation of this process:
+
+
The XML structure of an MS Word file, using segmentation rather than milestones, does simplify the query a bit, but still presents some radical complexities. The process starts by choosing all those w:p elements that were inserted by John Smith as well as all their previous and contiguous w:p elements that were deleted before or inserted after the first ones. In OOXML, each sequence of contiguous w:p elements implicitly represents one paragraph. Therefore, we can now take all the text fragments contained in each w:p sequence that were inserted before or deleted after the paragraph defined by the sequence itself. The following is the resulting XPath for an OOXML document.
+
+
The complexity of both XPath queries is due to the intrinsic complexity of the data structure on which the query has to work. Although the interface of OpenOffice or MS Word may provide tools to directly deal with these queries using specific strategies on the internal data structures, applications working directly on the XML structure have very little help in disentangling the mess of the data formats.
+On the other hand, since EARMARK documents are actually OWL files, it is possible to access and query them with plain Semantic Web tools. Powerful searches then can be performed without using niche‐specific tools or complex and long XPath expressions simply with mainstream technologies such as SPARQL (Garlik & Seaborne, + 2010).
+The corresponding SPARQL query for (“the textual content of all paragraphs inserted by John Smith”) therefore can be written as follows:
+
+
But EARMARK is useful for even more than querying: EARMARK also decreases the costs, in terms of efforts and lines of code, for manipulating documents.
+Let us consider the task of generating an intermediate version (i.e., neither the first nor the last one of a version chain) from a document that includes change‐tracking information about the whole document history.
+The process of rebuilding these versions by working on the XML structure without specific APIs is both complex and inefficient. For example, a basic XSLT that returns an XML document defining the desired version requires to at least:
+ define templates for all the elements actively involved in the change tracking (e.g., for ODT, text:changed‐region, text:change‐start, text:change‐end, and text:change and similarly for OOXML) to understand, by looking at their creation date, whether they must be considered or ignored when building the requested version. In particular, we must exclude insertions following and deletions preceding the version we are building; define templates for paragraphs to handle cases where the paragraph is the result of an insertion or a deletion of other paragraphs to identify whether it should be considered for the result and, in such case, finding out its real text content and remembering that in the following versions, such content may have spread out among other paragraphs; define templates for handling insertions/deletions for structures such as images, sections, lists, and tables; and define an identity template for the other elements to visit the entire document.
+
+
Even the most basic and incomplete implementation of such XSLT requires hundreds of lines of complex and convoluted code and a large number of ad hoc decisions based on the specificities of whether we start from ODT or OOXML. Note also that a Java‐based implementation (or in any other procedural language) of the same process would be equally or even more complex.
+The same result can be achieved on EARMARK documents with a few lines of Java code:
+
+
This approach uses the EARMARK Java API
+ 17
+
+
This method heavily uses Semantic Web technologies on the structures provided by EARMARK whose characteristics are always explicit and clear. In fact, since all versions coexist within the EARMARK document and each version can be encoded explicitly as a tree within the overall graph, this operation is straightforward and fast.
+EARMARK also can be exploited to improve semantic annotations. As noted earlier, there are in fact strong limitations in the same process of annotating web documents with semantic structures that overlap the structural ones. The same example—of vcards that cannot be created on the top of tables organized per rows—will be used in this section.
+We solve this by converting the web document with annotations into an EARMARK document, allowing both semantic and structural annotations to coexist. Through EARMARK, we can explicitly express both markup structures and vcard assertions. Figure + 4 shows how the vcard example can be modeled (Once again, we show a graphical representation for the sake of clarity.)
+The abstract model of the EARMARK document solving the microformats issue. [Color figure can be viewed in the online issue, which is available at
+
The textual content of the original table cells is now encoded in two different docuverses: one for the header (with roles) and one for the body (with names of committee members). Ranges r1, r2, …, r8 are then created to distinguish each role and name. Two independent and coexisting hierarchies are then built on top of the same set of ranges: the HTML table that includes one cell for each range (in blue) and the vcards about each person (in green) that include only the relevant ranges and overlap the previous one. Note also that the vcards are defined in such a way that does not interfere with the structural features of the table. The full linearization in OWL of this example can be found at
+
EARMARK can be used to improve wiki reversion mechanisms and overcome the limitations discussed earlier: The automatic filtering and merging of contributions from multiple versions of the same page are still a manual process, but it can be fully automatized if the overlapping structures buried in the whole history of the page become explicit.
+The role of EARMARK is to make those structures explicit and available for more sophisticated content manipulation. To understand the extent EARMARK structures can be derived from wikis and how they can be exploited by the final users, we use as our example the wiki platform MediaWiki
+ 18
+
+
MediaWiki offers sophisticated functionalities for creating diffs of wiki content. Users can compare any two revisions in the page history and highlight changes in a friendly interface that shows modifications with a word‐level granularity. Diff pages contain metadata about each compared version (when the version was created, who the author was, or which IP address an anonymous author was connected from, etc.) and a two‐column table showing the changes side by side. Changes are detected a posteriori by comparing two arbitrary versions, which are not even required to be temporally contiguous.
+The output of the MediaWiki diff engine has regularities that can be exploited to automatically build the overlapping structures of the diff and to express them in EARMARK. Let us consider a fictitious example summarized in Table + 1, where an initial text is revised three times by different authors.
+To display the differences between V1 ad V2, Mediawiki creates a page whose HTML code is as follows:
+ 19
+ For the sake of clarity, we removed all markup irrelevant to our discussion.
+
This is an HTML table of two rows, the first showing metadata (date and author of the modification), and the second showing the actual modifications. The first cell of the second row contains all the unmodified text and a del element for each inline fragment that was deleted. The second cell contains all the unmodified text and an ins element for each inline fragment that was inserted. Thus, these cells share exactly the same unmodified part(s) of the two compared versions.
+When the structure itself is modified rather than merely the text, the source code of the MediaWiki diff is slightly different. Thus, the diff between V3 and V4 (which splits a paragraph in two) is as follows:
+
+
The diff output is not complete or sophisticated, and of course, it is a completely different task to replan such an algorithm (but for a first idea of natural changes in diffing XML documents, see Di Iorio, Marchetti, Schirinzi, & Vitali, + 2009). Thus, limitations of that algorithm are inevitably shared by any EARMARK representation. Yet, this output is sufficiently rich to allow us to extract the overlapping information we need. For instance, the insertion of a nonbreaking space or a carriage return generates rows according to specific rules that can be easily detected to capture the actual change by the author.
+Figure + 5 shows the aforementioned example rebuilt in EARMARK. All versions are encoded in the same document by creating overlapping assertions over the docuverses. Metadata and RDF statements are layered on top of those assertions and create a rich knowledge‐base about the history of the documents and, in particular, about the history of each fragment.
+The wiki sample versions encoded in a single EARMARK document. [Color figure can be viewed in the online issue, which is available at
+
Due to the complexity of the example, we labeled arrows with numbers that indicate the position of each range within each markup item. Consider, for instance, Version V4: It is composed of two DIV elements, the first one containing the concatenation of “Bob was farming carrots” + “,” + “tomatoes” + “and” + “green” + “beans” + “.” and the second one containing the string “They are all tasteful.”
+Implementing a wiki content‐filtering mechanism on top of such a structure is rather simple. For instance, the removal of all the contributions of “Angelo Di Iorio” that leaves untouched all the content written (previously and subsequently) by “Silvio Peroni” and “Fabio Vitali” can be performed straightforwardly. Three steps are enough to apply such an intermediate content reversion:
+ the identification of the fragments written by “Angelo Di Iorio,” which is a straightforward SPARQL query on the embedded statements; the creation of a new version where references to those fragments are removed and references to fragments no longer in the document are correctly fixed; the translation of that document into an actual MediaWiki page through the serialization process described in Peroni and Vitali (
+ 2009).
+
+
Of course, an automatic process may generate ambiguities or even errors in the resulting content (i.e., some parts may become dangling, wrong, or unclear after removing text fragments elsewhere); grammar discrepancies also might be generated by the same approach. Linguistic and semantic problems, however, become a problem once the technical issues of managing independent, yet successive, edits are solved. What is important is that all the information about overlaps and dependencies among fragments is available in EARMARK and can easily be searched, filtered, and manipulated. Besides, foreseeing a manual intervention for checking and polishing automatically filtered content is perfectly in line with the wiki philosophy, so that the wiki community itself can wisely use the reversion tools to revise the content and adjust any intervening minor nuisances or imperfections. Such checks would still be far simpler and faster than would the manual process of partially reverting versions as we have today.
+Since we do not expect documents to be natively written in EARMARK or manually created by users, we need a way to extract EARMARK data structures from existing XML‐based resources, which is trivial when the XML is simple and clearly hierarchical and slightly more complex when the XML contains workarounds to force an intrinsically overlapping situation into a single hierarchy.
+We designed a reliable process to transform XML files into EARMARK documents that fully captures overlapping structures even when the overlaps are hidden in one the many well‐known workarounds. This approach takes as input an XML file and produces the corresponding EARMARK document in five steps: Read, Overhaul, Convert, Classify, and Organize (hence, the name ROCCO).
+Since ROCCO is not the main topic of this article, we very briefly discuss the issue of converting XML into EARMARK, explaining how each step works. The ROCCO algorithm performs five steps, described next.
+The first two steps consist of loading the XML source file and, if needed, adding information useful for further processing. In EARMARK, there is a clear distinction between the textual content of a document and the structures built on top of it: The content is stored as plain text—within docuverses—and all structures are externalized and expressed through OWL and RDF assertions.
+While OpenOffice stores all overlapping structures in the main document file, some other editors (e.g., MS Word) store overlaps in many different ways, even in a separate file. The overhaul step extracts such data and adds them to the main content document by exploiting format‐specific procedures, implemented via XSLT in most cases.
+The subsequent step consists of converting the XML source file into an early EARMARK document that expresses exactly the same information and hierarchies. No interpretation or disentanglement of workarounds is performed at this step.
+Since the input is XML, this translation can be performed directly via a generic XSLT stylesheet. It basically consists of a recursive algorithm that parses the source file and generates the corresponding instances in the EARMARK ontology. Such a translation is straightforward and not difficult.
+The “Classify” step extends the EARMARK document built so far with information about the workarounds used to encode overlaps. That information will be exploited in the subsequent steps to make those overlaps explicit.
+The basic idea is to exploit OWL reasoners to detect workarounds in an early EARMARK document D by:
+ defining an ontology O that models all the workarounds used by applications, such as milestones, stand‐off markup, etc.; these workarounds are specific to the data format used in the source document; specifying the EARMARK document D as an ABox for the ontology O; defining SWRL rules that capture the role of each element in D and check relationships between elements; running an OWL reasoner, such as Pellet, on D+O to create new OWL instances and properties that identify which workarounds are present.
+
+
The actual detection of workarounds is delegated to an external reasoner. Refining detection strategies and even adding new strategies for new formats all can be done via OWL and SPARQL. Indeed, tricky issues need to be addressed—mostly depending on the idiosyncrasies of the original formats—but no procedural code is required.
+The final step consists of building yet another EARMARK document that expresses the overlaps and metadata in an explicit way, based on the information collected by the previous steps. This phase consists of mapping operations from the native format into the EARMARK structure. Such conversion relies on the identification of metadata to classify the operations and to externalize relevant metadata in separate RDF statements.
+One of the most frequent criticisms when proposing a different approach to solving a well‐known problem in information and communication technology is that the new solution may simplify the difficulties of the specific problem, but brings with it hidden costs in terms of size of the data structure, computation efforts, or conversion restrictions that compensate the advantages. In our case, one of the anonymous reviewers of our article (Di Iorio, Peroni, & Vitali, + 2009) wondered whether a difference in file size could weigh in on the convenience of adopting EARMARK as opposed to working with the original files.
+As such, a discussion of cost functions of EARMARK versus other formats is in order. Yet, a systematic discussion of the relative costs (e.g., in byte size) of some original XML‐based data structures versus their EARMARK equivalent is an open‐ended undertaking that heavily depends on the original XML data structure and the specific features present in the document, and is badly defined anyway: While XML is a linearization format immediately expressible in actual bytes, OWL (or more precisely, RDF, the language in which OWL ontologies are expressed) is an abstract structure that allows a large number of linearization formats (including XML itself) with corresponding huge differences in the final byte counts.
+For these reasons, to provide at least an initial test of meaningful concepts, we selected two XML‐based data formats (OOXML and ODT) and, specifically, a set of documents where overlapping tricks were present (i.e., where change‐tracking was active). To bypass the size discussion, we decided not to test byte lengths (which are not meaningful and easily skewed, e.g., by reducing the string length of the element names or of the class names) but the number of nodes for XML documents and of triples for OWL documents. This comparison again is not particularly appropriate (Triples are naturally numerous in OWL ontologies, and it is customary to deal with hundreds of thousands and even millions of assertions in Semantic Web applications.) but closer to meaningfulness than is the mere byte count.
+Our comparison was carried on a small set of documents in ODT and OOXML that included change‐tracking information. As discussed in the previous sections, change‐tracking facilities generate rather complex overlaps even for basic operations on small text fragments, which in turn are expressed as a potentially huge number of standoffs and milestone markup within the XML hierarchy. The same documents were individually converted into EARMARK. We then charted how simple edits under change‐tracking affect the number of nodes in XML formats and of statements in OWL files.
+ 20
+ The full details about each version and each format also are available at
+
We created seven different versions, named after the “Seven Dwarfs” for recognizability, by applying very common edits (e.g., the insertion of few words, the deletion of some sentences, the split of a paragraph, etc.) on a small document, creating multiple overlaps. Figure + 6 shows the results of our comparison.
+A graph summarizing the results of the first experiment. [Color figure can be viewed in the online issue, which is available at
+
The overall trend is interesting and comforting: While in simple documents with no overlap the node count of XML is lower than is the assertion count of EARMARK triples, the presence of overlaps makes EARMARK and XML formats comparable. The growth of EARMARK statements is in fact very close to the growth of XML nodes when the number of overlaps increases. EARMARK is even more efficient than is XML for more complex documents.
+The measure for each format was done by counting only those nodes and statements instrumental to encode content and (overlapping) structures: We did not take into account either the presentational information for ODT and OOXML (Each file, for instance, includes a very long list of style definitions that are not relevant for the purposes of our analysis.) or namespace declarations (OOXML files, for instance, list all relevant namespaces for the Office toolkit.) or ignorable white spaces (that are only added to indent content and improve readability).
+Interestingly, EARMARK and ODT show a very similar increase in size while OOXML is much more verbose and grows faster. The content of the first version, for instance, is encoded using four nodes in ODT, 13 statements in EARMARK, and 54 nodes in OOXML; the last one contains 241 ODT nodes, 233 EARMARK statements, and 452 OOXML nodes. To return to our original inquiry, it is clear that the weight of EARMARK documents is very good compared to the other ones.
+Also note the regularity in the growth of EARMARK statements. Regardless of the actual modifications applied to the document, in fact, EARMARK adds about 40 statements for each edit. Both OOXML and ODT, on the contrary, show a more irregular “pace.” The reason for this is that EARMARK externalizes all assertions, so that all modifications (either to leaf nodes or to intermediate nodes in the original XML) are “flattened” onto the docuverses and do not depend on the complexity of the structure within which the edit took place.
+Figure + 7 shows the results of a similar comparison on a different set of documents and edits. We collected seven versions named after the days of the week and created by seven different authors when editing a very simple document. The overall trend does not change, and shows that EARMARK and ODT again have a comparable behavior, far better than that of OOXML.
+A graph summarizing the results of the second experiment. [Color figure can be viewed in the online issue, which is available at
+
In conclusion, although preliminary, this study shows clear trends of a very conservative behavior of EARMARK with respect to document size.
+Overlaps, far from being an obscure requirement for sophisticated functionalities of arcane markup languages, are a very frequent undertaking even in major data formats and in rather frequent situations. Yet, since the XML language does not allow them, consciously or not, designers of data formats have adopted a huge and entangled array of tricks, special cases, and workarounds that, although solving the actual problem of storing overlapping structures, open new and complicated ones when approaching even basic chores on documents containing them, such as queries.
+The EARMARK approach drastically reduces the efforts needed to perform such chores on overlapping structures since it does not allow the corresponding multiple trees to actually entangle and complicate the job. EARMARK is radically different from both special markup metalanguages that allow overlaps and the introduction of workarounds within the traditional tree‐oriented XML language because it treats multiple trees over the same content as first‐class citizens of the language, yet uses well‐known and standard W3C technologies and languages to perform all tasks. EARMARK documents, at the end, are OWL ontologies. Thus, any Semantic Web technology (e.g., SPARQL) can be used straightforwardly to perform operations on their content.
+Improving queries is not the only application of EARMARK. Validation is another interesting field that we are investigating. In fact, the same ontological framework can be used to prove properties concerning a document, such as validity against a schema, compliance to co‐constraint specifications, or adherence to structural patterns. Moreover, inspired by Marcoux and Rizkallah ( + 2009), in which they described an approach for defining natural‐language semantics for XML‐based languages, we also are developing an ontology‐based approach for encoding markup semantics—that is, the formal definition of meanings of markup elements, besides the syntactical structure of a markup document—within EARMARK documents.
+