Category Archives: TEI

More digital than thou

Posted on by

[16 December 2013]

An odd thing has started happening in reviews for the Digital Humanities conference: reviewers are objecting to papers if the reviewer thinks it has relevance beyond the field of DH, apparently on the grounds that the topic is then insufficiently digital. It doesn’t matter how relevant the topic is to work in DH, or how deeply embedded the topic is in a core DH topic like text encoding — if some reviewers don’t see a computer in the proposal, they want to exclude it from the conference.

[“You’re making this up,” said my evil twin Enrique. “That’s absurd; I don’t believe it.” Absurd, yes, but made up? no. The reviewing period for DH 2014 just ended. One paper proposal I reviewed addressed the conceptual analysis of … I’ll call the topic X, to preserve some fig leaf of anonymity here; X in turn is a core practice of many DH projects and an essential part of any account of the meaning of tag sets like that of the Text Encoding Initiative. I thought the paper constituted a useful contribution to an ongoing discussion within the framework of DH. Another reviewer found the paper interesting and thoughtful but found nothing specifically digital in the proposal [after all, X can arise in a pen and paper world, too, computers are not essential], graded it 0 for relevance to DH theory and practice, and voted to reject it. It should be submitted, this reviewer said, to a conference in [another field where X is a concern] and not in a DH conference. I showed this to Enrique. For once, he was speechless. Thank you, Reviewer 2!]

At some level, the question is what counts as work in the field of digital humanities. Is it only work in which computers figure in an essential role? Or is digital humanities concerned with the application of computers to the humanities and all the problems that arise in that effort? I think the latter. Some reviewers appear to think the former. If solving an essential problem turns out to involve considerations that are not peculiar to work with computing machines, however, what do we do? I believe that the problem remains relevant to DH because it’s rooted in DH and because those interested in doing good work in DH need the answer. The other reviewer seems to take the view that once the question becomes more general, and applicable to work that doesn’t use computers, the question is no longer peculiarly digital, and thus not sufficiently digital; they would like to exclude it from the DH conference on the grounds that it addresses a question of interest not only to DH but also to other fields.

[“Wait a second,” said Enrique. “You’re saying there’s a pattern here. Isn’t it just this one reviewer?” “No,” I said. “This line of thought also came up in at least one other review (possibly in a more benign form), and it has also been seen in past years’ reviews.” “Is that why you’re so touchy about this?” laughed Enrique. “Did they reject one of your papers on this account?” “Oh, hush.”]

The notion that there must be something about a topic that is peculiar to the digital humanities, as opposed to the broader humanities disciplines, makes sense perhaps if one believes that the digital humanities are intrinsically different from the pre-electronic humanities disciplines. On that view, any DH topic is necessarily distinct from any non-DH humanities topic, and once a topic is relevant to the broader humanistic fields (e.g. “what is the nature of X?”), it is ipso facto no longer a DH topic.

This is like arguing that papers about the concept of literary genre don’t belong at a conference about English literary history, because there is nothing peculiarly English about the notion of genre and any general discussion will (if it’s worth anything at all) also be relevant to non-English literatures. Or like trying to exclude work on the theory of computation from computer science conferences because it applies to all computation, not only to computation carried out by electronic stored-program binary digital computers.

[“I notice that leaders in the field of computer science occasionally feel obliged to remark that the name computer science is a misnomer,” said Enrique, “because computers are in no sense an essential element in the field.“ ”Perhaps they have the same problem, in their way,” I said. “My sympathy,” said Enrique.]

Another problem I see with the view my co-reviewer seems to hold is that some people believe that DH is not intrinsically different from the pre-electronic humanities disciplines. I didn’t get involved with computers in order to stop doing traditional philology; I got involved to do it better. Computers allow a much broader basis for our literary and linguistic argumentation, and they demand a higher degree of precision than philologists typically achieved in past decades or centuries, but I believe that digital philology is recognizably the same discipline as pre-digital philology. If the DH conference were to start refusing papers on the grounds that they are describing work that might be relevant to scholars working with pen and paper, then it would be presupposing an answer to the important question of how computers change and don’t change the world, instead of encouraging discussion of that question. (And it would be excluding people like me from the conference, or trying to.)

[“Trying in vain, I bet,” said Enrique. “Well, yeah. I’m here, and I’m not leaving.”]

These exclusionary impulses are ironic, in their way, because one of the motive forces behind the formation of scholarly organizations like the Association for Computers and the Humanities and the Association for Literary and Linguistic Computing (now the European Association for Digital Humanities) and behind their journals and conferences was that early practitioners in the field were often made to feel unwelcome in the conferences and journals of their home disciplines. The late Paul Fortier was eloquent on the subject of the difficulties he had trying to publish his computer-assisted work on Céline in conventional journals for Romance languages and literatures; he often spoke of Computers and the Humanities as having made it possible for him and others like him to have an academic career.

It will be a sad thing if the recent growth in degree programs devoted to digital humanities turns out to result in the field of DH setting up signs at all boundaries reading “Outsiders not wanted.”

Searching for patterns in XML siblings

Posted on by

[9 April 2013]

I’ve been experimenting with searches for elements in XML documents whose sequences of children match certain patterns. I’ve got some interesting results, but before I can describe them in a way that will make sense for the reader, I’ll have to provide some background information.

For example, consider a TEI-encoded language corpus where each word is tagged as a w element and carries a pos attribute. At the bottom levels of the XML tree, the documents in the corpus might look like this (this extract is from COLT, the Bergen Corpus of London Teenage English, as distributed by ICAME, the International Computer Archive of Modern and Medieval English; an earlier version of COLT was also included in the British National Corpus):

<u id="345" who="14-7">
<s n="407">
<w pos="PPIS1">I</w>
<w pos="VBDZ">was</w>
<w pos="XX">n't</w>
<w pos="JJ">sure</w>
<w pos="DDQ">what</w>
<w pos="TO">to</w>
<w pos="VVI">revise</w>
<w pos="RR">though</w>
</s>
</u>
<u id="346" who="14-1">
<s n="408">
<w pos="PPIS1">I</w>
<w pos="VV0">know</w>
<w pos="YCOM">,</w>
<w pos="VBZ">is</w>
<w pos="PPH1">it</w>
<w pos="AT">the</w>
<w pos="JJ">whole</w>
<w pos="JJ">bloody</w>
<w pos="NN1">book</w>
<w pos="CC">or</w>
<w pos="RR">just</w>
<w pos="AT">the</w>
<w pos="NN2">bits</w>
<w pos="PPHS1">she</w>
<w pos="VVD">tested</w>
<w pos="PPIO2">us</w>
<w pos="RP">on</w>
<w pos="YSTP">.</w>
</s>
</u>

These two u (utterance) elements record part of a conversation between speaker 14-7, a 13-year-old female named Kate of unknown socio-economic background, and speaker 14-1, a 13-year-old female named Sarah in socio-economic group 2 (that’s the middle group; 1 is high, 3 is low).

Suppose our interest is piqued by the phrase “the whole bloody book” and we decide we to look at other passages where we find a definite article, followed by two (or more) adjectives, followed by a noun.

Using the part-of-speech tags used here, supplied by CLAWS, the part-of-speech tagger developed at Lancaster’s UCREL (University Centre for Computer Corpus Research on Language), this amounts at a first approximation to searching for a w element with pos = AT, followed by two or more w elements with pos = JJ, followed by a w element with pos = NN1. If we want other possible determiners (“a”, “an”, “every”, “some”, etc.) and not just “the” and “no”, and other kinds of adjective, and other forms of noun, the query eventually looks like this:

let $determiner := ('AT', 'AT1', 'DD',
'DD1', 'DD2',
'DDQ', 'DDQGE', 'DDQV'),
$adjective := ('JJ', 'JJR', 'JJT', 'JK',
'MC', 'MCMC', 'MC1', 'MD'),
$noun := ('MC1', 'MC2', 'ND1',
'NN', 'NN1', 'NN2',
'NNJ', 'NNJ2',
'NNL1', 'NNL2',
'NNT1', 'NNT2',
'NNU', 'NNU1', 'NNU2',
'NP', 'NP1', 'NP2',
'NPD1', 'NPD2',
'NPM1', 'NPM2' )

let $hits :=
collection('COLT')
//w[@pos=$determiner]
[following-sibling::w[1][@pos = $adjective]
[following-sibling::w[1][@pos = $adjective]
[following-sibling::w[1][@pos = $noun]
]]]
for $h in $hits return
<hit doc="{base-uri($h)}">{
$h,
<orth>{
normalize-space(string($h/..))
}</orth>,
$h/..
}</hit>

Such searches pose several problems, for which I’ve been mulling over solutions for a while now.

  • One problem is finding a good way to express the concept of “two or more adjectives”. (The attentive reader will have noticed that the XQuery given searches for determiners followed by exactly two adjectives and a noun, not two or more adjectives.)

    To this, the obvious solution is regular expressions over w elements. The obvious problem standing in the way of this obvious solution is that XPath, XQuery, and XSLT don’t actually have support in their syntax or in their function library for regular expressions over sequences of elements, only regular expressions over sequences of characters.

  • A second problem is finding a syntax for expressing the query which ordinary working linguists will find less daunting or more convenient than XQuery.

    Why ordinary working linguists should find XQuery daunting, I don’t know, but I’m told they will. But even if one doesn’t find XQuery daunting, one may find the syntax required for sibling searches a bit cumbersome. The absence of a named axis meaning “immediate following sibling” is particularly inconvenient, because it means one must perpetually remember to add “[1]” to steps; experience shows that forgetting that predicate in even one place can lead to bewildering results. Fortunately (or not), the world in general (and even just the world of corpus linguistics) contains a large number of query languages that can be adopted or used for inspiration.

    Once such a syntax is invented or identified, of course, one will have the problem of building an evaluator for expressions in the new language, for example by transforming expressions in the new syntax into XQuery expressions which the XQuery engine or an XSLT processor evaluates, or by writing an interpreter for the new language in XQuery or XSLT.

  • A third problem is finding a good way to make the queries faster.

    I’ve been experimenting with building user-level indices to help with this. By user-level indices I mean user-constructed XML documents which serve the same purpose as dbms-managed indices: they contain a subset of the information in the primary (or ‘real’) documents, typically in a different structure, and they can make certain queries faster. They are not to be confused with the indices that most database management systems can build on their own, with or without user action. Preliminary results are encouraging.

More on these individual problems in other posts.

XForms and XQuery tutorials at TEI members’ meeting

Posted on by

[23 August 2010]

The TEI has published a list of workshops to be offered at the TEI Members’ Meeting this November in Zadar, Croatia.

Together with Syd Bauman of Brown University, I’m offering two tutorial workshops: one on XForms and one on XQuery. Each will last a day and a half, and involve some talking heads, some group discussion, and as much hands-on work as we can manage.

There are several other very good workshops on offer: Norm Walsh on XProc, the TEI@Oxford team on the ODD system, Elena Pierazzo and Malte Rehbein on the encoding of genetic editions, and Andreas Witt et al. on TEI for transcriptions of speech.

The organizers remind me that there is an early-bird discount for those who register before 31 August. There is some chance that tutorials which fail to attract enough participants will be canceled if they don’t get enough registration, so if you definitely want to come, you definitely want to register early, to help make sure your tutorial has enough registrants to make the cut.

Bare-bones TEI

Posted on by

[6 January 2010]

The markup vocabulary defined by the Text Encoding Initiative is, for good and sound reasons, rather large and in some ways rather complicated. From time to time, however, it’s useful to have a radically cut-down version of the TEI vocabulary. For people just learning TEI markup (to name just one instance), a cut-down version can simplify initial training and make the TEI feel a little less intimidating.

Many people use TEI Lite, which is much smaller than full TEI. But for some work I’m doing with Yves Marcoux and Claus Huitfeldt, we felt it would be handy to have an even smaller subset of TEI to work with. Years ago (1994), I defined a profile of TEI called ‘Bare Bones TEI’, intended not so much for serious use as for training and for thought experiments.

I had long regarded the full details of bare-bones TEI as lost to history: the documentation has been preserved by Robin Cover at the XML Cover Pages, but it didn’t include the DTD modification file showing the precise changes from the then-current TEI DTD. I had tried a few times, searching both the Web and my hard disk, to find the original data, but had had no luck. But the other day, for reasons I don’t think I can explain, an attempt at one more search eventually found an old copy of the documentation, the modification files, and the full DTD for bare-bones TEI.

I’ve now translated the original documentation to XML, added updated links to the various DTD files, and added parameter entities to the DTD to make it work both for SGML contexts and for XML. The documentation for Bare bones TEI is now available on this site, as is the modified DTD.

The current version of the bare-bones DTD is based on TEI P3, and the translation to XML loses a small amount of information involving the pb (page break) element. Eventually, perhaps, I’ll apply the bare-bones customization to TEI P5 and produce an updated version of the schema and documentation.

Note: in searching for Bare-bones TEI on the TEI Consortium site just now, I discovered that someone has produced a similar profile, called ‘Bare TEI’. [Further research shows that although the page on customizations does not identify the authors, the work was done originally by Laurent Romary and later edited by Syd Bauman, Lou Burnard, and possibly also Sebastian Rahtz.] This may be worth exploring, for those seeking a minimal profile of TEI. Unfortunately, I haven’t found any usable documentation for Bare TEI, so I don’t know the design principles that govern it, and the DTD is full of undefined ghost elements (or ‘zombies’), which renders it unfortunately cumbersome in a syntax-directed editor. So for now I’m sticking with bare-bones TEI.