Working on your Spanish

[18 November 2017; copy edits 18-20 November]

In just over seven months, the annual Digital Humanities conference will take place in Mexico City. I doubt that I’m the only digital humanist in the world who is surreptitiously trying to improve my Spanish before next June.

If you are trying to improve your Spanish (whether for that reason or for others), here are some things I am finding useful.

First, a textbook.

  • The French publisher Assimil has a Spanish volume in their sans peine series: L’espagnol sans peine. It’s available in several versions for speakers of different languages: English (Spanish with ease), German (Spanisch ohne Mühe), Italian, Dutch, and Portuguese. The series is described, quite accurately, as suitable for “beginners and pseudo-beginners” (débutants et faux-débutants).

    I bought the book and CDs direct from the Assimil web site and had them shipped to me in the U.S. without any difficulty; the only catch for some potential buyers is that the site is in French. Period. (Wait, aren’t they specialists in books for foreign-language learners? Don’t their web people realize the firm has non-Francophone target readers? Ah, well. There are some English-language resellers one can use, or you can grab a Francophone friend and make them babysit you through finding the book you want and making your purchase.)

    I won’t attempt to describe the Assimil method here. The linguist John McWhorter has given a good account of their results in his piece on the NPR web site (and there is some useful concrete advice at the site ‘How to Learn any Language’, for those who find the books’ instructions vague). I will say that for those who like me are attempting to learn (or re-learn) a language on their own and not in a class, the Assimil sans peine series has no equal that I know of.

    I bought the combination pack with the printed book, sound CDs with recordings of the dialogues, and a CD with MPEG versions of the recordings; I have downloaded the MPEG recordings to a directory on an Android tablet, imported the directory into the Podcast Addict app as an audio book, and I use Podcast Addict to play the day’s recording on continuous loop while I fetch the paper, wash dishes, etc. Listening in a podcast player has the advantage that I can speed up the early lessons to something approaching normal conversational speed.

Second, a supply of relatively easy reading and listening material. My searches for podcasts for learners of Spanish as a foreign languages turned up large numbers of results, some of which were obviously irrelevant and some of which I was able to delete without qualms after listening for a couple of minutes. I now listen to three:

  • Español automatico, prepared by a personable teacher of Spanish named Karo Martínez; in some installments, she speaks in relatively simple Spanish about assorted topics (the relative merits of the various actors who have played James Bond, the history of Catalonia, how to learn Spanish, and others; self-help topics of the how-to-be-more-organized variety are not uncommon), in others she specifically discusses issues of Spanish idiomatic usage and vocabulary. Episodes generally range from twenty to forty minutes (although when she got going on her introduction to the history of Barcelona, it ran over an hour). Transcripts and other additional materials appear to be available from the web site, some for purchase, which I hope pays for the cost of producing and publishing the podcast, and others for free (but you have to give them your email address, which induces a spasm of irrational privacy paranoia in me, so I have no idea what form the transcripts take). The only blemish for me is the repeated plea for listeners to file a five-star review of the podcast in iTunes.

  • El oso latino habla español — para mejorar su español, a quirky podcast put together in Sherbrooke, Québec, produced by a Québecois Spanish-learner named Pascal Dion and featuring the Peruvian Oswaldo Horna Montes (known, I gather, to his friends as El oso latino) as the main speaker. Episodes often include interviews with visitors from Latin America, with other Latin Americans living in Sherbrooke, or with anyone whom the host and producer think will be interesting; topics of discussion regularly include differences among varieties of Spanish and idioms peculiar to this or that regional variety. In one show, Montes narrated the preparation of a Peruvian dish for dinner. Dion appears in a regular segment on language-learner errors called Crónica del gringo, and Montes’s daughters in the segment Los chistes de Celia y de Marisol, which makes me laugh til I cry even though I have not yet understood a word of any of the jokes they have told. Lots of music. The most personal and thus the most memorable of these three podcasts. Generally around 30 minutes per episode.

  • News in slow Spanish, which is more or less what it sounds like: a weekly podcast with news stories in Spanish (there are similar News in Slow X podcasts for a number of different languages). There are several versions (intermediate vs. advanced, Castilian vs. Latin American), but oddly only one that I was able to locate from within Podcast Addict (Spain, Advanced, which proves not too advanced for me). News in Slow Spanish (Latino) does appear in the Tune-In Radio app (the only thing I miss there is continuous looping).

    I shied away from this at first; I have decades-old bad memories of unconvincing ‘newscasts’ specially for language learners filled with soft news (to give them a longer shelf life) and painful explanations of words. But I ended up trying the podcast, after I failed to find an app or podcast that would give me a conventional five- to fifteen-minute radio news broadcast once a day or once an hour (something along the lines of the NPR News app, or the Radio Canada app — still looking; if you know of anything let me know). And I was convinced. The news items are real, current, and interesting, and the editorial comment feels lively and intelligent. In the Latin-American version, particularly, it is interesting to listen to the friendly discussion between hosts with slightly different political leanings.

    I’ve been listening to the initial free portion of the podcast; a longer episode with more news items and some discussions of Spanish grammar and idioms is available as a paid service. The free portion runs about five minutes. (Every now and then there is either a slip or an intentional freebie, and the entire thirty-minute program is included.)

On easy reading material, I’m not doing too well. Children’s and young-adult books are an obvious choice, but I don’t know an easy way to know what’s worth buying and what’s not. Well, actually I do. The next time I’m at my public library I’ll ask at the information desk for recommendations.

And third, a supply of normal Spanish for listening and reading, ideally interesting and not over-challenging. Here, of course, the choice is limitless and the expanse of possibilities feels as trackless as Borges’s Library.

There is always the news (which tends to have a relatively manageable vocabulary, and to have a lot of short pieces). There are Android apps for any number of Spanish-language newspapers, most of which I haven’t heard of and some of which may or may not be worth reading. With my mind focused on Mexico City, I have looked only at apps and web sites for Mexican newspapers. A Mexican colleague (whom I thank, but who shall remain nameless here because I haven’t asked permission to name them here) has suggested:

  • Animal politico (left wing); the web site is fine on a desktop machine and a bit hard to navigate on a tablet. I didn’t find any app.
  • El Universal (center right); I did find an app but did not find it usable.
  • La Jornada (left wing); I find the Android app usable (though I wish it allowed me to adjust the font size), so I haven’t worked with the web site.

At the moment, I confess to finding Mexican newspapers slightly heavy going.

Since I’m a digital humanist, and I’m looking forward to DH 2018, I read the blog run by the Red de humanidades digitales with great interest, even if sometimes with imperfect comprehension.

Eventually, I’ll be looking for Spanish-language detective stories and the like: page-turners are a real boon for a foreign-language learner, so I will happily read many things in a foreign language whose English equivalents I wouldn’t normally be caught dead with. (I’m told that on the same principle, some adult literacy programs in this country do great work with Mickey Spillane.) Suggestions welcome.

Finding good podcasts aimed not at language learners but at intelligent adults has been a challenge, but looking around for podcasts on the sites of UNAM (Universidad Nacional Autónomo de México) and IMER (Instituto Mexicano de la Radio) has produced dividends, as have some journalistic think pieces I found on the Web on new media in Latin America. Right now my subscriptions include the following. (At the moment, all of these are tough sledding for me, but they repay repeated listening. The ability to slow the playback helps — it’s like being able to say “¡Demasiado rápido! ¡Más despacio, por favor!” and have the podcast nod and slow down.)

  • Azul Chiclamino, a podcast by Rodrigo Llop. I have no idea how to describe this; perhaps the subtitle will do: La realidad de lo absurdo. This is sometimes characterized as a humorous broadcast; for a sufficiently nuanced definition of “humor” (think Mark Twain) that’s probably true, but I find the podcast much more appealing than that label would lead me to expect.
  • Radio Ambulante, an NPR-affiliated podcast. Feels a bit like Radio Lab or This American Life, in Spanish: thoughtful, serious, well produced. Has the advantage that its stories are often about Hispanic affairs in the US, so I understand some of the background; has the drawback that its stories are often about the US, so I’m not learning about Latin America.
  • Ráfagas de pensamiento, a series of short pieces by the philosopher Ernesto Priani Saisó of UNAM, often reacting to a passage in an earlier philosopher (Nietzche, Husserl, Leibniz, More, …). Produced with atmospheric music and read by what sound like professional voice actors. Has the disadvantage for me that the background music can make the words harder to hear; has the advantage that it’s worth listening to. Usually 3-5 minutes per piece.
  • A multipart dramatization by Radio UNAM of Así asesinaron a Trotski by Leandro Sánchez Salazar (the man in charge of the investigation of Trotsky’s murder). I have no idea what Sánchez’s book is like as a historical source, but it has the virtue of strong narrative drive (even though I already know how it turns out). I may need to read the book in order to understand some of the broadcast.

Netflix and Amazon appear to have rather thin selections when it comes to Spanish-language films but they do have some. If anyone reading this knows an effective way to search by language on either, I’m all ears; surely searching for “Almodovar” should not be the only possibility (I am going to save Buñuel for later, when my Spanish is better and I can tell the difference between surrealism and not understanding the words). YouTube has a fair bit of Spanish content, though again I have not found any good way to find it except for searching on random Spanish words. An impulsive search on “Así asesinaron a Trotski” turned up several documentaries on Trotsky’s assassination, Trotsky’s life, Trotsky and Stalin, and Ramon Mercader (the man who killed Trotsky), as well as a few seminars on Trotskyite political theory.

[Addendum: on Netflix, selecting Browse / Audio & Subtitles takes the user to an interface where one can browse items with audio, or subtitles, in a given language. This is imperfect, but probably better than nothing. Looking for something to watch in the resulting display feels like looking for a book to read in a library arranged by color; for every ten times you feel irritated by its apparently random arrangement and the inconvenience of having to click on something every time you would like more information than is given in the icon, you may once or twice feel pleased by some serendipity.]

All of this is, of course, just my two cents. As may be clear from the above, my language learning work happens mostly on an Android tablet, not on a desktop machine.

Running the BaseX XQuery engine in the OpenShift cloud platform

[7 January 2016]

Late last year I worked through the process of making the BaseX XQuery engine run under Tomcat 6 at a commercial Java hosting provider. As I mentioned in that post, I also spent some time trying to make a cloud-services solution work, using OpenShift and Andy Bunce’s excellent Openshift quick start for BaseX. I ran into trouble then, because the Red Hat Cloud (rhc) tools refused to install on either of my two machines because the operating systems were more than twelve months old.

But during a quiet day or two last month I downloaded a new operating system for the newer machine (I’m exaggerating; it probably only took sixteen hours or so), and the other day I tried again.

The instructions for the quick-start set up are a bit terse, so for future reference (and for the benefit of any other XQuery developers out there who would like a little more detail in the instructions), this is my checklist for next time. Many of the same considerations apply as for installing BaseX under Tomcat; see the earlier post for more discussion.

Prerequisites

The following checklist assumes that:

  • You are reasonably comfortable with command-line tools.
  • You are reasonably comfortable with git, or can copy and paste git incantations without mucking them up.
  • You know what the basexhttp script is (or can live without knowing all the details of how things work).
  • It may be helpful if you’re familiar with ssh and scp, but it’s not essential.

If these don’t all apply, you will need to make some adjustments on the fly.

Preliminary preparation

  1. Sign up for an account at OpenShift.

    I don’t remember how long this took, maybe an hour. (It would go faster if it were easier to find the actual signup page and if one didn’t read the agreements one is consenting to.)

    I went through the “log in to the console and create an app” tutorial at OpenShift and “created” my first application (by clicking a button). I didn’t find this helpful or instructive, but YMMV.

    Remember your userid and password; you will need them repeatedly.

  2. Install the OpenShift / Red Hat Cloud command-line tools (rhc); instructions are on the Getting Started with OpenShift Online page at OpenShift.

    This also takes a little while, to download and install all the libraries on which the rhc command-line tools depend. My recollection is that it took half an hour or so. I expect it’s faster for people with faster connections than mine.

    (This is where things went terribly wrong for me in November, since Red Hat’s command-line tools refused to be installed in an operating system that shipped two years ago. Trying to solve the problem by upgrading the system’s Ruby interpreter landed me in dependency hell.)

  3. Decide in advance what userids you wish to specify for your BaseX database(s); for each userid specify the initial password and the initial permissions for each user. At the very least, determine what userid and password will be used as the default database user for the app.

    To allow myself to undertake each database operation with the lowest feasible level of privilege, I made myself an array of userids, one for each privilege level, and assigned them passwords generated by a simple random process:

    • Angie (admin privileges)
    • Chris (create privileges)
    • Will (write privileges)
    • Ralph (read privileges)
    • Nadine (no privileges)
  4. Prepare in advance a small XML file or two to use in creating a helloworld database to check that things are running as you expect.

    I use the documents in http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/ for making a helloworld example. When I use the REST interface to create a database and populate it, it’s simplest to retrieve the documents by URI; when I use the dba interface of BaseX 8.3.1, it proves simplest if I have copies of them on my hard disk.

Doing the deed

The basic instructions are all given in the quick-start readme file; they are, perhaps, a little terse there, for readers who haven’t worked with OpenShift before, so I’ll repeat them with some comments. We are going to create a do-nothing OpenShift application, copy Andy Bunce’s BaseX quickstart setup into it, and check it in. For concreteness, I will assume the app to be built is named “Allegheny”, and the OpenShift user’s domain is “AIK”.

  1. rhc app create -a Allegheny -t diy-0.1

    This creates a do-nothing app named Allegheny on the OpenShift server, using the diy-0.1 cartridge, then clones a git repository of the code into a new directory called Allegheny, on your hard disk. The DIY cartridge provides a sort of minimal environment for an app, on a do-it-yourself basis; fortunately for us, we do not have to do it ourselves, as Andy Bunce has done all the crucial bits for us.

    When I did it this morning, this step took a little over two minutes.

  2. cd Allegheny

    Move into the app directory.

    Do not omit this step; you will regret it, especially if you use git yourself as a backup or synching mechanism. (That is to say, when I forgot to do this, the next step put all of the quickstart code into my home directory. It took me a painfully long time to find and delete it all and get it back out of my git repository. And my .gitignore file seems to have lost both its content and its history. Fortunately, I do have backups.)

  3. git remote add upstream -m master https://github.com/Quodatum/openshift-basex-quick-start.git

    git pull -s recursive -X theirs upstream master

    Pull the quickstart code into your app. (If you want to understand in detail what these lines do, I recommend the git manual. I am not going to try to explain.)

    When I ran this this morning, it took a little under two minutes; it seemed longer.

    If you’re in a hurry, go ahead to the next step. Otherwise, now may be a good time to pause to look at the application’s directory structure, since it’s where you will be doing your development. The crucial bits appear (in the current state of my imperfect knowledge) to be:

    • config (A bash script which sets some variables to be used in other bash scripts, including BASEX_USER and BASEX_PASSWORD, which are used as the values of the -U and -P options in calls to the basexhttp script. The URI of the version of BaseX to install is also given here, as are the port numbers the server should use.)

    • .openshift/ (part of the OpenShift infrastructure)

      • action_hooks/ (this is where the application developer places code to be executed by the OpenShift infrastructure at predefined moments)

        • start (a bash script to be executed when the application needs to be started; the script provided by the quickstart calls basexhttp start)

        • stop (a bash script to be executed when the application needs to be stopped; the script provided by the quickstart calls pkill java to stop the server; in principle it would rather call basexhttp stop, but at the moment that doesn’t work)

      • cron/ (Directory for cron jobs to be executed on behalf of the application, with subdirectories hourly, daily, etc.; the README file has a pointer to the relevant OpenShift documentation. The weekly subdirectory appears to have a weekly statistics job in it.)

      • markers/ (I have no idea what this is; the README points to the documentation.)

    • basex/ (For XQuery developers, this is where the action is.)

      • lib/ (Contains a JAR file for Saxon HE 9.7.0.1.)

      • repo/ (The BaseX repository; contains XQuery modules installed using REPO INSTALL URI-of-module; for details see the BaseX wiki under Repository.)

      • webapp/

        • restxq.xqm/ (A sample RESTXQ function by Andy Bunce supplied as part of the quickstart framework; it generates the default welcome page.)

        • WEB-INF/ (Contains the configuration files governing BaseX’s behavior as a web service)

          • jetty.xml

            (Jetty-specific configuration. With any luck you’ll never need to change anything here. The BaseX documentation refers readers to the Jetty documentation for details, which in turn suggests that you consult the Jetty 9 documentation instead, and good luck to you if you actually need to understand the details. The salient item for the quickstart is setting the IP address and port for the server, using the environment variables OPENSHIFT_DIY_IP and OPENSHIFT_DIY_PORT.)

          • web.xml

            (Generic servlet configuration. By default, the quickstart has the REST and WebDAV interfaces turned off and the RESTXQ interface turned on; you’ll need to edit the web.xml file to turn the REST and WebDAV interfaces back on. This document also specifies the default user and password for the server; how the specification in the web.xml file relates to the values given by the -U and -P options passed to basexhttp remains a mystery to me.

            Since the default web.xml file for the quickstart does not set the RESTXQPATH or RESTPATH options, they default to “the standard web application directory”, which appears in this case to be the webapps directory in the root directory of the git repository. That would be consistent with the placement of restxq.xqm. The web.xml file also doesn’t specify the REPOPATH option; the documentation says that it defaults to {home}/BaseXRepo, but apparently {home}/repo is also a possibility; that would be consistent with the placement of the repo/ directory here.)

    The other files and directories either have obvious functions (.git, .gitignore, LICENSE, README.md) or appear to be just samples (diy/, misc/).

  4. You can now adjust things in the configuration, if you wish. I mostly wait until later.

  5. git push origin master

    This checks in your changes on the server. The OpenShift infrastructure will tell the running DIY application to shut down (i.e. it will run the script in the checked-in version of .openshift/action-hooks/stop), push the changes for your local git repository to the copy in the cloud, and restart the app (by running .openshift/action-hooks/start).

    When I ran it this morning, this step took just under three minutes, including fetching and installing the BaseX binary.

    (In my initial experiments, I ran into a problem here; there is some disagreement between BaseX and OpenShift regarding who can connect to what ports, and the result is that basexhttp stop doesn’t have the desired effect, which means the check-in fails. In the meantime, Andy Bunce has rewritten the quickstart code with a temporary workaround.)

  6. Set up userids.

    BaseX can now be configured, using the dba application that ships with BaseX 8, and which is conveniently linked from the default quickstart welcome page at http://allegheny-AIK.rhcloud.com/.

    The only essential configuration to do at this point is to change the admin password. I try to do this quickly, since until it is done, anyone who happens across this BaseX engine on the open Web can have admin privileges just by logging in using the default password.

    There are a couple of catches that make the process very slightly less straightforward than it might be.

    • The users.xml file will go into the OpenShift data directory, which means that you cannot conveniently put it in place before starting BaseX. (If inconvenience is no object, then by all means, be my guest: first prepare a users.xml file with another copy of BaseX, then scp it to the ./app-root/data/basex/data directory of your OpenShift app, before doing the push in the previous step.)

    • The config file needs to know the admin userid and password, in order to start and stop the server. (At least, I think it needs to know them; I haven’t actually tried putting gibberish into those variables to see.) If we change the admin password, we risk not being able to stop and start the server gracefully.

    This is the technique I’ve worked out for dealing with these catches. There are probably better ways.

    • Log in to the dba application using the default admin userid and password.
    • Add the users in the list prepared earlier, with the corresponding privileges and passwords. Note that this list includes a second user with admin privileges, here called Angie.
    • In the local repo, change the relevant parts of the config file to read as follow (substituting your userid and password of choice, of course):
      BASEX_USER="Angie"
      BASEX_PASSWORD="Where-is-the-devil-in-Evelyn-what's-it-doing-in-Angela's-eyes"
    • Check in your changes and push them to OpenShift (git add config; git commit -m "Changing userid and password"; git push origin master).
    • Log in to the dba application again (actually, you’re probably still logged in), and change the password of the admin user. At this point, we have closed the window of vulnerability we opened when we started the server with the default admin password. When I’m feeling paranoid, I take this moment to check the Databases, Users, Files, and Logs tabs to see whether any intruders actually showed up and did anything.
  7. Test

    To test that things are going as expected, I also create a database and do a few queries with the dba application and the REST interface.

So now I know three different ways to have access to an XQuery database over the web:

  • Run it as a server on a machine you control, or on which you can persuade the sysadmin to install it. (I assume that this can be done in the virtual private servers offered by many Web hosting providers, but I haven’t done it that way myself.)
  • Run it as a servlet running under Tomcat on a Java hosting provider.
  • Run it as an application in a cloud service.

My personal experience with these is all with BaseX, but of course all three methods will also work, at least in principle, for other XQuery engines like eXist and MarkLogic.

It’s always good to have more than one string to one’s bow.

Running the BaseX XQuery engine under Tomcat

[30 November 2015; notes on Tomcat 7 and BaseX 8.3.1 added 7 January 2016]

For a long time I have wished to have an XQuery engine I could use as a back end for Web applications.

Once upon a time, one could use 28msec’s Sausalito platform for this, but not since 28msec stopped supporting it. (In theory, one can still use their new tools to build XQuery-based applications, but I was not able to figure out how, the last time I tried.) But as I wrote when I first used Sausalito, “having an XML database on demand is a lot like having running water on demand — those who have never had it may think it’s a luxury anyone should be able to live without, but once you’ve had it, it can be hard to go back.”

I spent a couple days this past week trying again to choose among the various options (an XQuery server running in a shared hosting environment, running in a dedicated virtual private server, running in the cloud) and succeeded, after two days of work, in getting BaseX running under Tomcat at a Java hosting provider. (If you’re looking for Java hosting, I can say that I have been happy at Kattare.com.) Some of those two days went to attempting to make another solution work (but OpenShift appears to be unwilling to install its command-line tools on either of my current operating systems, and my network connection appears to be unequal to the task of downloading an OS upgrade for the sake of making OpenShift’s installation software happy), and some went to missteps.

If I could have spent time only on the things that turned out to work, it would only have taken a couple hours.

For the record, and for the benefit of any other XQuery developers out there who face the task of getting an XQuery engine running in a commercial hosting environment, this is the checklist I am making for myself for next time.

Prerequisites

The following checklist assumes that:

  • Your Java hosting provider provides you with Tomcat.
  • Your Java hosting provider provides you with ssh access.
  • You know how to ssh into your hosting provider and are comfortable using bash or another shell.
  • You have and can run a copy of BaseX installed on your local machine, which is the same version of BaseX as you are going to install in Tomcat. (Warning: if you are running a current BaseX locally, you may need to install a back-level BaseX for preparatory step 3 below.) It’s helpful to know where it’s installed, so you can launch the stand-alone version of BaseX successfully.
  • You are interested in getting an XQuery engine set up, so you can develop XQuery apps; you are not seeking to develop Java applications.
    (Too bad for you; in order to install anything at all in Tomcat you must understand a little about Tomcat, and all of the Tomcat documentation is aimed at Java developers, not at users of Java applications. Sorry about that; I don’t like it either. As Liam Quin recently remarked to me, our refrigerator salesmen manage to tell us how to install the thing without expecting us to read documentation about how to build a freon condenser; someday the people who build Java application containers like Tomcat will realize that their users include people who want to use not write Java programs. I can hardly wait for the time when Java infrastructure is documented as intelligently as refrigerators. But so far, the refrigerators are winning.)

If these don’t all apply, you will need to make some adjustments on the fly.

Preliminary preparation

  1. Review enough of the Tomcat configuration on your host (consulting the Tomcat documentation where necessary) to know in advance the answers to the following questions:

    • 1.a What version of Tomcat and Java are you using?

      If you’re running Tomcat 6 and Java 1.6, you’ll need a version of BaseX that runs under Java 1.6. The last of that line was BaseX 7.9.

      If you’re running Tomcat 7 and Java 1.7, you can use the current version of BaseX (as I write this, that’s BaseX 8.3.1).

      If you’re running some other combination, I have no idea whether you can run BaseX 8 or not; there are conflicting reports on the Web, and I’m not interested enough in Java application development to bother trying to track down the details. Either stick with BaseX 7.9 for safety or try BaseX 8 and see if it works.

      Trying to run BaseX 8.x under Tomcat 6 and Java 1.6 will not break anything (that I know of). But the servlet will not deploy, and you will find an error message in your Tomcat logs that (how can I put this tactfully?) appears to be aimed at Java developers and is not very helpful to other readers. In my case, trying to deploy BaseX 8.3.1 in a Tomcat 6 environment cost me a few hours and some hairpulling, because it took time to understand what was going wrong.

      The original version of this post discussed only installation of BaseX 7.9, but I had now upgraded to Tomcat 7 and can also report on the process for BaseX 8.3.1.

    • 1.b What directory serves as the base directory for Tomcat ($CATALINA_BASE)?

      On my hosting provider, it’s my home directory (i.e. ~) on the Java host.

    • 1.c What URI is used for the web application management interface (the “Manager application”) in your Tomcat?

      This was surprisingly time consuming for me; I host two domains at my Java hosting provider, which I’ll call able.example.com and baker.example.com here, and the Manager application was available only on one of them (the one configured by server.xml as the ‘default’ one). But eventually I found that http://baker.example.com/manager/html/list gave me results; the various candidates with paths like $host/manager/html, $host/manager, $host/server/manager/html/list, $host/server/manager/html, $host/server/manager, and any path at all on the host http://able.example.com/, all failed.

    • 1.d What userid and password do you use to sign in to the Manager application? Or equivalently for most purposes, where is your tomcat-users.xml file?

      Please tell me you and your hosting provider didn’t leave it at the Tomcat default. If you did, now would be a good time to change it, so that random strangers don’t have administration privileges in your Tomcat instance.

      In my installations, this information is in $CATALINA_BASE/conf/tomcat-users.xml; I find it convenient to have that file open in an emacs buffer before trying to navigate to the Manager application, so I can copy and paste the password instead of typing it.

    • 1.e What does the Manager application look like when things are normal?

      That is, look at it before hand, before you do anything. Otherwise, you’ll never be able to tell whether what you’re looking at afterwards is normal, or you broke something.

    • 1.f Where is the webapps directory for the Tomcat engine and host into which you wish to install?

      In my case, there is one for each virtual host, named with the virtual host name, a hyphen, and the string “webapps”: able_example_com-webapps and baker_example_com-webapps.

    • 1.g How do you stop and restart Tomcat?

    • 1.h Where does your Tomcat write its logs?

    • On my hosting provider, these go into ~/logs.

  2. Decide in advance what userids you wish to specify for your BaseX database(s); for each userid specify the initial password and the initial permissions for each user. At the very least, determine what userid and password will be used as the default database user for the servlet.

    For what it’s worth, I like to set the database up with two users, initially: an administrator userid with ADMIN permissions, whom I’ll call ‘Abel’ here, and a userid with READ permissions, whom I’ll call ‘Romeo’. For purposes of this exposition I’ll assume their passwords are ‘Elba1812’ and ‘Juliet1597’.

  3. Prepare a suitable file of userids, passwords, and privileges.

    How you do this varies between BaseX 7.9 and BaseX 8.0 and later.

    In BaseX 7.9, the documentation tells us that global users, passwords, and permissions are stored in a .basexperm file in the BaseX home directory. The calculation of what counts as the home directory varies depending on how BaseX is installed, and I am unable to deduce from the documentation what counts as the home directory when BaseX 7.9 is running under Tomcat. Experiment, however, shows that it’s apparently …/webapps/BaseX79/WEB-INF. (At least, that’s where the .basexperm file needs to go.)

    An examination of the .basexperm file on your local installation (look in your home directory) will show you (at least, it showed me) that it’s not intended for editing by a human. We will need to use BaseX to make a .basexperm file for us.

    You might be tempted to say “Wait, let’s get BaseX running on the server, then use the REST interface to create users Abel and Romeo and assign them passwords and permissions, then use the Abel userid to assign permission level NONE to the built-in userid ‘admin’. I like the way you think, but I don’t think that’s going to work. (Ask me how I know.) The server command for creating or altering passwords does not allow the password value to be given as an argument: instead, the server prompts you for the password. But there is no way for the REST interface to prompt you for a password.

    One workaround (this is apparently what the BaseX developers had in mind for this situation) is to use BaseX 7.9 on your local machine (we’ll use the stand-alone version, but if you’re adventurous you can use the client-server version or the GUI) to make a .basexperm (permissions) file, which you will then copy to the appropriate directory on the server.

    Since I don’t want the .basexperm file I create now to interfere with the existing set of users and passwords on my local machine, I want to run BaseX with a new home directory that is not the one usually used. So what I do in my bash shell is something like the following:

    # make a temporary place for a .basexperm
    # file to reside
    
    mkdir ~/basex-temp-home
    cd ~/basex-temp-home
    echo "# hi there" > .basexhome
    echo "DBPATH = `pwd`/data" > .basex
    
    # Now launch BaseX.  N.B.
    
    /opt/local/BaseX-7.9/basex/bin/basex
    

    In BaseX, I issue the following server commands to create the users I want, and assign a new password for the built-in admin user. I’d assign NONE to admin if I could, but I can’t, so I just reset the password.

    # Just check to see where we are to start with:
    SHOW USERS
    
    CREATE USER Abel
    # at this point, the server prompts for
    # the password 'Elba1812'
    CREATE USER Romeo
    # password:  Juliet1597
    GRANT ADMIN TO Abel
    GRANT READ TO Romeo
    ALTER USER admin
    # password:  Elba1812
    
    # now, just to make sure we are
    # where we want to be
    SHOW USERS
    
    EXIT
    

    If all has gone well, BaseX will have created a .basexperm file in directory ~/basex-temp-home.

    (Alternatively, I guess one might rename the existing .basexperm to .temp.basexperm.old, launch the GUI, issue the server commands given above, exit, move the new .basexperm out of ~ and into some other place, then give the old .basexperm its old name back. But I haven’t tried it this way.)

    In BaseX 8.3.1, the documentation tells us that users, passwords, and permissions are stored in a human-editable users.xml document in the database directory. Experiment shows that if in this environment we add a new user, BaseX 8.3.1 places the users.xml document at …/webapps/BaseX831/data.

    One could go through a process similar to that described above to create a users.xml file, but in practice I became lazy and handled things a slightly easier, slightly riskier way (see description of “the lazy way” below in steps 8 and 9).

  4. Prepare in advance a small Web-accessible XML file or two to use in creating a helloworld database to check that things are running as you expect.

    I use the documents in http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/ for making a helloworld example. When installing BaseX 8.3.1, it proves simpler to use XML documents on my hard disk.

Doing the deed

After the four preparation steps just described, the steps for actually installing the software are very simple in principle: first fetch a copy of the WAR file for BaseX into the appropriate webapps directory, adjust its configuration, and then restart Tomcat to deploy the app.

  1. Download the WAR file for BaseX.

    In one window, I use ssh to reach my hosting provider. In another, I navigate to the Downloads area on the BaseX web site.

    In the browser window, I right-click the WAR file and save its URI to the clipboard (in different browsers this is Copy Link, or Copy Link Address, or Copy Link Location).

    In the ssh window, I issue the following commands, using a Paste action (command-V) to insert the URI of the WAR file (substituting the BaseX version number for $vvv):

    mkdir incoming
    cd incoming
    curl --output BaseX$vvv.war $URI-of-WAR-file
    
  2. Open the Manager application in a browser window and keep an eye on it.

  3. Put the WAR file in the webapps directory

    If you do this while Tomcat is running, Tomcat will unpack the WAR file; this is convenient, because you need to edit files inside it. On the other hand, you don’t want BaseX to start up yet, so keep an eye on the Manager window to make sure that BaseX doesn’t suddenly show up there. If it does, use the Manager application to stop it.

  4. Configure BaseX

    The only essential configuration to do at this point is to install the userids prepared earlier and change the default username and password used when the HTTP request supplies none; otherwise anyone who happens across your BaseX engine on the open Web has admin privileges.

    • 8.a Copy the .basexperm or users.xml to your Java hosting provider. (Or, if we are doing things the lazy way, do nothing here.)

      It doesn’t much matter how you do this (but be aware that the file is binary; cut and paste is not going to work). I use scp; from my local machine I type:

      scp .basexperm able.example.com:incoming/.basexperm
      

      I could copy it straight to the webapps directory, but I like to keep a copy in ~/incoming in case I end up zapping the webapps subdirectory.

    • 8.b Place the .basexperm file in the …/webapps/BaseX79/WEB-INF directory, or the users.xml in the …/webapps/BaseX831/data directory. (Again, if we are doing things the lazy way, we do nothing here.)

      In my ssh window on the web host I type:

      cd ~/able_example_com-webapps/BaseX79
      cp ~/incoming/.basexperm .
      

      8.c Change the default user and password by editing the web.xml file. (Even if we are doing things the lazy way, this is worth doing.)

      No one who presents no credentials gets to create a new database or change the setup on our server. So the default user should not have admin permissions, only read permissions. It is this for which we made the read-only user Romeo.

      In …/webapps/BaseX79/WEB-INF/web.xml, close to the top of the file, the default user credentials are set:

      <!-- Set default credentials -->
      <context-param>
      <param-name>org.basex.user</param-name>
      <param-value>admin</param-value>
      </context-param>
      <context-param>
      <param-name>org.basex.password</param-name>
      <param-value>admin</param-value>
      </context-param>
      

      This changes to

      <!-- Set default credentials -->
      <context-param>
      <param-name>org.basex.user</param-name>
      <param-value>Romeo</param-value>
      </context-param>
      <context-param>
      <param-name>org.basex.password</param-name>
      <param-value>Juliet1597</param-value>
      </context-param>
      
    • 8.d Take care of anything else you need to configure.

      In my Tomcat 6 installation, there is already a servlet named ‘default’, so the rules at the end of web.xml for static content cause problems. For the moment, I comment them out. I may try to deal properly with them later, but I am looking for a back end, not a complete web server. If I have static content to be served, Apache will serve it, not BaseX.

      Under Tomcat 7, the rules for static content caused no trouble, so I didn’t comment them out.

  5. Flip the switch

    Restart Tomcat to cause it to deploy BaseX.

    You can in principle cause Tomcat to deploy the new application by issuing appropriate commands from the Manager application, and possibly even upload the WAR file from your machine, without logging in to your hosting provider. When I tried doing it that way, I ran into other problems, so I’ve never actually done it that way.

    Check in the Manager application to make sure BaseX appears in the list of servlets and is deployed.

    It’s at this point that I ran into failures with BaseX 8.3.1 running under Tomcat 6. The Tomcat logs (remember I told you to find out where your Tomcat logs are going?) said (among many other things):

    Nov 27, 2015 6:44:35 PM org.apache.catalina.startup.HostConfig deployWAR

    SEVERE: Error deploying web application archive BaseX831.war

    java.lang.UnsupportedClassVersionError: org/basex/http/SessionListener :
    Unsupported major.minor version 51.0 (unable to load class org.basex.http.SessionListener)

    Searching on the Web for the wording of the error message tells me that this is what happens when you try to run a Java program compiled with Java 7 (or 1.7? or 51.0? I wonder what drugs Sun engineers were on when they decided how to number Java versions? And how many drugs Oracle engineers have had to consume in order to be willing to continue following the same pattern?) under Java 6 (or 1.6). This is what alerted me to the statement that was there on the BaseX downloads site all along: “Versions before BaseX 8.0 can be run with Java 6.” Read with a heightened sensitivity to subtle entailments, we can infer that they mean: version 8.0 and later won’t run with Java 6.

    This analysis seems consistent with the fact that I get the same error message when I try to run BaseX 8.x on a system whose default Java is Java 6.

    If we are using BaseX 8.* and doing things the lazy way, it is now that we set up our users and passwords. Do this first thing, since until you do it, your server is open to anyone who happens by and tries “admin” as the userid and “admin” as the password.

    1. Log in to the dba application that comes with BaseX 8, using the default admin username and password. On my setup, the dba application is at http://baker.example.com/BaseX831/dba.
    2. Change the password for the admin account to Elba1812. (This closes the door to our casual intruders.)
    3. Glance at the dba application’s tabs for Databases, Users, Files, and Settings,
      just to make sure no one has exploited the window of vulnerability we opened by starting the server with the default admin password in place.
    4. Add the userid Abel with password Elba1812.
    5. Add the userid Romeo with password Juliet1597.
  6. Test

    To test that things are going as expected, the REST interface can be used (both in 7.9 and 8.3.1) to create and query a database. In 8.3.1, it’s easier to use the dba application. I’ll describe both methods.

    Using the REST interface

    First, a query that should work from the default user (without credentials):

    http://able.example.com/BaseX79/rest
    

    Hmm. This doesn’t in fact work for me: it prompts me for a userid and password. One possible explanation is that I mistyped the password either when creating the .basexperm file or in the web.xml file. Another is that I have once again managed to misunderstand what the BaseX wiki says about the USER and PASSWORD options. For the moment, however, I am going to ignore this problem of failure in setting the default user. When I do give the appropriate userid and password, I get the expected response from the REST server.

    Next, some queries that should require credentials; if you dereference this using curl or wget without specifying the –user option, you will get an “Access denied” message, and if you try in a browser the browser will normally prompt you for userid and password. Supply user Abel and password Elba1812.

    All of the following URIs contain characters that will need to be escaped; I find that the browser always takes care of that for me.

    http://able.example.com/BaseX79/rest?command=show users
    

    Then create a database and add some documents:

    http://able.example.com/BaseX79/rest?command=create db helloworld http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/doc1.xml
    http://able.example.com/BaseX79/rest/helloworld?command=add http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/doc2.xml
    http://able.example.com/BaseX79/rest/helloworld?command=add http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/doc3.xml
    http://able.example.com/BaseX79/rest/helloworld?command=add http://cmsmcq.com/2015/11/XQuery-over-HTTP/data/doc4.xml
    

    Now we can query them:

    http://able.example.com/BaseX79/rest/helloworld?query={/}
    

    Finally, we can delete the database:

    http://able.example.com/BaseX79/rest?command=drop db helloworld
    

    If the responses to dereferencing these URIs is as expected, then hurrah, it works, and you can now get on with your application development.

    If not, then check the logs (remember I told you to find out where your logs are going?) and good luck to you.

    Using the dba application in BaseX 8.3.1

    In the Databases tab, create a helloworld database.

    Then use the Add button to add resources to it. In 8.3.1, this expects
    you to upload files from your browser; it doesn’t seem to accept URIs. So I uploaded
    my local copies of the four hello-world documents, one at a time.

    In the Queries tab, type some queries in the query widget, click the Run button to evaluate them, and make sure the results are as expected.

    • collection('helloworld')/*
    • collection('helloworld')//greeting[@var='northern']

    The REST queries listed above should also work.

Four-up PDF slides from Slidy

[9 December 2009]

For the last few years, I’ve been using Dave Raggett’s tool HTML Slidy to prepare slides for presentations. This has a number of advantages: the slides can live happily on the Web, each slide has a distinct URI so they are easy to point at for purposes of discussion, etc., no proprietary slide format is involved, and I can write the text of the slides in TEI Lite.

From time to time, though, organizers of conferences or other gatherings where I speak ask for a PDF copy of my slides. It’s easy enough to print the slides from Slidy, but the result has generally been disappointing to the organizers, who are sometimes a bit distressed (or am I only projecting?) that my slides are so unlike everyone else’s.

So I generally try to coax my system into producing PDF pages with four slides on a page, preferably in little rectangles, so my slides look a little more like the conventional idea of a PDF dump of a slide set. The need arises seldom enough that I always forget how I did it last time. But sorting through some papers just now, I found that the last time I did this I took notes for the future. If I leave the notes only on paper I’ll never find them again when I want them. So, for the record, this is one way to make four-up slides in PDF from Slidy.

  1. In the HTML file, wrap the table of contents in a <div class='toc'>.
  2. Also make sure the header refers to a print stylesheet. (This ought to be generated by my translation from TEI Lite to HTML Slidy, but it wasn’t the last time I did this.) That is, make sure the header contains something like <link rel="stylesheet" type="text/css" media="print" href="http://www.w3.org/Talks/Tools/Slidy/print.css" />. This forces a new page for each slide. Not all browsers respect the media attribute, so you may need to try multiple browsers; I have had good results from Opera in this regard.
  3. In the browser’s page setup (or print dialog, wherever the browser puts it), disable the printing of headers and footers (except for page numbers, if you want them).
  4. In the print dialog box, when using Opera, I also find it helpful to check the Fix to paper width option. Backgrounds don’t seem to work, so I also tell Opera to print no background. (This too has troubled some organizers, but I can’t help them.)
  5. Still in the print dialog box, on Mac OS X, select the ‘Layout’ options and choose 4 pages per sheet, with the appropriate Z-shape layout of the sequence, with single thin line borders (or hairline — I haven’t noticed any difference in the output between the two choices). I have managed this on Windows as well, in the dim past, but I don’t remember how I got to the menu that let me select 4-up printing.

It’s helpful to preview the thing for bad page breaks, so you can insert forced page breaks or fiddle with the font size until you get better breaks, but I have no algorithm for this, just fiddle and check until you are satisfied or out of time. Sometimes Firefox has given me better page breaks than Opera, sometimes the reverse; dunno why.