The XML Schema Working Group is trying to move its focus to the Structures spec, whose Last Call comment period ended a couple months ago. (Work on the remaining Datatypes issues will continue in the background, but currently it is the editors, not the Working Group, who are the bottleneck on Datatypes. If the Datatypes editors manage to work effectively in the background, they may manage to use these next few weeks to produce wording proposals for the remaining issues.)

So I just finished reading through all 104 Last Call issues (so far) opened on the Structures spec, trying to impose some order on them.

Whenever I have a longish list of tasks, or of issues that need resolution, my fingers itch to classify them. Which ones are easy and will take just a few minutes of the editors’ time to fix? (Ha! in my experience, editors’ time doesn’t come in increments smaller than half an hour — once you factor in the time it takes to generate a new version of the spec and check to make sure you actually made the change correctly, even a simple fix to a broken paragraph can take forty-five minutes.) Which ones are hard and will take a lot of time and effort, either Working Group discussion time to reach consensus on the right thing to do, or editorial time to draft, or both? Which lie in the middle?

The process is a little like some of those classic AI programs discussed in textbooks (bagging groceries, for example): it’s hard to do perfectly, but even an imperfect classification can be much better than nothing. Of course, it’s even more like the process of deciding, after a catastrophe, which victims will benefit from medical attention, which victims are too far gone to expend resources trying to save, and which can get along without help. So I always think of the process of classifying issues as triage.

In scheduling issues for WG discussion, you want (in my experience) to put a few easy items on the agenda each week, so that every week the WG has the experience of nailing a few issues. But you can’t just sequence the list from easy to hard, because that leads to a dramatic slowdown as you get past the easy ones and move into the hard ones, which can have dreadful effects on morale. So in addition to the easy ones each week, you want to schedule some hard ones early on, so that the WG can spend the time it may take to understand them, develop solutions, argue about them, develop some better solutions, and eventually converge on a good solution, before you start getting into deadline pressure.

But every time I start to perform triage on a list by estimating how much time it will take, I am reminded that some items are important, and others are less important, and that importance doesn’t really correlate well with how long things will take. There is then a crisis while I contemplate some issue that is clearly important, but will take a while — or won’t take long but also won’t actually contribute much to making the spec better. And in the time-honored fashion, I compromise by trying to do both.

I almost always end up performing a sort of double triage, classifying things along two distinct axes:

• cost: hard, easy, work (as in “this will take some work”)
• importance: important, medium, or thimble (thimble? yes, thimble. I’ll explain in a minute)

Of course, the result is at best a rough and ready way of subdividing the problem space in order to control complexity and stave off despair. Different people will judge the importance of issues differently, likewise their cost, and even if everyone agreed on the importance or the likely cost of an issue, they could still be wrong. (For this reason, I encourage others in the WG to perform their own rough and ready classifications, but discourage the occasional attempt to discuss the correct categorization at length. By all means, tell me you think issue 666 is likely to be devilishly hard, so you think I’m wrong to class it easy. I may change it, then. But for heaven’s sake, let’s not waste time arguing about a back of the envelope scheduling estimate!)

I use the term thimble for items that aren’t either important or medium important, mostly because I find I can’t bear to call any problem in a spec “unimportant”. No issue raised by a reader is unimportant. And yet, some are more important than others. And if some are more important, then it seems to follow with logical necessity that some are (sigh) less important than others.

The image comes from a discussion of Dante’s Paradiso during my student days. Some students found it hard to come to terms with the fact that there are hierarchies even in Dante’s paradise: some of the blessed are in the inner circles, close to God, and others are, well, they are in the outer circles. This offended some readers’ egalitarianism (are they less virtuous than the other souls? less good? less deserving? Just where does God thinks he gets off, banishing some virtuous souls to the outer circles?!), and so we discussed it for a while. Eventually, someone said that when they had discussed this kind of thing in school, the nun teaching the class had finally taken up a thimble and a water tumbler and filled them with water. Each, she said, demonstrating, was full, as full as it could get. One, to be sure, held more water, but saying that the tumbler held more water did not entail saying that the thimble was not full. In a similar way, we can believe that all the souls in heaven are as good as they can be, while still recognizing that their capacity for goodness may vary.

A comment correctly pointing out that a particular sentence is malformed or confusing can never be unimportant. it is as important as it can be, even if the sentence in question describes a corner case seldom encountered and thus of comparatively little practical import, compared to (say) a problem in the definition of a concept that is constantly appealed to.

I call this process ‘triage“, but of course it works not with three classes but nine, in a three by three table. In a perfect world, of course, you’ll resolve all problems and the categorization is used only for scheduling. If, however, in this world of woe you or your Working Group sometimes miss that level of perfection, then it can matter which issues you address and which you end up closing with a laconic WONTFIX message. If you haven’t botched the categorization, you will get the best bang for the buck if the important, easy items have gotten done, and if you haven’t poured precious resources into unsuccessful attempts to resolve the items classed thimble, hard. Me, I figure you want to start at the top left of the table (important, easy) and move more or less systematically to the bottom right.

I still haven’t figured out how to decide between spending time on important, hard items or on medium, work items. The first are more important (d’oh!), but you’re likely to get fewer of them actually done. So it’s a judgement call, at best.

Unfortunately, I’m never happy even with a three by three classification of issues.

To understand any issue, you need to understand what the commenter claims the spec is doing, what is wrong with that, and how to fix it. But that doesn’t suffice to resolve it: before you touch the spec, you must decide whether you think the commenter is right. What did the working group intend to do here? And why? What’s the underlying design story? What does the current text actually do? Is the problem reported a symptom of some larger complex of problems? What are the options for changing things? What are the relevant design principles? Which invariants should the spec be seeking to achieve or preserve? If the issue is at all complex, it can take a while to get up to speed on the relevant information. So if there are several issues that deal with related topics, you really, really want to deal with them all at once, or in succession. (Few things sap morale more effectively than the discovery that in dealing with four related issues, at four separate times, the WG made four decisions which effectively contradict each other because it failed to spin up successfully or remember what it did earlier.)

So I almost always find that I also want a sort of topical clustering of issues. “If we’re going to deal with issue 2218, then we might as well deal with 5078 at the same time. And then the answers to 5163 will just fall out from the earlier decisions.” Perfect clustering involves perfect knowledge and perfect classification, so it doesn’t happen. And I often change my mind about what real issue a given problem ticket involves. So my attempts at topic clustering are even less stable and reproducible than my cost and value estimates. But failure to cluster is like non-locality of reference in a program: it leads to thrashing.

The XML Schema Working Group maintains its issue list in public, so for what it’s worth the current triage of Structures issues is visible. You have to make the Whiteboard column visible, and sort on it, to see the triage clearly.

Several questions arise. Other people presumably face this problem, just as I do. But I don’t hear them moaning and groaning about it the way I do. Have they found better, less painful ways of managing this process? Or are they just more stoic?

And why oh why do so few issue tracking systems make it convenient to add yet another dimension on which to classify issues? Bugzilla at least provides the Whiteboard field, where you can do pretty much anything you like, and then sort. But there isn’t a convenient way to say “sort first by cost and then by importance” or “sort first by importance and then by cluster and finally by cost”, etc. What would it take to make it better?