J. Nathan Matias pointed me to Mark Bernstein’s paper (‘paper’ is an inadequate label for the work) on literary criticism and hypertext, which Mark presented at the recent Hypertext 2010 conference. It’s a great piece of writing that ably defends the literary tradition from the Barbarians of mechanical evaluation. My summary of the paper cannot do it justice, what with its 93(!) references, quotes from Pope’s An Essay on Criticism, and Mark’s typical wit (“We cannot make feature films about vertebrate paleontology or test-driven software development; too few people are interested. The same audiences profitably support numerous books.”) The paper has a visual companion in the form of a SlideShare deck; the one aspect that appears not to have been preserved (for posterity and for those with limited travel budgets) is a recording of the actual presentation.
The comment that lead me to the paper seemed to offer it as a counter-example to Andrew Dillon’s thesis about methodological failures of the hypertext community to assess its impact on education. I don’t see it.
Both positions dismiss the evaluation of navigation as not inherently interesting. Bernstein writes:
Though these studies can provide useful information to system designers, the tasks they study have little relationship to our most important reading work. Much attention has been paid, for example, to helping people locate facts in a complex document, presumably in the belief that this is what knowledge workers do. I am skeptical that this assumption is valid even for clerical work. Researchers may do this when checking footnotes or confirming statistics, but no one receives tenure for their speed at locating a reference or dexterity at finding the population of Montenegro in 1912.
…but we do not, in fact, ask students to read biology textbooks in order for them to answer multiple-choice tests. The goal, surely, is to instill understanding of biology, to learn how biologists reason, what evidence they adduce, what arguments they accept.
Focusing narrowly on the (easily quantifiable) mechanics of link following isn’t that useful in the grand scheme of things; what’s more important is understanding how people react to the text and what they get out of it.
My critique of Criticism is that it takes too narrow a view of hypertext, privileging hypertext literature and narrative over hypertext interaction. Literary hypertext is about effective communication, about reading, about art. Hypertext interaction is about control, about ease of use, about intuitive design. Neither is necessary nor sufficient for the other: it’s possible to have compelling hypertext literature with pedestrian interaction (or with no explicit links at all), and it’s possible to have powerful and intuitive interaction without a shred of narrative.
One of the hallmarks of the hypertext academic community is a long-standing division between the engineers and the humanists, a division that undoubtedly contributed, for example, to scheduling Mark’s session against one that he wanted to attend. He notes:
One mild vexation was that my own paper on Criticism…was scheduled directly opposite one of the strongest sessions on adaptive hypertext systems, and so I couldn’t see several terrific papers. I suspect that this was a two-cultures issue; people assumed that systems builders would be less interested in critical theory. Oh well.
The sharing of the Hypertext term (and the conference) by these two different camps is a good thing, in my opinion, but it is also useful to remind ourselves periodically that sometimes we are talking about completely different things. When Mark starts his paper with a question “How do we know that a hypertext is a good hypertext?” he is already referring to a form of writing. To talk about interaction, he would also have had to answer (or at least ask) the utilitarian question “Good for what?”
While Bernstein and Dillon don’t disagree on the shortcomings of evaluation as it has typically been practiced, their approaches differ in that Dillon still wants to know whether those biology students actually learned something, and if not, why not. While it’s not clear what the relative importance of hypertext interaction and hypertext writing are for education, I argue that it is premature optimization to focus solely on the prose. But then I am an interaction guy.
Update: fixed typo in Bernstein quote (sorry!)