Achieving impact


The impact of academic computer-human interaction research on the real world has been debated repeatedly over the last few years. The criticism is that HCI research isn’t that relevant, and that really innovative interfaces (such as Apple’s iPhone) are designed by outsiders, without input from HCI researchers. My sense is that things are not so dire, that there is a trickle-down effect, and that practitioners do pay attention to research results when those results are packaged effectively.

But the criticism is not completely without merit, and only a few systems described in the CHI and UIST literature (to take two academic examples) actually make it into product. On the other hand, one finds examples of transformative work (e.g., Tim Berners-Lee’s framework for the World Wide Web) being rejected by top-tier conferences.

Thus it gives me great pleasure to point to an academic success that is also succeeding in the real world. I am talking about ShapeWriter.

ShapeWriter on the iPhoneShapeWriter is an embodiment of an interaction technique for text entry that allows words to be generated by drawing shapes on a touch-screen keyboard rather than tapping on individual keys. It’s easy to learn how to write this way, and it improves text entry rates considerably. It’s also fun to use.

One of the aspects I find intriguing about this style of writing is that the shapes you create, once you’ve practiced them, can be drawn independently of the location of the letters on the keyboard. In a way it transforms the alphabetic writing system of western languages into something that resembles a logographic writing system.

The basic idea was developed, implemented, evaluated, advocated, and turned into a product by Shumin Zhai. Here’s a brief run-down of the history of the project:

  • First research prototype and experiment completed in 2002
  • First paper published in CHI 2003, and over a dozen more subsequent scientific publications on or around ShapeWriter since then.
  • First public release of a fully functional gesture keyboard (dubbed SHARK) from IBM AlphaWorks in 2004.
  • Dozens of national and international press articles, including Time, BBC, Mercury News, New Scientist since 2003
  • Over a half dozen patents and patent applications granted or filed.
  • In 2007 ShapeWriter, Inc. is spun out of IBM to commercialize the technology.
  • In 2007 a ShapeWriter “WritingPad” on is released for the iPhone. It became Time’s top 11 iPhone must have, won many other awards, and has had over two million downloads.
  • In 2008, the ShapeWriter team won an Android Developer Challenge award. ShapeWriter was publicly released on Android Market in 2009.
  • In May 2010 Nuance Communications closed its acquisition of ShapeWriter.

Seven years of hard work by Shumin and by many others translated an idea into a product line, in the interim producing a range of research results and IP accomplishments. This is not only a testament to the principled application of HCI methodology to the design of software systems, but also a tribute to Shumin’s vision and tenacity.

It is also an example of how research results can be transferred from the lab to the product line. It takes hard work, perseverance, a clear vision, and, above all, a willingness for researchers to engage in the productization process directly. In my experience, transferring technology from the research lab to an engineering company without also transferring motivated personnel makes it virtually impossible to influence product. Perhaps that’s the bigger lesson for HCI researchers who would like to make an impact on the real world.

Postscript: As of this writing, Nuance has suspended the downloading of the iPhone and Android applications. I hope this is a temporary measure that is a side-effect of the acquisition, and that the applications will soon be available again.

Share on: 


  1. Regarding your postscript: Sounds like Rhetorical all over again:

    Nuance has their own competing product, “T9 Trace”. And both are like Swype:

    Even the picture illustrating the story looks the same!

  2. It’ll be interesting to see how the patent landscape shakes out… ShapeWriter patents should have priority over the other IP

  3. Shumin Zhai says:

    I would add that ShapeWriter was a team effort. Indeed it would not be able to go this far without at first my student Per Ola Kristensson and later the whole team in the company ShapeWriter Inc. My colleagues and particularly management at IBM Research supported the project because they sincerely believe in exploratory research. To get back to the main point of this post about HCI research, I should point out all of our academic publications on touchscreen text input ( and most of the literature references in these publications are in/from the HCI field. The field provided the academic culture, spirit, context and value system for projects such as ShapeWriter to take place. Long before ShapeWriter reached millions of download, it was appreciated and taught by many HCI professors in graduate classes.

  4. […] It will be interesting to see if the Android SDK makes it easier to integrate applications at varying levels of granularity, or if it will be possible to create and integrate more capable mini-apps. It would also be great to plug in alternate text entry widgets such as ShapeWriter. […]

Comments are closed.