Blog Author: Chunyuan Liao

An exploration of cross-media interaction

on Comments (1)

One of FXPAL’s papers at the ACM Multimedia conference this year describes FACT, an interactive paper system for fine-grained interaction with documents. The FACT system consists of a small camera-projector unit, a laptop, and ordinary paper documents. The system works as follows: a user makes pen gestures on a paper document in the view a of a camera-projector unit. FACT processes these gestures to select fine-grained content and to apply various digital functions. For example, the user can choose individual words, symbols, figures, and arbitrary regions for keyword search, copy and paste, web search, and remote sharing. FACT thus enables a computer-like user experience on paper. This paper interaction can be integrated with laptop interaction for cross-media manipulations on multiple documents and views. FACT can be used in the application areas such as document manipulation, map navigation and remote collaboration.

Paper UI reseach at FXPAL

on Comments (1)

Paper still plays an important role in many tasks even in this age of computers. This phenomenon can be attributed to paper’s unique advantages in display quality, spatial arrangement flexibility, instant accessibility and robustness, which the existing computers can hardly beat. However, paper lacks computational capability and does not render dynamic information. In contrast, cell phones are becoming powerful in computation and communication, providing a convenient access to dynamic information and digital services. Nevertheless, cell phones are constrained by their limited screen size, relatively lower display quality and cumbersome input methods. Combining the merits of paper and cell phones for rich GUI-like interactions on paper has become an active research area.

Here at FXPAL, the Paper UI group currently focuses on cell phone-based interfaces and their supporting techniques to link paper documents to digital information and enable rich digital interactions on physical paper through content-based image recognition algorithms. We started  research in this area several years ago (see our project page for more details), and our recent on-going projects include EMM and PACER.

Continue Reading