New Human-computer Interaction Techniques for the Digital Library

• Robert J.K. Jacob
  Horn-Yeu Shiaw
  Holly Taylor
  Gregory R. Crane

  • Department of Electrical Engineering and Computer Science
    Department of Psychology
    Department of Classics

  • Tufts University
    Medford, Mass. U.S.A.

• With thanks to:

  • Principal support: NSF and NEH DLI Phase 2 Program

  • Additional support: NSF Interactive Systems Program, Office of Naval Research, Berger Family Fund, Tufts Selective Excellence Fund

  • Graduate students: Leonidas Deligiannidis, Stephen Morrison, Vildan Tanriverdi

Invent New Interaction Techniques for Interacting with Digital Libraries

• Focus on spatial information

  • As ancillary aspect of text in library

  • Communicate some spatial information...

  • Without distracting from main reading

• "Lightweight," "non-command" interface

  • Follows user

  • Responds appropriately

  • Require little explicit action or attention

• User just reads text normally

  • Provide ancillary spatial information "for free"

Communicate Spatial Information

• Related to text user is reading

  • With minimal distraction to reader

• Conventional solution:

  • Traditional hypertext link in the text

  • Disrupt reading, lose context

• Lightweight Interaction

  • Get more information from user, without much effort from user

  • User not really give explicit commands

  • But system observes, guesses, infers, takes hints

• Examples

  • Physiological measurements

  • Behavioral measurements

  • Other, independent interactions

  • Affect

  • Mental models

  • Agents

• But:

  • Midas touch problem

  • Must use lightweight input judiciously

• Interaction Technique:

  • A way of using a physical device to perform an interaction task

Background Display

• Metaphor: Text on clear plastic

• Faint view behind

  • Toned down

  • Blurred

• Subtly provide approximate sense of place

  • Indoors or outdoors

  • Street or forest

  • Daytime or nighttime

Implementation

• Collect background images

• Mark each point in text

• Process image

  • Filter to 3x3 pixels

  • Enlarge 9-pixel image back to full size

  • Blur, convolve with filter whose base > newly-enlarged pixels

  • Increase brightness, reduce contrast

  • To make more subdued background

• "9-pixel walkthrough"

• Follows scroll bar input

  • Can also use eye tracker

Sample text in the foreground, blurred image behind

After user scrolls, new background appears

Peripheral Border Display

• Peripheral vision instead of background behind text

• Metaphor: Read text while riding bus

  • Without looking up from text, could still get rough sense place, through peripheral vision:

  • Indoors, outdoors, urban, rural, daytime, nighttime

• Reading task is primary

  • Use no real estate on main screen for spatial

• Spatial information projected onto a wall just behind screen

  • Need not ever look up at it

  • Can view peripherally

• Peripheral vision for general rough sense of place

  • Foveal vision for reading

Implementation

• Two workstations (SGI)

  • Main CRT display

  • Drive video projector

• Two Java apps

  • On the two workstations

  • Communicate via network socket

  • Notify when the background image should change

Photograph of setup for Peripheral Border Display

Schematic picture of Peripheral Border Display:
Outer window = image on video projector.
CRT icon = image on real CRT screen.

3-D London Walkthrough

• Generate spatial view from 3-D model, not 2-D images

• Tallis maps, Bolles Collection, Tufts University Archives

  • Use as texture maps for VR model

• Can view on screen, as "fishtank VR," or regular VR with head-mounted display

Example of original Tallis map

Screendump from walkthrough

Another screendump from walkthrough

Head-mounted display (with eye tracker)

Peripheral Workspace

• Metaphor: Large reading table

  • Can only read clearly in center

  • Use outer to maintain context: cross-references, dictionary, atlas,

Implementation

• Two workstations, video projector as previous

• Center workstation runs regular Unix Netscape

• Projector workstation communicates with it via command-line feature

Schematic of peripheral workspace prototype

3-D Drag

• Same approach

• But peripheral workspace contains full 3-D image, with 3-D "icons"

Schematic of 3-D drag prototype

Conclusions and Future Work

• Lightweight user interface approach

• Interaction techniques for spatial information in DL

  • While reading

  • Text is primary

  • Spatial is ancillary

  • Non-distracting

  • Feed to reader in background/periphery

• Prototype implementations of new interaction techniques

  • Showing technical feasibility

  • But not yet mature enough to evaluate usefulness

• Next step: Narrow down to most promising

  • Refine the prototypes

  • Test usability and usefulness to users

  • Psychological experiments on spatial awareness

• N.B. Perseus digital library is normally accessed from WWW browsers

  • But this research envisions future platforms