Video-based Human-Computer Interfaces
In recent years, there has been an effort to augment traditional human-computer interfaces like the keyboard and mouse with intelligent interfaces that allow users to interact with the computer more naturally and effectively. The goal is to develop computer vision systems that make computers perceptive to a user's natural communicative cues such as gestures, facial expressions, and gaze direction. Such systems are especially relevant for people who cannot use the keyboard or mouse due to severe disabilities.
In this talk, I will describe some of the interfaces my group has been developing. Among them is the Camera Mouse, a system that tracks a computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. It has been commercialized and is used by people with severe disabilities in many homes and educational facilities. Another system can detect a user's eye blinks and accurately measure their durations. Voluntary long blinks trigger mouse clicks, while involuntary short blinks are ignored. Other interfaces can detect a user's eyebrow raises, gaze direction, and hand signals to control the computer. I will also describe ongoing work on facial feature analysis for American Sign Language recognition.