Interactive Systems Program

The Interactive Systems Program supports scientific and engineering research fundamental to the design of technology and systems for human communication with computers and using computers. Research topics encompass but are not limited to: information access and usability; speech recognition and natural language understanding; alternative input/output modalities such as gesture, facial recognition, and haptics; interfaces for people with disabilities; and virtual environments, where natural and artificial entities interact with shared resources.

This program recognizes and encourages the emergence of new approaches and the use of novel and realistic environments instrumented to capture human expression and signals in order to explore and validate hypotheses about the laws governing human-computer interactions and the principles of their use in various domain tasks. It encourages the discovery or refinement of computer models of perceptual/sensory and cognitive human- computer interactions. The program scope includes visualization of and interaction with real or virtual, complex, high-dimensional numeric, symbolic, or pictorial knowledge in computer assisted environments, as well as measurement and evaluation of the performance of the models and of the methodologies used in human-computer interactive systems. The objectives of the program also include upgrading the human resources and infrastructure of human-computer interaction education and research, and encouragement to women, minorities, and persons with disabilities to participate in scientific research in this area.

Major Program Elements

1. Virtual Environments

This element supports research the representation and manipulation of complex, high-dimensional, physical or abstract information for the enhancement of human interaction with computers and human performance in general. This area includes studies on the scientific basis for visualization and virtualization, interface semiotics, visualization and manipulation of programming objects, visualization of task decomposition, principles of human exploration, comprehension, and understanding of representations. Some examples of investigative questions are those which address efficiency, transparency, the human sense of naturalness, fidelity, clarity, or other characteristics of interactions with representations. Some example domains of investigation include telerobotics, virtual prototyping, mapped interactions with very small scale or very large scale systems, or interactions with models of events that occur at past or future times.

2. Speech and Natural Language Understanding

This is one of the "Grand Challenge" areas, related to high performance computing and communication. Its eventual goal is reliable and robust human interaction with the computer through spontaneous, user- independent, natural language in real time, possibly in a multilingual environment. Many aspects of basic human-communication research are involved in this element of the program: Semantic aspects of speech and natural language; spontaneous speech and language, recognition, analysis, and synthesis; syntactic, semantic, pragmatic, and prosodic factors; signal processing, symbolic, and connectionist architectures; models of the auditory and vocal tracts and related cognitive functions as they are associated with machine recognition and synthesis of speech, and the automation of the processes of speech/language acquisition and adaptation; dialogue models and response generation to queries; and finally, their place in multimodal interactive systems.

3. Other Communication Modalities

This element focuses on determining and understanding basic principles of human expression for input and computer facial animation for output. It includes studies of human-generated or human-controlled sounds and vibrations, tones, music, handwriting and stylus interfaces, gestures, posture, body language, facial expression, tactile, haptic and other motor channels, chemical senses (olfaction, taste) and effectors, even electromagnetic input-output (e.g. electric, magnetic, or optical measurements) to detect human commands, human intent, human states of perception, cognition or affective states (e.g., attention, confusion, satisfaction, etc.) and their use to guide computer simulations or processes.

4. Adaptive Human Interfaces

This element supports basic research aimed at making computers adapt dynamically to human users to enhance task performance. The focus is on human physical, physiological, psychological, perceptual, or cognitive interactive behavior and their use in dynamically adaptive models of human-computer interactions. One example is intelligent automatic sequencing and spatial organization of visual or auditory information to match the expressed or implied needs and goals of the user based on the dynamic discovery of those decision and performance strategies that humans use in stressful or constrained situations. Other environments for intelligent agent research involve the retrieval of information from a characterization of the user habits in database search, in learning and educational environments, and in a variety of decision-making tasks.

5. Usability and User-Centered Design

This element includes the scientific study of the factors influencing the human productivity, acceptability, and comfort of human-computer interfaces and the incorporation of those factors and their assessment in the design process. This is particularly important in the prototyping and design of complex interactive systems which may be susceptible to human error and need to be designed with error avoidance and anticipation procedures in mind. This involves basic and experimental research on rapid prototyping, user-centered methodologies and testing, robustness assessment of interfaces, and cognitive ergonomics in the design of complex systems, command and control and decision aid systems, group interaction systems, and communication and distributed interactive systems.

6. Intelligent Interactive Systems for Persons with Disabilities

Basic research in intelligent interactive systems that has great potential for allowing persons with disabilities to not only participate fully in the workforce but also to improve their quality of life and is also supported as an element of this program. Such research might involve ways to transform information across modalities, such as among vision, touch, and speech, for more effective communication both with machines as well as with other humans. It may involve computer enrichment within a modality to overcome specific deficits of that modality. It may involve the invention of information prostheses to augment specific cognitive processes such as memory or sequencing. And, certainly, research can improve computer and communication networks to provide more effective virtual workplaces for persons with mobility impairments.