Interactive Systems Program
The Interactive Systems Program supports scientific and engineering
research fundamental to the design of technology and systems for human
communication with computers and using computers.
encompass but are not limited to: information access and usability;
speech recognition and natural language understanding; alternative
input/output modalities such as gesture, facial recognition, and
haptics; interfaces for people with disabilities; and virtual
environments, where natural and artificial entities interact with shared
This program recognizes and encourages the emergence of new approaches
and the use of novel and realistic environments instrumented to capture
human expression and signals in order to explore and validate hypotheses
about the laws governing human-computer interactions and the principles
of their use in various domain tasks.
It encourages the discovery or
refinement of computer models of perceptual/sensory and cognitive human-
The program scope includes visualization of and
interaction with real or virtual, complex, high-dimensional numeric,
symbolic, or pictorial knowledge in computer assisted environments, as
well as measurement and evaluation of the performance of the models and
of the methodologies used in human-computer interactive systems.
objectives of the program also include upgrading the human resources and
infrastructure of human-computer interaction education and research, and
encouragement to women, minorities, and persons with disabilities to
participate in scientific research in this area.
Major Program Elements
1. Virtual Environments
This element supports research the representation and manipulation of
complex, high-dimensional, physical or abstract information for the
enhancement of human interaction with computers and human performance in
This area includes studies on the scientific basis for
visualization and virtualization, interface semiotics, visualization and
manipulation of programming objects, visualization of task
decomposition, principles of human exploration, comprehension, and
understanding of representations.
Some examples of investigative
questions are those which address efficiency, transparency, the human
sense of naturalness, fidelity, clarity, or other characteristics of
interactions with representations.
Some example domains of
investigation include telerobotics, virtual prototyping, mapped
interactions with very small scale or very large scale systems, or
interactions with models of events that occur at past or future times.
2. Speech and Natural Language Understanding
This is one of the "Grand Challenge" areas, related to high performance
computing and communication.
Its eventual goal is reliable and robust
human interaction with the computer through spontaneous, user-
independent, natural language in real time, possibly in a multilingual
Many aspects of basic human-communication research are
involved in this element of the program:
Semantic aspects of speech and
natural language; spontaneous speech and language, recognition,
analysis, and synthesis; syntactic, semantic, pragmatic, and prosodic
factors; signal processing, symbolic, and connectionist architectures;
models of the auditory and vocal tracts and related cognitive functions
as they are associated with machine recognition and synthesis of speech,
and the automation of the processes of speech/language acquisition and
dialogue models and response generation to queries; and
finally, their place in multimodal interactive systems.
3. Other Communication Modalities
This element focuses on determining and understanding basic principles
of human expression for input and computer facial animation for output.
It includes studies of human-generated or human-controlled sounds and
vibrations, tones, music, handwriting and stylus interfaces, gestures,
posture, body language, facial expression, tactile, haptic and other
motor channels, chemical senses (olfaction, taste) and effectors, even
electromagnetic input-output (e.g. electric, magnetic, or optical
measurements) to detect human commands, human intent, human states of
perception, cognition or affective states (e.g., attention, confusion,
satisfaction, etc.) and their use to guide computer simulations or
4. Adaptive Human Interfaces
This element supports basic research aimed at making computers adapt
dynamically to human users to enhance task performance.
The focus is
on human physical, physiological, psychological, perceptual, or
cognitive interactive behavior and their use in dynamically adaptive
models of human-computer interactions.
One example is intelligent
automatic sequencing and spatial organization of visual or auditory
information to match the expressed or implied needs and goals of the
user based on the dynamic discovery of those decision and performance
strategies that humans use in stressful or constrained situations. Other
environments for intelligent agent research involve the retrieval of
information from a characterization of the user habits in database
search, in learning and educational environments, and in a variety of
5. Usability and User-Centered Design
This element includes the scientific study of the factors influencing
the human productivity, acceptability, and comfort of human-computer
interfaces and the incorporation of those factors and their assessment
in the design process.
This is particularly important in the
prototyping and design of complex interactive systems which may be
susceptible to human error and need to be designed with error avoidance
and anticipation procedures in mind.
This involves basic and
experimental research on rapid prototyping, user-centered methodologies
and testing, robustness assessment of interfaces, and cognitive
ergonomics in the design of complex systems, command and control and
decision aid systems, group interaction systems,
and communication and
distributed interactive systems.
6. Intelligent Interactive Systems for Persons with Disabilities
Basic research in intelligent interactive systems that has great
potential for allowing persons with disabilities to not only participate
fully in the workforce but also to improve their quality of life and is
also supported as an element of this program.
Such research might
involve ways to transform information across modalities, such as among
vision, touch, and speech, for more effective communication both with
machines as well as with other humans.
It may involve computer
enrichment within a modality to overcome specific deficits of that
It may involve the invention of information prostheses to
augment specific cognitive processes such as memory or sequencing.
certainly, research can improve computer and communication networks to
provide more effective virtual workplaces for persons with mobility