COOPERATIVE ASSISTANCE THROUGH VISUAL INTERACTION:
PERCEPTION AND PROBLEM-SOLVING
IN COMPLEX ENVIRONMENTS

Erika Rogers

Clark Atlanta University
Dept. of Computer and Information Science
223 James P. Brawley Dr., S.W.
Atlanta, GA 30314

CONTACT INFORMATION

E-mail: erika@pravda.cc.gatech.edu
Phone: (404) 880-6952
Fax: (404) 880-6963

WWW PAGE

http://www.cau.edu

PROGRAM AREA

Adaptive Human Interfaces.

KEYWORDS

Visual cognition, cooperative systems, intelligent agents, perception/problem-solving interaction, intelligent visual decision support

PROJECT SUMMARY

The proliferation of visualization capabilities in current technology has accelerated the need for a better understanding of how people utilize visually-displayed information to solve problems. Furthermore, such tools often increase the cognitive load on the user, who must master the toolset interface, as well as the particular skills of the task itself. Therefore, the work in this proposal addresses the nature of visual problem-solving and its impact on the design of cooperative systems which can assist and enhance human performance in complex task environments.

The first objective of the project is to extend the cognitive theory of how perception and problem-solving interact and exchange information in particular visual problem-solving tasks. Initial work involving analysis of extensive protocol data in diagnostic radiology will be extended to study task-related and experience-related trends, as well as dynamic problem-solving patterns using model animation techniques.

Additional data collection and analysis will be carried out in at least one more visual problem-solving application such as geographical information systems or power plant operations. The results of this work are expected to lead to a revised model of visual interaction, and a more comprehensive theory of how people use visual information to solve problems.

The second objective of the project is to utilize the cognitive theory as a foundation for the development of intelligent interactive systems (or agents) which can provide assistance to human decision-makers in these types of tasks. The system design will be instantiated in a number of working prototypes to demonstrate proof of concept as well as to provide platforms for testing and evaluation of underlying cognitive principles.

The research project will be carried out in the context of a comprehensive educational plan to develop the P.I.'s career as an academic professional. This plan emphasizes the development of supplemental visual problem-solving aids (including visualizations, animations and multimedia presentations) for specific core courses in an evolving computer science curriculum.

PROJECT REFERENCES

Rogers, E., 1995. "Cognitive Cooperation Through Visual Interaction. Knowledge-Based Systems 8, Nos. 2-3, April-June 1995, 117-125.

Rogers, E., 1995. "VIA-RAD: A blackboard-based system for diagnostic radiology". Artificial Intelligence in Medicine 7, 343-360.

Rogers, E., 1995. "A Cognitive Theory of Visual Interaction". In B. Chandrasekaran, J. Glasgow and N.H. Narayanan (eds.) Diagrammatic Reasoning: Computational and Cognitive Perspectives. AAAI/MIT Press: Menlo Park, CA.

Rogers, E., 1994. "Perception and Problem-Solving in Diagnostic Reasoning". Working Notes for Int'l Workshop on Principles for Diagnosis (DX-94), 252-260.

Rogers, E.,1994. "Cognitive Engineering for Intelligent Man-Machine Systems". Proc. of 1994 IEEE Int'l Conf. on Systems, Man and Cybernetics. IEEE, 2729-2734.

Rogers, E., 1994. "Intelligent Visual Interaction: A Cooperative Paradigm". Advances in Artificial Intelligence - Theory and Application II, Proc. of 7th Int'l Conf. on Systems Research, Informatics and Cybernetics. Int'l Inst. for Advanced Studies in Systems Research and Cybernetics: Windsor, Canada, 1-6.

Rogers, E. and Murphy, R.R., 1994. "Tele-Assistance for Semi-Autonomous Robots". Proc. of AIAA Conference on Intelligent Robots in Field, Factory, Service and Space. NASA Conference Publication 3251, 500-508.

Rogers, E., 1992. "Visual Interaction: A Link Between Perception and Problem-Solving". Ph.D. Dissertation, Tech Report No. GIT-CC-92/59, Georgia Inst. of Technology, Atlanta, GA.

AREA BACKGROUND

In order to build systems which can truly cooperate with humans, it is important to gain a better understanding of how humans actually perform some of these tasks. The primary technique used to collect such data is called "concurrent think-aloud reporting" -- basically we videotape subjects performing as close to their normal task as possible, and ask them to articulate their thoughts throughout the problem-solving session. These articulations are then transcribed and analyzed and incorporated into an information-processing type of cognitive model of visual interaction. Related work in cognitive studies, especially with respect to perceptual and problem-solving aspects, provides a good background for this part of the research.

Aspects of this model have been incorporated into a blackboard-style system design, which can be customized for the particular application. Blackboard architectures have been shown to be appropriate for complex domains which contain a great deal of knowledge, but not necessarily a great many "right answers" to the problems encountered. They work well with "opportunistic problem-solving", where the direction of reasoning may change as the person discovers more information and/or evidence about the problem. The knowledge bases which must be developed to support this system are significantly enhanced by information obtained during the earlier data collection phase. The intent of the system is to keep the human in the loop, and, by tracking where the person is in the perception-problem-solving cycle, provide appropriately enhanced images and/or solution hypotheses in an interactive, cooperative manner.

AREA REFERENCES

Chi, M.T.H., Glaser, R. and Farr, M.J. (1988). The Nature of Expertise. Hillsdale, NJ: Erlbaum.

Ericsson, K.A. and Simon, H.A. (1984). Protocol Analysis. Cambridge, MA: MIT Press.

Evans, D.A. and Patel, V.L., (eds.) (1989). Cognitive Science in Medicine. Cambridge, MA: MIT Press.

Fischer, G. and Reeves, B. (1992). "Beyond Intelligent Interfaces: Exploring, Analyzing, and Creating Success Models of Cooperative Problem Solving." Journal of Applied Intelligence 1, Boston: Kluwer, pp. 311-332.

Kosslyn, S.M., Flynn, R.A., Amsterdam, J.B. and Wang, G. (1990). "Components of high-level vision: A cognitive neuroscience analysis and accounts of neurological syndromes." Cognition 34 pp. 203-277.

Lesgold, A., Feltovich, P., Glaser, R. and Wang, Y. (1981). "The Acquisition of Perceptual Diagnostic Skill in Radiology." LRDC Technical Report No. PDS-1.

Nii, H.P. (1986). "Blackboard Systems: The Blackboard Model of Problem Solving and the Evolution of Blackboard Architectures." The AI Magazine, Summer 1986, pp. 38-53.

Posner, M.J., (ed.) (1989). Foundations of Cognitive Science. Cambridge, MA: MIT Press.

Norman, D.A. (1985). "Twelve Issues for Cognitive Science." In A.M. Aitkenhead, and J.M. Slack (eds.), Issues in Cognitive Modelling. Hillsdale, NJ: Erlbaum, pp. 309-337.

Woods, D.D. (1986). "Cognitive Technologies: The Design of Joint Human-Machine Cognitive Systems." The AI Magazine 6, No. 4, pp. 86-92.

RELATED PROGRAM AREAS

Usability and User-Centered Design

Intelligent Interactive Systems for Persons with Disabilities