Joseph K. Kearney
James Cremer

Department of Computer Science
University of Iowa
Iowa City, Iowa 52242

CONTACT INFORMATION, (319) 335-0713 phone, (319) 335-3624 fax, (319) 335-0736 phone, (319) 335-3624 fax



Virtual Environments


State machines, scenario control, real-time simulation, interactive simulation, autonomous agents, reactive systems.


This objective of this research is to develop a computational framework for behavior modeling adapted to problem of creating complex scenarios in large-scale virtual environments. The project will develop a modeling system based on hierarchical, concurrent state machines that supports the control of reactive behaviors and provides mechanisms for coordinating behaviors to create critical situations and events. A second research thrust is to create authoring tools to facilitate the design of virtual environments and the definition of scenarios in which computer generated agents play central roles. This technology has application in interactive, real-time simulation environments such as vehicle simulation for driver training, virtual prototyping, and experimental studies of driving safety that require dynamic situations and circumstances to be reproduced across trials without overly restricting subject's actions and while maintaining diversity, responsiveness, and spontaneity in agent behaviors.


J. Cremer, J.K. Kearney, and Y. Papelis, "HCSM: A Framework for Behavior and Scenario Control in Virtual Environments," to be published in ACM Transactions on Modeling and Simulation.

J. Cremer, J. Kearney, Y. Papelis, and R. Romano, "The Software Architecture for Scenario Control in the Iowa Driving Simulator," Proceedings of the 4th Computer Generated Forces and Behavioral Representation Conference, May 4-6, 1994, Orlando, FL.

J. Cremer and J. Kearney, "Scenario Authoring For Virtual Environments," Proceedings of IMAGE VI Conference, July 1994, Tucson, AZ.


Our research lies in the broad domain of scenario control for dynamic virtual environments. This body of work focuses on methods to create realistic behaviors for simulated agents populating virtual environments and to adaptively manage the environment so that human subjects experience an orderly progression of events. For example, a virtual environment for training emergency room physicians or air traffic controllers might involve a series of situations where the succession of circumstances is based on a student's decisions, responses, and actions in the virtual environment.

Virtual environments present what appear to be conflicting demands on scenario control. On one hand, computer generated agents must behave in consistent and believable ways in a complex, dynamic environment. They must interact with other simulated agents and subjects who have considerable freedom of action. On the other hand, experimental and training applications require that subjects be tested under controlled conditions. The essential aspects of events and situations must be repeated from trial to trial. The challenge we face is to create scenarios that reproduce the intended conditions without overly restricting the subject's actions and while maintaining dynamism, complexity, and spontaneity in agent behaviors.

For example, consider the problem of creating a crash threat on a simulated urban freeway for a virtual driving environment. Such a scenario might be part of an experiment to determine the influence of Alzheimer's disease on the driving ability. The scenario requires generation of dense traffic that provides a backdrop for a critical situation such as an abrupt stop or dangerous lane change. In order to compare the performance of subjects, the behavior of the vehicles surrounding the driver must be carefully orchestrated -- some vehicle, possibly determined on-line, must perform the threatening deed and gaps in the traffic that provide possible escape routes must be consistently presented. Moreover, this coordination must be done inconspicuously so as not to alert subjects to the upcoming event.

Research in this area encompasses technology for:

An important component of this research is the development of effective interfaces for authoring scenarios. This includes the design of basic behaviors, the specification of ambient characteristics of the environment, and the specification of critical events in the context of the scene in which they will take place.


Bruce M. Blumberg and Tinsley A. Galyean, "Multi-level direction of autonomous creatures for real-time virtual environments," Computer Graphics (SIGGRAPH 95), August, 1995.

Rodney A. Brooks, "A Robot that Walks: Emergent Behaviors from a Carefully Evolved Network," Proceedings of the 1989 IEEE International Conference on Robotics and Automation, May, 1989.

J. Granieri, W. Becket, B. Reich, J. Crabtree, and N. Badler, "Behavioral control for real-time simulated human agents," Proceedings of the 1995 Symposium on Interactive 3D Graphics, Monterey, CA, 1995

David Harel, Hagi Lachover, Amnon Naamad, Amir Pnueli, Michl Politi, Rivi Sherman, Aharon Shtull-trauring, and Mark Trakhtenbrot, "Statemate: A working environment for development of complex reactive systems", IEEE Transactions on Software Engineering, v. 16, n. 4, April, 1990.

Margaret Kelso and Peter Weyhrauch and Joseph Bates, "Dramatic Presence", PRESENCE: Teleoperators and Virtual Environments, v. 2, n. 1, 1993.

P. Maes, Designing Autonomous Agents, The MIT Press, 1990

M. Tambe, W. L. Johnson, R. Jones, F. Koss, J. Laird, P. Rosenbloom, and K. Schwamb, "Intelligent agents for interactive simulation environments", AI Magazine, v. 16, n. 1, 1995.

Recent workshops related to behavior modeling and interaction in virtual environments include:

From Animals to Animats 3, Proceedings of the Third International Conference on Simulation of Adaptive Behavior, edited by Dave Cliff, Philip Husbands, Jean-Arcady Meyer, and Stewart W. Wilson August 8-12, 1994, Brighton, England.

AAAI 1995 Spring Symposium on Interactive Story Systems: Plot and Character, Stanford, CA, March 1995.

The First Workshop on Simulation and Interaction in Virtual Environments, Iowa City, Iowa, July, 1995.


Adaptive Human Interfaces

Usability and User-Centered Design


Collaboration with researchers in the area of Adaptive Human Interfaces to apply of behavior modeling technology to adaptive human interfaces.

Collaboration with researchers in the area of Usability and User-Center Design to improve scene and scenario authoring interfaces. A second area of potential collaboration is the development of interfaces for interactive testing and inspection of complex environments densely populated with automonous agents. Related to this is experimenter-in-the-loop simulation in which human directors guide the evolution of the scenario by sending on-line instructions to simulated agents.