PMIW Documentation

Overview

This code provides a User Interface Management System that implements the model described in the paper: R.J.K. Jacob, L. Deligiannidis, and S. Morrison, "A Software Model and Specification Language for Non-WIMP User Interfaces," ACM Transactions on Computer-Human Interaction, vol. 6, no. 1, pp. 1-46, March 1999. [HTML]; [PDF].

It implements Links and Variables, EventHandlers, and state transition diagrams. It also provides the connection between hardware inputs (mouse, keyboard, future devices) and the UIMS.

To build a complete program, the user of this system (i.e., the programmer building a user interface with this system) would:

So the overall division of responsibilities is roughly that the UIMS handles input, receiving it and processing it through links, variables, and state diagrams; while the user's code handles graphics, output, and semantics.

Specifically, your code would create a window (X or GLX) and pass its X-window handle to the UIMS (so it can read events from it).

The UIMS then proceeds by reading events from your window, handling expose events by calling your redraw routine, and handling real input events either by sending them to a plugboard Variable or by running the appropriate discrete event handler. It will also read any other devices (such as Polhemus) and process them the same way. It recalculates the plugboard.

Then, you update the graphical output based upon changes in the plugboard Variables. If you are using Performer, you provide Update() routines that use the current values of the Variables to modify your scene graph.

To accommodate the stringent performance demands in VR, the system will ultimately afford the user control over when the redrawing and recalculation occurs and whether everything is recalculated or just some needed variables.

Connections between the UIMS and real input and output:



Installing the System

See installation instructions.

Compiling Your Programs

See compilation instructions.

To Use the System

To use this UIMS with Performer, you would do the following (See use.pf/example.cc for an example showing these steps):
  1. Instantiate a PfWindow object, which initializes Performer, creates a GLX window for it, and connects its input to the UIMS.
  2. Create your user interface objects (as described in a separate file) and install them into the Performer scene graph
  3. Execute a main loop that calls:


Using the UIMS without Performer

Since the job of this UIMS is mainly to handle input, it can also be used on its own, without Performer and even without X.

If you are not using a display list system like Performer, you must provide a redraw routine to be called in case get an expose event is received. The redraw routines would typically call further redraw methods on your user interface objects. Those methods would typically use the values contained in output Variables in redrawing the scene.

With X, you would thus write the redraw routine and install it as the callback when you create your DeviceXWindow. You can also call it directly yourself when a change in the data requires it. The UIMS (in class DeviceXWindow) will then remember the callback (and client data if any) that you passed it in its constructor and call it for you.

See use.x/examplex.cc for an example main program for using the UIMS with X. It handles the following tasks:

  1. Create a window.
  2. Write your redraw routine, which redraws the information that you want in the window, if necessary using GetE() to retrieve values from your Variables.
  3. Instantiate DeviceXWindow, and pass your X window and your redraw routine to its constructor. (If you are not using X, you would do something analogous for a different window system, and pass to your own subclass of DeviceBase)
  4. Define and instantiate your interface objects, links, and variables.
  5. Execute a main loop that calls:

It uses the XWindow convenience class in opt/XWindow to create and X window and get it connected to the UIMS. It will create the actual X window and its associated DeviceXWindow object, but you could do this manually if you prefer.

Note that XWindow code assumes there is only one instance of this class, and no tiles.

You can then draw on it yourself, via GetDisplay() and GetWindow()

For example:

		XDrawRectangle (your_xwindow->GetDisplay(), 
			your_xwindow->GetWindow(), 
			your_xwindow->GetGC(),
			x, y, width, height);

You still write the same redraw routine as with DeviceXWindow but now pass it to us instead, and we pass it to devicexwindow for you

Class XWindow doesn't get involved with redrawing in any other way..

See use.x/XSlider.cc for an example of a user interface object under X. It provides the same 8 functions described under "Example User Interface Object", except that (1) the graphics initialization is much simpler; and (2) the Update() method redraws the object on the window, rather than simply updating values in a scene graph.


Directories

All code is in /r/hci/pmiw/ (or the top level directory into which you unpacked the distribution tar file).

Directory: /r/hci/pmiw/basic/*

This is the main part of the UIMS. It reads input and uses it to operate plugboard and events.

Directory: /r/hci/pmiw/opt/*

Purpose: Optional utility features, built on top of basic/* These are things you can use or do the equivalent jobs yourself.
Links
A collection of pre-defined links for some common arithmetic operations
Quitter
A simple EventHandler object that accepts an ESCAPE key on the keyboard and exits the program.
PfWindow
A convenience class for use with Performer. It initializes performer and creates the window and creates the corresponding DeviceXWindow for you.
XWindow
A convenience class for use with X (but not Performer). It initializes X and creates one window and creates the corresponding DeviceXWindow for you.

Directory: /r/hci/pmiw/solve/*

Purpose: Flow graph solver algorithms or systems or support

Currently contains a correct but simple-minded and inefficient constraint solver. This (and other constraint solvers) can be written simply as subclasses of VariableBase and LinkBase. See discussion of Variable for more on what you need to do to subclass VariableBase.

The graph of Links and Variables you create must not have any cycles in it, and, obviously, an output Variable of a Link cannot also be an input Variable of that Link. A Link may, however, accumulate data in its own instance variables, for example, for a moving average calculation.

In general, you cannot assume when a Link will be evaluated, because the solver is free to defer evaluation when necessary to manage the available CPU time. However, a few situations require a Link that is evaluated with more reliable frequency. A special Link subclass, LinkStep, is provided for these cases.

Directory: /r/hci/pmiw/eye/*

Purpose: Code for reading and processing data received on serial port from ISCAN eye tracker (kept in separate directory, because most sites won't have this device). Our Polhemus is read via the eye tracker, so this code also obtains the Polhemus data.

Directory: /r/hci/pmiw/use.*/*

Purpose: The use.* directories all contain code that demonstrates the use of the UIMS, including links and variables, defined for each particular user interface you want to build.
use.x
Example and skeleton programs without Performer (using X or using no graphics)
use.pf
Example programs using Performer graphics, described in more detail in a separate file.
use.eye
Example programs using eye tracker (and Performer).
use.coop
Example program for multi-user interface (with Performer), described in more detail in a separate file.
use.test
Test programs, some are one-time, some are obsolete.
use.bare
Example program using the PMIW framework only for input and output, but without using the plugboard or the state diagrams. You would not normally want this.

Directory: /r/hci/pmiw/app (currently empty)

Purpose: User's simulation or other application semantics code, defined for each particular system you want to build

More Information