PATTERN-DRIVEN USER INTERFACES
Dan R. Olsen Jr.
Computer Science Department
Brigham Young University
Provo, UT 84602
CONTACT INFORMATION
Email: olsen@cs.byu.edu
Phone: 801-378-2225
Surface:
Computer Science Department,
Brigham Young University,
Provo, UT 84602
WWW PAGE
http://issl.cs.byu.edu/home.html
PROGRAM AREA
Adaptive Human Interfaces
KEYWORDS
Programming by demonstration, User Interfaces, Interaction, Pictures as
Data, Information presentation
PROJECT SUMMARY
Original goals
Direct manipulation user interfaces are difficult to build. User Interface
Management Systems (UIMS) have attempted to alleviate this problem by
providing models of the interactive dialog which can drive the interface
implementation.
Such models, however, have failed to directly express the changes of
information which comprise
most operations in direct manipulation interfaces. A data pattern /transformation
language based on unification matching will be developed as
a model of such additional processes.
Techniques will be developed for using this language as a user
interface design tool. These techniques will include approaches for
visually expressing the patterns / transformations interactively. This data
transformation language can then be used for creating visualizations of data.
Such a language can express
attribute modifications, style sheets, critics (which monitor and advise
users) and searches. Unifying the interaction in such a transformation language
provides a foundation for more intelligent tools to reason about the
behavior of the interface.
Results
In order to apply pattern-based transformations to user
interfaces and
the objects that they manipulate, it is important that there
be a
uniform model for all information in the system. It is critical that
any object in the system can be accessed through that model
regardless
of the object's implementation. Without such generalized access
to
information it is impossible to reason in an automatic way about
such
objects. We call our implementation of this concept NIC (Nucleus
for Interactive Computing).
Having defined and built such a model and
then having implemented a user
interface tool-kit within that model we
then apply unification-style patterns to such objects, including
user
interface objects. Based on this model we have been able to
compute search
patterns from examples, infer widgets by example from
drawings, infer
inductive relationships in pictures, and infer class
structures for data defined as pictures rather than in
traditional
programming languages. In addition we have developed a
pattern-based
agent architecture for networked information access.
Serendipity
Because of the general data model for representing user interfaces and
the
information they manipulate, we were able easily implement
distributed user
interfaces. In our model anything can be converted to
an intermediate form
and transmitted over the network. Based on this we
have developed interactive services that are
distributed over the
World-Wide-Web. We have also used this distribution of
interfaces to
implement a tool kit for sharing collaborative interfaces
across the
network. This allows collaborative applications to be built in NIC
with little more
effort than single-user interfaces.
PROJECT REFERENCES
Papers
"Interactive Net Services on the WWW" with K.
Rodham. INTERACT '95. (June1995).
"Distributable Interactive Objects" with K. Rodham. INTERACT '95. (June
1995).
"Pictures as Input Data" with D. Kohlert. Human Factors in Computing
Systems (May 1995)
"Building Geometry-based Widgets by Example" with B. Ahlstrom and D.
Kohlert. Human Factors in Computing Systems (May 1995)
"Smart Telepointers: Maintaining Telepointer Consistancy in the Presences
of User Customization" with K. Rodham. ACM Transactions on Graphics (July
1994)
"Automatic Generation of Interactively Consistent Search Dialogs", Human
Factors in Computing Systems (April 1994).
Theses and dissertations
(available through BYU library http://library.byu.edu)
"GUI Tool Templates from Examples" Jun Lu, MS Thesis, 1995.
"Techniques and Tools for Network-based Interacton" Kenneth J. Rodham, PhD
Dissertation, 1995.
"ADAPT: Application Data as Pictures on Terminals", Douglas C. Kohlert, PhD
Dissertation, 1995.
"AGENTMAN: An Agent Manager Which Allows New Agents to Benefit Preexisting
Application", Jeffery Kim Jensen, MS Thesis, 1995.
"LEMMING: Learning Easy Maps to Make Interesting New Gadgets", Bret D.
Ahlstrom, 1995.
"An Intelligent Drawing System Based on Induction by Example Technique",
Xinyu Deng, MS Thesis, 1994.
"Magic: Malleable Graphical Applications", Douglas C. Kohlert, MS Thesis,
1993.
"Searches by Example for GUI", Ming Cai, MS Thesis, 1993.
"A Nested Parametric Coordinate-based Interactive Interface Systems",
Jinyun Lin, MS Thesis, 1993.
AREA BACKGROUND
This project has its background in User Interface Management Systems.
The goal of such systems is to abstract the definition of a user interface
out of the code into an external specification. The purpose of extracting
the user interface is so that enhanced tools can be created to aid interface designers.
Originally this intermediate form was called the dialog specification when
it was believed that most user interface software problems are related to
the handling of input events. This external specification has gradually moved
to a higher level with much
emphasis being placed on connecting visual objects to the information that
they manipulate.
The project also has roots in the "by demonstration" community where
software is developed from examples of the
desired behavior rather than
by explicit programming. The problem is that
visual information such as
screen layouts and widget manipulations do not
lend themselves well to
textual specification in programming language. The hope is that such
information can
be inferred rather than encoded.
AREA REFERENCES
"Building Real Time Groupware with GroupKit, A Groupware Tool kit",
Roseman, Mark and Greenberg, Saul, ACM Transactions on Computer-Human
Interaction, March 1996.
"User Interface Management Systems: Models and Algorithms", Dan R.
Olsen Jr., Morgan-Kaufmann, San Mateo, CA, (1992).
"Watch What I Do: Programming by Demonstration", ed. Allen Cypher,
MIT Press, Cambridge, Mass. (1993).
POTENTIAL RELATED PROJECTS
The network-based user interface techniques used in NIC would be
highly amenable to virtual environments. Using distributed rendering the
NIC information update model would make developing such shared environments
quite easy.