KEYWORDS:
User interface software and technology, human-computer interaction,
future, prediction, UIST'2007
The conference this year is the tenth anniversary of UIST. The keynote talk by John Sibert discusses the history of UIST over the last ten years; this panel looks into the future of the field over the next ten. For example, will our field still exist as a separate discipline? Will it have expanded and divided into subdisciplines? Or become part of some other field? And what will we study in the field of User Interface Software and Technology? Would you expect to see a paper on a new toolkit for building GUI's at UIST'07? How about an automatic design method for generating user interfaces? Or a software formalism for handling neural implant interfaces?
Our panelists are all pioneers and leaders in the field of User Interface Software and Technology and are in a good position to take an insightful, long view of the future of our field. Each will describe his scenario for what life will be like when we meet for UIST'07 (ten years from now). Then, since their predictions can't turn out to be true simultaneously, they will have a chance to challenge or question each other's scenarios. This will be followed by open discussion with the audience.
To start the ball rolling, here are some slightly exaggerated extreme predictions:
Much has changed from UIST '88 to UIST '97. Research computing that was done mostly on workstations then is routinely performed on PCs now. The expensive high-end 3D graphics devices that were barely visible at UIST '88, pale in comparison to today's inexpensive 3D PC graphics cards. Laptops and cell phones are now ubiquitous, and PDAs have weathered their early disappointments to play an important, if limited, role for many of us. Hypertext, which was being popularized by single-user HyperCard stacks in the late 80's, is now synonymous with the mass confusion of the Web. The desktop GUI-only orientation of UIST '88 has metamorphosed into the rich mix of graphics, audio, speech, and haptics presented at this year's conference. On the other hand, much of this progress is incremental. Leafing through the UIST '88 and '96 proceedings, one of the biggest differences that I notice in the figures is quite literally in the window dressing: fake 3D window frames and buttons had yet to take over in 1988!
While we may not have been able to predict all these changes in the early years of UIST, they were foreshadowed by work from that time. Therefore, my predictions for UIST '07 have as a theme the mainstreaming of some of today's research, with a few surprises thrown in. Ten years from now, I expect that attendees of a very healthy and pluralistic UIST '07 will gather at a set of primary linked conference sites, supplemented by others who can participate only electronically. Here are some of the areas that I think they (we) will be exploring:
Wearable UIs. As PDAs grow far more powerful than current desktop machines, advances in batteries, wireless networking, displays, input devices, and packaging, will make it possible for them to be comfortably and affordably integrated into clothing. Wearable UIs that are to be used anytime and anywhere (and, for many of us, everytime and everywhere) will have to be very different from the mobile UIs of laptops or even current PDAs. This research area can be seen in part as the union of work on augmented reality and mobile computing.
Hybrid UIs. In addition to having high-quality, affordable, headworn displays, we will at last be surrounded by high-quality, affordable, wall-sized and desk-sized displays. UIST '07 will address new ways to accommodate this broad range of technologies, which will be incorporated into hybrid UIs that combine input and output devices that are big and little, 2D and 3D, stationary and mobile.
Environment management. Controlling dynamic, shared, distributed, information spaces presented on hybrid UIs will be far too daunting a task to accomplish with current low-level window managers. UIST '07 researchers will develop environment management software that will make possible high-level manipulation of complex information spaces.
Automated design and control. Faster processors will make it possible to use large knowledge-based and constraint-based systems in response-time-critical interaction loops. UIST '07 will include research on using these technologies to design and control customized, interactive, multi-user, multimedia presentations, building on current work in information visualization, sonification, and multimedia generation.
Retrofitting legacy UIs. Our investment in legacy applications and their (G)UIs will only increase over the next decade. One topic for UIST '07 will be how to develop ways to use these applications within future environments, both with their original UIs and and with new UIs that are efficiently retrofit to an application's underlying functionality.
Two questions are important here: what types of user interfaces will we be crafting in 10 years, and what types of software tools will be be using?
In 10 years, by far the most prevalent interfaces will be web-centric. The growth of the web will further accelerate both in users and uses and in ubiquity. Not only will we have the obvious GUI-style web interface of today's browsers and tomorrows's integrated desktop / file system / browser, but we will also have interfaces that have been scaled, up and down, to accomodate different bandwidths (from high-speed video-on-demand to low-speed cellular) and different sizes (from wrist-watch/pager size to wall-size). But, beyond that, the web will be everywhere and so too will our interfaces. The web will be in our homes (imagine each light switch and appliance having a URL to support home automation, imagine a web browser / bulletin board built into the refrigerator door) and in our cars (with the car navigation system and cellular phone and radio electronically integrated, with access to roadside and traffic and weather information via the web).
That is to say, many more of the interfaces will be of the ubiquitous, imbeddded, mobile appliance style as opposed to the explicit GUI style. Multimodal inputs (voice, vision-recognized gestures) and outputs will play more significant roles. Already, at MERL, we have used low-cost vision systems to control a TV set with hand gestures, to control a game with body movements, and to recognize the orientation of a face.
What we lack are software tools for such applications. We can expect papers on tools that support multi-modal, parallel inputs, that can integrate multiple inputs. We still need a UI toolkit that makes building "Put that there" a trivial exercise. There will be papers about this, and VUI (Vision User Interface) toolkits to support gesture and body/facial expression/orientation recognition and utilization.
We will see many more papers dealing with specialized domains, both vertical application domains where much more knowledge of application domain semantics is embedded in the toolkit, or where the semantics of a specific type of delivery package (Pilot, pager, wrist-watch, smart cellular phone) are embedded. Some papers will deal with adaptation or scaling to
"Publishing" a data base onto the web, and "fill in the form" input on the web are actually quite trivial interfaces compared to create / delete / drag / resize direct manipulation interfaces. Hence the web presents a real opportunity for automatic UI generation software to be widely deployed, simply because the semantics of the interface are simple. And, as Java-based toolkits are developed, there is fresh opportunity for their widgets to be built in ways that support knowledge-based automatic design. Similarly, we will be seeing papers dealing with automatic adaptation of web content for different types of application environments and delivery platforms.
Will all this really happen in ten years? Some of it will, but not always in the pervasive, rapid way that the web has reached most desktop computers. In the case of the web, the infrastructure development and deployment had been going on for 20 years. Appropriate infrastructures for some of the futuristic scenarios will take time to develop. Note that the reason the video-on-demand bubble burst a few years ago was that the infrastucture development costs were much higher than initially appreciated and the demand for VOD services was lower than predicted. A similar fate awaits future scenarios that depend on rapid changes in infrastructure, such as homes being wired with cat 5 or firewire (IEEE 1394) cables or expressways being equipped with sophisticated monitoring and control equipment.
User Interface innovation springs from two sources: technological advances and a deep understanding of human capabilities and requirements. Furthermore, successful innovation typically involves both. For example, the second generation of user interfaces, commonly called WIMP interfaces (Windows, Icons, Menus, Pointing), involved both the development of bitmapped displays (as a technological advance over character and vector displays) and the understanding that people more easily recognize commands than remember them.
I believe the critical human requirement that will drive UIST into its second decade is the increased integration of computation into our physical lives. After all, the power of computation is restricted if it can only be accessed by sitting at a desk manipulating a mouse and keyboard because only some of our cognition is solitary and desk-oriented, such as authoring documents. The rest of the time, computation and user interfaces can only be useful to us if they become part of our lives while we interact with each other and manipulate physical objects.
The technological trends are encouraging. Laptops continue to get smaller and more powerful. The Pilot has revived interest in stylus-based user interfaces and has demonstrated hotsyncing applications between computers. The World Wide Web is creating new models for the delivery of applications and their user interfaces, as well as creating a huge collection of online documents and services. Faster computers are beginning to support effective interpretation of sensor data and improved control of robotic manipulators. Clearly, effective speech understanding and vision interpretation will have a big impact on how we design our user interfaces.
One of the key research topics at UIST'007 will be the design and implementation of user interfaces that utilize a multitude of computers spread over space and time while giving users the experience of a single application that is integrated with their physical lives. One issue will be how to design and implement physical spaces that contain multiple computational appliances supporting a specific application or multiple applications. Another issue will be how to develop applications that follow users over time, adapting to changes in the computational infrastructure as users move through their day.
HCI is the future of computer science. We have spent the last 50 years learning how to create computers and how to program computers. The next frontier is in actually making computers serve and adapt to human needs rather than forcing humans to adapt. The ultimate purpose of computation is to leverage the human intellect across the barriers of time, space and force. Computing achieves this leverage by providing control, communication and information that is far beyond what a given individual can do in a given amount of time. Computation provides the connectivity among human intellect, information acquisition and effecting actions in the physical world.
Other disciplines can provide the advances in acquisition and advances in technology to act in the physical world. The constant in all of this is human ability to absorb information, make sense of it, decide on a course of action and express that action in a way that will effect the environment. As technology becomes cheaper the abilities of humans will remain constant. Understanding how humans interact with technology will continue to grow in importance as the human portions of the system continue to dominate in cost. Optimizing the human portions of this equation rather than the technical aspects will be increasingly important.
The interfaces of the future will be more closely adapted to the way people naturally work and live. There will be many more computers per person and they will be much more specialized in function. With the increase in specialized computers will come an increase in their integration and more flexible software architectures. We will need to design with interactive hardware with the same facility that we design GUI screens today. Much more of our interactions will be oriented towards signal-based technologies like voice and video as these are the natural modalities for human beings. Computer mediated communication must become a pervasive part of every interface because nobody works alone. Three dimensional interfaces will remain an important niche but it is not the dominant solution of the future. The dominant solution will be "habitable computers" which are part of the living and working environment which we need not wear but are always accessable.