Automating & Customizing the Web With Keyword Programming

September 4, 2008
2:50 pm - 4:00 pm
Halligan 111
Speaker: Rob Miller, MIT
Host: Rob Jacob

Abstract

The migration of applications to the World Wide Web opens up new opportunities for user interface customization. Applications that would have been uncustomizable on the desktop sprout numerous hooks for customization when implemented in a web browser, without any effort on the application developerís part. These hooks can be used not only for automating web user interfaces (clicking links, filling in forms, and extracting data) but also for customizing them (changing appearance, rearranging components, and inserting or removing user interface widgets or data).

The openness and flexibility of the web platform enables customizations that would not have been possible on the desktop. For example, we have customized several webmail clients (including GMail) to address a common but underappreciated security problem: sending email to the wrong recipients. Pressing Reply All by accident is one example of this problem. Our solution, a technique that displays the recipients' faces during email composition, demonstrates both web customization (changing the appearance of GMail) and automation (looking up face images automatically from a variety of web sources).

These kinds of customizations generally require scripting, but many web users do not know how to write script commands. By drawing from experience with search engines, however, we have found that users can write a set of keywords expressing a command, such as "click I'm Feeling Lucky button", "push the Lucky button", or even just "feeling lucky", which an interpreter can convert into an appropriate script command. We call this technique "keyword programming", since it relies only on keywords, and not on formal syntax or even natural language grammar.

This talk will discuss some of our explorations into keyword programming in the web automation domain, and also in other domains such as Java development. One surprising result is that programming language syntax often has relatively little information content, and can be inferred automatically from only a handful of keywords -- allowing us to design programming systems that reduce the learning and complexity burdens on their users.

BIO: Rob Miller is an associate professor in MIT's Department of Electrical Engineering and Computer Science, and a member of the Computer Science and Artificial Intelligence Laboratory. He earned his Ph.D. in Computer Science from Carnegie Mellon University in 2002, and his dissertation earned the CMU SCS Distinguished Dissertation award and received an honorable mention in the ACM Distinguished Dissertation competition. He received the NSF CAREER award in 2005. His research interests span human-computer interaction, user interfaces, software engineering, and artificial intelligence. His current research focus lies at the intersection of programming and user interfaces, with the goal of reducing the complexity barriers that make programming difficult for novices and experts alike.