Supporting Understanding and Appropriate Use for Human-AI Collaborative Systems

Abstract

This talk will discuss the design and evaluation of two human-AI collaborative systems for supporting understanding and appropriate use. For the first system, our goal was to study how human-AI collaboration can improve conflict resolution for data labeling, by enabling users to automate groups of conflict resolution tasks. Human data labeling with multiple labelers and the resulting conflict resolution remains the norm for many enterprise machine learning pipelines, despite the time and cost needed. We designed and prototyped a system where an AI would assist and then investigated how and when users rely on labeler and AI information and on automated group conflict resolution. For the second system, we explored whether explanations can support effective authoring of natural language utterances and how those explanations impact users' mental models in the context of a natural language system that generates small programs. We compared two main types of explanations: 1) system- focused, which provide information about how the system processes utterances and matches terms to a knowledge base, and 2) social, which provide information about how other users have successfully interacted with the system.

Bio:

Michelle Brachman is a Staff Research Scientist at IBM Research in Cambridge, MA, working in human-centered AI. Her work focuses on supporting people in understanding and working effectively with AI systems, such as through explanations. Prior to working at IBM Research, Michelle was an Assistant Professor of Computer Science at UMass Lowell. She received her PhD in Computer Science from Washington University in St. Louis, where she focused on designing and evaluating systems to support novice and end-user programming and did her undergraduate degree in Computer Science at Tufts.