Combining Crowd Worker, Algorithm, and Expert Efforts to Accurately and Efficiently Annotate Images
While traditional approaches to image analysis have typically relied upon either manual annotation by experts or purely-algorithmic approaches, the rise of crowdsourcing now provides a new source of human labor to create training data or perform computations at run-time. Given this richer design space, how should we utilize algorithms, crowds, and experts to better annotate images? As a case study, I focus on image segmentation, an important precursor to solving a variety of fundamental image analysis problems, including recognition, classification, tracking, registration, and 3D visualization. In this talk, I will discuss research to analyze and combine trained experts, crowdsourced non-experts, and algorithms to demarcate object boundaries in biomedical and everyday images. An exciting finding is that hybrid algorithm-crowdsourcing systems can be designed to produce segmentations indistinguishable from those of experts. More generally, this work suggests promising avenues for research at the intersection of computer vision, human computation, and medical image analysis which catalyzes new problems, methodologies, and applications in each field.
Danna Gurari is a senior PhD candidate at Boston University in the Image and Video Computing Group under the supervision of Dr. Margrit Betke. Her research interest spans computer vision, human computation, and medical image analysis with a focus on developing new tools to accelerate biomedical research. Her current work is on combining efforts of crowdsourcing and algorithms to extract accurate boundaries of objects in images. In 2007- 2010, Danna worked at Boulder Imaging building custom, high performance, multi-camera recording and analysis systems for military, industrial, and academic applications. From 2005-2007, she worked at Raytheon developing software for ground stations of satellite systems.
Danna earned her MS in Computer Science and BS in Biomedical Engineering from Washington University in St. Louis in 2005, with her thesis on ultrasound imaging. She was awarded the 2014 MICCAI IMIC Best Paper Award for Innovative Idea for research that demonstrates how to leverage crowdsourced workers in order to successfully apply level set based algorithms. She was awarded the 2013 WACV Best Paper Award for research that proposes how to articulate the performance of segmentation methods.