From Exploratory to Hypothesis-Driven Visual Data Analysis

Abstract

PhD Defense:

The prevalent exploratory-first approach to acquire insights through visual analysis hinges on the unstructured and unguided exploration of data. As a result, visualization methodologies often fail to incorporate a user's analysis objectives and invaluable domain knowledge, leading to missed opportunities for effective visual communication and informed decision-making. While a number of conceptual frameworks, cognitive models, and task taxonomies have been proposed to address these limitations, no formal approach has been contributed to actualize these theories into practice. In this dissertation, I begin to bridge this gap by introducing a hypothesis-driven analysis approach to complement traditional open- ended visual exploration. I start by providing empirical evidence of the shortcomings of exploratory analysis through two interview studies with data scientists and subject matter experts. From these findings, I propose a grammar of hypotheses that formalizes the notion of hypothesis-driven analysis, providing a mechanism for visualization designers to extract domain knowledge and analysis questions from users. I demonstrate the utility of our formalism with two distinct case studies. The first showcases how a hypothesis grammar can be leveraged to operationalize and automate visual analysis tasks. The second describes a hypothesis- driven evaluation framework that can be used to validate the efficacy of visualizations in supporting users through hypothesis verification and insight generation. Finally, I conclude with a discussion of how I envision hypothesis-driven analysis complementing current visualization practices, ultimately advancing the scientific rigor of our field.