Grad research talk: Methods and Optimizations for Interactive Big Data Exploratory Visualization Systems
Visual data exploration requires that the underlying data be available at interactive speeds. As the amount of data increases and can no longer fit in memory, the data must be stored on disk or in a remote database, delaying query response time and degrading the exploration process. Visualization applications are now becoming visualization systems, incorporating a frontend interface and backend database system that still need to provide a responsive user experience. This requires prioritizing speed over accuracy in data visualization, with common approaches being online aggregation, sampling, precomputation and prefetching of data. This talk will present research into techniques that improve database query response time in two ways. The first is by extending the latest in online aggregation to effectively support visualization tasks. This is done through importance sampling and enabling user control over the online aggregation process. The second is by integrating the use of database query planning tools in visualization systems. I will also discuss future work in how these new, faster systems can influence visualizations and user workflows, and also deciding when to use online aggregation, sampling, precomputation or prefetching to achieve interactivity.