PhD Defense: Towards Adaptive User Interfaces using Real Time fNIRS

May 28, 2010
10:30 am
Halligan 111A
Speaker: Audrey Girouard


Enhancing user experience is a constant goal for human computer interaction (HCI)researchers, and the methods to achieve this goal are widespread, from changing the properties of the interface to adapting the task to the user's ability level. By sensing user's cognitive states, such as interest, workload, frustration, flow, we can adapt the interface immediately to keep them working optimally. This new train of thought in the brain computer interfaces community considers brain activity as an additional source of information, to augment and adapt the interface in conjunction with standard devices, instead of controlling it directly with the brain.

To obtain measured of brain activity, I adopt the relatively less-explored brain sensing technique called functional near-infrared spectroscopy (fNIRS), a safe, non-invasive measurement of changes in blood oxygenation. This dissertation presents a body of technologies and tools that enable the use of real time measures of cognitive load for adaptive interfaces, to support that fNIRS is an input technology with potential for HCI, especially when applied to the general, healthy public as an additional input.

First, I discuss the practicality and applicability of the technology in realistic, desktop environments. Our work shows that fNIRS signals are robust enough to remain unaffected by typing and clicking but that some facial and head movements interfere with the measurements. Then, I investigate the use of fNIRS to obtain meaningful data related to mental workload. My studies progress from very controlled experiments that help us identify centers of brain activity, to experiments using simple user interfaces, showing how this technique may be applied to more realistic interfaces. Our first study distinguishes levels of workload and interaction styles, and our second differentiates levels of game difficulty. Statistical analysis and machine learning classification results show that we discriminate well between subjects performing a mentally demanding task or resting, and distinguish between two levels with some success. Finally, I present a real time analysis and classification system that can communicate user cognitive load information to an application. I categorize adaption of interfaces with brain as an input, and propose a series of adaptation possible using our system.