Computer Science 250-BCI
Brain-Computer Interaction
Fall, 2009
Syllabus

Instructor

Description

Textbook

WWW Page

Course Work

The course will begin with an overview of the area and our general goal, which is to understand what can be measured from the brain and synthesize it into designs for new forms of lightweight, adaptable brain-computer interfaces. The bulk of the course will be in the format of a graduate seminar. Students will read research papers in fields that underlie brain-computer interaction and present and discuss them in the seminar. There will be 1 or 2 presentations per class (starting in the second week and running through the 11th or 12th week). For each paper, one student will study the paper and present it to the class for discussion. The rest of the class is required to read the paper before class, bring a question about the paper to class to discuss, and participate in discussing and evaluating the paper.

Presentations are 25-30 minutes in length. A presentation should include a summary and analysis of the paper. Then try to put this paper in perspective with respect to our goals and framework for BCI and also with respect to other work in its field (this may require looking at other related papers). After the presentation, the presenter will lead a discussion on the merits of the paper.

We will also have some projects later in the semester, where you will plan and design new interfaces or experiments, and discuss, obtain feedback, and refine.

Prerequisite

Topic Outline and Preliminary Reading List

• Overview of brain (wikipedia)
• An introduction to EEG
• Paper on fMRI (sign on from tufts.edu for access)

• Read the following sections from this web site:
  • Machine Learning, Part I: Types of Learning Problems
  • Machine Learning, Part II: Supervised and Unsupervised Learning
  • Machine Learning, Part III: Testing Algorithms, and The "No Free Lunch Theorem"
  • Decision Trees, Part I: How Decision Trees Work
• Lee, J.C. and Tan, D.S. Using a low-cost electroencephalograph for task classification in HCI research Proceedings of the 19th annual ACM symposium on User interface software and technology, ACM Press, Montreux, Switzerland, 2006. [Link] (ACM digital library, access from tufts.edu or else via Tisch library web page to get the text)
• First 2 pages of: E.T. Solovey, A. Girouard, K. Chauncey, L.M. Hirshfield, A. Sassaroli, F. Zheng, S. Fantini, and R.J.K. Jacob, "Using fNIRS Brain Sensing in Realistic HCI Settings: Experiments and Guidelines," ACM UIST 2009 Symposium on User Interface Software and Technology, ACM Press (2009) [Link]

• Several presenters, it is a long comprehensive paper, split it into 2 or 3 subtopics to present Fairclough, S.H. Fundamentals of physiological computing. Interacting with Computers, 21 (1-2). 2009, 133-145. [Link, obtain via Tufts library]
• See also: Allanson, J. and Fairclough, S.H. A research agenda for physiological computing. Interacting with Computers, 16. 2004, 857-878.

• Ramnani, N. and Owen, A.M. Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat Rev Neurosci, 5 (3). 2004, 184. [Link]
• Ridderinkhof, K.R., van den Wildenberg, W.P.M., Segalowitz, S.J. and Carter, C.S. Neurocognitive mechanisms of cognitive control: The role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain and Cognition, 56 (2). 2004, 129-140. [Link]

• E.T. Solovey, A. Girouard, K. Chauncey, L.M. Hirshfield, A. Sassaroli, F. Zheng, S. Fantini, and R.J.K. Jacob, "Using fNIRS Brain Sensing in Realistic HCI Settings: Experiments and Guidelines," ACM UIST 2009 Symposium on User Interface Software and Technology, ACM Press (2009) [Link]

• IEEE Computer special issue on Brain Computer Interaction, volume 41, no. 10, Oct 2008 [Link] (IEEE Xplore digital library, access from tufts.edu or else via Tisch library web page)
• Ramsey, N.F., van de Heuvel, M.P., Kho, K.H., Leijten, F.S.S., Towards human BCI applications based on cognitive brain systems: an investigation of neural signals recorded from the dorsolateral prefrontal cortex, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2006, Volume 14, No. 2, pp. 214-217. [Link]
• Adams, R., Bahr, G.S. and Moreno, B., Brain Computer Interfaces: Psychology and Pragmatic Perspectives for the Future. in AISB 2008 Convention, (Aberdeen, Scotland, 2008).
• Cichocki, A., Washizawa, Y., Rutkowski, T., Bakardjian, H., Phan, A.-H., Choi, S., Lee, H., Zhao, Q., Zhang, L. and Li, Y. Noninvasive BCIs: Multiway Signal-Processing Array Decompositions. IEEE Computer, 41 (10). 2008, 34-42. (From IEEE Computer special issue on BCI listed above)

• General
  • Davidson, R.J., What does the prefrontal cortex "do" in affect: Perspectives on frontal EEG asymmetry research, Biological Psychology 67 (2004) 219-233. [Link to journal] (emailed to class)
  • Hongyu Yang, Zhenyu Zhoua, Yun Liu, Zongcai Ruana, Hui Gongb, Qingming Luob, Zuhong Lua, Gender difference in hemodynamic responses of prefrontal area to emotional stress by near-infrared spectroscopy. Behavioural Brain Research 178 (2007) 172-176. [Link to journal] (emailed to class)
• Workload / Working Memory
  • Smith, E, and Jonides, J. Storage and Executive Processes in the Frontal Lobes. Science Volume 283. 1999. 1657-1661. [Link]
  • Fairclough, S., Venables, L., Tattersall, A. The influence of task demand and learning on the psychophysiological response. International Journal of Psychophysiology 56 (2005) 171-184.
  • Ryu, K., Myung, R. Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics 35 (2005) 991-1009
  • Goldberg et al. Uncoupling Cognitive Workload and Prefrontal Cortical Physiology: A PET rCBF Study. NEUROIMAGE 7, 296-303 (1998).
  • Gevins, A and Smith, M. Neurophysiological Measures of Working Memory and Individual Differences in Cognitive Ability and Cognitive Style. Cerebral Cortex, Sep. 2000, 10: 829-839. [Link]
• Emotion / Affect
  • Davidson R.J., Sutton S.K. Affective neuroscience: the emergence of a discipline. Current Opinion in Neurobiology, Volume 5, Number 2, April 1995 , pp. 217-224(8)
  • Jose Leon-Carrion, Juan Francisco Martin-Rodriguez, Jesus Damas-Lopez, Kambiz Pourrezai, Kurtulus Izzetoglu, Juan Manuel Barroso y Martin, Maria Rosario Dominguez-Morales. A lasting post-stimulus activation on dorsolateral prefrontal cortex is produced when processing valence and arousal in visual affective stimuli. Neuroscience Letters 422 2007
  • K. Phan, S. Taylor, R. Welsh, L. Decker, D. Noll, T. Nichols, J. Britton, I. Liberzon. Activation of the medial prefrontal cortex and extended amygdala by individual ratings of emotional arousal: a fMRI study. Biological Psychiatry, Volume 53, Issue 3, Pages 211-215
  • Joseph E. LeDoux. EMOTION: Clues from the Brain (1995) Ann. Rev. Psych.
• Other areas?
  • Yoichi Miyawaki et al, Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders, Neuron 60, pp. 915-929, December 11, 2008 Elsevier Inc.

• How does fNIRS work?
  • Rolfe, P. In vivo near-infrared spectroscopy. Annual Review of Biomedical Engineering, August 2000, Vol. 2, Pages 715-754 [Link]
  • Bunce, S., Izzetoglu, M., Izzetoglu, K., Onaral, B. and Pourrezaei, K. Functional Near Infrared Spectroscopy: An Emerging Neuroimaging Modality., IEEE Engineering in Medicine and Biology Magazine, 25 (4), pp. 54-62.
  • Villringer, A. and Chance, B., Non-Invasive Optical Spectroscopy and Imaging of Human Brain Function, Trends in Neuroscience, 20, pp. 435-442.
  • Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C., Murray, T., Ovetsky, Y. and Thomas, R., A novel method for fast imaging of brain function, non-invasively, with light, Optics Express, 10 (2), pp. 411-423.
  Measuring brain activity with fNIRS
  • S. Coyle, T. Ward, C. Markham, G. McDarby. "On the Suitability of Near-Infrared Systems for Next Generation Brain Computer Interfaces". World Congress on Medical Physics and Biomedical Engineering, Sydney, Australia, IFMBE, 2003
  • Hoshi, Y. Tamura, M. Near-Infrared Optical Detection of Sequential Brain Activation in the Prefrontal Cortex during Mental Tasks. NEUROIMAGE 5, 292-297 (1997)
  • Herrmann, M.J., Ehlis, A. C., Fallgatter, A. J. Frontal activation during a verbal-fluency task as measured by near-infrared spectroscopy. Brain Research Bulletin 61 (2003) 52-56.
  • Nagamitsu, S., Nagano, M., Yamashita, Y., Takashima, S.Toyojiro Matsuishi. Prefrontal cerebral blood volume patterns while playing video games--A near-infrared spectroscopy study. Brain & Development 28 (2006) 315-321
  • "Spatial and temporal analysis of human motor activity using noninvasive NIR topography". Maki A. et al. Med. Phys. 22 (12), Dec 1995.
  • "Prefrontal Hypooxygenation during Language Processing Assessed with Near-Infrared Spectroscopy". Falgatter, A. J., Muller, Th. J., Strik, W. K. Neuropsychobiology 1998; 37: 215-218.
  BCI Studies with fNIRS
  • Shirley M Coyle, Tomas E Ward and Charles M Markham. Brain-computer interface using a simplified functional near-infrared spectroscopy system. 2007 J. Neural Eng. 4 219-226 [Link]

• Ferrez, P. Millan, J. You Are Wrong!--Automatic Detection of Interaction Errors from BrainWaves. in Proceedings of the 19th International Joint Conference on Artificial Intelligence, August 2005.
• Gerwin Schalk, Jonathan R. Wolpaw, Dennis J. McFarland, Gert Pfurtscheller. EEG-based communication: presence of an error potential. 2000. Clinical Neurophysiology 111
• Kiern, Z. A., Aunon, J. I. A New Mode of Communication Between Man and His Surroundings. IEEE Transactions on Biomedical Engineering, Vol. 37, No. 12. 1990
• C.W. Anderson and Z. Sijercic. Classification of EEG signals from four subjects during five mental tasks. Intl. Conf. on Engineering Applications of Neural Networks, 407--414, 1996. [Link]
• Kostov, A. Polak, M. Parallel Man-Machine Training in Development of EEG-Based Cursor Control. IEEE Trans. On Rehabilitation Engineering , Vol. 8, No. 2. 2000.
• Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for cursor control. Electroencephalogr Clin Neurophysiol. 1991 Mar; vol. 78, no. 3, pp. 252-9.
• Leeb et al., Walking from thoughts: Not the muscles are crucial, but the brain waves!, 8th Annual International Workshop on Presence, PRESENCE 2005, 21-23 September 2005, London. [Link]

• R.J.K. Jacob, "The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At is What You Get," ACM Transactions on Information Systems, Vol. 9(3) pp. 152-169 (April 1991).
• Pomplun, M. and Sunkara, S. Pupil Dilation as an Indicator of Cognitive Workload in Human-Computer Interaction. Human-Centred Computing: Cognitive, Social, and Ergonomic Aspects. Vol. 3 of the Proceedings of the 10th International Conference on Human-Computer Interaction, HCII 2003, Crete, Greece, 542-546. [Link]
• Iqbal, S. T., Zheng, X. S., and Bailey, B. P. 2004. Task-evoked pupillary response to mental workload in human-computer interaction. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM Press, New York, NY, 1477-1480[Link]
• Saulnier, P., Sharlin, E. and Greenberg, S. Using bio-electrical signals to influence the social behaviours of domesticated robots Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, ACM, La Jolla, California, USA, 2009.
• Krzysztof Z. Gajos, Katherine Everitt, Desney S. Tan, Mary Czerwinski, Daniel S. Weld, Predictability and Accuracy in Adaptive User Interfaces. ACM CHI 2008 paper (emailed to class)

• Parasuraman, R., Mouloua, M. and Molloy, R. Effects of adaptive task allocation on monitoring of automated systems. Human Factors, v38 (n4). p665(625). [Link]
• Wilson, G.F. and Russell, C.A. Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding.(Author abstract). Human Factors, 49 (6). 1005(1014). [Link]

• Izzetoglu M, Devaraj A, Bunce S, Onaral B, (2005). Motion Artifact Cancellation in NIR Spectroscopy Using Wiener Filtering. IEEE Transactions on Biomedical Engineering, 52(5):934-938. [Link]
• Gevins et al. Towards measurement of brain function in operational environments. Biological Psychoogy 1995 May;40(1-2):169-86.
• Noel, J., Baue, K., Lanning, J. Improving pilot mental workload classifcation through feature exploitation and combination: a feasibility study. Computers & Operations Research 32 (2005) 2713-2730.
• Coyle, S., Ward, T., Markham, C. Physiological Noise in Near-infrared Spectroscopy: Implications for Optical Brain Computer Interfacing. Proc. of the 26th Annual International Conference of the IEEE EMBS (2004)
• Haihong, Z., Cuntai, G. A Kernel-based Signal Localization Method for NIRS Brain-computer Interfaces. ICPR (2006)
• Sitaram et al. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 34 (2007) 1416-1427.