Computer Science 250-BCI
Brain-Computer Interaction
Fall, 2007
Syllabus


Instructor

 Robert J.K. Jacob
Dept. of Computer Science
Halligan Hall
http://www.cs.tufts.edu/~jacob/

Description

 This is a seminar course exploring new forms of human-computer interaction based on measurement of brain function and properties. Students will read research papers in several related disciplines and present and discuss them in the seminar. We will explore the use of brain sensing technologies to detect specific forms of brain activity, with a focus on functional near-infrared spectroscopy work at Tufts. We will then explore ways to use such measurements as input to new forms of lightweight, adaptable user interfaces.

Textbook

 None; individual articles to read.

WWW Page

 http://www.cs.tufts.edu/~jacob/250bci/

Course Work

The course will begin with an overview of the area and our general goal, which is to understand what can be measured from the brain and synthesize it into designs for new forms of lightweight, adaptable brain-computer interfaces. The bulk of the course will be in the format of a graduate seminar. Students will read research papers in fields that underlie brain-computer interaction and present and discuss them in the seminar. There will be 3 presentations per class (starting second week and running through the 11th or 12th week). For each paper, one student will study the paper and present it to the class for discussion. The rest of the class is required to read the paper before class and participate in discussing and evaluating it.

Each person is responsible for 3 presentations. Presentations are 25-30 minutes in length. A presentation should include a summary and analysis of the paper. Try to put this paper in perspective. (This may require looking at other related papers). After the presentation, the presenter will lead a discussion on the merits of the paper.

For your presenation, there are two options:
1) Pick a paper (or papers) from the syllabus for that day and present
2) Pick a paper (or papers) that you think are missing from the syllabus and present.
This paper must be related to the topic for that day. Before starting work on your presentation, the paper must be approved.

We will also plan potential future projects, which students will present, discuss, obtain feedback, and refine.
-Final Project: Course ends with experiment write up (in groups or individually)
-Last 2 weeks: Presentations of experimental write up during final weeks of class

Prerequisite

 Background in one or more of: human-computer interaction, psychology, cognitive science, or brain measurement techniques; or permission of instructor.

Topic Outline and Reading List
Week 1
(Sep 4)		 Introduction to Course
Week 2
(Sep 11)		 Audrey Girouard, Leanne Hirshfield, Erin Solovey
Non-invasive Brain Measurement with HCI
  • Lee, J.C. and Tan, D.S. Using a low-cost electroencephalograph for task classification in HCI research Proceedings of the 19th annual ACM symposium on User interface software and technology, ACM Press, Montreux, Switzerland, 2006.
  • Millan, J. d., Renkens, F., Mouriño, J., and Gerstner, W. 2004. Brain-actuated interaction. Artif. Intell. 159, 1-2 (Nov. 2004), 241-259.
  • St. John, M., Kobus, D. A., Morrison, J. G., Schmorrow, D. Overview of the DARPA Augmented Cognition Technical Integration Experiment International Journal of Human-Computer Interaction , 2004, Vol. 17, No.2, Pages 131-149. [Link]
  • (Optional paper) Izzetoglu, K., Bunce, S., Onaral, B., Pourrezaei, K. and Chance, B., Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks, International Journal of Human-Computer Interaction, 17 (2), pp. 211-231.
Week 3
(Sep 18) 		Krysta Chauncey, Martin Paczynski
Overview of Brain Physiology
Overview of Prefrontal Cortex

Readings:
Week 4
(Sep 25)		 What could we measure in the prefrontal cortex?
    Workload / Working Memory
    • [Kristoff Redei] Smith, E, and Jonides, J. Storage and Executive Processes in the Frontal Lobes. Science Volume 283. 1999. 1657-1661. [Link}
    • Fairclough, S., Venables, L., Tattersall, A. The influence of task demand and learning on the psychophysiological response. International Journal of Psychophysiology 56 (2005) 171-184.[Link]
    • [Anila Chowdhury] Ryu, K., Myung, R. Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics 35 (2005) 991-1009[Link]
    • Goldberg et al. Uncoupling Cognitive Workload and Prefrontal Cortical Physiology: A PET rCBF Study. NEUROIMAGE 7, 296-303 (1998).[Link]
    • [James Kebinger] Gevins, A and Smith, M. Neurophysiological Measures of Working Memory and Individual Differences in Cognitive Ability and Cognitive Style. Cerebral Cortex, Sep. 2000, 10: 829-839. [Link]
Week 5
(Oct 2)		 What could we measure in the prefrontal cortex?
    Emotion / Affect
    • [Kelly Moran] Davidson R.J., Sutton S.K. Affective neuroscience: the emergence of a discipline. Current Opinion in Neurobiology, Volume 5, Number 2, April 1995 , pp. 217-224(8)
    • Jose Leon-Carrion, Juan Francisco Martin-Rodriguez,Jesus Damas-Lopez, Kambiz Pourrezai, Kurtulus Izzetoglu,Juan Manuel Barroso y Martin, Maria Rosario Dominguez-Morales. A lasting post-stimulus activation on dorsolateral prefrontal cortex is produced when processing valence and arousal in visual affective stimuli. Neuroscience Letters 422 2007
    • [Bridget Flaherty] K. Phan, S. Taylor, R. Welsh, L. Decker, D. Noll, T. Nichols, J. Britton, I. Liberzon. Activation of the medial prefrontal cortex and extended amygdala by individual ratings of emotional arousal: a fMRI study. Biological Psychiatry, Volume 53, Issue 3, Pages 211-215
    • [Meredith Staszak] Joseph E. LeDoux. EMOTION: Clues from the Brain (1995) Ann. Rev. Psych.
    Other areas?
Week 6
(Oct 16)		 Introduction to functional near-infrared spectroscopy (fNIRS)
    How does fNIRS work?
    • Rolfe, P. In vivo near-infrared spectroscopy. Annual Review of Biomedical Engineering, August 2000, Vol. 2, Pages 715-754 [Link]
    • Bunce, S., Izzetoglu, M., Izzetoglu, K., Onaral, B. and Pourrezaei, K. Functional Near Infrared Spectroscopy: An Emerging Neuroimaging Modality., IEEE Engineering in Medicine and Biology Magazine, 25 (4), pp. 54-62.
    • Villringer, A. and Chance, B., Non-Invasive Optical Spectroscopy and Imaging of Human Brain Function, Trends in Neuroscience, 20, pp. 435-442.
    • [Anila Chowdhury] Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C., Murray, T., Ovetsky, Y. and Thomas, R., A novel method for fast imaging of brain function, non-invasively, with light, Optics Express, 10 (2), pp. 411-423.
    Measuring brain activity with fNIRS
    • [Jeremy Tyler] S. Coyle, T. Ward, C. Markham, G. McDarby. "On the Suitability of Near-Infrared Systems for Next Generation Brain Computer Interfaces". World Congress on Medical Physics and Biomedical Engineering, Sydney, Australia, IFMBE, 2003
    • [Nada Attar] Hoshi, Y. Tamura, M. Near-Infrared Optical Detection of Sequential Brain Activation in the Prefrontal Cortex during Mental Tasks. NEUROIMAGE 5, 292-297 (1997)[Link]
Week 7
(Oct 23)		 Measuring brain activity with fNIRS
  • [Martin Paczynski] Herrmann, M.J., Ehlis, A. C., Fallgatter, A. J. Frontal activation during a verbal-fluency task as measured by near-infrared spectroscopy. Brain Research Bulletin 61 (2003) 52-56.
  • [Lee Keyser-Allen] Nagamitsu, S., Nagano, M., Yamashita, Y., Takashima, S.Toyojiro Matsuishi. Prefrontal cerebral blood volume patterns while playing video games--A near-infrared spectroscopy study. Brain & Development 28 (2006) 315-321 [Link]
  • "Spatial and temporal analysis of human motor activity using noninvasive NIR topography". Maki A. et al. Med. Phys. 22 (12), Dec 1995.
  • "Prefrontal Hypooxygenation during Language Processing Assessed with Near-Infrared Spectroscopy". Falgatter, A. J., Muller, Th. J., Strik, W. K. Neuropsychobiology 1998; 37: 215-218.
BCI Studies with fNIRS
  • [Kristoff Redei] Shirley M Coyle, Tomas E Ward and Charles M Markham. Brain-computer interface using a simplified functional near-infrared spectroscopy system. 2007 J. Neural Eng. 4 219-226 [Link]
Week 8
(Oct 30) 		EEG and BCI
  • [Kelly Moran] Ferrez, P. Millan, J. You Are Wrong!--Automatic Detection of Interaction Errors from BrainWaves. in Proceedings of the 19th International Joint Conference on Artificial Intelligence, August 2005. [Link]
  • [Krysta Chauncey] Gerwin Schalk, Jonathan R. Wolpaw, Dennis J. McFarland, Gert Pfurtscheller. EEG-based communication: presence of an error potential. 2000. Clinical Neurophysiology 111
  • [Orit Shaer] Kiern, Z. A., Aunon, J. I. A New Mode of Communication Between Man and His Surroundings. IEEE Transactions on Biomedical Engineering, Vol. 37, No. 12. 1990
  • [Nada Attar] C.W. Anderson and Z. Sijercic. Classification of EEG signals from four subjects during five mental tasks. Intl. Conf. on Engineering Applications of Neural Networks, 407--414, 1996. [Link]
  • [Jeremy Tyler] Kostov, A. Polak, M. Parallel Man-Machine Training in Development of EEG-Based Cursor Control. IEEE Trans. On Rehabilitation Engineering , Vol. 8, No. 2. 2000.
  • Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for cursor control. Electroencephalogr Clin Neurophysiol. 1991 Mar;78(3):252-9.
  • Leeb et al., Walking from thoughts: Not the muscles are crucial, but the brain waves!, 8th Annual International Workshop on Presence, PRESENCE 2005, 21-23 September 2005, London. [Link]
Week 9
(Nov 6) 		Other non-invasive, subtle or lightweight user interfaces
  • [Lee Keyser-Allen] R.J.K. Jacob, "The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At is What You Get," ACM Transactions on Information Systems, Vol. 9(3) pp. 152-169 (April 1991).
  • [Bridget Flaherty] Pomplun, M. and Sunkara, S. Pupil Dilation as an Indicator of Cognitive Workload in Human-Computer Interaction. Human-Centred Computing: Cognitive, Social, and Ergonomic Aspects. Vol. 3 of the Proceedings of the 10th International Conference on Human-Computer Interaction, HCII 2003, Crete, Greece, 542-546. [Link]
  • [Meredith Staszak] Iqbal, S. T., Zheng, X. S., and Bailey, B. P. 2004. Task-evoked pupillary response to mental workload in human-computer interaction. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM Press, New York, NY, 1477-1480[Link]
Week 10
(Nov 13) 		Machine Learning (and data processing) with fNIRS and EEG data
  • Izzetoglu M, Devaraj A, Bunce S, Onaral B, (2005). Motion Artifact Cancellation in NIR Spectroscopy Using Wiener Filtering. IEEE Transactions on Biomedical Engineering, 52(5):934-938. [Link]
  • [Orit Shaer] Gevins et al. Towards measurement of brain function in operational environments. Biological Psychoogy 1995 May;40(1-2):169-86. [Link]
  • [Anila Chowdhury] Noel, J., Baue, K., Lanning, J. Improving pilot mental workload classifcation through feature exploitation and combination: a feasibility study. Computers & Operations Research 32 (2005) 2713-2730. [Link]
  • [James Kebinger] Coyle, S., Ward, T., Markham, C. Physiological Noise in Near-infrared Spectroscopy: Implications for Optical Brain Computer Interfacing. Proc. of the 26th Annual International Conference of the IEEE EMBS (2004)
  • Haihong, Z., Cuntai, G. A Kernel-based Signal Localization Method for NIRS Brain-computer Interfaces. ICPR (2006)
  • [Audrey Girouard] Sitaram et al. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 34 (2007) 1416-1427. [Link]
Week 11
(Nov 20) 		Assignment 2 Experiment Development: Initial discussions
Week 12
(Nov 27) 		Assignment 2 Experiment Development: Final presentations (1)
Week 13
(Dec 4) 		Assignment 2 Experiment Development: Final presentations (2)