Alternate Modes of Human-Computer Interaction:
EEG Recognition with Neural Networks

Charles W. Anderson

Department of Computer Science
Colorado State University
Fort Collins, CO 80523


Department of Computer Science, Colorado State University, Fort Collins, CO 80523,

970-491-7491, FAX: 970-491-2466



Other Communication Modalities


EEG, human-computer interface, neural networks, pattern recognition, signal processing


The purpose of this project is to explore the practicality of EEG recognition as a new mode of communication for severely disabled persons. A system that can identify, on-line, which of several mental tasks a person is performing would provide an alphabet with which the person can compose commands to devices like a wheel chair. Some success has been achieved in distinguishing mental tasks using EEG waves, but the limitations of current recognition schemes preclude their use in on-line systems. Most research has focussed on recognition accuracy, not on fast, real-time responses. The goals of this project encompass both the need for high recognition accuracy and for fast response.

Critical to the success of on-line EEG recognition is the selection of a representation consisting of features that are low in number and in computationally complexity, yet capture information related to the mental state of a person in a way that is invariant to time and subject. The goal of this project is to evaluate a variety of EEG signal representations of data recorded from subjects performing several mental tasks. Representations are evaluated according to how well the EEG samples can be classified as examples of the correct mental task. Representations considered to-date include the raw signals, Karhunen-Loeve and Fourier Transforms, and scalar and multivariate AR (autoregressive) models.

Data was obtained using the following procedure. Subjects were seated in a sound controlled booth with dim lighting. Six electrode positions were recorded using an Electro-Cap elastic electrode cap: C3, C4, O1, O2, P3, and P4 according to the 10-20 system of electrode placement. Data was recorded for 10 seconds during each task and each task was repeated five times per session. Most subjects attended two such sessions recorded on separate weeks. Currently we are studying data from four subjects, each performing two tasks: the Baseline Task , for which the subjects were not asked to perform a specific mental task, but to simply relax as much as possible, and the Math Task , for which the subjects were given nontrivial multiplication problems, such as 49 times 78, and were asked to solve them without vocalizing or making any other physical movements. The 10 seconds of data from each recording trial are divided into 1/2 second windows and windows that contain eye blinks are discarded.

Classification is performed with a neural network trained via standard error backpropagation. Cross-validation is used to control for the over-fitting in the following way. The data is divided into eight partitions, each of which contains data from unique recording trials. Six parts are randomly chosen to train the network classifier and the remaining seventh and eighth parts are designated as validation and test sets. After each epoch, or pass through the training data, the squared error on the seventh part is calculated. After the network has converged, the network's weights at the epoch having the lowest validation set error are saved. The generalization ability of this network is tested by calculating the error on the test set. Thus, 7/8ths of the data is used to train the network and select the best network weights, while 1/8th of the data is used to test the performance. This procedure is repeated for all 56 assignments of data partitions to training, validation and test sets.

Experiments show that the AR and Fourier representations produce the most reliable classifications. Our latest results are the following. For three of the subjects, 80-85% of the data are correctly classified, while for the fourth subject only 70% is correctly classified. Currently we are investigating other representations and additional tasks and subjects.


Anderson, Charles, W., Stolz, Erik A., and Shamsunder, Sanyogita, and Martin, Charles. (submitted) Multivariate Autoregressive Models for Classification of Spontaneous Electroencephalogram During Mental Tasks. Submitted to IEEE Transactions on Biomedical Engineering.

Anderson, C. W., Devulapalli, Sai, and Stolz, Erik. (1995) Determining Mental State from EEG Signals Using Neural Networks, Scientific Programming, Special Issue on Applications Analysis, Vol. 4, No. 3, Fall, 1995, 171--183.

Anderson, Charles W., Devulapalli, Saikumar V., and Stolz, Erik A. (1995) EEG Signal Classification with Different Signal Representations. In Neural Networks for Signal Processing V, ed. by F. Girosi, J. Makhoul, E. Manolakos, E. Wilson, IEEE Service Center, Piscataway, NJ, pp. 475--483.

Anderson, Charles W., Stolz, Erik A., and Shamsunder, Sanyogita. (1995) Discriminating Mental Tasks Using EEG Represented by AR Models. Proceedings of the 1995 IEEE Engineering in Medicine and Biology Annual Conference, Sept 20--23, 1995, Montreal, Canada.

Anderson, C. W., Devulapalli, Sai, and Stolz, Erik. (1994) EEG as a Means of Communication: Preliminary Experiments in EEG Analysis Using Neural Networks. Proceedings of ASSETS'94, the First International ACM/SIGCAPH Conference on Assistive Technologies, pp. 141--147.


EEG signals have been scrutinized for decades to find patterns that correlate to disorders like epilepsy, sleep disorders, tumors, and more recently, schizophrenia. EEG have also been used to suggest or validate models of cognitive behavior and the dynamics of the brain.

This project uses EEG in yet a third way---to extract information that correlates to mental decisions with the long-term goal of providing a new means of input from humans to computers. Considerable work has been done on event-related effects. For example, EEG just prior to movement of a joystick contains information related to the direction of the movement. Our work attempts to find information in spontaneous, on-going EEG.

On December 2, 1995, at the Neural Information Processing Systems (NIPS) Workshops in Vail, CO, a workshop will be held titled ``Online Neural Information Processing Systems: Prospects for Neural Human-System Interfaces'', chaired by S. Makeig of the Cognitive Psychophysiology Laboratory at the Naval Health Research Center in San Diego, CA. Talks will be given by B. Taheri, SRI Int., on a new EEG electrode technology, by A. Gevins, director of the EEG Systems Lab in San Francisco, and by the PI of this project on our work with EEG, plus others.


J. Barlow, The Electroencephalogram: Its Patterns and Origins, MIT Press, Cambridge, Massachusetts, 1993.

A. Gevins and R. Remond, eds., Handbook of Electroencephalography and Clinical Neurophysiology: Methods of Analysis of Brain Electrical and Magnetic Signals, Elsevier, 1987.

S. Haykin, Neural Networks: A Comprehensive Foundation, Macmillan College Publishing Company, Inc., New York, New York, 1994.

N. Masic, G. Pfurtscheller, and D. Flotzinger, Neural network-based predictions of hand movements using simulated and real EEG data, Neurocomputing, 7, Feb., 1995, pp. 259--274.

P. Nunez, Neocortical Dynamics and Human EEG Rhythms, Oxford University Press, New York, 1995.


One future application of this work is to the control of wheelchairs for paralyzed persons, thus the work is related to Topic 6, Intelligent Interactive Systems for Persons with Disabilities.


Information extracted from EEG provides a new input modality that can be combined with any other modalities to increase reliability of automatic recognition of user's intent. For example, acoustic and visual information have been combined to increase the reliability of speech recognition and perhaps the reliability can be increased even more with additional information from EEG.