ON APPLICATIONS OF BRAIN-COMPUTER INTERFACE

. Brain-computer interface (BCI) applications implement a direct communication path between the brain and the computer. This paper deals with the fundamentals of BCI systems and the experience of the neuroinformatics team with the design and implementation of various BCI applications. Their advantages, drawbacks and suitability are discussed in multiple contexts.


Introduction
Jonathan R. Wolpaw formalized a brain-computer interface (BCI) system [1] as a communication or control system in which the user's messages or commands do not depend on the brain's normal output channels.Nerves and muscles do not carry messages, and neuromuscular activity is not needed to produce the activity that does carry the message.In other words, BCI systems mediate direct communication between the (human) brain and computer.Apart from invasive BCIs, its non-invasive forms utilize surface electroencephalography (EEG) and event-related potential (ERP) methods; the scalp-recorded electrical activity of the human brain is acquired to control an application or environment.
Research on BCIs has a long history, but its ultimate result, BCI systems working successfully and in the long-term in everyday life, has not been achieved so far.BCI paradigms, techniques and workflows focused on signal detection, off-line preprocessing and processing of the signal have been developed.However, standardized implementation, actual online deployment, testing, and customization of BCI systems on target users have not achieved the expected results.The persistent problems such as low classification accuracy and information transfer bit-rate still prevent these systems from becoming more widespread and competing with other communication solutions such as eye-tracking or voice recognition systems [2].On the other hand, there are no other usable means of communication for locked-in people.
Current BCI systems rely on several paradigms such as detecting the brain frequencies, event-related components, steady-state visual evoked potentials (SSVEPs), visual evoked potentials (VEPs) or motor imagery signals.A comprehensive review of EEGbased BCI paradigms is provided in [3].Current signal sensing technologies and computational intelligence approaches (including machine learning and deep learning algorithms) in BCI applications are reviewed in [4].
The goal of this paper is to acquaint readers with the fundamentals of BCI systems and the experience of the neuroinformatics team at the University of West Bohemia with the design and implementation of various brain-computer interface applications, shortly present the outcomes of these applications and debate the advantages, drawbacks and suitability of BCI systems in multiple contexts.
The paper is organized as follows.The Materials and methods section deals with the fundamentals of the electroencephalography method, event-related potentials and steady-state visual evoked potentials broadly used in BCI paradigms and applications.BCI design is shortly presented, related work is included, the neuroinformatics lab at the University of West Bohemia is introduced, and the methodology of presenting individual BCI experiments and applications is provided.The section BCI experiments presents five designed and implemented BCI applications.While the section Results provides individual results of and experience with the BCI applications, the section Conclusions summarizes experience from all BCI experiments/applications and gives prospects for the future.

Materials and methods
This section provides an essential insight into the fundamental concept of BCI.The basics of collecting the scalp-recorded electrical activity of the human brain using the EEG/ERP methods and techniques are described.It is extended by explaining ERP components, steady-state visual evoked potentials (SSVEPs), and BCI design.Then a short overview of related BCI research is provided.Finally, the neuroinformatics laboratory at the University of West Bohemia in the Czech Republic is introduced.Here, the BCI applications presented further have been developed, and related experiments have been carried out.

vol. 39/2022
On Applications of Brain-Computer Interface
The EEG method has many advantages: affordability, non-invasiveness, routine examination protocols, and the opportunity to measure spontaneous activity.However, it also has a significant disadvantage; the resulting picture of brain activity (the EEG signal) is rough since it represents many sources of neuronal activity.Then it is challenging to derive individual neurocognitive processes from the measured brain activity.This significant limitation must be considered in the design of BCI systems.

Brain Oscillations
Brain oscillations (also called brainwaves), a rhythmic and repetitive brain electrical activity generated spontaneously or in response to stimulation, play a key role in brain sensory-cognitive processes and neural communication and create a prevalent feature of brain recordings.These oscillations (frequencies) are associated, e.g. with relaxation/concentration states, and their detection is utilized in simple BCI systems and neurofeedback applications.However, two basic paradigms are used for more complex BCI applications: event-related potentials and steady-state visual evoked potentials.

Event-related potentials
The event-related potential (ERP) method initially focused on identifying and understanding ERP components (these are explained later) and was used as an alternative to measurements of the speed and accuracy of motor responses in paradigms with discrete stimuli and responses.Later, ERP research changed from identifying and recognizing individual components to answering questions of broader scientific interest, such as processing visual and auditory information in the brain, and research into human attention and human behaviour when performing parallel tasks in the brain.This experience has been subsequently used in BCI research.
With the development of imaging methods and techniques (fMRI -functional magnetic resonance imaging and PET -positron emission tomography), a gradual attenuation of the ERP method was predicted.However, this method has become an essential part of BCI and cognitive neuroscience experiments due to the relative affordability and high temporal resolution of the EEG signal that imaging methods lack.
ERPs have two advantages compared to behavioural methods.They help determine which stage or stages of processing are influenced by a given experimental manipulation; for a detailed set of examples, see [5].They can also provide an online measure of stimuli processing, even when there is no behavioural response [6].
As in the case of the EEG method, the ERP method does not require insertion of the electrodes directly into the brain, as the change in potential is recorded directly on the scalp.
Based on the information above, we can state that the ERP method is suitable for solving issues such as 'which neurocognitive process is affected by a given experimental protocol' [6].
In ERP research, ERP components must be observed in the EEG signal.An ERP waveform consists of a series of peaks and troughs, but these voltage deflections reflect the sum of several relatively independent underlying or latent components [5].These latent components are complicated to isolate, as the maximum and minimum amplitude values and their latencies in the observed signal may not necessarily be the right guide for determining them.Limited possibilities of their independent measurement are the single biggest roadblock to designing and interpreting ERP experiments [5].It is, therefore, necessary to distinguish between observable maximum signal values and latent components.The ERP component is then defined as scalp-recorded neural activity generated in a given neuroanatomical module when a specific computational operation is performed [5].
ERP components are obtained from the EEG signal by averaging the epochs around the events.Thus, observable local positive and negative maxima of voltage values found in the resulting ERP signal are referred to as components.
The designation of the component consists of a character indicating the polarity of the local maximum (P -positive local maximum of voltage value, N -negative local maximum of voltage value) and the order of local maxima (for positive and negative local maxima separately, e.g., P1, N1, P3).Instead of indicating the order of the local maximum, a timestamp is sometimes used to indicate the latency of the component in milliseconds, e.g., P300 or N400.The order of ERP components reflects the flow of information through the brain.
The P300 (also denoted as P3) component depends entirely on the task performed by the subject and is not directly influenced by the physical properties of the stimulus.It is sensitive to various global factors, such as time since the last meal, weather, body temperature, and even the time of day or year [5].Although thousands of experiments related to the P300 component have been published, we still do not know what the P300 component really means.The proposal that the P300 component is related to a process called 'context updating' seems approximately correct.
However, there are known factors that influence the amplitude and the latency of the P300 component.The P300 component is sensitive to the probability of the target stimulus.P300 amplitude increases when the probability of the target stimulus class decreases and when more non-target stimuli precede the target stimulus.P300 amplitude is also larger when the subject pays more attention to a task.On the other hand, P300 amplitude is smaller if the subject does not know whether a given stimulus is/is not a target.It means that more complex tasks can increase P300 amplitude because the subject pays more attention to these tasks and simultaneously decrease it because the subject is not certain of the stimulus category [5].
As can be seen, the experimental design of EEG/ERP and BCI experiments is a challenging and critical step that influences other technical issues related to BCI research.The P300 component is widely used in BCI experiments.

Steady-state visual evoked potentials
Steady-state visually evoked potentials (SSVEP) are natural oscillatory cortical responses to visual stimulation at specific frequencies.As an alternative to ERPs and P300 BCIs, SSVEPs can be used for BCI systems because they are easy to detect (they have an excellent signal-to-noise ratio and resistance to artifacts) and stable across participants.The relative stability of SSVEPs under different perturbations, such as speaking, listening, or thinking, is also often highlighted [7].
On the contrary, a significant disadvantage is a certain discomfort towards the subjects because the flickering of visual stimuli is disturbing.

BCI systems design
All of the paradigms presented above (brain oscillations, P300 ERP component and SSVEPs) require little to no training and can easily fit the purpose of BCI systems.They all need to use an appropriate acquisition system and an experimental protocol that allows collecting, analyzing, and interpreting the EEG/ERP recordings to establish communication between the brain and computer.One of the challenges BCI systems face is their ability to classify the EEG signal in real-time.BCI pipelines (workflow management systems) and technical solutions, such as cloud environments in the training phase, have been proposed and tested.The BCI experiments section presents five various BCI applications, from simple to more complex ones, to demonstrate their opportunities and drawbacks.[10].This company also provides complete MATLAB-based research and development systems.BCI systems focusing on the P300 component and SSVEPs paradigms are developed by the BrainTech company [11].A framework of hardware and software components for BCI research is developed by the BrainProducts company [12].The EMOTIV company offers BCI devices paired with EmotivBCI software [13].Mobile wireless EEG devices are produced by the mBrainTrain company [14].Finally, we need to also mention the Neuralink project [15] that is on the border of invasive and non-invasive brain data collection.

Related studies
When focusing on scientific studies, we need to mention, in particular, a 10-year update of a review of classification algorithms for EEG-based BCIs [16].At this time, deep learning methods still had not shown convincing improvement over state-of-the-art BCI methods.Another systematic review of hybrid deep learning approaches in BCI systems is provided in [17].BCI paradigms, signal processing, feature extraction methods, hybrids BCIs, and design of the synchronous/asynchronous BCIs are reviewed in [18].Progress of EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms is summarized in [19].Finally, as the result of the IEEE working group, a first version of the standardized BCI glossary for a community review is presented in [20].

Neuroinformatics laboratory
When performing BCI applications/experiments, operating a laboratory with appropriate BCI infrastructure is important.This includes mainly: • EEG acquisition system to collect brain recordings, • equipment/tools for stimuli presentation, in case of the P300 paradigm, accurate time synchronization with the EEG recording system is required, • a computer (computers) for processing EEG recordings (and stimuli) and establishing the communication path to the brain, • devices that are controlled by the BCI application, • a pipeline (workflow management system) for processing/managing the whole BCI communication.
Not all parts of this infrastructure are always used in the BCI applications presented below.

Methodology
Five various BCI experimental applications were selected to demonstrate the opportunities and experience from the real use of such systems.The BCI Experiments section briefly provides specific goals, application design, used equipment, and basic information about experiments.While the Smart train application used a simple acquisition headset and was developed to show BCI to the public, the following four applications were designed and developed to achieve specific goals in research projects.The Results section then provides experience with designing and using these BCI applications to collect BCI data; i.e. specific scientific or educational findings are not focused on in this paper.

BCI Experiments
This section introduces five BCI experiments/applications that use various BCI paradigms described above.

Smart Train
A simple BCI system Smart Train was developed to demonstrate to students and the interested public the basic principles and practical utilisation of brain oscillations (Figure 1).
The system is based on acquiring the participant's EEG signal, finding its frequency, and controlling the model train depending on the frequency.Brain signal is collected, and brain frequencies are analyzed with the headset; the train goes faster with a higher concentration level (higher frequency of the brain signal).The eye blinking signal is processed to change the direction of the model train.The users can observe their brain signals and interpretation in the graphical user interface.
The Neurosky headset (as the EEG acquisition system and signal processing unit) is used to control the speed of the model train (a controlled device).

BCI for Developmental Coordination Disorder Experiment
A BCI system was designed and developed to investigate developmental coordination disorder (DCD).DCD is described as a motor skill disorder characterized by a marked impairment in the development of motor coordination abilities.The main research goal was to contribute to a diagnosis of this disorder using the ERP method.The next goal included annotating the raw data with relevant metadata and providing them publicly for further analysis.
The experimental protocol was based on the ERP paradigm; a combined auditory and visual stimulation was used.Visual stimuli were represented by pictures of animals.The corresponding auditory stimuli were represented by the sounds of animals that occurred in synchronization with the visual stimuli.
Participants were asked to respond to various stimuli combinations by pressing two different buttons during the experimental session.
Standard gel EEG caps, the international 10-20 system (that describes the location of EEG electrodes on the scalp) and the BrainAmp DC amplifier were used to collect EEG data (the EEG acquisition system).The sampling frequency was set to 1 kHz.The raw EEG signal was filtered using an analogue band-pass filter with cut-off frequencies of 0.1 and 250 Hz.The experiments were performed in a sound-and electrically shielded booth (Figure 2).The data were collected from 32 school children (16 with possible DCD and 16 in the control group).
The experimental work, collected data and data validation process were described in detail and published in [21].

BCI for Guess the number experiment
Guess the number is a simple P300-based BCI experiment [22].It aimed to demonstrate another simple BCI application to school-age children and students.
In parallel, it aimed to collect a large amount of BCI data in a noisy environment, annotate them properly, create a large publicly available BCI dataset and use machine/deep learning methods for P300 component classification.
Visual stimuli (the numbers between 1 and 9) were presented on the screen, the participants picked one of these numbers, and experimenters tried to guess the number thought while they were observing ERP waveforms online (Figure 3).
EEG data were collected from three midline EEG channels (frontal, central and parietal electrodes of the 10-20 international system).The common gel electrodes and the BrainProducts V-amp amplifier were used.
The experiment was carried out in elementary and secondary schools in the Czech Republic on 250 participants.

BCI for Drivers' attention investigation
About twenty experimental protocols (some experiments were published), each performed on 10 participants on average, were designed and run to investigate drivers' attention during simulated driving.The driving conditions varied and included, e.g., heavy workload put on the drivers, various types of disturbance or, on the other hand, driving on a monotonous track.Different stimuli (both visual and auditory) were used; the P300 component paradigm mainly was applied.The stimuli were presented either using a custom programmable hardware stimulator (for visual stimulation with LEDs) placed on the car windshield or the Presentation software tool (for  The car simulator (Figure 4, a front part of a real Škoda Octavia car with the Logitech G27 wheel, accelerator, and brake connected to the control computer via the USB port) was used.The tracks for driving simulations were prepared mainly using the World Racing 2 game produced by the Synetic Company.The track was projected on the wall in front of the car simulator.Gel EEG caps (the 10-20 international system) and the V-amp amplifier were used to collect EEG data.The amplifier also served as an input unit for collecting additional biosignals.

BCI for people with limited mobility
The BCI application for people with limited mobility focused on designing, developing, and testing an open and affordable prototype of a BCI system built on lowcost hardware and open-source software components.The prototype allows people with limited mobility to control their basic home environment.The project added the concept of the cloud for remote BCI computations and relied on testing and customization of the whole BCI system.
Experimental protocols were based on the P300 component and SSVEP paradigms.In the case of the P300-based experiment, the pictures corresponding to the needs/activities of the end-user were presented sequentially or in a matrix.In the case of the SSVEP experiment 6, three pictures corresponding to endusers needs/activities (turn on the radio -15 Hz, turn on the light -12 Hz, make a phone call -10 Hz) were selected.
At first, we used common gel electrodes and the V-amp amplifier within this BCI system.Later, hardware components for signal acquisition, designed and developed by Sensorik-Bayern GmbH, were tested.These components included dry electrodes (Figure 5), a head-mounted device, and a base station.Wireless EEG signal transmission via Bluetooth connection was implemented.The hardware components were supple-  mented with software components for local execution of online BCIs; these included mainly an analysis library and workflow designer.
The application was tested on 10 participants in laboratory conditions, but its parts (mainly a variety of P300 paradigms) were also tested in hospitals on 20 people.

Results
This section summarizes the experience with designing and using the BCI applications and performing individual experiments described above.In general, we can state that the neuroinformatics laboratory has been working successfully, and the conducted experiments have contributed both to scientific results and the everyday operation of the laboratory by including advances achieved in the BCI data lifecycle (data collection, annotation, storing, preprocessing, analysis, visualization, interpretation and publication).
While experimenting with the Smart Train BCI application (approx.five years of experimenting during excursions, science and technology days, and exhibitions), we found that most people could control the model train after a short training.There have been occasional technical difficulties with the headset set- tings, but this simple BCI system can be successfully used for such a purpose.It turned out to be very important and beneficial that the basics of BCI can be simply explained to BCI non-professionals (usually students and their teachers).
When designing and developing the BCI application for the DCD experiment, we experienced difficulties obtaining clean data, even in laboratory conditions and sound-and electrically shielded booth.This experiment also set a comparison base to evaluate various EEG acquisition systems (mainly various types of EEG electrodes) to collect clean EEG data.The used gel electrodes require skillful personnel to apply them, but at the same time, their impedance remained consistently low during the whole experiment when compared to dry and semi-dry electrodes.Finally, the responses to auditory stimuli measured by ERPs at pre-attentive and attentive levels between children with DCD and children with normal motor development (NDC) were compared.The child's crying (a stimulus) evoked a significant P300 response, composed of the early P3a peak and late P3b peak (P3a and P3b are P300 subcomponents).The P3a component had its maximum amplitude in the central region, while the P3b component had maximum amplitude in the parietal region of the brain.No significant differences were observed between the DCD and NCD groups in the amplitude and latency of the P3a component.In the case of the P3b component, significantly lower amplitude in the parietal region was found in the DCD group (Figure 7).In contrast, no significant difference between the groups was found in the component latency [24].
During the Guess the Number experiment, we experienced that BCI applications can also be run, with some limitations, in a noisy environment.Midline electrodes were used to analyze ERPs and the P300 component was used to guess the number thought.The number of participants also showed a variety of P300 representations; researchers highly improved their skills in identifying the P300 component.Moreover, an online P300 component classifier (used in the following BCI applications) based on machine learning methods was trained on the obtained recordings.The accuracy of up to 79.4 % was achieved when using stacked autoencoders for classification, while the researchers guessed the numbers manually by observing gradually averaging ERP waveforms with 64.4 % accuracy [25].When using convolutional neural networks to classify the P300 component, the was 62-64 % for single trials and 76-79 % for averaged trials [26].The accuracy achieved for averaged trials was comparable with other state-of-the-art methods used for this dataset.A comparison of target and non-target epoch grand averages is shown in Figure 8.
The most important experience we gained during driver's attention experiments was properly designing the experimental protocol since obtaining clear and interpretable data was difficult.Besides that, we had to cope, e.g. with the troubles related to participants' willingness to wear an EEG cap for a longer time or the necessity to reduce drivers' movements.We verified that the principles and rules for designing BCI experiments based on the ERP paradigm had to be rigorously followed, although many other scientific studies often violate them.
An example result [27] of driver's ERP data validation (based on the objective that target and non-target trials are expected to be associated with differently shaped ERP components) from a successful experiment using a stacked autoencoder (as an important step preceding data analysis) is provided in Figure 8.If the classification of a specific dataset from a participant yields low error rates, the objective of the odd-ball paradigm is considered to be fulfilled.The error rate was calculated as: where t p is the number of correctly classified targets, t n is the number of correctly classified non-targets, f p -is the number of misclassified non-targets, and f n is the number of misclassified targets.As a result, error rates indicate the extent to which the classifier was unable to separate target and non-target single trials.
In the case of the BCI application for people with limited mobility, we experienced that a BCI system can be built on low-cost devices for EEG signal acquisition and amplification.Eye blinking and alpha activity were clearly observable, especially when gel and dry electrodes with long pins were used to collect data.SSVEPs were clearly observable, independently of frequency (frequencies between 8.5 Hz and 20 Hz were evaluated).Using the SSVEP paradigm brought more reliable results, but end-users could not be exposed to SSVEPs for a long time.A trainingless classifier for online SSVEP classification was developed.Six out of ten participants could control the system online, achieving more than 70 % accuracy.
Table 1 shows the results of SSVEP online detection for each participant when the spectral difference (SD) method and canonical correlation analysis (CCA) were used.
On the other hand, we failed to evoke an observable P300 component when dry electrodes were tested.It can be explained by a generally low P300 amplitude and a relatively low signal-to-noise ratio.

Conclusions
This paper presented the fundamentals of BCI systems and five specific BCI experiments/applications designed and performed in the neuroinformatics laboratory of the University of West Bohemia.We experienced the advantages and disadvantages of several BCI paradigms, coped with BCI design principles and their limitations, used various kinds of EEG acquisition systems, and brought advances to the whole lifecycle of BCI data.This experience accompanied scientific and educational goals achieved.
We experienced that simple EEG devices (headsets, headbands) work reasonably well when brain frequencies are evaluated and controlling any end device is not a critical step.Following design principles while building BCI systems is essential; their violation leads to uninterpretable results.We found out that gel electrodes generally worked better than dry electrodes, but the latter ones worked better in the case of the SSVEP paradigm used.
Based on our experience, we are convinced that BCI systems and applications are promising for future use, although they seem beneficial for a very limited group of people (especially those in the locked-in state) when used as a primary communication path.However, some issues such as low information-transfer bit-rate and lower performance persist.We also believe that

Figure 1 .
Figure 1.Smart train -a model train controlled with brain oscillations.

Figure 3 .
Figure 3. Researchers are observing ERPs during the Guess the Number experiment [22].

Figure 4 .
Figure 4.A participant during simulated driving in a drivers' attention experiment.
auditory stimulation) produced by Neurobehavioral Systems, Inc [23].The drivers were asked to respond to stimuli by pressing the wheel buttons.

Figure 5 .
Figure 5. Dry electrodes (made by the Sensoric-Bayern company) developed and tested during the BASIL project.

Figure 6 .
Figure 6.A participant while testing the SSVEP protocol.

Figure 7 .
Figure 7. Mean scalp potentials distributions of the P3b response in DCD (two left columns) and NDC children (two right columns).Adopted from [24].
The experiment data, stored in the custom EEG/ERP Portal, were collected and annotated respecting the outcomes of the International Neuroinformatics Coordinating Facility (INCF) Program on Standards for Data Sharing and the group developing the Ontology for Experimental Neurophysiology.

Figure 8 .
Figure 8.Comparison of target and non-target epoch grand averages.There is a large P300 component following the target stimuli.Adopted from [26].
non-invasive BCI systems.Wireless transmission of the EEG signal in BCI systems is widespread; it can provide more comfort to endusers.Various EEG electrodes (gel, semi-dry, dry), EEG caps, headsets and headbands as a part of BCI acquisition systems are designed and developed.
BCI systems are designed and developed at many workplaces around the world.Although they are still considered more as research topics, there is already a market dominated by