Evaluating portable EEG: a comparison between two wireless systems (EPOC Flex and LiveAmp) and the wired BrainAmp system

TL;DR

This study compares portable and wired EEG systems to see how well they capture brain signals in different tasks, finding that portable systems can work well if enough data is collected.

Contribution

The study evaluates the performance of portable EEG systems in real-world cognitive neuroscience tasks, providing guidance for their use in research.

Findings

All systems captured resting state alpha power and SSVEP responses, though EM showed lower spectral effects.

ERPs for active and passive processes were recorded across all systems, with EM and LA showing lower N170 amplitudes compared to BA.

Dataset reduction affected EM's N170 signal more than LA or BA, and EM had shorter ERP latencies except for MMN.

Abstract

Recent advances in equipment miniaturization have led to low-cost, portable electroencephalography (EEG) systems that facilitate data collection in real-world settings and with larger samples. Although wireless EEG systems were originally developed for non-research applications, recent studies have provided valuable information to help researchers make informed choices, particularly about participant comfort, mobility during recordings, and data validity. This study aimed to assess the impact of portability by comparing the performance of portable consumer- and research-grade systems (EPOC Saline Flex, EM; LiveAmp, LA) with fixed research-grade systems (BrainAmp, BA).

Continuous EEG was recorded with each system in healthy adults performing five benchmark tasks in fundamental and clinical cognitive neuroscience. Mental states (alpha power variations in open/closed eyes) and unconscious perception (steady-state visual evoked potential, SSVEP) were analyzed through time/frequency methods, while active (N200 and P300 components during active listening and N170 component during face recognition) and passive cognitive processes (Mismatch negativity, MMN component during passive listening) were examined using time/amplitude analyses (event-related potential, ERPs). Our analyses compared system efficiency at native and equalized sampling rates and examined 100%, 75%, and 50% of the datasets to determine the required trial number for satisfactory signal quality.

Despite the smaller amount of signal retained for EM, all systems recorded the expected resting state alpha power decrease and SSVEP responses, with EM showing lower spectral effects ([EM < (LA≈BA)]). ERPs for active (N170, N200, P300) and passive (MMN) processes emerged across all systems, with EM and LA showing lower amplitudes only for N170 compared to BA. Furthermore, the dataset reduction resulted in a decreased N170 at P7 only for EM ([EM < LA < BA]). EM also exhibited shorter latencies for all ERPs except for MMN.

This study provides concrete guidance for designing EEG experiments in real-world settings, with significant potential for investigating children and vulnerable populations. The efficiency of the three EEG systems is more influenced by task duration than sampling rates. A wireless EEG device, such as the EM, can effectively support both time/frequency and time/amplitude analyses in cognitive science, provided that the number of trials is sufficient and latencies are controlled.

Figures

Electrode names and labels for the BA system (A), the LA and EM systems (B).

Amplitude spectra expressed as SNR across occipital electrodes (O1, Oz, O2) for the three systems and three conditions (fundamental component = 6, 10 or 15 Hz) in Pack-100a.

Group N170 ERP waveforms for the three systems at P7 and P8 electrode sites in response to face (top) and texture (bottom).

Topographic distribution of the signal for the difference between face and texture conditions.

Group N200 and P300 waveforms for the three systems at Fz, Cz and Pz electrodes (A). Topographic distribution of N200 (B) and P300 (C) ERP signals for the three systems.

(A) Group MMN ERP waveforms for the three systems at Fz, Cz and Pz electrode sites. (B) Topographic distributions of MMN ERP signals for each system.

Full text

Introduction

The measure of brain activity with electroencephalography (EEG) originated with Hans Berger’s human recordings in 1929. It has since established itself as a popular method among neuroscientists for its non-invasiveness and high temporal resolution. While advances in technology have contributed to improve instrument sensitivity and signal processing, research-grade EEG systems still require extensive wiring. This restricts their use to laboratory settings. Heavy equipment makes it difficult to use EEG for the assessment of naturalistic interactions, sport practice, restless patients, young, as well as for long-recordings or large-scaled studies (Lau-Zhu, Lau & McLoughlin, 2019).

The recent emergence of consumer devices and portable systems has represented a significant advance in EEG research. They are easy to transport, require no application of electrolytic gel and subsequent hair washing, can be easily and quickly installed even outside the laboratory, and allow the participant to move. Therefore, even with a limited budget, EEG data acquisition becomes feasible under ecological conditions (for a review, see Cannard et al., 2020), which encourages the recording of large samples (e.g., 505 participants, Kounios et al., 2024), or hitherto little-tested populations, including those at school or at home. The biotech company EMOTIV Systems was among the first to launch consumer EEG devices with the EPOC system. Though the system was initially designed as a hands-free device for video games, it has since found its place in the brain-computer device (BCI) category (Vasiljevic & de Miranda, 2020; Antoniou et al., 2021). This has prompted researchers to evaluate the efficiency of the EPOC system as a research tool. For instance, previous studies concluded that the EMOTIV system can be reliably used in neuroscience research (Sawangjai et al., 2020; Sabio et al., 2024), despite lower signal-to-noise ratio (SNR) of mobile EEG systems than their research-grade counterparts (Radüntz, 2018; Mahdid, Lee & Blain-Moraes, 2020).

In a recent article, Niso et al. (2023) showed the diversity of portable EEG systems on the market and attempted to highlight the characteristics of 48 wireless systems and objectively listed the specific features of each one regarding electrodes, amplifier, and analog-to-digital converter. However, the authors pointed out imprecise information and inconsistences across methodologies in most of the validation studies, suggesting that standardized testing protocols are needed. They also remarked on the lack of direct comparison of portable EEG systems, taking account both portability and signal quality. Limitations are also due to variations in the location and number of electrodes, the sample sizes, and the experimental paradigms used to elicit event-related potentials (ERPs) or changes in neural frequencies. Other authors expressed the need of validation studies including time/frequency signal analyses, and information about the necessary number of trials to obtain expected ERP components (Lau-Zhu, Lau & McLoughlin, 2019). In line with these recommendations, recent efforts have been undertaken to directly compare portable and research-grade EEG systems, but they encountered various obstacles.

For instance, (1) Williams et al. (2020) compared the EMOTIV EPOC with a research-grade system (Neuroscan SynAmps2) using five different paradigms in 20 adults. However, they opted for simultaneous assessment with priority given to EMOTIV positioning at sites of interest—as Neuroscan is widely accepted in research (Badcock et al., 2015)—at the expense of accuracy in electrode location for Neuroscan. This could explain intriguing higher N170 amplitude recorded with EMOTIV than Neuroscan. This limitation could be avoided by an intra-individual sequential assessment of the different EEG systems with a counterbalanced order. (2) Another issue that Ries et al. (2014) faced when comparing the research-grade Biosemi system with the portable Advanced Brain Monitoring (ABM) and EMOTIV systems was the differences in configuration and number of electrodes between the systems. The authors attempted to overcome this limitation by testing a rather large set of participants (N = 16) compared to other studies. (3) The validation studies also made it possible to overcome the limitation due to very small numbers of electrodes. For example, Ratti et al. (2017) used intra-individual sequential recordings to compare portable EEG systems: two multichannel medical systems with wet electrodes (B-Alert, Enobio, Barcelona, Spain) and two limited-channel consumer systems with dry electrodes and no impedance control (Muse, Mindwave). The authors found differences in signal quality and greater variability for Muse. However, the signal was only compared from one common site (Fp1) between the four EEG systems. (4) Another limitation in comparing an ambulatory EEG device (OptiEEG, referenced to A1 and A2 positioned at the ear) to a clinical reference device (Natus, referenced to Cz) was differences in the location of the reference electrodes (Shivaraja et al., 2023).

In the present study, we aimed to compare the performance of two portable EEG systems—the EMOTIV EPOC Flex (EM) and the BrainProducts LiveAmp (LA)—to the lab-based BrainProducts BrainAmp system (BA), while addressing many of the shortcoming of previous validation studies highlighted above. The two portable EEG systems were chosen because they were well-suited for the type of developmental cognitive neuroscience research conducted at our research center. Specifically, they are portable and allow for rapid installation because they have wet electrodes (both of which are desirable for testing in schools or at home). Additionally, they have 32 electrodes, which allows for recording of the components that are most studied in our research center and in cognitive psychology and neuroscience generally. Additionally, we were familiar with the BrainAmp system and chose its portative version (i.e., LiveAmp) from BrainProduct’s offer. EMOTIV EPOC Flex was selected because of its low cost and the fact that some researchers in our center had already some experience with it. The price difference between the EPOC Flex and the other systems, with a ratio of around 10, prompted us to carry out these investigations in order to measure the advantages and limitations of each system. Among the growing number of validation studies for mobile EEG (for reviews, see Lau-Zhu, Lau & McLoughlin, 2019; Browarska et al., 2021; Sabio et al., 2024), the present one combined a set of conditions that are rarely found together in validation studies: (1) two portable EEG systems have been compared with each other, but also with a reference wired system, which has rarely been done ((but see Khng & Mane, 2020), for a medical-grade system as reference), or with few participants (Ratti et al., 2017); (2) mobile EEG systems were pitted against each other for their efficacy in replicating four expected ERPs, but also two well-established frequency-based (i.e., brain oscillations) classical data; (3) all three systems had the same number of electrodes; (4) to compare signals recorded at the same sites, it followed an intra-individual design with fixed order of the tasks, and EEG systems tested in separate sessions with counterbalanced order, instead of simultaneously recording with two systems (Williams et al., 2020); (5) the minimal number of items required to capture the expected changes in brain oscillations and ERPs was assessed by analyzing either 100% or the first 75% and 50% of the data; (6) visual and auditory paradigms were used to provide the scientific community with various and concrete guidelines for implementing experimental protocols with portable EEG systems.

Portable EEG systems were compared here to a lab-based system according to three main dimensions. First, we evaluated whether the three EEG systems could allow for time/frequency analyses of brain activity to infer reliable information about conscious states. Measurable increase in amplitude of alpha power at occipital sites with closed eyes as compared with open eyes (Berger effect) is typically considered to reflect resting state. This spectral feature has already been extracted with mobile EEG systems (Johnstone, Blackman & Bruggemann, 2012; Debener et al., 2015; Cannard, Wahbeh & Delorme, 2021; An et al., 2022), even in children (Bhavnani et al., 2022) and epilepsy patients (Shivaraja et al., 2023). We expected to replicate this effect in healthy adults using our three systems.

Second, we compared the performance of the three systems with measures of steady state visually evoked potential (SSVEP, Regan, 1966). Like the Berger effect, the SSVEP overcomes one of the main weaknesses of conventional ERP experiments, namely the long acquisition time due to a low signal to noise (SNR) (Norcia et al., 2015; Stothart, Smith & Milton, 2020). Participants looked at a red square flashing at three different rates. Analyses focused on the electrophysiological responses in the occipital cortex at the exact frequencies of the displayed stimuli, as already observed for face, word and Arabic numeral processing (Liu-Shuang, Norcia & Rossion, 2014; Lochy, Van Belle & Rossion, 2015; Dzhelyova, Jacques & Rossion, 2016) in both adults and children. A validation study showed comparable SSVEP with the research-grade g.Hlamp and EM systems (Grummett et al., 2015). However, because noise floor was higher with EM, there were concerns about assessing low-frequency spectra (Sabio et al., 2024).

Third, systems were compared based on four ERP components. The first component was the N170, best-known in the domain of face processing (Bentin et al., 1996; Eimer, 2011). Specifically, the N170 was expected to be maximal at occipitotemporal sites in response to facial stimuli compared to aggregates of random features in a discrimination task (Rossion, 2014). The N170 component is relatively easy to observe in children and adults (Taylor, Batty & Itier, 2004), and it has already been recorded with EM (De Lissa et al., 2015; Wehrman et al., 2021). Three other components of interest were also investigated: the N200, the P300, and the mismatch negativity (MMN). To include a perceptual modality other than visual, we chose two auditory paradigms. A Go/Nogo test was used to evaluate variations in N200 and P300 amplitudes and provide indications of voluntary inhibition of responses to rare (deviant) phonemes in a stream of frequent (standard) phonemes (Bedoin, Krzonowski & Ferragne, 2013). These stimuli were also used in a passive listening task to assess the MMN, peaking around 200 ms after a deviant onset (Näätänen, 2009) and indexing the automatic detection of a unexpected sounds. N200 and P300 have already been successfully recorded using EM, but simultaneously with another system, preventing from comparisons at the same sites (Badcock et al., 2013). Mixed results have been reported for the P300, which was lower when recorded with EM than a medical-grade EEG system, suggesting that numerous trials are necessary (Duvinage et al., 2013). MMN is indicative of phonological development (Linnavalli et al., 2017; Stekić et al., 2023) and it is therefore interesting to know the conditions under which it can be recorded with mobile EEG systems, to favor extensive investigation of speech development in children, phonological deficiencies in patients, or second language learning. Previous studies have indicated a low SNR for the MMN recorded with EM (Badcock et al., 2013) and more trials had to be rejected to record it than with a traditional EEG system (Barham et al., 2017). Therefore, in the present study, analyses of the spectral and ERP recordings conducted on 100%, 75% and 50% of the data could be informative about the minimal number of stimuli enabling researchers to evaluate these various indices of cognitive mechanisms with each of the three investigated EEG systems.

Methods

Each section closely adheres to the recommendations of the MEEG Committee on Best Practice in Data Analysis and Sharing (COBIDAS) established by the Council of the Organization for Human Brain Mapping (OHBM) (Pernet et al., 2018). The raw data, the output data obtained after processing the EEG files and the scripts used for statistical analyses are available at Zenodo at the link https://zenodo.org/records/15698441.

Participants

Twenty-one French volunteers, free from neurological or psychiatric conditions and in good health, participated in the study. Data from two participants were excluded due to technical issues and experimenter error, resulting in a final sample of 19 individuals (15 females, four males), aged 19 to 56 years (M = 27.6; SD = 9.3). All participants provided written informed consent, and the study was approved by the University of Lyon ethics committee (CER-UdL n° 2023-06-15-003).

General design of the study

The study employed a within-subjects design, divided into three sessions, each featuring a different EEG system. Each participant carried out the three sessions (one or two per day). Each session included five tasks administered in the same order for all EEG systems to minimize variability between sessions: (1) SSVEP, (2) active auditory oddball (standard vs. deviant stimuli), (3) passive auditory oddball, (4) face perception, and (5) resting state. Alternating between active and passive tasks aimed at preventing boredom and maintaining participant attention during the approximately 35-min session. In contrast, the order of the EEG systems was counterbalanced to ensure that the three systems were used similarly in the first, second, and third sessions. The total duration of EEG acquisition for one participant, excluding installation time, lasted around 105 min.

Stimuli

The experiment was conducted in a dedicated sound-attenuated room. Participants were comfortably seated in a chair, with the ventilation system and lights turned off, the curtains drawn, and their phones set to airplane mode throughout the EEG recordings. Stimuli and instructions were delivered using Presentation software (version 23.1) on a laptop with a 15.6″ Full HD (1,920 × 1,080) screen (60 Hz refresh rate). Participants were seated 60 cm from the screen, and auditory stimuli were played through Sennheiser IE 100 Pro Clear in-ear headphones at a comfortable listening level. We used the ERP CORE (Compendium of Open Resources and Experiments) resource material as a starting point to create our variant of the standard ERP paradigms (Kappenman et al., 2021).

Resting state paradigm

The task consisted of four 1-min blocks alternating between eyes-open (EO) and eyes-closed (EC) conditions. In the EO condition, participants were shown images of the cosmos. The total duration of the paradigm was 4 min.

Steady state visual evoked potential paradigm

The task consisted of six 30-s blocks, each starting with a 2-s white fixation ellipse at the center of the screen (8 × 13 pixels). Stimuli were red squares (200 pixels) on a gray background, flashing at fixed frequencies: 6 Hz (two blocks), 10 Hz (two blocks), or 15 Hz (two blocks). Consequently, squares were displayed for 83.3 ms at 6 Hz, 50 ms at 10 Hz, and 33.3 ms at 15 Hz. The blocks were presented in random order, and participants were instructed to focus on the center of the screen throughout the task, which lasted 3.5 min.

Face perception paradigm (N170)

The task consisted of 300 trials, featuring 150 human face images and 150 texture images (400 × 500 pixels) created from scrambled face pixels (from the FACES platform, Ebner, Riediger & Lindenberger, 2010). The faces included young and middle-aged individuals, with equal gender distribution and neutral expressions. Each image was displayed for 300 ms, followed by a 1,000-ms fixation ellipse. Participants pressed a key (Q for face, M for texture) using their left and right index fingers, with key assignments counterbalanced. Two self-paced breaks were provided during the task, which began with a practice sequence of five faces and five textures not included in the main trials. Response times were recorded. The task lasted 7 min.

Active auditory oddball paradigm (N200 and P300)

A total of 400 trials were presented, consisting of audio excerpts of the standard /yty/ (80%) and the deviant /ysy/ (20%) pronounced by a female voice. Each trial featured one sound (320 ms) followed by a 1,000-ms interval. Stimuli were pseudo-randomized for each participant, ensuring at least one standard was presented between two deviants. A white fixation ellipse was centered on the screen, and participants were instructed to press the space key for /yty/ and to refrain from pressing for /ysy/. Reaction times for correct responses were recorded. The task was divided into two equal parts with a self-paced break, each beginning with at least four standards, and it lasted 9 min.

Passive auditory oddball paradigm (MMN)

Participants were exposed to a sequence of 674 pure tones, each lasting 70 ms. The sequence included 562 tones of 500 Hz (standards, 83.4%) interspersed pseudo-randomly with 112 tones of 550 Hz (deviants, 16.6%) each. Inter-trial interval varied randomly between 550 to 650 ms. The same sequence was used for all participants, with at least two standards between deviants. As in typical settings in passive oddball task (Winkler, 2007), participants watched a silent video throughout the task. By focusing on the movie, the participants engaged attention away from the sounds, which allowed us to measure non-controlled cerebral responses to deviants and standards. There was no break during the MMN task, which lasted 8 min.

EEG systems

Table 1 presents the characteristics of each system, including channels, electrodes, headset, amplifier resolution, sampling rate, and connectivity.

Table 1: Features and comparison of the three EEG systems.

LiveAmp & EMOTIV wireless systems

The LiveAmp (LA) research-grade system included a LiveAmp amplifier and a 32-channel R-Net headset (both from Brain Products GmbH). The compact, wireless amplifier communicated with the acquisition computer via Wi-Fi, while the R-NET cap used passive wet electrodes arranged according to the international 10-10 system, secured by a flexible silicone structure.

The EM consumer-grade system featured EPOC Flex Saline sensors, a Flex control unit for amplification, an EasyCap cap, and an EMOTIV (EM) Extender. EEG data were wirelessly transmitted via Bluetooth from the EPOC Flex control unit to the acquisition computer. The EasyCap system used passive wet electrodes, which were manually positioned according to the international 10-10 system. The EMOTIV Extender received event markers from the laptop presenting the stimuli. EMOTIV products are CE certified and comply with the Radio Equipment Directive (2014/53/EU), but are not intended for medical diagnostics, as stated in EU Directive 93/42/EEC.

BrainAmp wired system

The BrainAmp (BA) system included a BrainAmp amplifier, an actiCap cap, and a USB adapter for data transmission and event marker reception. EEG data were acquired using the BrainAmp amplifier with a 32-channel cap featuring active electrodes positioned according to the international 10-20 system and using conductive gel (Supervisc 2). Data were transmitted to the USB adapter via optical fiber. Data was sent to the USB adapter through optical fiber. Brain Products GmbH meets the requirements of Annex II (excluding ‘Discussion’) of the Medical Devices Directive 93/42/EEC for neurophysiological research and has implemented a quality assurance system for the design, production, and final inspection of its devices.

Procedure

Electrode positions

EEG recordings for each system were acquired on a Windows 11 Pro Dell Precision 3571 laptop, separate from the stimulus presentation computer, using the respective native software (BrainVision Recorder for Brain Products, GmbH and EmotivPRO for EM). Continuous EEG was recorded from 32 scalp sites for the EM and LA systems, with F9, P9, F10, and P10 replaced by FT9, TP9, FT10, and TP10 for the BA system (Fig. 1). In the EM layout, reference electrodes were CMS (Fpz) and DRL (FCz), while LA and BA used Fpz as the ground and FCz as the reference. Impedance was maintained below 50 kΩ for LA and 20 kΩ for BA. Following Williams et al. (2020), Flex electrodes were adjusted until impedance values turned ‘green’, indicating levels below 20 kΩ.

Electrode names and labels for the BA system (A), the LA and EM systems (B).

Event-marking

Event tagging was performed using a USB-6501 trigger box from NATIONAL INSTRUMENTS-NI, which emitted triggers from the stimulus presentation computer. The triggers were sent via a 2.5 mm audio jack to the LiveAmp amplifier for LA and the EMOTIV Extender for EM. As the amplifier had no trigger input channel, the extender box was used to receive the triggers. The extender was connected to the NI board via two wires and to the EMOTIV amplifier via a USB Mini-A port. For the BA system, triggers were sent to the recording computer via a USB adapter. These configurations enabled single-bit event coding, which required an additional step in the processing pipeline to identify all event codes.

Offline EEG general processing

Python and the MNE library (Gramfort, 2013; https://doi.org/10.5281/zenodo.592483) were used to match event codes in the EEG recordings with log files from the stimuli presentation software. All subsequent processing steps were conducted using BrainVision Analyzer software (version 2.2).

System sampling rate and dataset

BA and LA data were downsampled to 128 Hz to match the EM’s native sampling rate, referred to as Pack-100a. For the three ERP tasks, an additional analysis retained LA and BA recordings at their native rates of 500 and 250 Hz, respectively (Pack-100b). Further analyses on raw data included the first 75% or 50% of trials for each task (Pack-75, Pack-50).

Common steps for signal processing

The EEG signal was filtered using a bandpass filter (0.016 to 30 Hz for Resting State, Active Auditory, and Passive Auditory tasks; 0.01 to 55 Hz for SSVEP; 0.1 to 40 Hz for Face Perception) and a 50 Hz notch filter. The data was then re-referenced to the common average.

Resting state processing

Continuous EEG was segmented into two 60-second segments for the eyes-closed (EC) and eyes-open (EO) conditions. A wavelet transform using Morlet complex waveforms (1 to 40 Hz with Gabor normalization) was applied to these segments. Spectral power from the two segments in each condition was averaged afterwards, and alpha band power (8.5–12.7 Hz) was extracted for each participant and system at the O1, Oz, and O2 electrodes before final averaging.

SSVEP processing

Continuous EEG was segmented for 6, 10 and 15 Hz conditions, resulting in two 30-s segments for each one, which have been finally averaged for each condition at O1, Oz and O2. Spectral power over these electrodes was calculated with a Fast Fourier Transform (0.05 Hz resolution using a Hanning window). For each frequency bin, a SNR was calculated by dividing the amplitude of a frequency bin by the average of the 20 surrounding bins (10 on each side, excluding the two immediate neighbors).

ERP processing

For ERPs, artifacts corresponding to deflection greater than 500 µV over 800 ms were marked as large muscular events in the raw data for each electrode. Electrodes with artifacts longer than 5% of the total recording time were deemed faulty and removed. No interpolation was performed. The data was epoched from −200 to 800 ms relative to stimulus onset, except for the passive auditory task, where epochs were from −50 to 560 ms due to variable inter-stimulus intervals (ISI) that could cause overlap.

An independent component analysis (ICA) was conducted and components that were manually identified as blinks or saccade artifacts were corrected. After data had been epoched, a second automatic artifact rejection excluded epochs with deflections exceeding 200 µV. No other noise reduction step was conducted afterwards.

N170

One subject was excluded from all systems due to a high rejection rate (94.3%) in the EM recordings. For baseline correction, the mean voltage of the 200-ms prestimulus interval was subtracted from each epoch. Face and texture epochs were averaged separately for each participant across the three systems.

The N170 peak was identified at P7 and P8 electrodes as the most negative voltage between 100 and 160 ms, with temporal windows set to 120–160 ms for LA and BA and 100–140 ms for EM. The P100 peak was determined as the maximum positive voltage between 60 and 120 ms at P7 and P8, with windows of 80–120 ms for LA and BA and 60–100 ms for EM. In the face condition, the N170 peak latency and the peak-to-peak (P2P) amplitude difference between P100 and N170 were extracted for each participant and system.

N200 and P300

The mean voltage from the 200-ms prestimulus interval was subtracted from each epoch. ERPs time-locked to stimulus onset were averaged separately for standards and deviants at every electrode, system, and participant. Standards immediately following a deviant were excluded (Bedoin, Krzonowski & Ferragne, 2013). The average waveforms for standards were then subtracted from those for deviants to extract ERP responses for each participant. N200 intervals were determined from the grand average ERP at Fz, peaking around 270 ms for BA and LA, and 260 ms for EM, with temporal windows set to 200–280 ms for BA, LA and EM. The P300 peak was identified at around 380 ms for BA, LA and EM, using a time window of 280–420 ms for all systems. N200 and P300 peak amplitude and latency were analyzed at Fz, Cz, and Pz. One participant was excluded from the analysis for all systems due to excessive faulty electrodes at Fz, Pz, and 11 adjacent electrodes during EM processing.

MMN

Frequencies below 0.5 Hz and above 20 Hz were filtered out, and baseline correction was performed by subtracting the mean voltage from the 50 ms prestimulus interval. ERPs time-locked to sound onset were averaged for deviants and standards preceding a deviant to compute the MMN as the difference signal (deviant waveform minus standard waveform). Visual inspection of the average difference waveform at Fz indicated that the MMN peaked around 190 ms for LA, 200 ms for BA and 140 ms for EM, leading to the selection of a 100–250 ms time window. The area under the curve (AUC) has been chosen as the amplitude measure for the MMN (Pekkonen et al., 1993). It was computed following Beauchemin & De Beaumont (2005) guidelines over a 50 ms window centered on the MMN peak in the grand average ERP. Statistical analyses were conducted on peak amplitude, peak latency, and AUC across Fz, Cz, and Pz, with time windows adjusted for each system and pack (Table 2). One participant was excluded from all analyses due to numerous noisy electrodes at Fz, Pz, and 11 adjacent sites during the processing steps of the EM recording.

Table 2: Mean MMN peak latency across all analysis and time windows for area calculation.

Data analyses

Statistical analyses were conducted using Python with the pingouin and scipy.stats libraries. Expected phenomena (alpha power, harmonic responses, ERPs) were assessed with two-tailed one-sample t-tests against zero. Repeated-measures ANOVAs were performed, followed by two-tailed bilateral t-tests with Bonferroni adjustment. Effect sizes were reported for ANOVAs (partial eta-squared, η^2^; small = 0.01, medium = 0.06, large = 0.14) and t-tests (Cohen’s d; small = 0.2, medium = 0.5, large = 0.8), with all results expressed as absolute values.

Resting state

The difference alpha power between EO and EC conditions was computed for each participant and compared to zero using a two-tailed t-test. An ANOVA was conducted on this difference with system (BA, EM, LA) as within-subject factor.

SSVEP

The SNR at the fundamental components and their first two harmonics (12, 18 Hz for the 6 Hz condition; 20, 30 Hz for the 10 Hz condition; 30, 45 Hz for the 15 Hz condition) were averaged. The presence of harmonic responses was assessed using t-tests against zero for the pooled SNR values. A two-way ANOVA was performed on the SNR using system (EM, LA, BA) and condition (6, 10, 15 Hz) as within-subject factors.

N170

The P100-N170 difference in peak amplitude was compared to zero by a t-test at P7 and P8. ANOVAs using the EEG system (EM, LA, BA) as within-subject factor were conducted on N170 peak latency and P100-N170 peak amplitude difference, at P7 and P8.

N200 and P300

The presence of significant N200 and P300 waves was assessed using t-tests against zero for peak amplitude at Fz, Cz and Pz. ANOVAs were performed on N200 and P300 latencies and amplitudes at Fz, Cz and Pz with EEG system as within-subject factor (three levels).

MMN

Significant MMN was assessed with one-tailed t-tests comparing the AUC and mean peak amplitude to zero at Fz, Cz and Pz. ANOVAs were performed on latency, amplitude, and AUC of the MMN at these electrode sites using EEG system (three levels) as within-subject factor.

Supplementary analyses

We conducted a series of additional analyses to further assess the reliability and precision of the ERP measures obtained across the three EEG systems. First, Pearson correlations were calculated to assess subject-by-subject measurement consistency across the three systems. Second, we computed the standardized measurement error (SME) for each ERP component, following the procedure described by Luck et al. (2021). The SME provides an estimate of the trial-level variability in amplitude and latency measures, and thus complements the correlation analysis by quantifying measurement precision at the individual level. The SME for each system was compared with the benchmark from Zhang & Luck (2023) (when available at the same electrode sites) using Mann-Whitney tests. The systems were also pitted against each other using Friedman tests.

Results

Signal quality

As can be seen in Table 3, the EM system yielded less exploitable data than LA and BA systems. The EM headset had to be changed in the middle of a session for two participants due to a lost connection between the amplifier and the acquisition computer. More triggers were lost and more electrodes were declared faulty within the region of interest with EM than the other systems, leading to exclude one participant from the analysis of N200, P300 and MMN for every system. Another participant has been excluded for N170 for every system, because of a high percentage of rejected epochs with EM system.

Table 3: ERP signal quality of EM, LA and BA systems. Standard errors are in brackets.

Resting state results

When compared to zero, the difference in alpha power between the EO and EC conditions (Berger Effect) was significant and large-sized for each EEG system whether calculated in Pack-100, 75, or 50 (Cohen’s ds > 1.03, Table 4, left).

Table 4: Mean difference in alpha power for each pack and each system as compared to zero (left panel), and between-system comparisons for the Berger Effect (right panel).

The significant main effect of system was large in size for Pack-100 (F(2, 36) = 7.04, p = 0.007, η_p_^2^ = 0.28), Pack-75 (F(2, 36) = 6.62, p = 0.008, η_p_^2^ = 0.27), and Pack-50 (F(2, 36) = 6.36, p = 0.011, η_p_^2^ = 0.26), reflecting a lower Berger Effect with EM than LA and BA (Table 4, right).

SSVEP results

The SNR was significantly higher than zero in the 6, 10, and 15 Hz conditions for every system (Fig. 2, Table 5), and the effect sizes were large (ds > 1.25) for every system and Pack.

Amplitude spectra expressed as SNR across occipital electrodes (O1, Oz, O2) for the three systems and three conditions (fundamental component = 6, 10 or 15 Hz) in Pack-100a.

Table 5: Comparisons of the SNR value to zero, for each pack, system, and condition.

In Pack-100, the significant main effect of system (F(2, 36) = 23.93, p < 0.001, η_p_^2^ = 0.57) was due to lower SNR with EM than LA (t(18) = 8.17, p < 0.001, d = 0.97) and BA (t(18) = 5.45, p < 0.001, d = 1.11). There was no main effect of condition neither Condition × System interaction. However, the SNR was very low with EM at 10 Hz condition for the fundamental component and its harmonics (Fig. 2).

In Pack-75, there was only a significant main effect of system (F(2, 36) = 18.15, p < 0.001, η_p_^2^ = 0.50), reflecting lower SNR with EM than LA (t(18) = 6.71, p < 0.001, d = 0.93) and BA (t(18) = 4.94, p < 0.001, d = 1.05). Similarly, in Pack-50, the main effect of system (F(2, 36) = 16.44, p < 0.001, η_p_^2^ = 0.48) reflected lower SNR with EM than LA (t(18) = 6.53, p < 0.001, d = 0.98) and BA (t(18) = 4.77, p < 0.001, d = 1.03).

N170 results

To assess the absence of any bias due to the EEG system on overall response speed, mean response times and SD were submitted to Wilcoxon tests, corrected for multiple comparisons. No significant difference was observed neither for response speed (EM-LA, V = 82, p = 0.615; EM-BA, V = 111, p = 0.841; LA-BA, V = 107, p = 0.956) nor speed variability (EM-LA, V = 108, p = 0.927; EM-BA, V = 139, p = 0.216; LA-BA, V = 145, p = 0.143).

In face condition and Pack-100 with native sampling rate (Pack-100a for EM, Pack-100b for LA and BA), the P2P amplitude difference (P100-N170, hereafter, N170) significantly differed from zero at P7 and P8, with large effect sizes for every system (1.22 < all ds < 2.06) (Table 6). N170 remained large-sized in Packs-75 and 50 for the three systems (1.12 < all ds < 2.00), and with lower sampling rate (Pack-100a) for LA and BA (1.22 < all ds < 1.93).

Table 6: Peak latency of the N170 for face condition, and comparisons of the Peak-to-Peak (P2P = P100-N170) amplitude difference to zero, for each pack, sampling rate, system, and electrode site.

For Pack-100 and native sampling rate, the significant main effect of system on P2P amplitude at P7 (F(2, 34) = 14.81, p < 0.001, η_p_^2^ = 0.47) and P8 (F(2,34) = 14.57, p < 0.001, η_p_^2^ = 0.46) reflected higher N170 for BA than EM and LA, but no EM-LA difference (Fig. 3, Table 6). This pattern of results was confirmed by the main effect of system with a lower sampling rate (Pack-100a: P7, F(2, 34) = 12.36, p < 0.001, η_p_^2^ = 0.42; P8, F(2, 34) = 13.29, p < 0.001, η_p_^2^ = 0.44), and Pack-75 (P7, F(2, 34) = 13.30, p < 0.001, η_p_^2^ = 0.44; P8, F(2, 34) = 13.73, p < 0.001, η_p_^2^ = 0.45). For Pack-50, the significant main effect of system at P7 (F(2, 34) = 13.82, p < 0.001, η_p_^2^ = 0.45) reflected higher N170 for BA than EM and LA, but at P8 (F(2, 34) = 13.89, p < 0.001, η_p_^2^ = 0.45) it reflected higher N170 with BA than LA and higher N170 with LA than EM (Table 6).

Group N170 ERP waveforms for the three systems at P7 and P8 electrode sites in response to face (top) and texture (bottom).

Regarding N170 latency for face condition, the significant main effect of system in Pack-100 at native sampling rate (P7, F(2, 34) = 76.08, p < 0.001, η_p_^2^ = 0.82; P8, F(2, 34) = 71.62, p < 0.001, η_p_^2^ = 0.81) reflected shorter latencies for EM than LA and BA (Table 6, Fig. 3). This pattern remained in Pack-100a, according to the main system effect (P7, F(2, 34) = 77.45, p < 0.001, η_p_^2^ = 0.82; P8, F(2, 34) = 66.64, p < 0.001, η_p_^2^ = 0.80), as well as in Pack-75 (P7, F(2, 34) = 60.14, p < 0.001, η_p_^2^ = 0.78; P8, F(2, 34) = 52.32, p < 0.001, η_p_^2^ = 0.75), and Pack-50 (P7, F(2, 34) = 41.64, p < 0.001, η_p_^2^ = 0.71; P8, F(2, 34) = 39.34, p < 0.001, η_p_^2^ = 0.70).

The typical parietal N170 saliency for faces was observed with LA and B—less with EM—, at least when the face-texture difference was represented (Fig. 4).

Topographic distribution of the signal for the difference between face and texture conditions.

In texture condition, N170 for Pack-100a significantly exceeded zero at P7 and P8 (Table 7), and the effect size was large for LA and BA, and medium for EM. The examination of Fig. 3 suggested the classical decrease in amplitude and increase in latency of the N170 at posterior electrodes in texture condition as compared with face condition (Rossion, 2014), especially for LA and BA.

Table 7: Peak latency of the N170 for texture condition, and comparisons of the Peak-to-Peak (P2P = P100-N170) amplitude difference to zero, for each system and electrode site, with equated sampling rates (Pack-100a).

The main effect of system for Pack-100a was significant at P8 (F(2, 34) = 8.38, p < 0.001, η_p_^2^ = 0.33), reflecting lower N170 amplitude for EM than BA (t(17) = 3.49, p = 0.008, d = 0.79) and LA (t(17) = 4.51, p < 0.001, d = 0.65). No difference reached significance at P7.

N170 latency was modulated by the main effect of system (P7, F(2, 34) = 26.12, p < 0.001, η_p_^2^ = 0.61; P8, F(2, 34) = 30.12, p < 0.001, η_p_^2^ = 0.64). It was shorter for EM than LA (P7, t(17) = 6.29, p < 0.001, d = 1.85; P8, t(17) = 6.06, p < 0.001, d = 1.83) and BA (P7, t(17) = 6.15, p < 0.001, d = 1.79; P8, t(17) = 5.36, p < 0.001, d = 1.69) without LA-BA difference.

N200 & P300 results

Like the face perception task, the active auditory oddball task did not give rise to differences between the systems regarding response speed (EM-LA, V = 89, p = 0.571; EM-BA, V = 116, p = 0.701; LA-BA, V = 100, p = 0.869) nor speed variability (EM-LA, V = 83, p = 0.430; EM-BA, V = 105, p = 0.999; LA-BA, V = 101, p = 0.898).

The N200 amplitude in Pack-100 and native sampling rate significantly differed from zero for all three systems (Table 8). It was large in size for BA (medium in size at Pz) and LA, and medium in size for EM, with prominence at Fz for every system (Fig. 5). Despite reduced datasets (Packs-75, 50) for every system or lowered sampling rate (LA and BA), the N200 effect size remained large and prominent at Fz. N200 amplitude was not significantly affected by a main effect of system in Packs-100a, 100b, 75 and 50, at Fz, Cz or Pz (all ps > 0.415).

Table 8: Peak latency of N200, and comparisons of the N200 amplitude to zero for each pack, sampling rate, system, and electrode site.

Group N200 and P300 waveforms for the three systems at Fz, Cz and Pz electrodes (A). Topographic distribution of N200 (B) and P300 (C) ERP signals for the three systems.

Regarding N200 latency, there was no significant main effect of system at the native sampling rate (Pack-100b) (Fz, F(2, 34) = 3.50, p = 0.064, η_p_^2^ = 0.17; Cz, F(2, 34) = 0.51, p = 0.606; Pz, F(2, 34) = 0.90, p = 0.416). The marginal effect at Fz site reflected shorter latency for EM than BA (t(17) = 4.67, p < 0.001, d = 1.12). A main effect of system appeared with a lower sampling rate (Pack-100a: Fz, F(2, 34) = 7.66, p = 0.006, η_p_^2^ = 0.31; Cz, F(2, 34) = 5.73, p = 0.016, η_p_^2^ = 0.25; Pz, F(2, 34) = 3.51, p = 0.041, η_p_^2^ = 0.17), reflecting significantly shorter latency with EM than BA (Fz, t(17) = 6.51, p < 0.001, d = 1.44; Cz, t(17) = 5.25, p < 0.001, d = 1.04; Pz, t(17) = 3.07, p = 0.021, d = 0.77). For Pack-75, the system effect was significant at Fz site (F(2, 34) = 4.77, p = 0.031, η_p_^2^ = 0.22) reflecting the EM < BA pattern (t(17) = 5.69, p < 0.001, d = 1.40). For Pack-50, there was no significant main effect of system, but the EM < BA pattern remained at Fz (t(17) = 2.84, p = 0.034, d = 0.60).

P300 amplitude significantly exceeded zero at Fz and Cz, with a large effect size for every pack and system (Table 9). It was prominent at Cz (except with LA in Packs-75 and 50) and overall smaller in size, but still significant, at Pz.

Table 9: P300 amplitude and peak latency comparisons to zero, for each pack, sampling rate, system, and electrode site.

These was no main effect of system on P300 amplitude in all Packs and electrodes.

In Pack-100b, the main effect of system was significant on P300 latency (Fz, F(2, 34) = 3.90, p = 0.030, η_p_^2^ = 0.19; Cz, F(2, 34) = 4.57, p = 0.018, η_p_^2^ = 0.21), reflecting shorter latency for EM than BA (Fz, t(17) = 2.75, p = 0.041, d = 0.63; Cz, t(17) = 3.18, p = 0.016, d = 0.81) and LA at Fz (t(17) = 2.71, p = 0.045, d = 0.66). In Pack-100a, the main effect of system at Fz (F(2, 34) = 5.57, p = 0.008, η_p_^2^ = 0.25) was explained by a shorter latency for EM than LA (t(17) = 2.82, p = 0.035, d = 0.76). The system effect at Fz was replicated in Pack-75 (F(2, 34) = 7.87, p = 0.002, η_p_^2^ = 0.32), with an EM < BA pattern (t(17) = 5.29, p < 0.001, d = 1.03), but there was no system effect in Pack-50.

MMN results

The AUC of the MMN in Pack-100 at native sampling rate significantly exceeded zero for every system at Fz (Table 10); the effect size was large (LA, BA), or medium (EM). It also exceeded zero significantly at Cz, medium-sized for LA and BA and small for EM (Fig. 6), while it was not significant at Pz. It remained the highest at Fz, even in Pack-100a, for all three systems.

Table 10: Peak latency of the MMN, Area Under the Curve of MMN (AUC), and comparisons between MMN amplitude to zero (t-tests) at Fz, Cz and Pz electrodes, for the three EEG systems, each pack and sampling rate.

(A) Group MMN ERP waveforms for the three systems at Fz, Cz and Pz electrode sites. (B) Topographic distributions of MMN ERP signals for each system.

For BA, over Packs-100b, 100a, 75 and 50, the significant MMN amplitude was medium in size at Cz, and large at Fz. For LA, the significant MMN was medium in size over every pack at Cz, whereas it decreased sharply from large over Pack-100b, 100a and 75 towards medium size in Pack-50 at Fz. For EM, at Cz, the significant MMN was small in Pack-100a and -50, but medium in Pack-75, while it increased from medium size in Pack-100a to large size in Pack-75 and -50 at Fz. No main effect of system, and no two-by-two system comparison reached significance, whatever the sampling rate or the pack, at any electrode site.

There was no system effect on the MMN latency, peaking around 180–200 ms for every dataset (Table 10). Detailed two-by-two comparisons did not reveal any significant difference.

Between-system correlations

Correlations calculated to assess measurement consistency across participants between the three systems (LA/BA, EM/LA, and EM/BA) are detailed in Table 11. Theses correlations reached significance (p < 0.05) when exceeding r = 0.47. With respect to time/frequency measures (alpha-power and SSVEP), most correlations were significant or near-significant (even with reduced data-sets), with the notable exception of the EM-BA relation for 15 Hz condition. With respect to ERP components, the systems were highly correlated for both the N170 amplitude and latency at P7, but only for latency at P8. For the N200 as P300, few correlations were significant for latencies, but correlations were largely significant for amplitude. Taken together, various indices supported relatively high measurement consistency between participants for time/frequency measures across the systems, and moderate consistency for ERP components with the exception of the MMN, (see Figs. S1 to S6).

Table 11: Across participants correlation values (Pearson) between the measures (time/frequency or ERPs) recorded with the three systems (EM, LA, BA) in every task and in every data-set (Pack).

Standardized measurement error

Data are shown in Table 12. Key findings were (1) systematically lower SME for the LA system measuring MMN, N170, N2 and P3 amplitude (LA < BA < EM), (2) no difference between systems when measuring latencies except the EM > LA = BA pattern at P8 and the EM > BA at P7 for the N170, (3) regarding the N170, each system had a lower SME than the ERP CORE dataset for both amplitude and latency, whereas EM exhibited a higher SME that the ERP CORE, (4) as to the MMN latency, higher SME was observed for each of the three systems as compared with the ERP CORE dataset (see Tables S1, S2 and Fig. S7).

Table 12: Between-system comparisons of the standardized measurement error (SME) for each ERP component for Pack-100b. Post-hoc comparisons are not reported when Friedman test is not significant.

Effect sizes for every system, according to task duration/number of trials

Table 13 provides an overview of the effect sizes for each task, system and Pack (expressed in seconds or number of trials). This synthesized presentation meets our objective regarding guidelines for researchers.

Table 13: Effect size (Cohen’s d) of the expected spectral features of the neural signal as compared to zero in two paradigms (Resting State, SSVEP) and of the four studied ERPs, as a function of the size of the dataset (Pack-100%, Pack75%, Pack 50%) and the EEG system.For SSVEP, d values for the fundamental frequency and its 2 first harmonics have been averaged; for N170, d values in face condition have been averaged over P7 and P8; for N200, P300 and MMN, d values have been averaged over Fz, Cz, and Pz. Duration or number of trials are presented on the gray lines (s = second).

Discussion

We aimed to compare how three EEG systems—a mobile consumer-grade system (EM), a portable research-grade system (LA), and a non-portable research-grade system (BA)—perform on measures of well-established electrophysiological indices of mental states, sensory, and cognitive processes. All three systems succeeded in showing, (1) alpha power changes in resting state, (2) spectral variations guided by the frequency of flickering stimuli, (3) ERP responses to faces (N170), and to controlled (N200, P300) or automatic (MMN) auditory ERPs. Therefore, the results support our three hypotheses and highlight how effective but also limited may be wearable EEG systems.

Spectral analyses

Consistent with our hypothesis, all three EEG systems captured the occipital Berger Effect already observed in adults (Johnstone, Blackman & Bruggemann, 2012; Cannard, Wahbeh & Delorme, 2021; An et al., 2022) and children (Barry et al., 2009; Bhavnani et al., 2022; Edgar et al., 2023). The effect was stable and large in size even in the 30-s Pack-50 with every system, though it was of larger size with LA and BA compared to the consumer-grade EM system. These differences in spectral responses could not be attributed to discrepancy between recording resolution, as the three systems were equated in sampling rate. Though the Berger Effect has previously been reported with very minimalist devices (Johnstone, Blackman & Bruggemann, 2012; Debener et al., 2015; Shivaraja et al., 2023), mixed results were obtained from a 14-electrode EM system (Grummett et al., 2015; Mahdid, Lee & Blain-Moraes, 2020; Kounios et al., 2024). In the current study, the 32-electrode mobile EM system proved effective in assessing oscillations and succeeded in demonstrating the Berger Effect.

We also found that flickering stimuli evoked oscillatory brain responses exactly at the periodic stimulation frequency and its two first harmonics (SSVEP) across every EEG system. It should be noted that, with LA, these effects were large in size even on a short recording (15-s), highlighting the efficiency of this portable research-grade system in assessing oscillations, but a drop in effect size was however observed in Packs-75 and -50 compared to Pack-100 for LA, not for BA. Consequently, using only 15- or 22.5-s recording, comparable effect sizes were obtained with LA and EM systems, in line with similar spectral effects shown using EM and a medical-grade EEG system during emotional clips perception (Katsigiannis & Ramzan, 2018).

Despite showing lower SNR responses than LA and BA, the EM system revealed effective for spectral analyses, probably due to the choice of wet electrodes for EM and LA. Indeed, systems with dry electrodes have been found less efficient (Xing et al., 2018; Ladouce et al., 2022). However, there are two restrictions. Firstly, for SSVEP, increasing the dataset duration up to 30 s led to higher effect size with LA, but not EM. Therefore, longer stimulation did not compensate for less exploitable signal from EM. Secondly, EM captured the expected SNR increase at the 6 or 15 Hz fundamental frequency of a periodic visual stimulation and its harmonics, but it failed to do so in response to 10 Hz-stimulation, contrary to LA and BA. This may come from a temporal instability of SSVEP responses, as a decline of amplitude specifically after 10 and 15 Hz has been documented (Łabęcki et al., 2024). Another possibility is that 10 Hz is part of the alpha range (8–12 Hz), which is vulnerable to changes in endogenous attentional states. Endogenous attention has been shown to interfere with SNR variations and decrease their amplitude in response to flickers at alpha rhythm (e.g., 10 Hz) (Gulbinaite, Roozendaal & VanRullen, 2019). To avoid alpha wave contamination, slightly higher rates (12–13 Hz) are preferred in SSVEP experiments (Yang, Paller & Van Vugt, 2022). In sum, EM was found to record satisfactory SSVEP signature, but caution is advised when EM is used to assess the brain’s ability to discriminate images below the threshold of consciousness.

ERP analyses

Compared to spectral aspects, ERPs require time/amplitude analyses on very short signal durations, and the ability of consumer-grade systems to do so has been rarely questioned. Below, we discuss our results with respect to the N170, N200, P300, and MMN.

First, all three EEG systems showed a large N170, peaking at its typical posterior site (P7 and P8) in the face condition, with a much smaller amplitude, texture conditions, consistent with its well-known association with face perception (Bentin et al., 1996). This N170 was robust enough to remain large after LA and BA downsampling, or dataset reduction (e.g., Pack-50, 150 trials). The EM system detected an N170 that was only slightly smaller than with BA (but comparable to the LA system, except in the very short 150-trials dataset). Its efficiency in detecting the N170 was in line with previous studies examining similar systems (De Lissa et al., 2015; Soto et al., 2018; Wehrman et al., 2021; Sagehorn et al., 2024). Taken together, the results contribute to validate the use of mobile, wireless and low-cost EEG systems for research about N170. Note, however, that the posterior N170 prominence was less typical from EM than from LA and BA. Additionally, an overall reduction in latency occurred using EM for most of the ERP components, including N170. EM users are therefore encouraged to focus on ERP amplitude, and should be aware of shortcomings regarding spatial distribution and temporal precision.

Second, although the N200 has not always been obtained in auditory oddball studies (Falkenstein, Hoormann & Hohnsbein, 1999, 2002; Smith, Johnstone & Barry, 2004), all three EEG systems were able to detect it as a medium-to-large cerebral response with consistent frontal prominence in our auditory oddball task, despite dataset reduction for every system and sampling rate lowering (LA and BA). N200 amplitude did not significantly differ between systems. This effectiveness of the EM system was consistent with a previous study (Barham et al., 2017). Like the N170, the N200 peaked earlier using the EM than the BA system, reflecting lower time precision for EM.

Third, the P300 also appeared as large-sized with the three systems at the expected fronto-central site, replicating (Badcock et al., 2015). LA contrasted with EM and BA by failing to show Cz prominence for P300 in short datasets (Packs-75 and -50). EM and BA systems succeeded in recording P300 of similar size and were unaffected by dataset reductions. The efficiency of the EM system here contrasts with a previous study comparing EM with a standard system (Duvinage et al., 2013). However, it is worth noting that this previous study, which evaluated EM for medical diagnosis, differed in several aspects from ours (e.g., participants were walking, deviants were frequent, and EM was compared to a medical-grade EEG device). Nevertheless, like the N200 and N170, the P300 appeared earlier with EM than LA and BA.

Fourth, in line with our hypothesis, a significant fronto-central MMN was observed across all systems and all sampling rates, with stable prominence in effect size at Fz over Cz and Pz. At Fz site, the MMN was stable in amplitude (medium to large) over all systems, in line with MMN comparison with another research-grade system (Neuroscan) (Williams et al., 2020). Mixed results were previously obtained using EM to assess the MMN (Badcock et al., 2013, 2015). Note the decrease in the MMN size with the dataset reduction from Pack-75 (505 trials) to Pack-50 (337 trials) using EM and LA, contrary to BA. As only LA also exhibited sensitivity to the number of trials for P300, users should be aware of the risk of employing few trials to assess ERPs with this system. In contrast with the other investigated components, the MMN was not observed earlier using EM than the other systems. Therefore, the MMN was of satisfactory amplitude, latency, and topography even when recorded with a wireless and consumer grade EEG system.

Overall, one key finding is that the EM system recorded ERPs of reasonable quality. Each targeted ERP component had satisfactory amplitude and intra-individual variability measured by SME reached an appropriate level as compared with standards (Zhang & Luck, 2023). This is in relative contrast with previous reports (Grummett et al., 2015). We speculate that this might be due to the use of a custom acquisition software to improve synchronization between stimuli presentation and EEG recording (Tahmasebzadeh, Bahrani & Setarehdan, 2013). Note, however, that shorter latencies were observed for three of the four assessed ERPs using EM compared to both LA and BA. The observed lack of time precision replicates previous results obtained when the EPOC-Flex was compared to the Biosemi System (Ries et al., 2014). More generally, discrepancies related to latencies have been shown with portable EEG systems (e.g., Nautilus) compared with literature data (Melnik et al., 2017), though reports are inconsistent (David Hairston et al., 2014; De Lissa et al., 2015; Williams et al., 2020). Here, we can rule out that this limitation is due to a difference in electrolyte (Ries et al., 2014), because saline solution was used with EM as well as with LA. As a tentative explanation, it can be suggested that time imprecision could be partly due to either the method used to synchronize the EEG data with the events of interest or the internal filters of the EM amplifier. Furthermore, we could find no indication in our results to attribute this difference to a participant effect or task order. In addition, we used robust paradigms adapted from ERP CORE (Kappenman et al., 2021).

In line with Williams, McArthur & Badcock (2021), the low-cost mobile EM system can be recommended for spectrum analyses of EEG signal recorded over relatively long intervals, but it is less precise for time and has to be used with caution to assess some ERPs.

Interestingly, as a portable research-grade system, LA provided robust results with respect to ERP detection. Only minor limitations have been observed regarding LA, (1) for N170, it was less effective than BA in the smallest dataset, and it did not outperform the consumer-grade EM system in terms of effect sizes in larger datasets, (2) slight topographical variability was noticed for P300 in small datasets, (3) the MMN amplitude dropped with the LA and EM systems in the smallest dataset. Some of these weaknesses could be attributed to impedance maintained below 20 kΩ for BA, but only 50 kΩ for LA. Interestingly, no latency difference was observed between ERPs recorded with LA and BA, which suggests a higher temporal precision than with EM.

Taken together, the LA system appears to record reliable signal even with lowered sampling rate, saving energy and memory space, and to be effective except with very few trials.

General considerations regarding the mobile consumer-grade EM and the portable research-grade LA

Over the last few years, EM has been the prevalent choice for cognitive scientists using consumer-based EEG systems (Sawangjai et al., 2020; Sabio et al., 2024; Gramann, 2024), probably because of its large number of electrodes ensuring wider scalp coverage, and its attractive price. In our study, its signal quality yielded less exploitable data than research-grade systems. Additionally, a major incident (lost amplifier/acquisition computer connection leading to change the headset) occurred twice only with EM. However, with 90% of participants in the final dataset and enough epochs to conduct analyses, EM exhibited overall satisfactory signal quality and can be seen as a reliable tool for researchers, as emphasized in reviews (Lau-Zhu, Lau & McLoughlin, 2019; Niso et al., 2023; Sabio et al., 2024; Jacobsen et al., 2024; Gramann, 2024) and validation studies (Maskeliunas et al., 2016; Melnik et al., 2017; Sawangjai et al., 2020; Khng & Mane, 2020; Williams et al., 2020; Browarska et al., 2021; Williams, McArthur & Badcock, 2021).

In a previous study, Niso et al. (2023) investigated every characteristic of 48 portable systems, highlighting notable advantages of LA over EM system, justifying a price ten times higher for LA than EM. This is consistent with the score for system specifications (CoME S) proposed by Bateson et al. (2017), which considers electrode type, bit resolution, sampling rate, and battery life, on a 4 to 20 scale. According to this scale, LA obtained a higher score with 15-10 than 8-7 for EM. Furthermore, SME measurements have provided evidence for fewer variability in LA data than in the other systems data (often including the BA system). Some higher performance observed from LA over EM system can be attributed to transmission mode (Wi-Fi used for LA is better than Bluetooth used for EM), discrepancies in hardware filter responses, and in amplifiers. Regarding the latter, the bandwidth (Hz) was higher for LA than for EM (131 Hz vs 0.16–43 Hz), and the common mode rejection ratio (CMRR) is 80 dB for LA, but it is unknown for EM. Both characteristics could impact the signal amplitude and SNR, which was lower in EM than in LA. The impact of these features, emphasized by Gramann (2024), increases in brief experiments.

More generally, although EM can certainly be used as a research tool (Maskeliunas et al., 2016), it is important to keep in mind that, as well as other consumer-grade systems, EM does not provide raw data and the signal is already preprocessed in untraceable way (Browarska et al., 2021). Despite the exhaustive study by Niso et al. (2023), some technical information is not known and could explain temporal imprecision and slightly lower amplitude of the ERP.

In summary, the portable LA system meets the requirements of a research-grade system with good signal quality and temporal accuracy, but it still underperforms its wired counterpart (BA). Furthermore, although the EM system is definitely more affordable and offers good scalp coverage, as well as a portability score equivalent to that of the LA (CoME D = 3, Niso et al., 2023), it only provides preprocessed data, has a lower SNR, and has shown technical problems such as amplifier-computer disconnections, which need to be taken into account, particularly for time-sensitive measures such as ERPs.

Limitations

This study contributes to the growing body of research assessing the validity of portable EEG systems to measure neural oscillations and ERPs, and is notable for its analysis of the minimum test duration required to obtain usable data. However, the current findings should be interpreted in light of some limitations.

First, while the sample size was rather large in the context of studies comparing the validity of wireless vs wired EEG systems, it remained limited. It might have prevented the detection of small differences between the systems due to insufficient power. Nevertheless, Melnik et al. (2017) calculated that 8% and 9% of variance in ERP measures can be respectively explained by variability between subjects and EEG systems if 16 participants were tested. Thus, our 19-participant group accounted for even smaller part of ERP variance in the present study. It is possible that inter-individual variability may come from contextual differences, since participants were tested at different times of the day, and were not forbidden from drinking coffee or smoking before EEG recordings. For instance, Chen et al. (2020) showed that caffeine intake may enhance the efficiency of individual automatic responses and early cognitive processes. Cui et al. (2020) also showed that abstinence in young smokers may have an impact on the effect of neurophysiological indicators of performance control. However, as shown by the relatively high between participants correlation values observed across systems for time/frequency measures and most ERPs, measures were relatively stable between systems. The lack of correlation for the MMN can tentatively be explained by the specificity of the electrode reference (the common average) used to calculate this ERP component, because of the differential changes in frontal and sub-temporal components of mismatch negativity (Baldeweg, Williams & Gruzelier, 1999).

Second, another limitation concerns comparisons of the SME data for the three systems to the ERP CORE dataset due to few compatibilities between electrode sites of interest. This calculation conveys mitigated conclusions since variability was overall lower than the ERP CORE dataset for the N170 amplitude and latency, while variability was higher than this reference for the MMN latency.

Third, due to a technical problem, the BA system was not used with its optimal sampling rate, which was paradoxically lower than for the LA system. This problem was indirectly solved by analyses conducted with an equalized 128 Hz-sampling rate, initially designed to rule out an interpretation of between-system differences based solely on recording resolution.

Fourth, although researchers may be inclined to use mobile systems, particularly for real-life studies, our study focused on immobile participants. Future research could explore the suitability of these EEG systems in real-world settings, which would involve addressing artifacts caused by large gestures and making informed decisions about preprocessing (Jacobsen et al., 2024). Guidelines on the acceptability of these systems (Bhavnani et al., 2022) and the selection of hardware are available elsewhere (Sawangjai et al., 2020; Niso et al., 2023; Gramann, 2024).

Finally, future studies might extend the comparison of these systems towards tasks involving later cognitive processing, such as sensitivity to semantic or syntactic violations through N400 and P600 components, or addressing spectral power in other bands.

Practical guidelines

Based on the present study, the following points can be offered as guidelines to researchers wishing to use a portable EEG system to address scientific issues. First, not only the LA, but also the EM consumer-grade EEG system can be recommended if time/frequency analyses on brain oscillations are planned, and no increase in the recording duration is needed. For instance, large changes in the alpha-band, and typical responses to the frequency of flickering stimuli and its harmonics can be observed from 30- and 15-s recordings. Second, whether the system was wired or portable, ERP amplitude did not significantly decrease with the reduction of the number of trials (150, 200 and 300 trials for the N170, N200 and P300, respectively). Except for N200 (and N170 in the texture condition), which can be expected to be of medium size from EM, large-sized ERPs can be recorded using wireless as wired systems. Third, studying ERP latencies is possible with LA, but is compromised by temporal imprecision of EM. Fourth, capturing the MMN with the EM or LA portable systems proved possible, but to a lesser extent than other ERPs, and the number of trials should not be lower than 505 to expect a medium-sized MMN using EM system, while it can be reduced towards 337 using LA or BA.

Conclusions

The present results were obtained from one fixed and two portable EEG devices outside a specialized EEG recording booth. By replicating several classic ERP components reflecting automatic and voluntary processing of auditory and visual stimuli, as well as neural changes in cerebral rhythms during resting states with open vs closed eyes in adults, they provide novel evidence in favor of the validity of such cost-effective, user- and participant-friendly tools. Despite the limitations outlined above, portable EEG systems such as LA and EM may contribute to research in real life conditions. Future comparison studies should be extended to pediatric populations as wireless EEG systems may be particularly useful in developmental research in real-world settings.

Supplemental Information

10.7717/peerj.20416/supp-1Supplemental Information 1Mean bootstrapped standard measurement error (bSME) over all participants for latency and amplitude measures for each system, waveform and electrode site in pack-100b.

10.7717/peerj.20416/supp-2Supplemental Information 2Statistical comparisons of the SME values to those obtained with ERP CORE data.* p ≤ 0.05, ** p ≤ 0.01, *** p ≤ 0.001.

10.7717/peerj.20416/supp-3Supplemental Information 3Between-system correlations of the mean SNR response to the SSVEP paradigm for each pack and each stimulation condition.

10.7717/peerj.20416/supp-4Supplemental Information 4Between-system correlations of the difference in alpha power during the resting state task for each pack.

10.7717/peerj.20416/supp-5Supplemental Information 5Between-system correlations of the peak amplitude and latency of the N170 ERP at P7 and P8 for each pack.

10.7717/peerj.20416/supp-6Supplemental Information 6Between-system correlations of the peak amplitude and latency of the N200 ERP at electrode sites of interest (Fz, Cz) for each pack.

10.7717/peerj.20416/supp-7Supplemental Information 7Between-system correlations of the peak amplitude and latency of the P300 ERP at electrode sites of interest (Fz, Cz) for each pack.

10.7717/peerj.20416/supp-8Supplemental Information 8Between-system correlations of the area under the curve (AUC) and latency of the MMN ERP at electrode sites of interest (Fz, Cz) for each pack.

10.7717/peerj.20416/supp-9Supplemental Information 9Distribution of the SME for N170 amplitude and latency, MMN latency, compared to the distribution of the SME ERP CORE dataset at the electrode sites of interest.