BCI FII Corpus Benchmark

This document reports the results of a benchmark run on the FII BCI Corpus using Eegle.

Only subjects/sessions that passed all exclusion criteria described in the @discarded.md file (see it for MI and P300) were included in this benchmark.

Individual subject/session accuracies are included in the .yml metadata files and are also available in CSV format in this GitHub repository. The repository also hosts the scripts used to make this benchmark and more information.

This document reports the mean(±sd) balanced accuracies for each database according to the analyzed tasks: - MI: "righthand" vs. "feet" and "lefthand" vs. "right_hand" - P300: target vs nontarget.

Available Classifiers

The benchmark has employed three Riemannian classifiers, all adopting the affine-invariant (Fisher-Rao) metric:

Native classifiers (acting on the manifold); the metric is used for computing distances and barycenters:

  • MDM (Minimum Distance to Mean).

Tangent space classifiers; the metric is used to compute a barycenter for projecting the data onto the tangent space:

  • ENLR (Elastic Net Logistic Regression)
  • Linear SVM (Linear Support Vector Machine).

Processing Pipelines

For both MI and P300 the default pipeline implementd by the crval function has bene used with the following two customizations:

  • Stratified K-Fold (K=10) instead of the default 8-fold
  • Seed set to 109 (instead of default 1234).

Motor Imagery (MI) Benchmark Results

Task: right_hand vs. feet

DatabaseMDM (%)ENLR (%)SVM (%)
AlexMI-None73.75 ± 18.0380.62 ± 14.075.31 ± 13.39
BNCI2014001-None85.13 ± 12.287.7 ± 9.6586.33 ± 11.05
BNCI2014002-Test80.77 ± 15.8181.54 ± 16.8881.92 ± 14.58
BNCI2014002-Train77.07 ± 11.477.54 ± 11.3874.07 ± 13.87
BNCI2015001-None81.62 ± 13.0884.53 ± 10.2378.94 ± 12.42
Schirrmeister2017-None86.13 ± 12.6796.48 ± 4.5297.07 ± 3.62
Weibo2014-None61.94 ± 9.5686.5 ± 8.981.72 ± 8.32
Zhou2016-None87.33 ± 6.7692.89 ± 3.6687.45 ± 8.41

Task: lefthand vs. righthand

DatabaseMDM (%)ENLR (%)SVM (%)
BNCI2014001-None77.75 ± 13.9978.26 ± 15.3378.85 ± 14.27
BNCI2014004-Test78.9 ± 12.4180.95 ± 11.5378.59 ± 12.58
BNCI2014004-Train66.15 ± 9.9568.73 ± 9.7466.92 ± 9.63
Cho2017-None64.96 ± 9.4273.7 ± 10.2669.54 ± 10.07
GrosseWentrup2009-None60.12 ± 8.3886.21 ± 10.7382.23 ± 10.81
Lee2019_MI-Train66.22 ± 13.2379.74 ± 11.6475.35 ± 13.21
PhysionetMI-Task 255.1 ± 16.4876.77 ± 11.5862.99 ± 12.83
SPSM2025-None62.49 ± 7.566.67 ± 7.9160.08 ± 9.14
Schirrmeister2017-None65.26 ± 16.5183.11 ± 11.9282.08 ± 12.21
Shin2017A-None61.67 ± 18.5879.74 ± 15.7773.08 ± 17.46
Weibo2014-None55.14 ± 11.9382.0 ± 10.3674.52 ± 16.58
Zhou2016-None84.52 ± 7.7686.92 ± 8.1882.46 ± 9.31

P300 Benchmark Results

DatabaseMDM (%)ENLR (%)
BNCI2014009-None80.11 ± 7.6782.87 ± 6.22
BNCI2015003-Test77.18 ± 9.1872.89 ± 6.89
BNCI2015003-Train77.76 ± 6.2372.65 ± 6.29
Cattan2019-Personal Computer (PC)81.93 ± 7.7682.1 ± 8.36
Cattan2019-Virtual Reality (VR)79.07 ± 6.881.07 ± 6.38
EPFLP300-Run 1 - Television68.51 ± 10.7568.75 ± 7.65
EPFLP300-Run 2 - Telephone68.25 ± 10.7571.49 ± 8.58
EPFLP300-Run 3 - Lamp68.51 ± 13.7471.84 ± 11.07
EPFLP300-Run 4 - Door68.06 ± 9.4966.25 ± 7.17
EPFLP300-Run 5 - Window69.8 ± 9.1776.18 ± 8.25
EPFLP300-Run 6 - Radio67.92 ± 9.7770.4 ± 11.08
bi2012-Online77.66 ± 5.1582.9 ± 7.69
bi2012-Training73.52 ± 3.2880.55 ± 5.38
bi2013a-Adaptative - Online86.95 ± 4.7183.65 ± 6.23
bi2013a-Adaptative - Training84.74 ± 5.4984.47 ± 5.51
bi2013a-Non Adaptative - Online86.64 ± 6.3483.09 ± 6.27
bi2013a-Non Adaptative - Training84.58 ± 5.0884.37 ± 5.28
bi2014a-None74.49 ± 7.3377.58 ± 6.66
bi2014b-Solo72.76 ± 13.6773.78 ± 12.67
bi2015a-Flash Duration 110ms80.01 ± 6.3381.84 ± 6.5
bi2015a-Flash Duration 50ms79.96 ± 6.8581.46 ± 6.85
bi2015a-Flash Duration 80ms79.38 ± 6.0781.53 ± 6.59