close
Skip to main content
Springer Nature Link
Log in

Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems

Guidelines Derived from Simulation and Real-World Data

  • Original Article
  • Published:
Save article
View saved research
Neuroinformatics Aims and scope Submit manuscript

Abstract

We report on novel supervised algorithms for single-trial brain state decoding. Their reliability and robustness are essential to efficiently perform neurotechnological applications in closed-loop. When brain activity is assessed by multichannel recordings, spatial filters computed by the source power comodulation (SPoC) algorithm allow identifying oscillatory subspaces. They regress to a known continuous trial-wise variable reflecting, e.g. stimulus characteristics, cognitive processing or behavior. In small dataset scenarios, this supervised method tends to overfit to its training data as the involved recordings via electroencephalogram (EEG), magnetoencephalogram or local field potentials generally provide a low signal-to-noise ratio. To improve upon this, we propose and characterize three types of regularization techniques for SPoC: approaches using Tikhonov regularization (which requires model selection via cross-validation), combinations of Tikhonov regularization and covariance matrix normalization as well as strategies exploiting analytical covariance matrix shrinkage. All proposed techniques were evaluated both in a novel simulation framework and on real-world data. Based on the simulation findings, we saw our expectations fulfilled, that SPoC regularization generally reveals the largest benefit for small training sets and under severe label noise conditions. Relevant for practitioners, we derived operating ranges of regularization hyperparameters for cross-validation based approaches and offer open source code. Evaluating all methods additionally on real-world data, we observed an improved regression performance mainly for datasets from subjects with initially poor performance. With this proof-of-concept paper, we provided a generalizable regularization framework for SPoC which may serve as a starting point for implementing advanced techniques in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. To simplify the notation, epoched data X(t,e) will further on be written as a matrix X(e).

References

  • Arvaneh, M., Guan, C., Ang, K.K., Quek, C. (2011). Optimizing the channel selection and classification accuracy in EEG-based BCI. IEEE Transactions on Biomedical Engineering, 58(6), 1865–1873. https://doi.org/10.1109/TBME.2011.2131142.

    Article  PubMed  Google Scholar 

  • Arvaneh, M., Guan, C., Ang, K.K., Quek, C. (2013). Optimizing spatial filters by minimizing within-class dissimilarities in electroencephalogram-based brain-computer interface. IEEE Transactions on Neural Networks and Learning Systems, 24(4), 610–619. https://doi.org/10.1109/TNNLS.2013.2239310.

    Article  PubMed  Google Scholar 

  • Bai, Z, & Silverstein, JW. (2009). Spectral analysis of large dimensional random matrices. Springer Science & Business Media.

  • Bartz, D., & Müller, K.-R. (2014). Covariance shrinkage for autocorrelated data. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (Eds.) Advances in neural information processing systems, (Vol. 27 pp. 1592–1600): Curran Associates Inc.

  • Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Müller, K.-R. (2008). Optimizing spatial filters for robust EEG single-trial analysis. Signal Processing Magazine, IEEE, 25(1), 41–56.

    Article  Google Scholar 

  • Blankertz, B., Sannelli, C., Halder, S., Hammer, E.M., Kübler, A., Müller, K.-R., Curio, G., Dickhaus, T. (2010). Neurophysiological predictor of SMR-based BCI performance. NeuroImage, 51(4), 1303–1309. https://doi.org/10.1016/j.neuroimage.2010.03.022.

    Article  PubMed  Google Scholar 

  • Blankertz, B., Acqualagna, L., Dähne, S, Haufe, S., Schultze-Kraft, M., Sturm, I., Ušćumlic, M., Wenzel, M.A., Curio, G., Müller, K.-R. (2016). The berlin brain-computer interface: progress beyond communication and control. Frontiers in Neuroscience, 10, 530. https://doi.org/10.3389/fnins.2016.00530.

    Article  PubMed  PubMed Central  Google Scholar 

  • Castaño-Candamil, J.S., Meinel, A., Dähne, S., Tangermann, M. (2015). Probing meaningfulness of oscillatory EEG components with bootstrapping, label noise and reduced training sets. In 2015 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 5159–5162). IEEE.

  • Castaño-Candamil, S., Meinel, A., Tangermann, M. (2017). Post-hoc labeling of arbitrary EEG recordings for data-efficient evaluation of neural decoding methods. arXiv:171108208.

  • Chen, Y., Wiesel, A., Hero, A.O. (2011). Robust shrinkage estimation of high-dimensional covariance matrices. IEEE Transactions on Signal Processing, 59(9), 4097–4107. https://doi.org/10.1109/TSP.2011.2138698.

    Article  Google Scholar 

  • Cheng, M., Lu, Z., Wang, H. (2017). Regularized common spatial patterns with subject-to-subject transfer of EEG signals. Cognitive Neurodynamics, 11(2), 173–181. https://doi.org/10.1007/s11571-016-9417-x.

    Article  PubMed  Google Scholar 

  • Cho, H., Ahn, M., Kim, K., Jun, S.C. (2015). Increasing session-to-session transfer in a brain–computer interface with on-site background noise acquisition. Journal of Neural Engineering, 12(6), 066,009. https://doi.org/10.1088/1741-2560/12/6/066009.

    Article  Google Scholar 

  • Clerc, M., Bougrain, L., Lotte, F. (2016). Brain-computer interfaces 2: technology and applications. Wiley.

  • Dähne, S., Meinecke, F.C., Haufe, S., Höhne, J, Tangermann, M., Müller, K.-R., Nikulin, V.V. (2014). SPoC: a novel framework for relating the amplitude of neuronal oscillations to behaviorally relevant parameters. NeuroImage, 86(0), 111–122. https://doi.org/10.1016/j.neuroimage.2013.07.079.

    Article  PubMed  Google Scholar 

  • de Cheveigné, A., & Parra, L.C. (2014). Joint decorrelation, a versatile tool for multichannel data analysis. NeuroImage, 98(Supplement C), 487–505. https://doi.org/10.1016/j.neuroimage.2014.05.068.

    Article  PubMed  Google Scholar 

  • De Bie, T., Cristianini, N., Rosipal, R. (2005). Eigenproblems in pattern recognition. In Handbook of geometric computing (pp. 129–167). Springer.

  • De Vos, M., Riès, S., Vanderperren, K., Vanrumste, B., Alario, F.X., Huffel, V.S., Burle, B. (2010). Removal of muscle artifacts from EEG recordings of spoken language production. Neuroinformatics, 8(2), 135–150. https://doi.org/10.1007/s12021-010-9071-0.

    Article  PubMed  Google Scholar 

  • Devlaminck, D., Wyns, B., Grosse-Wentrup, M., Otte, G., Santens, P. (2011). Multisubject learning for common spatial patterns in motor-imagery BCI. Intelligence Neuroscience, 2011, 8:8–8:8. https://doi.org/10.1155/2011/217987.

    Article  Google Scholar 

  • Engemann, D.A., & Gramfort, A. (2015). Automated model selection in covariance estimation and spatial whitening of MEG and EEG signals. NeuroImage, 108, 328–342. https://doi.org/10.1016/j.neuroimage.2014.12.040.

    Article  PubMed  Google Scholar 

  • Farina, D., Jiang, N., Rehbaum, H., Holobar, A., Graimann, B., Dietl, H., Aszmann, O.C. (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(4), 797–809. https://doi.org/10.1109/TNSRE.2014.2305111.

    Article  PubMed  Google Scholar 

  • Farquhar, J., & Hill, N.J. (2013). Interactions between pre-processing and classification methods for event-related-potential classification. Neuroinformatics, 11(2), 175–192. https://doi.org/10.1007/s12021-012-9171-0.

    Article  CAS  PubMed  Google Scholar 

  • Farquhar, J., Hill, N., Lal, T.N., Schölkopf, B. (2006). Regularised CSP for sensor selection in BCI. In Proceedings of the 3rd international BCI workshop.

  • Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.

    Article  Google Scholar 

  • Frey, J., Daniel, M., Hachet, M., Castet. J., Lotte, F. (2016). Framework for electroencephalography-based evaluation of user experience. InProcedings of CHI (pp. 2283–2294).

  • Grosse-Wentrup, M., Liefhold, C., Gramann, K., Buss, M. (2009). Beamforming in non invasive brain-computer interfaces. IEEE Transactions on Biomedical Engineering, 56(4), 1209–1219.

    Article  PubMed  Google Scholar 

  • Hyvarinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626–634. https://doi.org/10.1109/72.761722.

    Article  CAS  Google Scholar 

  • Kang, H., Nam, Y., Choi, S. (2009). Composite common spatial pattern for subject-to-subject transfer. IEEE Signal Processing Letters, 16(8), 683–686. https://doi.org/10.1109/LSP.2009.2022557.

    Article  Google Scholar 

  • Kenney, J.F. (2013). Mathematics of statistics. Toronto: D. Van Nostrand Company Inc. Princeton; New Jersey; London; New York,; Affiliated East-West Press Pvt-Ltd; New Delhi.

    Google Scholar 

  • Koles, Z.J. (1991). The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalography and clinical Neurophysiology, 79(6), 440–447.

    Article  CAS  PubMed  Google Scholar 

  • Krusienski, D., Grosse-Wentrup, M., Galán, F., Coyle, D., Miller, K., Forney, E., Anderson, C. (2011). Critical issues in state-of-the-art brain-computer interface signal processing. Journal of Neural Engineering, 8(2), 025,002.

    Article  Google Scholar 

  • Ledoit, O., & Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365–411. https://doi.org/10.1016/S0047-259X(03)00096-4.

    Article  Google Scholar 

  • Lotte, F. (2015). Signal processing approaches to minimize or suppress calibration time in oscillatory activity-based brain-computer interfaces. Proceedings of the IEEE, 103(6), 871–890. https://doi.org/10.1109/JPROC.2015.2404941.

    Article  Google Scholar 

  • Lotte, F., & Guan, C. (2010). Learning from other subjects helps reducing brain-computer interface calibration time. In 2010 IEEE International conference on acoustics, speech and signal processing (pp. 614–617). https://doi.org/10.1109/ICASSP.2010.5495183.

  • Lotte, F., & Guan, C. (2011). Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Transactions on Biomedical Engineering, 58(2), 355–362. https://doi.org/10.1109/TBME.2010.2082539.

    Article  PubMed  Google Scholar 

  • Lu, H., Eng, H.L., Guan, C., Plataniotis, K.N., Venetsanopoulos, A.N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936–2946. https://doi.org/10.1109/TBME.2010.2082540.

    Article  PubMed  Google Scholar 

  • Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S. (2018). Applications of deep learning and reinforcement learning to biological data. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2063–2079. https://doi.org/10.1109/TNNLS.2018.2790388.

    Article  PubMed  Google Scholar 

  • Makeig, S., Debener, S., Onton, J., Delorme, A. (2004). Mining event-related brain dynamics. Trends in Cognitive Sciences, 8(5), 204–210. https://doi.org/10.1016/j.tics.2004.03.008.

    Article  PubMed  Google Scholar 

  • Makeig, S., Kothe, C., Mullen, T., Bigdely-Shamlo, N., Zhang, Z., Kreutz-Delgado, K. (2012). Evolving signal processing for brain-computer interfaces. Proceedings of the IEEE, 100(Special Centennial Issue), 1567–1584. https://doi.org/10.1109/JPROC.2012.2185009.

    Article  CAS  Google Scholar 

  • Mattout, J., Phillips, C., Penny, W.D., Rugg, M.D., Friston, K.J. (2006). MEG source localization under multiple constraints: an extended Bayesian framework. NeuroImage, 30(3), 753–767. https://doi.org/10.1016/j.neuroimage.2005.10.037.

    Article  PubMed  Google Scholar 

  • Meinel, A., Castaño-Candamil, S, Reis, J., Tangermann, M. (2016). Pre-trial EEG-based single-trial motor performance prediction to enhance neuroergonomics for a hand force task. Frontiers in Human Neuroscience, 10, 170. https://doi.org/10.3389/fnhum.2016.00170.

    Article  PubMed  PubMed Central  Google Scholar 

  • Meinel, A., Lotte, F., Tangermann, M. (2017). Tikhonov regularization enhances EEG-based spatial filtering for single-trial regression. In Proceedings of the 7th Graz brain-computer interface conference 2017 (pp. 308-313). https://doi.org/10.3217/978-3-85125-533-1-57.

  • Millán, J.d.R., Rupp, R., Mueller-Putz, G., Murray-Smith, R., Giugliemma, C., Tangermann, M., Vidaurre, C., Cincotti, F., Kübler, A, Leeb, R., Neuper, C., Müller, K.-R, Mattia, D. (2010). Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Frontiers in Neuroscience, 4, 161.

    PubMed  PubMed Central  Google Scholar 

  • Nicolae, I.E., Acqualagna, L., Blankertz, B. (2017). Assessing the depth of cognitive processing as the basis for potential user-state adaptation. Frontiers in Neuroscience, 11. https://doi.org/10.3389/fnins.2017.00548.

  • Park, S.H., Lee, D., Lee, S.G. (2017). Filter bank regularized common spatial pattern ensemble for small sample motor imagery classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, PP(99), 1–1. https://doi.org/10.1109/TNSRE.2017.2757519.

    Article  Google Scholar 

  • Parra, L.C., Spence, C.D., Gerson, A.D., Sajda, P. (2005). Recipes for the linear analysis of EEG. NeuroImage, 28(2), 326–341.

    Article  PubMed  Google Scholar 

  • Ramanathan, C., Ghanem, R.N., Jia, P., Ryu, K., Rudy, Y. (2004). Noninvasive electrocardiographic imaging for cardiac electrophysiology and arrhythmia. Nature Medicine, 10(4), nm1011. https://doi.org/10.1038/nm1011.

    Article  CAS  Google Scholar 

  • Ramoser, H., Muller-Gerking, J., Pfurtscheller, G. (2000). Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Transactions on Rehabilitation Engineering, 8(4), 441–446.

    Article  CAS  PubMed  Google Scholar 

  • Reuderink, B., & Poel, M. (2008). Robustness of the common spatial patterns algorithm in the BCI-pipeline. Tech rep. HMI, University of Twente.

  • Samek, W., Vidaurre, C., Müller, K.-R., Kawanabe, M. (2012). Stationary common spatial patterns for brain-computer interfacing. Journal of Neural Engineering, 9(2), 026,013. https://doi.org/10.1088/1741-2560/9/2/026013.

    Article  Google Scholar 

  • Samek, W., Meinecke, F.C., Müller, K.-R. (2013). Transferring subspaces between subjects in brain-computer interfacing. IEEE Transactions on Biomedical Engineering, 60(8), 2289–2298. https://doi.org/10.1109/TBME.2013.2253608.

    Article  PubMed  Google Scholar 

  • Samek, W., Kawanabe, M., Müller, K.-R. (2014). Divergence-based framework for common spatial patterns algorithms. IEEE Reviews in Biomedical Engineering, 7, 50–72. https://doi.org/10.1109/RBME.2013.2290621.

    Article  PubMed  Google Scholar 

  • Schäfer, J., & Strimmer, K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 4, 1.

    Article  Google Scholar 

  • Schultze-Kraft, M., Dähne, S, Gugler, M., Curio, G., Blankertz, B. (2016). Unsupervised classification of operator workload from brain signals. Journal of Neural Engineering, 13(3), 036,008. https://doi.org/10.1088/1741-2560/13/3/036008.

    Article  Google Scholar 

  • Tian, T.S., Huang, J.Z., Shen, H., Li, Z. (2013). EEG/MEG source reconstruction with spatial-temporal two-way regularized regression. Neuroinformatics, 11(4), 477–493. https://doi.org/10.1007/s12021-013-9193-2.

    Article  PubMed  Google Scholar 

  • Tikhonov, A.N. (1963). Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 4, 1624–1627.

    Google Scholar 

  • Úbeda, A., Azorín, J.M., Chavarriaga, R., Millán, J.d.R. (2017). Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques. Journal of NeuroEngineering and Rehabilitation, 14, 9. https://doi.org/10.1186/s12984-017-0219-0.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang, H., & Li, X. (2016). Regularized filters for L1-norm-based common spatial patterns. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(2), 201–211. https://doi.org/10.1109/TNSRE.2015.2474141.

    Article  PubMed  Google Scholar 

  • Winkler, I., Brandl, S., Horn, F., Waldburger, E., Allefeld, C., Tangermann, M. (2014). Robust artifactual independent component classification for bci practitioners. Journal of Neural Engineering, 11(3), 035,013.

    Article  Google Scholar 

  • Wu, D., King, J.T., Chuang, C.H., Lin, C.T., Jung, T.P. (2017). Spatial filtering for EEG-based regression problems in brain-computer interface (BCI). IEEE Transactions on Fuzzy Systems, PP(99), 1–1. https://doi.org/10.1109/TFUZZ.2017.2688423.

    Article  Google Scholar 

Download references

Acknowledgements

This work was fully supported by BrainLinks-BrainTools Cluster of Excellence funded by the German Research Foundation (DFG), grant number EXC1086. For the data analysis, the authors acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) by grant no. INST 39/963-1 FUGG. Fabien Lotte received research support from the French National Research Agency with the REBEL project (grant ANR-15-CE23-0013-01) and the European Research Council with the BrainConquest project (grant ERC-2016-STG-714567). For parts of the data analysis, the Matlab-based BBCI toolbox was utilized (Blankertz et al. 2016). The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Brain State Decoding Lab, Cluster of Excellence BrainLinks-BrainTools, Department of Computer Science, Albert-Ludwigs-University, Freiburg, Germany

    Andreas Meinel, Sebastián Castaño-Candamil & Michael Tangermann

  2. Neurotechnology Dept., Technical University of Berlin, Berlin, Germany

    Benjamin Blankertz

  3. Potioc project team, Inria, Talence, France

    Fabien Lotte

  4. LaBRI (University of Bordeaux, CNRS, INP), Talence, France

    Fabien Lotte

Authors
  1. Andreas Meinel
    View author publications

    Search author on:PubMed Google Scholar

  2. Sebastián Castaño-Candamil
    View author publications

    Search author on:PubMed Google Scholar

  3. Benjamin Blankertz
    View author publications

    Search author on:PubMed Google Scholar

  4. Fabien Lotte
    View author publications

    Search author on:PubMed Google Scholar

  5. Michael Tangermann
    View author publications

    Search author on:PubMed Google Scholar

Corresponding authors

Correspondence to Andreas Meinel or Michael Tangermann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meinel, A., Castaño-Candamil, S., Blankertz, B. et al. Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems. Neuroinform 17, 235–251 (2019). https://doi.org/10.1007/s12021-018-9396-7

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s12021-018-9396-7

Keywords