Multimodal EMG-EEG Biosignal Fusion in Upper-Limb Gesture Classification

Abstract

Upper-limb gesture identification is an important problem in the advancement of robotic prostheses. Prevailing research into classifying electromyographic (EMG) muscular data or electroencephalographic (EEG) brain data for this purpose is often limited in granularity of gestures classified, the extent to which generalisation is demonstrated, and methodological rigour. This work proposes three architectures for multimodal fusion of EMG & EEG data in gesture classification, including techniques grounded in literature precedent and a novel “Hierarchical” strategy. Classification systems of these architectures are designed via Combined Algorithm Selection & Hyperparameter Optimisation (CASH) to ensure comparisons between the approaches are unbiased; likely this methodology’s first application to the biosignal classification domain. All architectures are demonstrated suitable for use in a same-hand multi-gesture classification problem less separable than seen in much Brain-Computer-Interface research. Fusion of EMG & EEG is shown to provide significantly higher (p<0.05) subject-independent classification accuracy (73.4%) than an equivalent single-mode EMG model, when tested on unseen individuals’ data. Subject-independent single-mode EEG classification achieved accuracies (51.9%) competitive with those reached by many subject-specific systems in the literature on similar, or more separable, problems. The efficacy of CASH optimisation as a means of determining modelling choices — over inferring such decisions from literature — is also evidenced. A desire to minimise the burden placed on potential prosthesis users motivates investigation of cross-subject and cross-session classification. Strategies for minimising per-session calibration, including through transfer learning, are explored. Results demonstrate that less session-specific data is needed to adapt a model pre-trained on an individual’s previous-session data than would be needed to train a session-specific classifier to a similar accuracy (85%). Domain transfer using data collected from other individuals as the basis for adaptation is proven capable of accuracies (83%) nearing those of the subject-specific approach, laying the groundwork for future developments in low-calibration gesture classification systems.