Interpretable Machine Learning for Mental Workload Detection: From Binary to Multi-Class with Multimodal Physiology Signals

Abstract

Accurate detection of mental workload is essential for developing adaptive human–machine interaction (HMI) systems that respond to cognitive states of users in real time. Therefore, this study aimed to develop as well as evaluate machine learning models for classifying mental workload into binary, namely load vs. no load, and multi-class comprising low, medium, and high levels using multimodal physiological signals. Thirty-four participants completed two cognitively demanding tasks, and heart rate variability (HRV) as well as electrodermal activity (EDA) were recorded during the tasks. Signals were segmented into 1-min non-overlapping windows, with features extracted from the time, frequency, and nonlinear HRV domains, as well as tonic and phasic EDA indices. Moreover, three feature selection strategies, correlation-based, domain-expert, and minimum redundancy maximum relevance (mRMR), were evaluated across multiple classifiers, including gradient boosting, XGBoost, random forest, k-nearest neighbor, and multilayer perceptron. The model performance was evaluated using an 80/20 subject-wise train–test split, with 10-fold cross-validation conducted on the training set. Additionally, oversampling was applied exclusively to the training data to address the class imbalance. The result showed that multimodal fusion of HRV and EDA consistently outperformed single-modality models, achieving the strongest performance with XGBoost and correlation-based feature selection (binary: F1 = 0.787, AUC = 0.847; multi-class: Macro-F1 = 0.722, AUC = 0.862). The Shapley additive explanations (ShAP) analysis identified nonlinear HRV indices and phasic EDA features as the dominant predictors. These outcomes showed the importance of multimodal integration and data-driven feature selection for robust mental workload detection in HMI.