Enhancing EEG-Based Cognitive State Prediction with AutoML-Driven Stacked Ensemble Models

Murad Ali Khan

Murad Ali Khan^1*

¹Department of Computer Engineering, Jeju National University, Jeju 63243, Republic of Korea.

Abstract

This research examines the effectiveness of a stacked ensemble model based on AutoML for predicting cognitive states from EEG data, in comparison with conventional machine learning approaches. Through an extensive evaluation using several metrics—MAE, MSE, RMSE, MAPE, and R2 Score—the study underlines the advanced performance of the new model. With considerably lower error rates (MAE = 0.08, MSE = 0.10, RMSE = 0.32, MAPE = 0.85) and the highest R2 Score of 0.96, the model marks a substantial improvement over previous method. This contribution is significant in the field of neuroscience, illustrating how AutoML can improve the precision and efficiency of models dealing with complex EEG datasets.

Keyword: AutoMl; Stacked Ensemble Models; EEG Analysis; Cognitive State Prediction; Machine Learning in Neuroscience.

^*Corresponding author: Ali MK, Department of Computer Engineering, Jeju National University, Jeju 63243, Republic of Korea.

Received: 14 January 2025; Accepted: 27 January 2025; Published: 31 January 2025

Citation: Ali MK (2025) Enhancing EEG-Based Cognitive State Prediction with AutoML-Driven Stacked Ensemble Models. Glob J Med Biomed Case Rep 1: 003.

Introduction

Predicting cognitive states through EEG data is a significant breakthrough in neuroscience and machine learning applications. Recent research underscores the complex, high-dimensional, and non-linear characteristics of EEG signals, which require sophisticated analytical methods for accurate analysis [1]. Automated Machine Learning (AutoML) plays an essential role by allowing individuals without specialized expertise to deploy intricate machine learning models for neurological data analysis. AutoML simplifies the model selection and tuning process, making it highly effective for EEG data interpretation. This automation reduces the necessary time and expertise to obtain meaningful results [2,3].

Ensemble learning methods, particularly stacking ensembles that combine various predictive models, have proven highly effective in improving prediction accuracy, outshining single models in multiple studies [4,5]. These techniques are increasingly applied in fields like emotion recognition and cognitive load assessment using EEG data, where they address model variance and bias issues [6]. The fusion of AutoML with stacking ensembles presents a promising research avenue, exploiting the strengths of various sophisticated models within an automated, cohesive framework [7].

Stacking ensembles enhance robust generalization by integrating multiple learning algorithms, crucial for managing the complex patterns found in EEG data. Prior studies have shown that stacking ensembles, composed of AutoML- selected base learners, achieve enhanced classification accuracy by effectively synthesizing diverse data interpretations and predictive strengths [8,9]. This methodology is particularly apt for the growing complexity of EEG-based applications, which range from simple emotion classification to complex cognitive state assessments involving attention and mental workload [10].

Building on these insights, the current study aims to push the boundaries of EEG cognitive state prediction by developing and implementing a stacked ensemble model optimized through AutoML. By comparing this model with conventional machine learning models and other ensemble techniques, this research not only seeks validation but also aims to refine the integration of AutoML and ensemble learning in EEG data analysis. The expected result is a more precise, dependable, and user-friendly tool for cognitive state prediction, potentially transforming clinical diagnostics and personalized medicine [1,10].

Related Work

Over the past decade, there has been a significant rise in research integrating EEG signals with machine learning for predicting cognitive states and diagnosing neurological disorders. Initial studies, such as the one by Mo et al. (2016), highlighted the effectiveness of machine learning in classifying motor imagery EEG, using optimized support vector machines for better classification performance [11]. As research in the field progressed, the focus shifted towards the capabilities of ensemble learning to manage the high-dimensional and non-linear nature of EEG data. A key study by Sun et al. (2007) showcased the superiority of ensemble methods compared to single classifier systems, demonstrating enhanced robustness and accuracy in EEG signal classification [12].

Recent advancements in ensemble methods have been well-documented, with studies demonstrating their effectiveness in combining multiple machine learning models to enhance diagnostic accuracy. For example, ensemble learning methods that employ wavelet transform for feature extraction have been particularly effective in managing the complexity of EEG data, as shown by Adeli et al. (2007) [13]. These methods have been applied across various EEG classification tasks such as emotional state detection and abnormal signal identification, showcasing their adaptability and robustness in diverse scenarios [14,15]. Further research by Fan et al. (2019) on gradient boosting-based ensemble methods highlighted their flexibility and efficacy in addressing EEG classification challenges [16].

The introduction of Automated Machine Learning (AutoML) into this domain has been revolutionary, streamlining the workflow and lessening the need for deep technical expertise in deploying effective models. Choubey et al. (2019) demonstrated how AutoML can automatically determine the best feature extraction and classification techniques, customizing model setups for specific EEG datasets [17]. Moreover, ensemble strategies like stacking have not only enhanced classification accuracy but also provided a sturdy defense against common EEG analysis problems such as model overfitting and variance [18].

In the realm of brain-computer interfaces and cognitive load assessments, these innovations have shown remarkable potential. Research by Singh et al. (2023) has highlighted the use of AutoML-enhanced ensemble learning in developing more intuitive and efficient brain-computer interfaces, underlining their practical value [19]. As advancements continue, the integration of AutoML and ensemble learning is expected to further refine predictions of cognitive states from EEG data, improving clinical outcomes and user interaction with technology [20].

Proposed Methodology

The proposed methodology section will detail the comprehensive approach undertaken to investigate the EEG Cognitive State Prediction Techniques, as illustrated in (Figure 1). This section is organized into several key subsections: Data Description, AutoML for Models Extraction, Stacked Ensemble Model Construction, Comparative Analysis.

Figure 1: Overview of the proposed framework for the prediction of EEG Cognitive State.

Each subsection will provide a detailed description of the specific components and steps involved in our study.

1) Data Description

The initial stage of our proposed methodology focuses on the collection and preprocessing of EEG channel data. We utilize a dataset consisting of multi-channel EEG recordings, which capture the brain's electrical activity through various electrodes positioned on the scalp. Each recording in this dataset is a representation of voltage fluctuations from different cerebral regions, subsequently digitized for analysis purposes. Our study concentrates on a specific subset of channels that are particularly relevant to the cognitive states under investigation.

The preprocessing steps are critical for ensuring the quality and consistency of the EEG data and include:

Signal Filtering: Noise is removed from the EEG signals to enhance clarity and reduce potential distortions in data interpretation. Normalization: We standardize the amplitude scales of the EEG signals across all recordings to address variations in signal intensity, facilitating more uniform analysis.

Segmentation: The continuous EEG recordings are segmented into smaller, manageable chunks, making them easier to analyze and correlate with specific cognitive states.

These preprocessing techniques are essential for preparing the EEG data, setting the stage for subsequent analyses aimed at predicting cognitive states.

2) AutoML for Models Extraction

Following the preprocessing stage, the dataset is introduced into an Automated Machine Learning (AutoML) framework. This framework is designed to streamline the model selection and hyperparameter tuning processes, thereby enhancing efficiency and effectiveness in model development. The AutoML system rigorously evaluates various machine learning models to identify those that best fit the task, utilizing predefined optimization metrics such as accuracy, computational efficiency, and resistance to overfitting. In our research, the AutoML pipeline tests a variety of models, including:

Random Forest (RF): A versatile ensemble learning method that uses multiple decision trees to improve prediction accuracy and control overfitting.
Gradient Boosting (GB): An approach that builds an additive model in a forward stage-wise fashion, allowing for optimization of arbitrary differentiable loss functions.
Extreme Gradient Boosting (XGBoost): An efficient and scalable implementation of gradient boosting that has proven effective in numerous machine learning competitions.
Decision Trees (DT): A non-parametric supervised learning method used for classification and regression, offering clear interpretation and handling of non-linear relationships within data.

Each model's performance is meticulously optimized considering constraints such as computational time and cost. This optimization ensures that the models not only perform well in terms of prediction accuracy but are also practical in terms of resource usage. The best-performing models are then stored in a model repository for future use, providing a resource of finely-tuned models ready for deployment in EEG cognitive state prediction tasks. This systematic approach facilitated by the AutoML framework significantly aids in harnessing the power of advanced machine learning techniques efficiently and effectively.

3) Stacked Ensemble Model Construction

Once the individual models are optimized and validated, the process moves to the construction of a stacked ensemble model. This sophisticated model construction technique integrates the outputs from the previously selected models—Random Forest (RF), Gradient Boosting (GB), Extreme Gradient Boosting (XGBoost), and Decision Trees (DT)—as inputs into a final ensemble classifier.

Stacking Methodology:

Integration of Model Outputs: The outputs from each of the base models (RF, GB, XGBoost, DT) serve as input features for a meta-classifier.
Meta-Classifier Function: The meta-classifier is a higher-level model that learns how to optimally combine the predictions from the base models. Its role is to analyze the patterns in the predictions made by each base model and determine the best way to synthesize these predictions to maximize accuracy.
Leveraging Strengths and Mitigating Weaknesses: This stacking approach capitalizes on the strengths of each base model while compensating for their individual limitations. For instance, where one model may be more robust against noise, another might excel in capturing subtle patterns in the data.

By utilizing a meta-classifier to make the final prediction, the stacked ensemble model not only improves accuracy but also enhances the robustness and generalizability of the predictions. This method effectively combines multiple learning algorithms, thereby providing a powerful tool for complex prediction tasks like EEG cognitive state prediction, where diverse data characteristics and intricate model dynamics are involved. This strategy ensures that the ensemble model achieves superior performance compared to any single model or simpler ensemble techniques.

3) Comparative Analysis

The final phase of our methodology entails a comprehensive comparative analysis to evaluate the performance of the stacked ensemble model in relation to each individual model generated by the AutoML process. We employ a suite of metrics to quantitatively assess and compare their performance, including Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), and the R2 Score. These metrics offer a well-rounded evaluation of model accuracy, reliability, and precision:

MSE and RMSE: Provide measures of the average squared difference between the estimated values and what is estimated. These metrics are sensitive to outliers and emphasize larger errors, making them particularly useful for identifying models that may underperform in predicting extreme cases.
MAE: Measures the average magnitude of the errors in a set of predictions, without considering their direction. It’s a straightforward representation of average error magnitude and is particularly interpretable.
MAPE: Expresses accuracy as a percentage, and can be more intuitive when assessing model performance, especially for non-technical stakeholders.
R2 Score: Indicates the proportion of variance in the dependent variable that is predictable from the independent variables, offering insights into the goodness of fit of the model predictions to the actual outcomes.

This analysis not only highlights the individual and comparative accuracies of each model but also sheds light on their reliability, precision, and trustworthiness for making predictions about cognitive states from EEG data. This proposed methodology integrates cutting-edge machine learning techniques with EEG data analysis to potentially enhance the predictive accuracy and reliability of cognitive state assessments. By combining AutoML for optimal model selection and hyperparameter tuning with ensemble methods that correct and compensate for potential errors, we present a robust framework capable of navigating the complexities of EEG data interpretation in cognitive state prediction. This holistic approach ensures that the models developed are not only technically sound but also practically viable for real-world applications, enhancing both clinical outcomes and research capabilities in neuroscience.

Experiment Setup and Results

This section discuss the experiments setup and results of the experiments, individual models and proposed ensemble model.

Experiment Setup

The study was carried out on a computer running Windows 10, with 64 GB of RAM and an Intel® Core™ m3-7X31 CPU. This processor operates at a base frequency of 2.5 GHz, features 1508 MHz, 2 physical cores, and 6 logical processors. Python was the programming language chosen for this study, employing libraries such as Sklearn for machine learning, H2O for implementing AutoML, and Matplotlib alongside Seaborn for data and statistical visualization, respectively. The data for the experiment was organized and maintained using MS Excel, ensuring a stable and capable computational environment for the required tasks. (Table 1)

Tools	Description
OS	Window 10
RAM	64 GB
CPU	Intel® Core™ m3-7X31 CPU @ 2.5 GHz, 1508 Mhz, 2 Core(s), 6 Logical Procesor(s).
Language	Python
Packages	Sklearn, H2O, Matplotlib, and Seaborn.
Database	MS Excel

Table 1: Tools and technologies used in the experiments.

Results

In the results section, we conduct an in-depth evaluation of five distinct predictive models applied to our EEG dataset, including Random Forest, Gradient Boosting, XGBoost, Decision Tree, and the Proposed Ensemble model. The effectiveness of each model is quantitatively assessed using a comprehensive suite of error metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), and R2 Score, as detailed in (Table 2). This assessment is critical for comparing the performance of these models, providing insights into their capabilities to accurately predict cognitive states from EEG data. Through this analysis, we aim to understand which models perform best under specific conditions and metrics, helping to guide future research and application in cognitive state prediction using EEG.

Models	MAE	MSE	RMSE	MAPE	R²
RF	0.12	0.13	0.36	1	0.93
GB	0.11	0.12	0.35	0.98	0.94
XGBoost	0.14	0.17	0.41	1.15	0.91
DT	0.1	0.11	0.33	0.92	0.945
Proposed Model	0.08	0.1	0.32	0.85	0.96

Table 2: Performance of RF model on each imputed data.

The Random Forest model, renowned for its robustness and capability to manage overfitting, displays commendable results with an MAE of 0.12 and an R² Score of 0.93, indicating its effectiveness in capturing a significant portion of the variance in the EEG dataset. Despite its advantages, the inherent complexity and randomness in Random Forest might lead to slightly higher errors compared to models that are more finely tuned.

Gradient Boosting shows a slight improvement over Random Forest, highlighting its efficiency in processing sequential errors with lower MAE, MSE, and RMSE values, and a slightly improved R² Score of 0.94. This improvement suggests a tighter fit to the data, likely because Gradient Boosting optimally reduces both bias and variance.

XGBoost, typically robust and quick, experiences a performance dip in this study, registering the highest errors across all metrics and the lowest R² Score of 0.91. This underperformance might stem from overfitting or indicate that the default hyperparameters are not ideally suited to this specific dataset.

Surprisingly, the Decision Tree model, often valued for its interpretability and speed, outperforms the more complex Random Forest and XGBoost. It achieves the lowest MAE and RMSE among the traditional models and an impressive R² Score of 0.945. This performance suggests that the simpler Decision Tree model effectively captures critical features in the EEG data without the burden of added complexity.

The Proposed Ensemble Model markedly surpasses all other models, achieving the lowest MAE, MSE, RMSE, and MAPE values, along with the highest R2 Score of 0.96. This exceptional fit to the EEG data likely results from integrating features of ensemble learning with advanced, possibly more customized mechanisms that adeptly handle the specifics of EEG data, thus significantly boosting predictive accuracy.

This enhancement is evident from the traditional models to the Proposed Model, as demonstrated in Figure 2. The Proposed Model shows reductions in MAE and MSE ranging from approximately 0.02 to 0.04 and 0.01 to 0.07, respectively, compared to other models. RMSE improvements also range from 0.01 to 0.09, showcasing more precise predictions across the board. These substantial advancements underscore the power of the Proposed Model as a highly effective tool for EEG-based cognitive state prediction, enhancing the accuracy and reliability of the results. This significant improvement highlights the benefits of employing advanced modeling techniques that perhaps leverage aspects of machine learning not fully utilized by traditional models. (Figure 2)

Figure 2: Visual illustration of the model performance for each model.

Conclusion

Our study presents a comprehensive comparative analysis of EEG-based cognitive state prediction models, culminating in a significant breakthrough with the development of an AutoML-based stacked ensemble model. This innovative model demonstrates marked superiority over traditional approaches, as evidenced by its exceptional performance metrics—MAE of 0.08, MSE of 0.10, RMSE of 0.32, MAPE of 0.85, and an outstanding R2 Score of 0.96. Such robust performance illustrates the model's enhanced capability to accurately capture the complex patterns inherent in EEG data, surpassing conventional models.

The results affirm the proposed AutoML-based stacked ensemble model as a potent tool for cognitive state prediction, with promising applications in clinical diagnostics and extensive neuroscientific research. This study not only advances the field of EEG data analysis but also underscores the importance of integrating advanced AutoML techniques to enhance predictive accuracy.

Looking ahead, we anticipate further refinements to this model, particularly its real-time application capabilities and its utility in a broader range of neurological studies. This research establishes a new benchmark in the application of machine learning in neuroscience, paving the way for future innovations that fully exploit the capabilities of AutoML and ensemble learning in brain-computer interface technologies.

Review Article