Machine learning classifies metabolic syndrome
An extreme gradient boosting machine learning model achieved the highest classification performance among 10 algorithms, with 93% accuracy and an area under the receiver operating characteristic curve of 0.986 in distinguishing pre-metabolic syndrome from metabolic syndrome, according to a retrospective study published in PLOS One.
"These findings indicate that algorithm choice may influence classification performance in metabolic classification tasks," the authors wrote.
The single-center, retrospective cross-sectional study evaluated 657 adults attending an outpatient clinic at HRH Princess Maha Chakri Sirindhorn Medical Center, Srinakharinwirot University, Thailand, between January 2021 and December 2022. Of these, 359 had pre-metabolic syndrome (pre-MetS), and 298 had metabolic syndrome (MetS).
The researchers compared machine learning (ML) algorithms using routinely collected clinical variables, identified the features contributing most strongly to classification, and examined whether overlap with established diagnostic criteria influenced model performance.
Study design and model development
Participants were classified using an operational definition based on International Diabetes Federation (IDF) criteria for Asian populations. MetS required central obesity, defined as a waist circumference of at least 90 cm in men or 80 cm in women, plus at least two additional metabolic abnormalities involving triglycerides, high-density lipoprotein cholesterol, blood pressure, or fasting plasma glucose or previously diagnosed type 2 diabetes.
Pre-MetS was defined as the presence of at least one metabolic abnormality without meeting the complete criteria for MetS. Central obesity was not required for classification as pre-MetS.
The researchers initially evaluated 27 candidate variables and used the Boruta feature-selection algorithm to retain 24 predictors. The dataset was divided into a training set of 525 participants (80%) and an internal holdout test set of 132 participants (20%). Hyperparameters were optimized using repeated 10-fold cross-validation with 10 repetitions. Feature selection, normalization, and model development were performed exclusively within the training dataset to prevent data leakage.
XGBoost achieves the highest classification performance
Among the 10 supervised ML algorithms, XGBoost achieved the highest discrimination in the holdout test set, with an area under the receiver operating characteristic curve (AUROC) of 0.986 and an accuracy of 93%.
The model demonstrated a sensitivity of 88%, specificity of 97%, positive predictive value of 97%, and negative predictive value of 91%. Its Brier score was 0.050, the lowest among the evaluated models, indicating the most accurate probability estimates by that measure.
The neural network and random forest models followed, with AUROCs of 0.948 and 0.942, respectively. Their classification accuracies were 88% and 89%. Across all 10 algorithms, AUROCs ranged from 0.804 to 0.986, demonstrating variability in classification performance depending on the modeling approach.
Clinical variables contribute to classification
The researchers used Shapley Additive Explanations analysis to identify the variables contributing most strongly to XGBoost classification.
Systolic blood pressure was the most influential feature, followed by fasting plasma glucose, waist circumference, sex, and the triglyceride-glucose waist-to-height ratio. Other influential predictors included high-density lipoprotein cholesterol, triglyceride-glucose waist circumference, triglycerides, the triglyceride-to-high-density lipoprotein cholesterol ratio, and waist-to-height ratio.
Participants with MetS generally had higher body mass index, waist circumference, visceral fat, systolic blood pressure, fasting plasma glucose, glycated hemoglobin, triglycerides, and insulin resistance measures than those with pre-MetS. High-density lipoprotein cholesterol was lower in the MetS group. However, several influential predictors were components of the IDF diagnostic criteria or derived from those components.
Removing diagnostic variables reduces performance
To assess whether overlap between predictors and established diagnostic criteria influenced model performance, the researchers conducted three sensitivity analyses that progressively excluded overlapping variables.
Removing 3 composite metabolic indices reduced XGBoost discrimination from an AUROC of 0.986 to 0.912. Excluding all variables directly included in the IDF diagnostic criteria for MetS, along with related derived indices, further reduced the AUROC to 0.875. After additionally excluding the waist-to-height ratio, the AUROC declined to 0.861.
The findings indicated that diagnostic overlap contributed substantially to the model's high discrimination in the primary analysis, although the model retained acceptable discrimination after the overlapping variables were removed. The researchers emphasized that the results did not establish that ML was superior to established diagnostic criteria.
Limitations and clinical implications
The researchers identified several limitations, including the retrospective cross-sectional design, recruitment from a single tertiary-care center, a relatively small internal holdout test dataset of 132 participants, and a lack of external validation. The models classified existing metabolic status but could not predict future progression or cardiometabolic outcomes. Detailed medication records were unavailable, potentially contributing to misclassification, and residual overlap between predictors and diagnostic criteria could not be excluded despite the sensitivity analyses.
The authors called for larger, multicenter prospective studies using predictor sets that minimize diagnostic overlap, incorporating comprehensive medication data, and conducting external validation and assessments of calibration and clinical utility.
"Whether machine-learning approaches provide clinically meaningful benefit beyond established IDF diagnostic criteria remains uncertain," they wrote.
The authors reported receiving no specific funding and declared no competing interests.
AACE Endocrine AI is published by Conexiant under a license arrangement with the American Association of Clinical Endocrinology, Inc. (AACE®). The ideas and opinions expressed in AACE Endocrine AI do not necessarily reflect those of Conexiant or AACE. For more information, see Policies.