3.1

Data set

The original data set used in this study has a total of 211,833 data samples, including 25 indicators, and the data have been desensitized. The dataset supporting the conclusions of this article is available in the DRYAD repository [unique persistent identifier and hyperlink to dataset in https://doi.org/10.5061/dryad.ft8750v].

Furthermore, to ascertain the model’s reliability, this study employed a local dataset for training purposes. The dataset comprises physical examination records from Jiading District Central Hospital in Shanghai, spanning a three-year period. It has been meticulously desensitized and subjected to ethical scrutiny, encompassing a substantial 556,497 data samples across 116 distinct physical examination indicators.

3.2

Method

Figure 1 outlines the comprehensive methodology employed in this study, presented as a streamlined flowchart. The journey begins with rigorous data preprocessing, which encompasses the elimination of invalid data, the meticulous handling of abnormal values, and the strategic imputation of missing values to ensure data integrity. This foundation is then strengthened through feature engineering, leveraging both the chi-square test and the mutual information method to meticulously select the most predictive features. Armed with the refined feature set, the study advances to the model training phase, where an array of six sophisticated machine learning models and a pair of deep learning models are deployed. The efficacy of these models is meticulously appraised through a suite of six pivotal metrics: accuracy, precision, recall, F1 score, ROC-AUC, and the loss function. This multifaceted evaluation offers a comprehensive assessment of their predictive prowess.

Figure 1.

Research Process Framework

3.2.1

Data processing and feature engineering

In this study, we meticulously processed and curated features from a comprehensive dataset encompassing 211,833 data samples with 25 initial variables, culminating in the extraction of 135,620 refined training datasets characterized by 8 pivotal feature variables.

Our initial step involved rigorous data preprocessing, where we excluded samples with a missing data ratio exceeding 10%, yielding a refined dataset of 135,620 data points. Subsequently, we imputed the remaining missing values with the respective average, ensuring data integrity.

For feature selection, we initially eliminated 5 variables that were deemed irrelevant to our research objectives. We then leveraged the chi-square test [16-19], a statistical technique renowned for its efficacy in analyzing categorical data. The chi-square statistic escalates with sample size, making it an optimal choice for the preliminary feature screening phase of our study.

Subsequently, we employed the Mutual Information (MI) method [20-22], which assesses feature significance by quantifying the mutual information between each feature and the target variable. This metric serves as a robust measure of the interdependence between variables. Following the initial chi-square screening, which discarded features with minimal impact, we retained 16 features. Building upon this foundation, the MI method further refined our selection, identifying and retaining features with substantial influence on the target variable.

Ultimately, through a stringent correlation threshold of 0.7, we pruned highly correlated features, resulting in a final set of 8 features: Age, Height, Weight, SBP (Systolic Blood Pressure), DBP (Diastolic Blood Pressure), FPG (Fasting Plasma Glucose), ALT (Alanine Aminotransferase), and CCR (Creatinine Clearance Rate). The interrelations among these features are visually represented in a heat map, as depicted in Figure 2.

Figure 2.

Characteristic correlation heat map

Guided by feature selection principles aimed at preventing overfitting, this study implemented a rigorous approach, setting a correlation threshold of 0.7 to eliminate features exhibiting strong inter-dependencies among the eight considered attributes. Subsequently, the refined feature set was visually represented through a distribution interval diagram, as depicted in Figure 3, offering a clear and insightful view of the final feature landscape.

Figure 3.

Feature distribution interval

Category imbalance [23], a prevalent challenge in machine learning, was encountered in the dataset derived from the aforementioned operations, where the count of normal samples significantly outweighed that of sick samples, with 133,685 normal instances compared to just 1,935 sick instances. To address this imbalance, SMOTE oversampling was strategically employed, resulting in balanced datasets of 133,685 instances for both normal and sick categories. This balanced representation facilitated more equitable model training and prediction, mitigating the effects of class imbalance.

In this study, the dataset is divided into training set and test set in 7:3 (70% training set and 30% test set), the test set is used to train the model and the test set is used to evaluate the model and to measure the model’s ability to generalize over different instances.

The local dataset underwent identical processing, yielding a total of 22,284 samples characterized by 13 distinct features. These include pulse rate, blood pressure (systolic and diastolic), waist circumference, hemoglobin, white blood cell count, platelet count, glycated hemoglobin, serum alanine aminotransferase (ALT), serum glutamic oxaloacetic transaminase (AST), total bilirubin, serum creatinine, and urinary glucose levels. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) was applied, resulting in an equitable distribution of 20,108 samples for both the control and case groups.

4.1

Signature analysis

In this study, a dual-approach methodology was employed for feature selection. Initially, the chi-square test was applied to the pre-processed data to filter out features with negligible correlation. Subsequently, the mutual information (MI) technique was engaged to refine the selection, identifying features with significant correlation. The process culminated with the exclusion of features that exhibited a correlation coefficient exceeding the threshold of 0.7. To mitigate the effects of data imbalance, the dataset underwent oversampling, and the data was partitioned to facilitate model training.

Table 1 presents the results of both univariate and multivariate regression analyses for the identified characteristics. Notably, in the multivariate regression analysis, the P-value associated with fasting plasma glucose (FPG) is markedly below the threshold of 0.005, underscoring its substantial influence on diabetes. FPG, derived from direct measurement of plasma glucose levels, is grounded in extensive demographic data and serves as a critical and widely utilized metric for assessing glucose metabolism. The FPG measurement is a direct reflection of current blood glucose levels and is pivotal for diagnosing metabolic glucose disorders. It is important to note that the accuracy of FPG testing can be influenced by a spectrum of factors, including hepatic function, hormonal regulation, and neurological elements.

Table 1.

Single-factor and multifactor regression analysis Variable

CI=Confidence Interval, 2*p<0.05,**p<0.01,***p<0.001

Furthermore, the P-values for fasting plasma glucose (FPG), body weight, age, and alanine aminotransferase (ALT) are notably lower, aligning with the feature importance rankings depicted in Figure 4. The t-values of these parameters in the univariate regression analysis corroborate their substantial influence on diabetes risk, echoing findings from the study by Lara Lama et al. [26]. ALT, synthesized and secreted by hepatocytes, serves as a key biomarker for liver function, and its levels can modulate FPG concentrations. Obesity, a prevalent symptom across numerous pathologies, is also reflected in body weight, which is a component of the body mass index (BMI). An elevated body weight, indicative of obesity, exerts a significant effect on FPG levels. Excess adipose tissue can release inflammatory cytokines, impairing the functionality of pancreatic beta cells. Concurrently, elevated blood lipids can induce insulin resistance, precipitating a cascade of metabolic disorders such as hypertension and hyperlipidemia, which in turn can disrupt blood glucose regulation.

Figure 4.

XGBoost feature importance ranking

Figure 5 elucidates the diabetes machine learning investigation through the lens of SHAP (SHapley Additive exPlanations) values [27]. These SHAP values serve as a mechanism to interpret the predictive outcomes of the machine learning model, providing a quantifiable measure of each feature’s influence on the model’s diabetes diagnosis. Notably, fasting plasma glucose (FPG) exhibits a robust positive correlation with diabetes, signifying its substantial risk factor status. Concurrently, diastolic blood pressure (DBP) and alanine aminotransferase (ALT) are also identified as having significant correlations with elevated diabetes risk. DBP, a metric of arterial vascular compliance, is pivotal in the assessment of cardiovascular health, a domain closely intertwined with metabolic conditions that can influence blood glucose dynamics. This correlation aligns with the prior feature importance findings, substantiating the precision and broad applicability of the employed feature selection techniques and predictive models in diabetes risk assessment.

Figure 5.

SHAP waterfall chart explaining XGBoost

In this study, the selection process meticulously curated the features by eliminating those with low and high correlations. This strategic refinement has bolstered the predictive prowess of the model, simultaneously enhancing its robustness and its capacity for generalization across diverse datasets.

4.2

Research results

This study conducts a comprehensive performance assessment of eight predictive models, scrutinizing five key classification metrics. The conclusive findings demonstrate that the XGBoost model outperforms its counterparts, exhibiting remarkable efficacy and resilience in the prediction of diabetes. A detailed account of the model’s performance, alongside the evaluative metrics, is meticulously documented in Table 2.

Table 2.

Multi-model comparison results on Dryad dataset

Table 2 offers a clear insight into the performance disparities among various models, with tree-based machine learning models, particularly XGBoost, LightGBM, and CatBoost, outshining the rest. The underwhelming performance of the deep learning model may stem from its relatively low complexity, failing to capture the intricacies of the data. Among the top performers, XGBoost emerges as the clear winner, boasting exceptional metrics and robustness. With an accuracy rate of 99.1%, precision of 99.0%, recall of 99.2%, F1 score of 99.1%, and an AUC value soaring at 0.991, XGBoost demonstrates unparalleled classification prowess. CatBoost and LightGBM follow closely, with LightGBM achieving the highest accuracy among the three. The classification capabilities of the machine learning models employed in this study, along with their impressive evaluation metrics, solidify the soundness of our feature selection method and underscore the reliability of XGBoost for diabetes prediction.

Table 3 presents the statistical outcomes of the validation experiments utilizing the Jiading Hospital dataset, revealing a consistency with the results depicted in Table 2. The trio of XGBoost, LightGBM, and CatBoost models continue to demonstrate superior performance, with the RF, LR, and DT models trailing in succession. Comparatively, the deep learning models have shown an improvement in performance over the prior results, yet on average, machine learning models still outperform them. Notably, the XGBoost model stands out with its robust and stable results, boasting an accuracy of 96.9%, a precision of 96.5%, a recall of 97.3%, an F1 score of 96.9%, and an AUC value of 0.969. These findings from this dataset reaffirm the prowess of the XGBoost model in the realm of diabetes prediction and classification.

Table 3.

Multi-model comparison results on local datasets