Machine learning for high-risk hospitalization prediction in outpatient individuals with diabetes at a tertiary hospital

ABSTRACT

Objective:  To characterize, via a predictive model using real-world data, patients with diabetes with a heightened probability of hospitalization.

Methods:  At the Endocrinology Unit of a tertiary public hospital in Rio Grande do Sul, Brazil, a retrospective cohort study analyzed initial consultations from January 1, 2015, to December 31, 2017, focusing on 617 patients with diabetes. Within this group, 82.98% (512 patients) did not require hospitalization, while 17.02% (105 patients) were hospitalized at least once. Multiple machine learning algorithms were tested, and the combination of XGBoost and Instance Hardness Threshold models displayed the best predictive performance. The SHapley Additive exPlanations method was used for result interpretation.

Results:  The most optimal performance was observed by combining the XGBoost and Instance Hardness Threshold models, resulting in the highest sensitivity (0.93) in accurately classifying hospitalization events, with an acceptable area under the curve of 0.72. Key predictive features included the number of outpatient visits, amplitude of estimated glomerular filtration rate, and age (individuals below 24 years old and between 65 to 70 years old had higher hospitalization likelihood).

Conclusion:  The proposed model demonstrated high predictive capability and may help to identify patients with diabetes who should be more closely monitored to reduce their risk of hospitalization.

Keywords
Diabetes mellitus; Hospitalization; Machine learning

INTRODUCTION

Diabetes mellitus is a disease characterized by chronic hyperglycemia due to the impaired release and action of insulin, as well as a failure to regulate hepatic glucose production (¹). The two most prevalent types are type 1 and type 2, which account for about 10% and nearly 90% of all cases, respectively (²). The prevalence of diabetes mellitus has been increasing steadily in recent years and, by 2021, about 537 million people were estimated to have this condition (³). In Brazil, 12% of the population is diagnosed with diabetes (⁴), the sixth country with the highest number of adults with diabetes in the world (³).

Patients with diabetes are at greater risk of developing chronic complications and related diseases and require more access to health services than those without diabetes (⁵,⁶), representing one of the leading causes for hospital admissions and outpatient visits. It was estimated that the economic burden in Brazil reached US$ 2.15 billion in 2016, of which 70.6% are indirect costs related to premature deaths, absenteeism, and early retirement. If the growth rate of diabetes prevalence continues in Brazil, the direct and indirect costs of diabetes will be more than double by 2030 (an increase of 133.4% or 6.2% per year) (⁷).

Artificial intelligence (AI) is a growing field and its applications to diabetes could reshape the management of this chronic condition. Artificial intelligence algorithms have been used to develop predictive models for the risk of developing diabetes and its complications and to optimize the use of healthcare resources (⁸,⁹,¹⁰,¹¹,¹²).

Electronic medical records (EMRs) allow the consistent and homogeneous gathering of data, enabling the repository use to train and develop algorithms (¹⁰,¹³). Such records have been used in medical studies with several objectives, including the prediction of hospitalization using the patient’s first record in the emergency room (¹⁴). The possibility of predicting which patients with diabetes are at greater risk of hospitalization and mortality via their characteristics would lead to early interventions that reduce risk, optimize treatments, and better prepare hospital resources to provide adequate care. Few studies evaluate the prediction of hospitalization of patients with diabetes (¹⁵,¹⁶,¹⁷,¹⁸).

This study aimed to characterize, via a predictive model using real-world data, patients with diabetes with a heightened probability of hospitalization.

MATERIALS AND METHODS

Study database

This was a retrospective cohort study using a database composed of EMRs of patients from the outpatient Endocrinology Unit of a tertiary public hospital from Southern Brazil. The complete dataset consists of EMRs of patients who had their first medical appointment in the period between January 1, 2015, and December 31, 2017, totaling 2,973 patients. Data within the 2-year period after the first medical appointment were used. Only patients diagnosed with diabetes were selected. They were identified via information from the International Classification of Diseases (ICD) of the first medical appointment and/or result of the first plasma glucose (≥ 126 mg/ dL) and/or the first glycated hemoglobin (HbA1c) (≥ 6.5%) measurements. The final dataset contained 617 patients, 512 (82.9%) were not hospitalized within the 2-year period and 105 (17.0%) were hospitalized at least once during that period. The EMR contained relevant information such as age, skin color, gender, number of outpatient visits, and laboratory tests (creatinine, plasma glucose, HbA1c, and urinary albumin concentration (UAC). The presence of diabetic kidney disease (DKD) was assessed with UAC and estimated glomerular filtration rate (eGFR) calculation using the CKD-EPI equation (¹⁹). Diabetic kidney disease was defined as an eGFR < 60 mL/min/1.73 m² and/or UAC from a single urinary sample ≥ 14 mg/L (²⁰,²¹,²²). All textual records were written in Brazilian Portuguese.

This study was approved by the hospital’s Ethical Committee under number 43431521.0.0000.5327. The data were obtained via anonymized query and the consent form was waived.

Intelligent system protocol

We adopted a five-step method (Figure 1) to predict the occurrence of hospitalization in patients with diabetes from their EMRs. In the first step, we gathered, pre-processed the data, and discarded repeated observations. Missing data were imputed using the k-Nearest Neighbor method (²³,²⁴). After pre-processing, the dataset was rescaled using the max-min scaling in Eqn. 1, the values of all continuous features were in the range [0, 1]. In the equation, X represents the feature’s values, and X_max and X_min are the largest and smallest feature values in the dataset.

Figure 1
Outline of the proposed method.

Eqn. 1 $Scaling=\left(X-X_{\min }\right)/\left(X_{\max }-X_{\min }\right)$

In step two, we divided the complete dataset to obtain train and validation portions using the κ-fold cross-validation technique (²⁵). The complete dataset is partitioned into five subsets (κ = 5), and the model is trained on four of them while being validated on the remaining subset. This process was iterated five times, ensuring each fold acted as validation set at least once. At each fold, we used a stratified randomized sampling approach to ensure proportional representation of each class (hospitalized and not hospitalized) and reflect the class proportions of the complete sample (²⁶). To obtain a better generalization of the model, we repeated the five-fold cross-validation process 20 times. In each repetition, we randomly shuffled the dataset before dividing it into five folds for cross-validation. This ensured the model was trained and validated on multiple different combinations of data sets, allowing it to capture more general and robust patterns in the data. At the end of these 20 repetitions, we obtained 100 validation results (five folds × 20 repetitions).

Our goal was to correctly identify patients at risk of hospitalization, i.e., the minority class. Therefore, in step three of the method we applied resampling techniques to the training portion. These techniques are recommended when dealing with highly unbalanced class problems, in which results can be influenced by the majority class. We tested six resampling techniques: Instance Hardness Threshold (IHT), Random Under Sampler (RUS), Synthetic Minority Oversampling Technique (SMOTE), adaptive synthetic sampling (ADASYN), synthetic minority oversampling technique and edited nearest neighbor (SMOTEENN) and synthetic minority oversampling technique with Tomek links technique (SMOTETomek) (Table S1). Resampling was not applied to the validation set, as we wanted to evaluate classification results in a reallife situation.

In step four, we performed feature selection using a wrapper method (²⁷). In step five, we used the best subset of features in step four in the validation portion of the dataset for each machine learning algorithm tested; they are logistic regression (LR), K-nearest neighbors (KNN), support vector machine (SVM), Extreme Gradient Boosting (XGBoost), and Bagging Classifier using Decision Trees (Table S2). We averaged the one hundred validation results for the following performance metrics: accuracy, positive and negative predictive values (PPV and NPV, respectively), sensitivity, specificity, F1-Score, and are under the Receiver Operating Characteristic (ROC) curve.

We finally selected the model with the best predictive performance and used the SHapley Additive exPlanations (SHAP) method to analyze the results. This involves calculating SHAP values, which are obtained using a game-theoretic approach (²⁸). The calculation of SHAP values involves evaluating the contribution of each feature to the model’s prediction by comparing the prediction with and without the feature, while considering all possible feature combinations. This process is applied to every feature and observation in the dataset, producing a matrix of SHAP values that reveals the relative importance of each feature to each observation. That enables a deeper understanding of the importance of each feature in the prediction and allows identifying features with the greatest impact on the model’s output (²⁹,³⁰).

Statistical analysis

A convenience sample was used. The algorithm performance was measured by the area under the ROC curve generated by plotting sensitivity versus one minus specificity. Based on the two operating points, 2×2 tables were developed to characterize the sensitivity and specificity of the algorithm. All statistical analyses, methods, techniques, and machine learning algorithms were implemented via Python (version 3.9.12).

RESULTS

Figure 2 shows the features used as predictors and the frequency in which they were selected by the best performing classification algorithm. Table 1 reports the validation results, with the best performers for each metric highlighted in bold.

Figure 2
Features used as inputs and frequency with which they were retained in the one hundred validations of the best predictive model.

Table 1
Average predictive performance and standard deviations obtained from one hundred replicates of the dataset validation portion for different combinations of resampling technique and classification algorithm

Resampling technique	Classification algorithm	Performance metrics
		Area under the ROC curve	Sensitivity	Specificity	Negative predictive value	Positive predictive value	F1_score	Accuracy
Instance Hardness Threshold	LR	0.7280 ± 0.0519	0.6428 ± 0.1044	0.8133 ± 0.0430	0.9179 ± 0.0218	0.4181 ± 0.0604	0.5036 ± 0.0660	0.7842 ± 0.0362
	KNN	0.7141 ± 0.0398	0.8890 ± 0.0816	0.5393 ± 0.0686	0.9611 ± 0.0262	0.2859 ± 0.0289	0.4313 ± 0.0355	0.5988 ± 0.0521
	SVM	0.7099 ± 0.0429	0.8609 ± 0.0705	0.5588 ± 0.0617	0.9518 ± 0.0232	0.2881 ± 0.0311	0.4308 ± 0.0393	0.6102 ± 0.0507
	XGBoost	0.7202 ± 0.0397	0.9333 ± 0.0538	0.5071 ± 0.0617	0.9740 ± 0.0205	0.2818 ± 0.0284	0.4323 ± 0.0358	0.5797 ± 0.0514
	Bagging	0.7123 ± 0.0373	0.9266 ± 0.0688	0.4979 ± 0.0609	0.9720 ± 0.0244	0.2760 ± 0.0238	0.4247(0.0313	0.5709 ± 0.0477
Random Under Sampler	LR	0.7184 ± 0.0481	0.6028 ± 0.0967	0.8340 ± 0.0383	0.9114 ± 0.0191	0.4317 ± 0.0638	0.4997 ± 0.0652	0.7947 ± 0.0326
	KNN	0.6901 ± 0.0528	0.6680 ± 0.1002	0.7122 ± 0.0521	0.9132 ± 0.0241	0.3257 ± 0.0488	0.4359 ± 0.0581	0.7047 ± 0.0440
	SVM	0.7343 ± 0.0506	0.6623 ± 0.1088	0.8062 ± 0.0524	0.9217 ± 0.0219	0.4190 ± 0.0625	0.5085 ± 0.0636	0.7817 ± 0.0403
	XGBoost	0.7180 ± 0.0589	0.7161 ± 0.1096	0.7198 ± 0.0558	0.9257 ± 0.0270	0.3479 ± 0.0568	0.4660 ± 0.0664	0.7191 ± 0.0481
	Bagging	0.7124 ± 0.0478	0.7128 ± 0.0965	0.7120 ± 0.0495	0.9242 ± 0.0226	0.3394 ± 0.0442	0.4579 ± 0.0526	0.7122 ± 0.0398
Synthetic Minority Oversampling Technique	LR	0.7408 ± 0.0479	0.6442 ± 0.0965	0.8374 ± 0.0357	0.9203 ± 0.0196	0.4524 ± 0.0611	0.5286 ± 0.0636	0.8046 ± 0.0308
	KNN	0.6887 ± 0.0569	0.6147 ± 0.1110	0.7626 ± 0.0516	0.9066 ± 0.0243	0.3513 ± 0.0629	0.4441 ± 0.0696	0.7375 ± 0.0436
	SVM	0.7505 ± 0.0485	0.6609 ± 0.1024	0.8402 ± 0.0375	0.9241 ± 0.0203	0.4636 ± 0.0614	0.5411 ± 0.0630	0.8097 ± 0.0303
	XGBoost	0.6820 ± 0.0511	0.4919 ± 0.0982	0.8721 ± 0.0333	0.8935 ± 0.0189	0.4460 ± 0.0787	0.4643 ± 0.0776	0.8074 ± 0.0317
	Bagging	0.6697 ± 0.0485	0.4928 ± 0.0962	0.8466 ± 0.0417	0.8909 ± 0.0182	0.4039 ± 0.0754	0.4397 ± 0.0718	0.7864 ± 0.0352
Adaptive synthetic sampling	LR	0.7479 ± 0.0523	0.6680 ± 0.0986	0.8278 ± 0.0368	0.9243 ± 0.0209	0.4470 ± 0.0654	0.5331 ± 0.0697	0.8006 ± 0.0346
	KNN	0.7049 ± 0.0552	0.6614 ± 0.1100	0.7484 ± 0.0429	0.9157 ± 0.0250	0.3519 ± 0.0501	0.4576 ± 0.0628	0.7336 ± 0.0371
	SVM	0.7630 ± 0.0499	0.6919 ± 0.1008	0.8342 ± 0.0356	0.9301 ± 0.0210	0.4650 ± 0.0578	0.5531 ± 0.0623	0.8100 ± 0.0310
	XGBoost	0.6744 ± 0.0554	0.5147 ± 0.1107	0.8340 ± 0.0402	0.8938 ± 0.0215	0.3926 ± 0.0721	0.4417 ± 0.0771	0.7797(0.0352
	Bagging	0.6716 ± 0.0589	0.5033 ± 0.1111	0.8399 ± 0.0429	0.8921 ± 0.0222	0.3983 ± 0.0865	0.4406 ± 0.0858	0.7826 ± 0.0398
Synthetic minority oversampling technique and edited nearest neighbor	LR	0.7245 ± 0.0520	0.7709 ± 0.0937	0.6781 ± 0.0594	0.9356 ± 0.0247	0.3332 ± 0.0470	0.4635 ± 0.0557	0.6939 ± 0.0497
	KNN	0.6813 ± 0.0513	0.7185 ± 0.1060	0.6441 ± 0.0508	0.9188 ± 0.0280	0.2939 ± 0.0355	0.4159 ± 0.0484	0.6568 ± 0.0405
	SVM	0.7306 ± 0.0495	0.7566 ± 0.0956	0.7045 ± 0.0472	0.9344 ± 0.0237	0.3466 ± 0.0431	0.4739 ± 0.0531	0.7134 ± 0.0397
	XGBoost	0.7234 ± 0.0525	0.7061 ± 0.1014	0.7406 ± 0.0453	0.9252 ± 0.0239	0.3608 ± 0.0468	0.4757 ± 0.0575	0.7347 ± 0.0390
	Bagging	0.7020 ± 0.0564	0.6528 ± 0.1067	0.7511 ± 0.0488	0.9138 ± 0.0241	0.3533 ± 0.0593	0.4562 ± 0.068	0.7344 ± 0.0427
Synthetic minority oversampling technique with Tomek inks technique	LR	0.7400 ± 0.0469	0.6419 ± 0.0934	0.8382 ± 0.0343	0.9198 ± 0.0190	0.4522 ± 0.0588	0.5280 ± 0.0623	0.8048 ± 0.0301
	KNN	0.7016 ± 0.0535	0.6366 ± 0.0988	0.7666 ± 0.0504	0.9117 ± 0.0221	0.3639 ± 0.0631	0.4604 ± 0.0674	0.7445 ± 0.0437
	SVM	0.7496 ± 0.0477	0.6557 ± 0.0991	0.8435 ± 0.0394	0.9233 ± 0.0197	0.4684 ± 0.0658	0.5422 ± 0.0625	0.8115 ± 0.0321
	XGBoost	0.6825 ± 0.0556	0.5019 ± 0.1046	0.8630 ± 0.0351	0.8944 ± 0.0202	0.4341 ± 0.0883	0.4620 ± 0.0861	0.8016 ± 0.0346
	Bagging	0.6696 ± 0.0583	0.4895 ± 0.1227	0.8497 ± 0.0372	0.8910 ± 0.0226	0.4027 ± 0.0753	0.4372 ± 0.0839	0.7884 ± 0.0314

• Results expressed as mean ± standard deviation
• The applied Classification Algorithms encompassed logistic regression, k-nearest neighbors, support vector machine, extreme gradient boosting and bagging classifier
• ROC: receiver operating characteristic; LR: logistic regression; KNN: K-nearest neighbors; SVM: support vector machine; XGBoost: Extreme Gradient Boosting

Combining the XGBoost and IHT models yielded the highest sensitivity (value of 0.93) in correctly classifying hospitalization events and an acceptable AUC of 0.72. In addition, it yielded the lowest standard deviation for sensitivity and AUC values, indicating more generalizable results compared to other models. We used SHAP to interpret the features most frequently selected by the XGBoost-IHT combination and provide insights about the importance of each feature (Figure 3) and its effect on the classification result (Figure 4).

Figure 3
SHapley Additive exPlanations values (features impacts on model outputs). SHapley Additive exPlanations values were computed across the entire dataset using the XGBoost-IHT combination model. Each point on the graph corresponds to a single data observation. The distribution of SHapley Additive exPlanations values is displayed along the horizontal axis through violin plots. Positive SHapley Additive exPlanations values indicate features that contribute to the accurate classification of hospitalization cases, with larger values meaning greater impact. Conversely, negative SHapley Additive exPlanations values are associated with features influencing the prediction of non-hospitalization cases. The color spectrum in the graph represents the actual values of data observations, transitioning from blue to red as the value increases.

Figure 4
Relationship between the features' real values (displayed along the horizontal axes) and their respective SHapley Additive explanations values (displayed along the vertical axes).

Figure 3 allows a visual assessment of the features’ importance for the classification of hospitalization cases. Outpatient visits in the 2-year period and amplitude of eGFR were important hospitalization predictors, as demonstrated by their high importance values in Figure 3. Regarding the outpatient visits in the 2-year period, it is clear that many visits (turning red) are associated with larger SHAP values, i.e., it increases the probability of hospitalization. The amplitude behavior of eGFR feature is similar, suggesting that the greater the difference in exam results, the higher the probability of hospitalization. Although in Figure 3 some features show relatively low importance, it is important to assess their clinical relevance in conjunction with other features and make informed decisions. A case in point is the DKD, which, despite its relatively modest SHAP contribution, is informative due to a shift in SHAP values from negative to positive when transitioning from the presence to absence of DKD.

The graphs of Figure 4 show in a clearer way the relationship between features’ values and the probability of hospitalization. This is because the horizontal axes represent the actual values of each feature, while the vertical axes represent their corresponding SHAP values. Age is the third most important feature in predicting hospitalizations. As shown in Figure 4, SHAP values for age vary according to the patient's age group. For patients under 24 years old, positive SHAP values indicate a higher probability of hospitalization for this age group. In opposition, mostly negative SHAP values associated with the age group between 25 and 65 years old suggest lower probability of hospitalization for this age group. Finally, for patients over 65 years old the relation between feature and prediction is not clear, varying according to other features present in the model.

DISCUSSION

Considering the limited number of characteristics, we were able to analyze with the available data, the area under the ROC curve reported for the constructed classifier is indeed a considerable achievement for predicting the hospitalization of patients with diabetes, showing that with few easy-to-obtain EMR data it is possible to predict the probability of the patient’s hospitalization, which enables to identify the patient at greatest risk, allowing the allocation of available resources before the outcome occurs.

The results of our study showed that the top three most important features for predicting hospitalization are: number of outpatient visits, amplitude of eGFR, and age (patients under 24 years old and 65 to 70 years old have a higher risk).

The classifier showed that patients who had the highest number of medical consultations during the last 2 years were those with the highest risk of hospitalization. The number of outpatients visits can correlate with the complexity of the patient’s health status, since patients who have more comorbidities or more serious illnesses are those who have a higher number of consultations. A study that evaluated the magnitude and predictors of hospital admission in patients with type 2 diabetes at public hospitals of Eastern Ethiopia showed that medical conditions, including the number of comorbidities and the presence of chronic diabetes complications, are determinant for hospital admission (¹⁵). The number of inpatient healthcare visits can also predict the chance of readmission within 30 days after hospital discharge (³¹).

The contribution of eGFR amplitude to predict hospitalization is because patients with varying eGFR may have presented a worsening of their renal function in the last 2 years. A study that evaluated the variability in eGFR in patients with diabetes showed they were at greater risk of major clinical outcomes (major macrovascular events, new or worsening nephropathy, and all-cause mortality) (³²). The study assessed the association between 20-month eGFR variability and the risk of major clinical outcomes in type 2 diabetes among 8,241 patients. Variability in eGFR was calculated from three serum creatinine measurements over 20 months. Compared with low variability, greater 20-month eGFR variability was independently associated with higher risk of the primary outcome with evidence of a positive linear trend (p = 0.015).

As our study showed, age is a known predictor for hospitalization in patients with diabetes. The study revealed that patients who are younger than 24 years old were more likely to be hospitalized, as well as patients between 65 and 70 years old. The fact that younger people are more likely to be hospitalized may represent patients with type 1 diabetes who have diabetic ketoacidosis (DKA). The precipitating factors of DKA were evaluated in a public Brazilian hospital, showing that the mean age of patients with DKA from January 2005 to March 2010 was 26 ± 13 years (³³), remaining at 26.2 ± 14.5 years in the period from April 2010 to January 2017 (³⁴), treatment noncompliance being the leading precipitating factor in both periods (³³,³⁴). A study that evaluated care indicators for patients with diabetes in our country showed that in 2019, worse indicators were observed for younger individuals (³⁵).

Although patients between 65 and 70 years old had a higher risk of hospitalization than those over 70 years old, we were unable to identify differences in this subgroup. Dennis et al. showed in their predictive model that hospitalized patients with diabetes were older. In addition, age can also predict length of stay (³⁶) and is an important feature for predicting 1-year mortality (³⁷).

Our study has limitations. First, from a clinical perspective, our data do not include patients’ medical history such as insulin use, type of diabetes, and duration of diabetes. Second, the database lacks information on important comorbidities and anthropometry. Third, the database had limited sociodemographic information; previous studies showed that low education, low socioeconomic status, high alcohol use, longer diabetes mellitus duration can predict hospitalization (¹⁵,¹⁶,¹⁷). Moreover, patients attending our hospital’s outpatient Endocrinology clinic could have been hospitalized in another hospital and this would not have been identified because the search for outcomes was done exclusively in our hospital database.

Despite these limitations, it is essential to consider the financial implications of implementing predictive models in healthcare settings. Implementation costs can vary significantly; according to Al Meslamani (³⁸), these considerations encompass various expenses, including the initial model development, its real-world implementation, user training, and ongoing maintenance. Cost estimates can vary widely; for instance, simple models may incur operational costs ranging from $60,750 to $94,500, whereas training more complex models can cost tens of millions. The sustainability of funding, coupled with potential secondary savings—such as increased efficiency for providers, reduced hospitalizations, and shorter lengths of stay—is vital for justifying investments in these models. For example, Brisimi et al. revealed that in 2012 the average hospitalization cost in the United States was $9,500. A predictive model with an 81% detection rate could reduce preventive measure costs to just $320 per patient, potentially saving up to $1 billion in avoidable hospitalizations across the Unites States (³⁹).

CONCLUSION

The proposed model demonstrated predictive capability and may help identify patients with diabetes who are at higher risk of hospitalization. That allows paying special attention to these patients during outpatient follow-up and identify patients with greatest risk upon arrival at the emergency room, optimizing resources allocation. The factors that most contribute to the prediction are the number of outpatient visits, amplitude of estimated glomerular filtration rate and age (patients under 24 years old and 65 to 70 years old present higher probability).

Appendix

Table S1
Overview of resampling techniques

Techniques	Description	References
IHT	Undersampling technique that removes observations from the majority class that are considered difficult to classify correctly, based on a measure of “instance hardness”. The objective is to keep only the instances that are easiest to classify, so that the model can focus on the most important instances and improve performance in the classification	Smith et al. (40)
RUS	Undersampling technique that reduces the number of observations in the majority class at random until the sample size matches that of the minority class. A major disadvantage is that important information from the sample may be lost due to the random removal of observations	Hanafy et al. (41)
SMOTE	Oversampling technique that involves selecting a random observation from the minority class and then randomly choosing one of its k nearest neighbors. The Euclidean distance between the chosen observation and its nearest neighbor is then multiplied by a random value between zero and 1 and added to the chosen observation, resulting in a synthetic observation. This process is repeated until the desired level of oversampling is achieved. By creating synthetic observations in this way, the model can learn from additional instances of the minority class, leading to better overall performance	Wei et al. (42) and Chawla et al. (43)
ADASYN	Oversampling technique that generates synthetic instances for the minority class using a density distribution, aiming to create more observations in regions where the density is low. By generating more synthetic instances in these regions, ADASYN adapts the classification decision boundary towards the difficult observations, thereby improving model performance on imbalanced datasets	Wei et al. (42)
SMOTEENN	SMOTEENN is a technique that combines oversampling through the SMOTE technique and undersampling through the ENN technique. First, SMOTE is used to generate synthetic observations for the minority class. Next, ENN is applied to remove noisy observations that may have been introduced during the oversampling process. ENN removes any observation whose class is different from the majority class of its nearest neighbors	Chawlaet al. (44)
SMOTETomek	SMOTETomek combines oversampling through the SMOTE technique and undersampling through the Tomek links technique. First, SMOTE is used to generate synthetic observations for the minority class. Next, Tomek selects random observations from the majority class and identifies any pairs of observations (one from the majority class and one from the minority class) that form a Tomek link, i.e., a pair of points from different classes that are each other’s nearest neighbors. Links are finally removed to help increase the separation between minority and majority classes	Chawla et al. (44) and Batista et al. (45)

• IHT: Instance Hardness Threshold; RUS: Random Under Sampler; SMOTE: Synthetic Minority Oversampling Technique; ADASYN: adaptive synthetic sampling; SMOTEENN: synthetic minority oversampling technique and edited nearest neighbor; ENN: Edited Nearest Neighbors; SMOTETomek: synthetic minority oversampling technique with Tomek links technique.

Table S2
Overview of machine learning algorithms for classification

Method	Description	References
LR	LR is a statistical technique used to predict a binary dependent event, and it gets its name from the logistic function it employs, which produces values between zero and one. As a popular probabilistic classification method for binary classification tasks, logistic regression models are obtained by minimizing the regularized negative log-likelihood function to achieve optimal results	Godoy et al. (46)
KNN	KNN works by finding the k nearest observations in the training set to an unclassified observation and assigning the class label of the majority of the k-nearest neighbors to the unclassified observation	Wu et al. (47)
SVM	SVM finds a decision boundary (or hyperplane) between two classes of observations in a dataset by maximizing the margin of separation between the hyperplane and the closest instances, known as support vectors. This leads to a more generalized model that can perform well on new, unseen data. SVM can perform nonlinear classification using the kernel trick, which maps observations onto a higher-dimensional feature space to find the optimal separating hyperplane. Different types of kernel functions can be used with SVM, such RBF, polynomial, and	Wu et al. (47) and Álvarez-Alvarado et al. (48)
XGBoost	XGBoost is a machine learning algorithm that uses an ensemble of decision trees to make predictions for both regression and classification problems. It stands out from other decision tree algorithms by training a sequence of decision trees that correct the errors of the previous tree. XGBoost optimizes the loss function to minimize prediction errors and improve accuracy, while also incorporating different regularization techniques to prevent overfitting	Grollmuss et al. (49) and Samih et al. (50)
Bagging Classifier	The Bagging Classifier is an ensemble method that trains multiple independent decision trees on bootstrapped subsets of the data, grows each tree to maximum depth to minimize bias, and combines their predictions to form a robust, lower-variance final model that improves overall performance and reduces overfitting	Bbeiman (51)

• LR: logistic regression; KNN: K-nearest neighbors; SVM: support vector machine; RBF: radial basis function; XGBoost: Extreme Gradient Boosting.

• Funding:
  this study was funded by the Research Incentive Fund of the Hospital de Clínicas de Porto Alegre and the Graduate Program in Medical Sciences: Endocrinology of the Faculty of Medical Sciences of the Universidade Federal do Rio Grande do Sul. The research was partially funded by the Brazilian Federal Agency for Support and Evaluation of Graduate Education (Capes) – Brazil – Financial Code 001; the Brazilian National Council for Scientific and Technological Development (CNPq); the Instituto de Avaliação de Tecnologia em Saúde (IATS); and the Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS).
• Consent for publication:
  all authors revised the final version of the manuscript and agree with the publication of the results presented.

Availability of data:

dataset that supports the findings of this study are available from the corresponding author, upon reasonable request.

REFERENCES

• ¹ Magliano DJ, Boyko EJ. IDF Diabetes Atlas 10th Edition. Bruxelas: International Diabetes Federation; 2022.
• ² Brazilian Society of Diabetes. Available from: https://diabetes.org.br Accessed on: 2024 Oct 20. doi:10.1080/13696998.2023.2285186
  » https://doi.org/10.1080/13696998.2023.2285186» https://diabetes.org.br
• ³ Sun H, Saeedi P, Karuranga S, Pinkepank M, Ogurtsova K, Duncan BB, et al. IDF Diabetes Atlas: Global, regional and country-level diabetes prevalence estimates for 2021 and projections for 2045. Diabetes Res Clin Pract. 2022;183:109119. doi: 10.1016/j.diabres.2021.109119. Erratum in: Diabetes Res Clin Pract. 2023 Oct;204:110945. doi: 10.1016/j.diabres.2023.110945
  » https://doi.org/10.1016/j.diabres.2021.109119
• ⁴ Telo GH, Cureau FV, De Souza MS, Andrade TS, Copês F, Schaan BD. Prevalence of diabetes in Brazil over time: A systematic review with meta-analysis. Diabetol Metab Syndr. 2016;8⁽¹⁾ doi:10.1186/s13098-016-0181-1
  » https://doi.org/10.1186/s13098-016-0181-1
• ⁵ Alonso-Morán E, Orueta JF, Esteban JI, Axpe JM, González ML, Polanco NT, et al. Multimorbidity in people with type 2 diabetes in the Basque Country (Spain): Prevalence, comorbidity clusters and comparison with other chronic patients. Eur J Intern Med. 2015 Apr;26(3):197-202. doi: 10.1016/j.ejim.2015.02.005
  » https://doi.org/10.1016/j.ejim.2015.02.005
• ⁶ Struijs JN, Baan CA, Schellevis FG, Westert GP, van den Bos GA. Comorbidity in patients with diabetes mellitus: impact on medical health care utilization. BMC Health Serv Res. 2006 Jul 4;6:84
• ⁷ Pereda P, Boarati V, Guidetti B, Duran AC. Direct and Indirect Costs of Diabetes in Brazil in 2016. Ann Glob Health. 2022 Mar 3;88(1):14. doi: 10.5334/aogh.3000
  » https://doi.org/10.5334/aogh.3000
• ⁸ Ellahham S. Artificial Intelligence: The Future for Diabetes Care. Am J Med. 2020 Aug;133(8):895-900. doi: 10.1016/j.amjmed.2020.03.033
  » https://doi.org/10.1016/j.amjmed.2020.03.033
• ⁹ Contreras I, Vehi J. Artificial Intelligence for Diabetes Management and Decision Support: Literature Review. J Med Internet Res. 2018 May 30;20(5):e10775. doi: 10.2196/10775
  » https://doi.org/10.2196/10775
• ¹⁰ Kavakiotis I, Tsave O, Salifoglou A, Maglaveras N, Vlahavas I, Chouvarda I. Machine Learning and Data Mining Methods in Diabetes Research. Comput Struct Biotechnol J. 2017 Jan 8;15:104-16. doi: 10.1016/j. csbj.2016.12.005
  » https://doi.org/10.1016/j. csbj.2016.12.005
• ¹¹ Vijiyakumar K, Lavanya B, Nirmala I, Caroline SS. Random Forest Algorithm for the Prediction of Diabetes. In: Proceeding of International Conference on Systems Computation Automation and Networking . ; 2019.
• ¹² Mujumdar A, Vaidehi V. Diabetes Prediction using Machine Learning Algorithms. In: Procedia Computer Science. Vol 165. Elsevier B.V.; 2019:292-9. doi:10.1016/j.procs.2020.01.047
  » https://doi.org/10.1016/j.procs.2020.01.047
• ¹³ Singla R, Singla A, Gupta Y, Kalra S. Artificial intelligence/machine learning in diabetes care. Indian J Endocrinol Metab. 2019;23(4):495-7. doi:10.4103/ijem.IJEM_228_19
  » https://doi.org/10.4103/ijem.IJEM_228_19
• ¹⁴ Lucini FR, Fogliatto FS, da Silveira GJ, Neyeloff JL, Anzanello MJ, Kuchenbecker RS, et al. Text mining approach to predict hospital admissions using early medical records from the emergency department. Int J Med Inform. 2017 Apr;100:1-8. doi: 10.1016/j. ijmedinf.2017.01.001
  » https://doi.org/10.1016/j. ijmedinf.2017.01.001
• ¹⁵ Regassa LD, Tola A. Magnitude and predictors of hospital admission, readmission, and length of stay among patients with type 2 diabetes at public hospitals of Eastern Ethiopia: a retrospective cohort study. BMC Endocr Disord. 2021;21⁽¹⁾ doi:10.1186/s12902-021-00744-3
  » https://doi.org/10.1186/s12902-021-00744-3
• ¹⁶ Sajjad MA, Holloway KL, de Abreu LLF, Mohebbi M, Kotowicz MA, Pedler D, et al. Comparison of incidence, rate and length of all-cause hospital admissions between adults with normoglycaemia, impaired fasting glucose and diabetes: a retrospective cohort study in Geelong, Australia. BMJ Open. 2018 Mar 23;8(3):e020346. doi: 10.1136/ bmjopen-2017-020346
  » https://doi.org/10.1136/ bmjopen-2017-020346
• ¹⁷ Begum N, Donald M, Ozolins IZ, Dower J. Hospital admissions, emergency department utilisation and patient activation for self-management among people with diabetes. Diabetes Res Clin Pract. 2011;93(2):260-7. doi: 10.1016/j.diabres.2011.05.031
  » https://doi.org/10.1016/j.diabres.2011.05.031
• ¹⁸ Dennis S, Taggart J, Yu H, Jalaludin B, Harris MF, Liaw ST. Linking observational data from general practice, hospital admissions and diabetes clinic databases: can it be used to predict hospital admission? BMC Health Serv Res. 2019;19(1):526. doi: 10.1186/s12913-019-4337-1
  » https://doi.org/10.1186/s12913-019-4337-1
• ¹⁹ Stevens PE, Levin A; Kidney Disease: Improving Global Outcomes Chronic Kidney Disease Guideline Development Work Group Members. Evaluation and management of chronic kidney disease: synopsis of the kidney disease: improving global outcomes 2012 clinical practice guideline. Ann Intern Med. 2013;158(11):825-30. doi: 10.7326/0003-4819-158-11-201306040-00007
  » https://doi.org/10.7326/0003-4819-158-11-201306040-00007
• ²⁰ Incerti J, Zelmanovitz T, Camargo JL, Gross JL, de Azevedo MJ. Evaluation of tests for microalbuminuria screening in patients with diabetes. Nephrol Dial Transplant. 2005;20(11):2402-7. doi: 10.1093/ ndt/gfi074
  » https://doi.org/10.1093/ ndt/gfi074
• ²¹ Zelmanovitz T, Gross JL, Oliveira JR, Paggi A, Tatsch M, Azevedo MJ. The receiver operating characteristics curve in the evaluation of a random urine specimen as a screening test for diabetic nephropathy. Diabetes Care. 1997 Apr;20(4):516-9. doi: 10.2337/diacare.20.4.516
  » https://doi.org/10.2337/diacare.20.4.516
• ²² ElSayed NA, Aleppo G, Aroda VR, Bannuru RR, Brown FM, Bruemmer D, et al., on behalf of the American Diabetes Association. 11. Chronic Kidney Disease and Risk Management: Standards of Care in Diabetes-2023. Diabetes Care. 2023;46(Suppl 1):S191-S202. doi: 10.2337/dc23-S011
  » https://doi.org/10.2337/dc23-S011
• ²³ Alwan JK, Jaafar DS, Ali IR. Diabetes diagnosis system using modified Naive Bayes classifier. Indonesian Journal of Electrical Engineering and Computer Science. 2022;28(3):1766-74. doi:10.11591/ijeecs.v28. i3.pp1766-1774
  » https://doi.org/10.11591/ijeecs.v28. i3.pp1766-1774
• ²⁴ Blume CA, Brust-Renck PG, Rocha MK, Leivas G, Neyeloff JL, Anzanello MJ, et al. Development and Validation of a Predictive Model of Success in Bariatric Surgery. Obes Surg. 2021;31(3):1030-7. doi: 10.1007/ s11695-020-05103-0.
  » https://doi.org/10.1007/ s11695-020-05103-0
• ²⁵ Wu SL, Christian Hospital C, Batur Çolak A. Risk factor identification and prediction models for prolonged length of stay in hospital after acute ischemic stroke using artificial neural networks. Front Neurol. 2023;14(1085178.). http://taiwanstrokeregistry.org/
  » http://taiwanstrokeregistry.org/
• ²⁶ Kumar M, Ang LT, Ho C, Soh SE, Tan KH, Chan JKY, et al. Machine Learning-Derived Prenatal Predictive Risk Model to Guide Intervention and Prevent the Progression of Gestational Diabetes Mellitus to Type 2 Diabetes: Prediction Model Development Study. JMIR Diabetes. 2022 Jul 5;7(3):e32366. doi: 10.2196/32366
  » https://doi.org/10.2196/32366
• ²⁷ Bolón-Canedo V, Rego-Fernández D, Peteiro-Barral D, Alonso-Betanzos A, Guijarro-Berdiñas B, Sánchez-Maroño N. On the scalability of feature selection methods on high-dimensional data. Knowl Inf Syst. 2018;56(2):395-442. doi:10.1007/s10115-017-1140-3
  » https://doi.org/10.1007/s10115-017-1140-3
• ²⁸ Mangalathu S, Hwang SH, Jeon JS. Failure mode and effects analysis of RC members based on machine-learning-based SHapley Additive exPlanations (SHAP) approach. Eng Struct. 2020;219. doi:10.1016/j. engstruct.2020.110927
  » https://doi.org/10.1016/j. engstruct.2020.110927
• ²⁹ Ullah I, Liu K, Yamamoto T, Zahid M, Jamal A. Modeling of machine learning with SHAP approach for electric vehicle charging station choice behavior prediction. Travel Behav Soc. 2023;31:78-92. doi:10.1016/j.tbs.2022.11.006
  » https://doi.org/10.1016/j.tbs.2022.11.006
• ³⁰ Yang C, Chen M, Yuan Q. The application of XGBoost and SHAP to examining the factors in freight truck-related crashes: An exploratory analysis. Accid Anal Prev. 2021;158:106153. doi: 10.1016/j. aap.2021.106153
  » https://doi.org/10.1016/j. aap.2021.106153
• ³¹ Eby E, Hardwick C, Yu M, Gelwicks S, Deschamps K, Xie J, et al. Predictors of 30 day hospital readmission in patients with type 2 diabetes: a retrospective, case-control, database study. Curr Med Res Opin. 2015;31(1):107-14. doi: 10.1185/03007995.2014.981632
  » https://doi.org/10.1185/03007995.2014.981632
• ³² Jun M, Harris K, Heerspink HJ, Badve SV, Jardine MJ, Harrap S, et al; ADVANCE Collaborative Group. Variability in estimated glomerular filtration rate and the risk of major clinical outcomes in diabetes: Post hoc analysis from the ADVANCE trial. Diabetes Obes Metab. 2021;23(6):1420-5. doi: 10.1111/dom.14351
  » https://doi.org/10.1111/dom.14351
• ³³ Weinert LS, Scheffel RS, Severo MD, Cioffi AP, Teló GH, Boschi A, et al. Precipitating factors of diabetic ketoacidosis at a public hospital in a middle-income country. Diabetes Res Clin Pract. 2012 Apr;96(1):29-34. doi: 10.1016/j.diabres.2011.11.006
  » https://doi.org/10.1016/j.diabres.2011.11.006
• ³⁴ da Rosa Carlos Monteiro LE, Garcia SP, Bottino LG, Custodio JL, Telo GH, Schaan BD. Precipitating factors of diabetic ketoacidosis in type 1 diabetes patients at a tertiary hospital: a cross-sectional study with a two-time-period comparison. Arch Endocrinol Metab. 2022;66(3):355-61. doi: 10.20945/2359-3997000000480
  » https://doi.org/10.20945/2359-3997000000480
• ³⁵ Malta DC, Ribeiro EG, Gomes CS, Alves FT, Stopa SR, Sardinha LM, et al. Indicators of the line of care for people with diabetes in Brazil: National Health Survey 2013 and 2019. Epidemiol Serv Saude. 2022;31(spe1):e2021382. doi: 10.1590/SS2237-9622202200011. especial
  » https://doi.org/10.1590/SS2237-9622202200011
• ³⁶ Barsasella D, Bah K, Mishra P, Uddin M, Dhar E, Suryani DL, et al. A Machine Learning Model to Predict Length of Stay and Mortality among Diabetes and Hypertension Inpatients. Medicina (Kaunas). 2022;58(11):1568. doi: 10.3390/medicina58111568
  » https://doi.org/10.3390/medicina58111568
• ³⁷ Alimbayev A, Zhakhina G, Gusmanov A, Sakko Y, Yerdessov S, Arupzhanov I, et al. Predicting 1-year mortality of patients with diabetes mellitus in Kazakhstan based on administrative health data using machine learning. Sci Rep. 2023;13(1):8412. doi: 10.1038/ s41598-023-35551-4
  » https://doi.org/10.1038/ s41598-023-35551-4
• ³⁸ Al Meslamani AZ. Beyond implementation: the long-term economic impact of AI in healthcare. J Med Econ. 2023;26(1):1566-9. doi: 10.1080/13696998.2023.2285186
  » https://doi.org/10.1080/13696998.2023.2285186
• ³⁹ Brisimi TS, Xu T, Wang T, Dai W, Paschalidis IC. Predicting diabetes-related hospitalizations based on electronic health records. Stat Methods Med Res. 2019;28(12):3667-82. doi: 10.1177/0962280218810911
  » https://doi.org/10.1177/0962280218810911
• ⁴⁰ Smith MR, Martinez T, Giraud-Carrier C. An instance level analysis of data complexity. Mach Learn. 2014;95(2):225-256. doi:10.1007/ s10994-013-5422-z
  » https://doi.org/10.1007/ s10994-013-5422-z
• ⁴¹ Hanafy M, Ming R. USING MACHINE LEARNING MODELS TO COMPARE VARIOUS RESAMPLING METHODS IN PREDICTING INSURANCE FRAUD. J Theor Appl Inf Technol. 2021;30(12). http://www.jatit.org
  » http://www.jatit.org
• ⁴² Wei Z, Zhang L, Zhao L. Minority-prediction-probability-based oversampling technique for imbalanced learning. Inf Sci (N Y). 2023;622:1273-95. doi:10.1016/j.ins.2022.11.148
  » https://doi.org/10.1016/j.ins.2022.11.148
• ⁴³ Chawla N V, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: Synthetic Minority Over-Sampling Technique. Vol 16.; 2002.
• ⁴⁴ Chawla N V, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: Synthetic Minority Over-Sampling Technique. Vol 16.; 2002.
• ⁴⁵ Batista GE, Prati RC, Monard MC. A Study of the Behavior of Several Methods for Balancing Machine Learning Training Data. Sigkdd Explorations. 2004;6(1):20-9. doi: https://doi.org/10.1145/1007730.1007735
  » https://doi.org/10.1145/1007730.1007735
• ⁴⁶ Godoy LC, Farkouh ME, Austin PC, Shah BR, Qiu F, Sud M, et al. Predicting left main stenosis in stable ischemic heart disease using logistic regression and boosted trees. Am Heart J. 2023 Feb;256:117-127. doi: 10.1016/j.ahj.2022.11.004
  » https://doi.org/10.1016/j.ahj.2022.11.004
• ⁴⁷ Wu X, Kumar V, Ross QJ, et al. Top 10 algorithms in data mining. Knowl Inf Syst. 2008;14(1):1-37. doi:10.1007/s10115-007-0114-2
  » https://doi.org/10.1007/s10115-007-0114-2
• ⁴⁸ Álvarez-Alvarado JM, Ríos-Moreno JG, Obregón-Biosca SA, Ronquillo-Lomelí G, Ventura-Ramos E, Trejo-Perea M. Hybrid techniques to predict solar radiation using support vector machine and search optimization algorithms: A review. Applied Sciences (Switzerland). 2021;11(3):1-17. doi:10.3390/app11031044
  » https://doi.org/10.3390/app11031044
• ⁴⁹ Grollmuss O, Zhu B, Demetrio P, Zhang H. The predictive value of pressure recording analytical method for the duration of mechanical ventilation in children undergoing cardiac surgery with an XGBoost-based machine learning model. Front Cardiovasc Med. 2022;9.
• ⁵⁰ Samih A, Ghadi A, Fennan A. Enhanced sentiment analysis based on improved word embeddings and XGboost. International Journal of Electrical and Computer Engineering. 2023;13(2):1827-1836. doi:10.11591/ijece.v13i2.pp1827-1836
  » https://doi.org/10.11591/ijece.v13i2.pp1827-1836
• ⁵¹ Bbeiman L. Bagging Predictors. 1996 [cited 2025 Feb 4];24:123-140. Available from: https://link.springer.com/article/10.1007/BF00058655
  » https://link.springer.com/article/10.1007/BF00058655

Publication Dates

• Publication in this collection
  02 May 2025
• Date of issue
  2025

History

• Received
  02 Aug 2024
• Accepted
  05 Dec 2024