Exp-Bagging: A Validation-Based Exponential Weighting Strategy for Decision Tree Ensembles

Abstract

Decision tree ensemble methods significantly enhance model generalization by integrating multiple base learners, with wide applications in medical diagnosis and financial risk control. However, traditional Bagging and its variants (e.g., Random Forest, RF) use static equal voting, failing to distinguish contributions of high-accuracy and low-quality subtrees and adapt to concept drift or high-dimensional sparse data. This study aims to develop a dynamic weighted ensemble framework to boost computational efficiency while maintaining classification accuracy. We propose the Exp-Bagging method, which adopts an exponential decay weighting mechanism based on validation set performance. It quantifies subtree classification errors via Mean Squared Error (MSE) and dynamically assigns voting weights—smaller MSE corresponds to higher weight. Theoretically, we analyze weight convergence and its role in optimizing generalization error bounds. Experiments were conducted on Climate Model Simulation Crashes, WDBC, and Diabetes datasets, using Accuracy, Precision, Recall, and F1-score for evaluation, with comparisons to RF and AdaBoost. Results show: Exp-Bagging improved F1-score by 6.5% and Accuracy by 5.3% on WDBC compared to RF; its throughput was 2.75 – 3.42 times that of traditional RF. This confirms dynamic weighting’s effectiveness in balancing efficiency and accuracy, providing a basis for real-time ensemble learning deployment.