Stock market decision support modeling with tree-based AdaBoost ensemble machine learning models
Forecasting stock market behavior has received tremendous attention from investors, and researchers for a very long time due to its potential profitability. Predicting stock market behavior is regarded as one of the extremely challenging applications of time series forecasting. While there is divided opinion on the efficiency of markets, numerous empirical studies which are widely accepted have shown that the stock market is predictable to some extent. Statistical based methods and machine learning models are used to forecast and analyze the stock market. Machine learning (ML) models typically perform better than those of statistical and econometric models. In addition, performance of ensemble ML models is typically superior to those of individual ML models. In this paper, we study and compare the efficiency of tree-based AdaBoost ensemble ML models (namely, AdaBoost-DecisionTree (Ada-DT), AdaBoost-RandomForest (Ada-RF), AdaBoost-Bagging (Ada-BAG), and Bagging-ExtraTrees (Bag-ET)). Ten stock data sets randomly collected from three different stock exchanges (NYSE, NASDAQ, and NSE) are used for the study. Forty technical indicators are computed and used as input features. The performance of the models is evaluated using accuracy, precision, recall, F-measure, specificity. And AUC metrics. Also, Kendall W test of concordance is used to rank the performance of the different models. The experimental results show that AdaBoost- ExtraTree (Ada-ET) model is the highest performer among the tree-based AdaBoost ensemble models studied.
Abu-Mostafa, Y. S., & Atiya, A. F. (1996). Introduction to financial forecasting. Applied Intelligence, 6(3), 205–213.
Alkhatib, K., Najadat, H., Hmeidi, I., & Shatnawi, M. K. A. (2013). Stock price predic- tion using k-nearest neighbor (knn) algorithm. International Journal of Business, Humanities and Technology, 3 (3), 32–44.
Araújo, R. d. A., Oliveira, A. L., & Meira, S. (2015). A hybrid model for high-frequency stock market forecasting. Expert Systems with Applications, 42 (8), 4081–4096.
Bacchetta, P., Mertens, E., & Van Wincoop, E. (2009). Predictability in financial mar- kets: What do survey expectations tell us? Journal of International Money and Finance, 28 (3), 406–426.
Ballings, M., Van den Poel, D., Hespeels, N., & Gryp, R. (2015). Evaluating multiple classifiers for stock price direction prediction. Expert Systems with Applications, 42 (20), 7046–7056.
Bergstra J. S., Bardenet, R., Bengio, Y., Ke̗gl, B. (2011). Algorithms for hyperparameter optimization. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 24, 2546–2554.
Bollerslev, T., Marrone, J., Xu, L., & Zhou, H. (2014). Stock return predictability and variance risk premia: Statistical inference and international evidence. Journal of Financial and Quantitative Analysis, 49 (03), 633–661.
Booth, A., Gerding, E., & Mcgroarty, F. (2014). Automated trading with performance weighted random forests and seasonality. Expert Systems with Applications, 41 (8), 3651–3661.
Breiman, L. (1996). Bagging predictors. Mach Learn 24, 123–140.
Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.
Campbell, J. Y., & Hamao, Y. (1992). Predictable stock returns in the united states and japan: A study of long-term capital market integration. The Journal of Finance, 47 (1), 43–69.
Chen, A.-S., Leung, M. T., & Daouk, H. (2003). Application of neural networks to an emerging financial market: Forecasting and trading the taiwan stock index. Computers & Operations Research, 30 (6), 901–923.
Chen, Y., Yang, B., & Abraham, A. (2007). Flexible neural trees ensemble for stock index modeling. Neurocomputing, 70 (4), 697–703.
Chong, E., Han, C., & Park, F. C. (2017). Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies. Expert Systems with Applications, 83, 187–205.
Enke, D., & Mehdiyev, N. (2013). Stock market prediction using a combination of stepwise regression analysis, differential evolution-based fuzzy clustering, and a fuzzy inference neural network. Intelligent Automation & Soft Computing, 19 (4), 636–648.
Feurer M., Hutter F. (2019) Hyperparameter Optimization. In: Hutter F., Kotthoff L., Vanschoren J. (eds) Automated Machine Learning. The Springer Series on Challenges in Machine Learning. Springer, Cham
Feuerriegel, S., & Gordon, J. (2018). Long-term stock index forecasting based on text mining of regulatory disclosures. Decision Support Systems 112: 88–97
Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. In machine learning. proceedings of the thirteenth international conference (ICML ’96). 148–156. Bari, Italy.
Geurts P., Ernst, D., Wehenkel L. (2006). Extremely randomized trees, Mach Learn (2005) 63: 3–42
Guresen, E., Kayakutlu, G., & Daim, T. U. (2011). Using artificial neural network models in stock market index prediction. Expert Systems with Applications, 38 (8), 10389–10397.
Granger, C. W. J., & Morgenstern, O. (1970). Predictability of stock market prices: 1. DC Heath Lexington, Mass.
Hassan, M. R., Nath, B., & Kirley, M. (2007). A fusion model of hmm, ann and ga for stock market forecasting. Expert Systems with Applications, 33 (1), 171–180.
Hsu, M.-W., Lessmann, S., Sung, M.-C., Ma, T., & Johnson, J. E. (2016). Bridging the di- vide in financial market forecasting: Machine learners vs. financial economists. Expert Systems with Applications, 61, 215–234.
Huang, C.-J., Yang, D.-X., & Chuang, Y.-T. (2008). Application of wrapper approach and composite classifier to the stock trend prediction. Expert Systems with Applications, 34(4), 2870–2878.
Jolliffe, I., T. (2002). Principal Component Analysis, Second Edition. New York: Springer.
Kim, J. H., Shamsuddin, A., & Lim, K. P. (2011). Stock return predictability and the adaptive markets hypothesis: Evidence from century-long us data. Journal of Empirical Finance, 18 (5), 868–879.
Khansa, L., & Liginlal, D. (2011). Predicting stock market returns from malicious attacks: A comparative analysis of vector autoregression and time-delayed neural networks. Decision Support Systems, 51 (4), 745–759
Lin X., Yang Z., & Song Y. (2009). Short-term stock price prediction based on echo state networks. Expert Systems with Applications, 36: 7313–7317.
Meesad, P., & Rasel, R. I. (2013). Predicting stock market price using support vector regression. In Informatics, electronics & vision (iciev), 2013 international confer- ence on (pp. 1–6). IEEE.
Nayak, A., Pai M., M., M., & Pai R., M. (2016). Prediction Models for Indian Stock Market. Twelfth International Multi-Conference on Information Processing-2016 (IMCIP-2016). Procedia Computer Science 89 441 – 449.
Patel, J., Shah, S., Thakkar, P., & Kotecha, K. (2015a). Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Expert Systems with Applications, 42 (1), 259–268.
Patel, J., Shah, S., Thakkar, P., & Kotecha, K. (2015b). Predicting stock market index using fusion of machine learning techniques. Expert Systems with Applications, 42 (4), 2162–2172.
Phan, D. H. B., Sharma, S. S., & Narayan, P. K. (2015). Stock return forecasting: Some new evidence. International Review of Financial Analysis, 40, 38–51.
Rajashree D., & Pradipta K., D. (2016). A hybrid stock trading framework integrating technical analysis with machine learning techniques. The Journal of Finance and Data Science, 2: 42-57.
Rather, A. M., Agarwal, A., & Sastry, V. (2015). Recurrent neural network and a hybrid model for prediction of stock returns. Expert Systems with Applications, 42 (6), 3234–3241.
Rokach, L., & Maimon, O. Z. (2008). Data mining with decision trees: theory and applications (Vol. 69). World scientific
Tan, T. Z., Quek, C., & See, Ng. G. (2007). Biological brain-inspired genetic complementary learning for stock market and bank failure prediction. Computational Intelligence, 23(2), 236–261.
Thawornwong, S., & Enke, D. (2004). The adaptive selection of financial and economic variables for use with artificial neural networks. Neurocomputing, 56, 205–232.
Tsai, C.-F., Lin, Y.-C., Yen, D. C., & Chen, Y.-M. (2011). Predicting stock returns by classifier ensembles. Applied Soft Computing, 11 (2), 2452–2459.
Tsai, C.-F., & Hsiao, Y.-C. (2010). Combining multiple feature selection methods for stock prediction: Union, intersection, and multi-intersection approaches. Decision Support Systems, 50 (1), 258–269.
Wang, J.-J., Wang, J.-Z., Zhang, Z.-G., & Guo, S.-P. (2012). Stock index forecasting based on a hybrid model. Omega, 40 (6), 758–766.
Wang, J.-Z., Wang, J.-J., Zhang, Z.-G., & Guo, S.-P. (2011). Forecasting stock indices with back propagation neural network. Expert Systems with Applications, 38 (11), 14346–14355.
Wang, L., Zeng, Y., & Chen, T. (2015). Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications, 42 (2), 855–863.
Weng, B., Lu L., Wang, X., Megahed, F., M., Martinez, W. (2018). Predicting short-term stock prices using ensemble methods and online data sources. Expert Systems with Applications, 112: 258–273.
Zhang, Y., & Wu, L. (2009). Stock market prediction of s&p 500 via combination of improved bco approach and bp neural network. Expert Systems with Applications, 36 (5), 8849–8854.
This work is licensed under a Creative Commons Attribution 3.0 License.