Intelligent System for Automatic Recognition of Environmental Sounds Using Optimal Feature Fusion and Ensemble Deep Learning Technique
Abstract
Environmental sound classification (ESC) is a challenging task due to the unstructured and overlapping nature of ambient sounds, which differ significantly from speech and music. Problems such as class imbalance, limited labeled samples, and high inter-class similarity hinder the performance of traditional classifiers. In this study, we propose a robust ESC system that combines optimal spectrum feature fusion with a stacked ensemble learning strategy. Specifically, we extract three types of spectral features—log Mel spectrum, log–log Mel spectrum, and Mel spectrograms—from environmental audio signals using the DenseNet-161 architecture. These features are then optimally fused using the Boosted Reptile Squirrel Search (BRSS) algorithm to capture both fine- and coarse-grained frequency patterns. For classification, we employ a two-level ensemble model: four classical machine learning classifiers (Linear Regression, Decision Tree, Random Forest, and Support Vector Machine) in the first stage, followed by a Bayesian Tensorized Neural Network (BTNN) for final prediction. Experimental results on three benchmark datasets—ESC-10, ESC-50, and UrbanSound8K—demonstrate that our fused spectrum feature approach achieves an accuracy of 98.98%, surpassing individual feature types and outperforming state-of-the-art models such as Convolutional Recurrent Neural Network (CRNN), EnvNet, and DualResNet. These results highlight the effectiveness and superiority of our proposed method for environmental sound classification.References
References
Mirbeygi, M., Mahabadi, A., &Ranjbar, A. (2022). Speech and music separation approaches-a survey. Multimedia Tools and Applications, 81(15), 21155-21197.
Kheddar, H., Hemis, M., &Himeur, Y. (2024). Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey. arXiv preprint arXiv:2403.01255.
Zaheer, R., Ahmad, I., Habibi, D., Islam, K. Y., &Phung, Q. V. (2023). A survey on artificial intelligence-based acoustic source identification. IEEE Access.
Chen, Z., Xie, G., Chen, M., &Qiu, H. (2023). Model for Underwater Acoustic Target Recognition with Attention Mechanism Based on Residual Concatenate. Journal of Marine Science and Engineering, 12(1), 24.
Saini, H., Srinivasan, N., Sedajova, V., Majumder, M., Dubal, D. P., Otyepka, M., ...&Jayaramulu, K. (2021). Emerging MXene@ Metal–organic framework hybrids: design strategies toward versatile applications. ACS nano, 15(12), 18742-18776.
Verma, V., Benjwal, A., Chhabra, A., Singh, S. K., Kumar, S., Gupta, B. B., ...& Chui, K. T. (2023). A novel hybrid model integrating MFCC and acoustic parameters for voice disorder detection. Scientific Reports, 13(1), 22719.
Zaman, K., Sah, M., Direkoglu, C., &Unoki, M. (2023). A Survey of Audio Classification Using Deep Learning. IEEE Access.
Siddique, M. F., Ahmad, Z., Ullah, N., & Kim, J. (2023). A Hybrid Deep Learning Approach: Integrating Short-Time Fourier Transform and Continuous Wavelet Transform for Improved Pipeline Leak Detection. Sensors, 23(19), 8079.
Tanveer, M., Rastogi, A., Paliwal, V., Ganaie, M. A., Malik, A. K., Del Ser, J., & Lin, C. T. (2023). Ensemble deep learning in speech signal tasks: A review. Neurocomputing, 126436.
Navarro, J. M., & Pita, A. (2023). Machine Learning Prediction of the Long-Term Environmental Acoustic Pattern of a City Location Using Short-Term Sound Pressure Level Measurements. Applied Sciences, 13(3), 1613.
Mehrish, A., Majumder, N., Bharadwaj, R., Mihalcea, R., &Poria, S. (2023). A review of deep learning techniques for speech processing. Information Fusion, 101869.
Bansal, A., &Garg, N. K. (2023). Robust technique for environmental sound classification using convolutional recurrent neural network. Multimedia Tools and Applications, 1-18.
Coelho, J. S., Machado, M. R., Dutkiewicz, M., & O. Teloli, R. (2024). Data-driven machine learning for pattern recognition and detection of loosening torque in bolted joints. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 46(2), 75.
Mou, A., &Milanova, M. (2024). Performance Analysis of Deep Learning Model-Compression Techniques for Audio Classification on Edge Devices. Sci, 6(2), 21.
Sharma, S., Sato, K., &Gautam, B. P. (2023). A Methodological Literature Review of Acoustic Wildlife Monitoring Using Artificial Intelligence Tools and Techniques. Sustainability, 15(9), 7128.
Balasubramanian, S. B., Balaji, P., Munshi, A., Almukadi, W., Prabhu, T. N., Venkatachalam, K., &Abouhawwash, M. (2023). Machine learning based IoT system for secure traffic management and accident detection in smart cities. PeerJ Computer Science, 9, e1259.
Rani, S., &Srivastava, G. (2024). Secure hierarchical fog computing-based architecture for industry 5.0 using an attribute-based encryption scheme. Expert Systems with Applications, 235, 121180.
Wazirali, R., Yaghoubi, E., Abujazar, M. S. S., Ahmad, R., &Vakili, A. H. (2023). State-of-the-art review on energy and load forecasting in microgrids using artificial neural networks, machine learning, and deep learning techniques. Electric power systems research, 225, 109792.
Moreno, R., Bianco, F., Carpita, S., Monticelli, A., Fredianelli, L., &Licitra, G. (2023). Adjusted controlled pass-by (CPB) method for urban road traffic noise assessment. Sustainability, 15(6), 5340.
Cao, K., Zhang, T., & Huang, J. (2024). Advanced hybrid LSTM-transformer architecture for real-time multi-task prediction in engineering systems. Scientific Reports, 14(1), 4890.
Yildirim, M., Kiziloluk, S., Aslan, S., &Sert, E. (2024). A new hybrid approach based on AOA, CNN and feature fusion that can automatically diagnose Parkinson's disease from sound signals: PDD-AOA-CNN. Signal, Image and Video Processing, 18(2), 1227-1240.
Mekruksavanich, S., &Jitpattanakul, A. (2023). Hybrid convolution neural network with channel attention mechanism for sensor-based human activity recognition. Scientific Reports, 13(1), 12067.
Ansari, S., Alnajjar, K. A., Khater, T., Mahmoud, S., &Hussain, A. (2023). A robust hybrid neural network architecture for blind source separation of speech signals exploiting deep learning. IEEE Access.
Wang, X., Wang, Y., Liu, D., Wang, Y., & Wang, Z. (2023). Automated recognition of epilepsy from EEG signals using a combining space–time algorithm of CNN-LSTM. Scientific Reports, 13(1), 14876.
Rashmi, P., & Singh, M. P. (2023). Convolution neural networks with hybrid feature extraction methods for classification of voice sound signals. World Journal of Advanced Engineering Technology and Sciences, 8(2), 110-125.
Jahangir, R. (2023). CNN‐SCNet: A CNN net‐based deep learning framework for infant cry detection in household setting. Engineering Reports, e12786.
Ullo, S. L., Khare, S. K., Bajaj, V., &Sinha, G. R. (2020). Hybrid computerized method for environmental sound classification. IEEE Access, 8, 124055-124065.
Liu, S., Fu, X., Xu, H., Zhang, J., Zhang, A., Zhou, Q., & Zhang, H. (2023). A fine-grained ship-radiated noise recognition system using deep hybrid neural networks with multi-scale features. Remote Sensing, 15(8), 2068.
Chen, X., Yang, Q., Wu, J., Li, H., & Tan, K. C. (2024). A hybrid neural coding approach for pattern recognition with spiking neural networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 46(05), 3064-3078.
Demir, F., Abdullah, D. A., &Sengur, A. (2020). A new deep CNN model for environmental sound classification. IEEE Access, 8, 66529-66537.
Zhang, D., Zhong, Z., Xia, Y., Wang, Z. and Xiong, W., 2023. An Automatic Classification System for Environmental Sound in Smart Cities. Sensors, 23(15), p.6823.
Piczak, K.J., 2015, September. Environmental sound classification with convolutional neural networks. In 2015 IEEE 25th international workshop on machine learning for signal processing (MLSP) (pp. 1-6). IEEE.
Chen, Y., Guo, Q., Liang, X., Wang, J. and Qian, Y., 2019. Environmental sound classification with dilated convolutions. Applied Acoustics, 148, pp.123-132.
Tokozume, Y. and Harada, T., 2017, March. Learning environmental sounds with end-to-end convolutional neural network. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 2721-2725). IEEE.
Hojjati, H. and Armanfard, N., 2022, May. Self-supervised acoustic anomaly detection via contrastive learning. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 3253-3257). IEEE.
Sang, J., Park, S. and Lee, J., 2018, September. Convolutional recurrent neural networks for urban sound classification using raw waveforms. In 2018 26th European Signal Processing Conference (EUSIPCO) (pp. 2444-2448). IEEE.
DOI:
https://doi.org/10.31449/inf.v49i36.8805Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
