Music Genre Classification via Time-Frequency Dual-Stream Neural Networks and SimCLR-based Self-Supervised Learning

Abstract

Against the backdrop of explosive growth in digital music data, traditional music classification methods suffer from high cost of manual feature extraction and poor generalization, while existing deep learning methods lack optimization of music time-frequency two-dimensional features and face the challenge of high cost of large-scale data annotation. This study addresses four core research questions: how to design a time-frequency dual stream network (using two layers of LSTM to capture rhythm dynamics for the time stream and two 5 × 5 convolutional layers+two sampling layers to extract timbre harmonic features for the frequency stream) and an effective feature fusion strategy to improve the classification accuracy of complex music; Which music specific data augmentation strategies and hyperparameter optimization enhance the generalization of SimCLR contrastive learning in unlabeled data scenarios; There are differences between these two methods in terms of adapting to data volume, genre complexity, and annotation constraints when executing across datasets (small-scale tagging GTZAN and large-scale MSD) (GTZAN outperforms SimCLR in terms of time-frequency collaboration, while SimCLR slightly outperforms MSD with no significant difference between the two). Its key indicators include classification accuracy, recall, and F-value (for example, time-frequency dual stream achieves 82.4% accuracy, 81.7% recall, and 82.0% F-value on GTZAN, with the best accuracy of 86.5% for pop music classification; SimCLR achieved an accuracy of 79.5%, a recall of 78.8%, and an F-value of 79.1% on MSD, and designed a time-frequency dual stream model with two layers of LSTM (time stream), two convolutional layers+two sampling layers (frequency stream), and an intermediate fusion module; SimCLR with data augmentation (time stretching, pitch adjustment, random cropping, reverberation, etc.), CNN encoder, and InfoNCE loss function is used to verify their effectiveness in music classification through 5-fold cross validation. This scheme complements each other's advantages and provides technical support for music classification and related applications.