Transformer-Based Fake News Classification: Evaluation of DistilBERT With CNN-LSTM and GloVe Embedding - Chen, Yin; Yin, Bingqi (Informatica, 2025)
Social media networks have changed the face of communication in recent times, but at the same time, they have brought various challenges, such as disseminating fake news and information. NLP and machinelearning algorithms try to meet these challenges by structuring online information, although dataset bias remains a critical concern. SA has helped people gain insight into the context of news dissemination. Still, sham news dissemination-often by fake accounts represents a great hazard not only to users but to the stability of society. Several researchers have tried to assess the credibility of information and reduce sham data flow. In this work, the datasets of 17,903 fake news and 20,826 real news from Kaggle will be used. Preprocessing steps included text normalization, removal of punctuation, links, usernames, and nonalphabetic characters to prepare the data for analysis. This study explored categorizing fake and true news using advanced NLP techniques, transformer-based architectures, and deep learning models. A focus on improving classification accuracy and addressing dataset bias was achieved through models like DistilBERT, CNN, and LSTM. DistilBERT demonstrated remarkable performance, achieving an accuracy of 99.65%, with precision, recall, F1-score, and ROC-AUC values of 0.992188, 1, 0.996078, and 0.996894, respectively, outperforming the other models. The study’s novelty lies in its detailed evaluation of DistilBERT, which showed significant improvements in accuracy, recall, and AUC while mitigating dataset bias. The results highlight the potential of DistilBERT for robust and reliable fake news classification, addressing critical limitations in existing approaches.