Context-Enriched Dynamic Graph Word Embeddings for Robust NLP Applications

Abstract

Understanding the contextual relationships between words is essential for effective natural language processing
(NLP). Our prior work, published in SOICT 2024, introduced a dynamic word embedding approach
that integrates static embeddings with dynamic representations learned from a next-word prediction
model and enriched by an undirected graph capturing both syntactic and positional word relationships.
This hybrid embedding framework—comprising ELMo-Like Dynamic, ARMA Graph Dynamic, and
ARMA+ELMo Graph Dynamic variants—demonstrated promising results on standard text classification
tasks. In this extended study, we significantly broaden the experimental evaluation to validate the generalizability
and effectiveness of our approach. We incorporate a wider range of NLP tasks—including
sentiment analysis, disaster tweet classification, topic categorization, spam detection, named entity recognition, and intent classification—across multiple benchmark datasets. Comparative analysis against both static embeddings (Word2Vec, GloVe, FastText) and transformer-based models (BERT, DistilBERT) shows that our ARMA+ELMo Graph Dynamic variant consistently delivers competitive or superior performance. Notably, our method achieves a classification accuracy of 93.2% on the AG News topic classification task and an F1-score of 94.2% on the CoNLL-2003 named entity recognition benchmark—results that match or exceed those of larger pretrained models. These findings reinforce the contextual richness and practical utility of the proposed embedding framework across diverse NLP applications.