T-BiLSTM-CCO: A Transformer-BiLSTM framework with Cuckoo Catfish Optimization for multilingual image description localization

Abstract

Image description localization benefits from integrating textual and visual cues. However, conventional neural machine translation models often overlook fine-grained grounding. This research introduces aNeural Machine Translation Model (NMT) named Transformer integrated Bidirectional Long Short-Term Memory Network-tuned with Cuckoo catfish optimizer (TBiLSTM-CCO) to enhance translation accuracy while preserving image-caption semantic congruence in multilingual captioning. Multilingual image-caption pairs were gathered from the Multi30 and MS-COCO datasets, each containing aligned captions in multiple languages. Visual features are extracted using ResNet, producing high-dimensional semantic embeddings representing objects and scenes. The proposed hybrid architecture combines the CCO with a Transformer-enhanced Bi-LSTM network. This configuration enhances sequence modeling and attention capabilities, optimally tuning parameters to improve translation fidelity, multimodal feature fusion, and localized visual alignment during description generation. The proposed T-Bi-LSTM-CCO achieves higher BLEU-1 to BLEU-4 scores of 0.90, 0.89,0.87, and 0.85 values than the existing methods. Python was used to conduct experiments on the MS-COCO and Multi30k datasets, utilizing Word2Vec for text embeddings and ResNet-152 for visual features. The CCO was trained for over 50 epochs. The hybrid T-Bi-LSTM-CCO-model consistently achieved better alignment between descriptions and image regions, validating its effectiveness for multimodal translation and grounding tasks. The multimodal NMT framework combines Deep Learning (DL) and visual features, improving translation quality and image-description localization, showing robustness for real-world multilingual applications.