Context-Aware Extractive Summarization Using Pretrained Models for Long Documents - Abdulkareem, Sammer; Feizi-Derakhshi, Mohammad-Reza (Informatica, 2026)
Recently, significant advancements have been made in models for automatic text summarization, primarily because of the emergence of modern technologies such as transformers and pre-training. However, handling long documents remains a major challenge for most models. Current research addresses this challenge either by designing recurring chained models or by truncating the text to fit the input size. Insufficient attention has been given to preserving the context of the original text and its role in summary generation. We propose a model that captures consecutive sentence contexts and processes them as coherent units. Using an unsupervised summarization pipeline that requires no task-specific fine-tuning, the model produces extractive summaries composed of the most informative sentences from the source document. We evaluate the proposed pipeline on the PubMed and ArXiv long-document benchmarks using ROUGE, achieving ROUGE-1/ROUGE-2/ROUGE-L of 53.0/24.32/47.7 on PubMed and 53.7/20.40/48.4 on ArXiv.