CTMOT: A CNN-Transformer Framework for Real-Time Multi-Ob-ject Tracking

Abstract

With the increasing demand for intelligent visual surveillance and autonomous systems, multi-object tracking (MOT) has become a critical research focus. To address challenges in identity preservation and real-time inference, this paper proposes CTMOT, a novel tracking framework that fuses convolutional neural networks (CNN) and vision Transformers via a Two-Way Bridge Module (TBM) for joint detection and tracking. The model features a dual-branch CNN-Transformer backbone and a parallel decoder de-sign with distinct object and track queries, enabling robust appearance modeling and temporal continuity. The TBM introduces grouped bidirectional attention to facilitate local–global feature fusion. Experi-mental results show that CTMOT achieves a MOTA of 76.4 and an IDF1 of 71.3 on the MOT17 dataset, and 66.3/67.1 respectively on MOT20, outperforming several state-of-the-art trackers. On the KITTI and UA-DETRAC vehicle benchmarks, CTMOT reaches 92.36 and 88.57 MOTA, while maintaining real-time speed at 35 FPS on an RTX 3090 GPU. Ablation studies confirm the effectiveness of the TBM design and the contribution of temporal query persistence, which reduces ID switches by 12.5%. These results demon-strate the potential of CTMOT as a reliable and efficient solution for dense and dynamic tracking scenar-ios.