Transformer-Augmented Deep Reinforcement Learning for MultiObjective Path Co-Optimization in Same-Day Delivery with Hybrid Fleets

Abstract

The rapid expansion of e-commerce and local services has intensified the demand for instant delivery, particularly same-day delivery (SDD), presenting significant challenges in operational efficiency and multi-objective cost management. Traditional logistics systems often struggle with dynamic order arrivals, complex routing decisions, and the effective integration of hybrid fleets comprising both dedicated and crowdsourced couriers. This paper investigates the multi-objective path co-optimization problem within logistics hyper-automation, proposing an innovative Transformer-Augmented Policy Optimization (TAPO) Double Layers Optimization Framework. This framework addresses the dynamic dispatching and routing challenges in SDD scenarios. The upper layer employs the TAPO agent, where a Transformer model processes complex spatio-temporal dependencies from dynamic order data to enrich state representations for a Deep Reinforcement Learning (DRL) agent (based on Proximal Policy Optimization - PPO). This TAPO agent learns a sophisticated policy for synergistic dispatch-delay decisions. The lower layer utilizes efficient heuristic algorithms (GCIH and VNS) for static vehicle routing and order assignment when a dispatch action is chosen. The primary goal is to co-optimize multiple objectives, focusing on minimizing total fulfillment costs (encompassing mileage and delay penalties) while enhancing service efficiency. A Markov Decision Process (MDP) models the sequential decision-making problem. Numerical experiments on diverse, realistic instances demonstrate the TAPO framework's superiority. Results show that the TAPO framework achieves an average total cost reduction of approximately 5-11% compared to a Myopic policy and 3-12% against an Urgent-Based Policy (UBP). The framework also exhibits robust generalization to varying order volumes and sensitivity to fleet composition changes, underscoring the significant potential of advanced AI techniques in achieving logistics hyper-automation.