Highway merging control using multi-agent reinforcement learning: exploring centralized and decentralized schemes

Abstract

This dissertation addresses critical challenges in autonomous vehicle (AV) control, focusing on complex highway merging control during lane reduction using multi-agent reinforcement learning (MARL). Both centralized and decentralized approaches are presented. For the centralized approach, Proximal Policy Optimization based approach is employed to learn optimal merging policies for a fixed number of vehicles in two platoons. To scale up for the large number of AVs in a typical highway scenario, a decentralized approach is investigated, where each AV acts independently based on local observations. Furthermore, as the number of AVs in a traffic flow can vary, a self-attention network is used to handle the varying number of AVs. Several reward functions are explored and compared, including global speed, local speed, fuel consumption, and ride comfort. Novel quantitative metrics are introduced to evaluate the fairness and efficiency of the learned merging strategies. Both proposed MARL approaches consistently outperform benchmark RL method and a rule-based zipper merge strategy across various metrics, including up to 60.14% increase in traffic flow at higher speeds, among many other advantages. Finally, the generalizability of the framework is demonstrated by training the MARL model using low speed scenario and testing the learned policy using high speed scenario.