Large-Sized Network: Independent AV agents

In this tutorial we use a big sized-network for agents navigation. The chosen origin and destination points are specified in this file, and can be adjusted by users. In parallel, we implement the learning process of the automated vehicles (AVs) using the TorchRL library.

---

Network Overview

In these notebooks, we utilize the Ingolstadt network within our simulator, SUMO. Since agents exhibit selfish behavior, we employ independent learning algorithms to model their decision-making.

Users can customize parameters for the TrafficEnvironment class by consulting the routerl/environment/params.json file. Based on its contents, they can create a dictionary with their preferred settings and pass it as an argument to the TrafficEnvironment class.

Included Tutorials:

• IQL Tutorial. Uses Independent Q-Learning (IQL) (IQL) as an initial baseline for training decentralized policies.

• IPPO Tutorial. Implements Independent Proximal Policy Optimization (IPPO) (IPPO), which has demonstrated strong benchmark performance in various tasks (paper1, paper2).

• ISAC Tutorial. Uses Independent SAC (ISAC), the multi-agent extension of Soft Actor-Critic (SAC) (SAC), which balances exploration and exploitation using entropy-regularized reinforcement learning.

---

Ingolstadt Network Visualization