This is a list of projects using stable-baselines3. Please tell us, if you want your project to appear on this page ;)

RL Reach

A platform for running reproducible reinforcement learning experiments for customisable robotic reaching tasks. This self-contained and straightforward toolbox allows its users to quickly investigate and identify optimal training configurations.

Authors: Pierre Aumjaud, David McAuliffe, Francisco Javier Rodríguez Lera, Philip Cardiff

Generalized State Dependent Exploration for Deep Reinforcement Learning in Robotics

An exploration method to train RL agent directly on real robots. It was the starting point of Stable-Baselines3.


A solution to the second project of the Udacity deep reinforcement learning course. It is an example of:

  • wrapping single and multi-agent Unity environments to make them usable in Stable-Baselines3

  • creating experimentation scripts which train and run A2C, PPO, TD3 and SAC models (a better choice for this one is

  • generating several pre-trained models which solve the reacher environment


A simple interface to instantiate RL environments with SUMO for Traffic Signal Control.

  • Supports Multiagent RL

  • Compatibility with gym.Env and popular RL libraries such as stable-baselines3 and RLlib

  • Easy customisation: state and reward definitions are easily modifiable

Author: Lucas Alegre


PyBullet Gym environments for single and multi-agent reinforcement learning of quadcopter control.

  • Physics-based simulation for the development and test of quadcopter control.

  • Compatibility with gym.Env, RLlib’s MultiAgentEnv.

  • Learning and testing script templates for stable-baselines3 and RLlib.


SuperSuit contains easy to use wrappers for Gym (and multi-agent PettingZoo) environments to do all forms of common preprocessing (frame stacking, converting graphical observations to greyscale, max-and-skip for Atari, etc.). It also notably includes:

-Wrappers that apply lambda functions to observations, actions, or rewards with a single line of code. -All wrappers can be used natively on vector environments, wrappers exist to Gym environments to vectorized environments and concatenate multiple vector environments together -A wrapper is included that allows for using regular single agent RL libraries (e.g. stable baselines) to learn simple multi-agent PettingZoo environments, explained in this tutorial: