Stable-Baselines3 requires python 3.6+ and PyTorch >= 1.8.1.

Windows 10

We recommend using Anaconda for Windows users for easier installation of Python packages and required libraries. You need an environment with Python version 3.6 or above.

For a quick start you can move straight to installing Stable-Baselines3 in the next step.


Trying to create Atari environments may result to vague errors related to missing DLL files and modules. This is an issue with atari-py package. See this discussion for more information.

Stable Release

To install Stable Baselines3 with pip, execute:

pip install stable-baselines3[extra]


Some shells such as Zsh require quotation marks around brackets, i.e. pip install 'stable-baselines3[extra]' More information.

This includes an optional dependencies like Tensorboard, OpenCV or atari-py to train on atari games. If you do not need those, you can use:

pip install stable-baselines3


If you need to work with OpenCV on a machine without a X-server (for instance inside a docker image), you will need to install opencv-python-headless, see issue #298.

Bleeding-edge version

pip install git+

Development version

To contribute to Stable-Baselines3, with support for running tests and building the documentation.

git clone && cd stable-baselines3
pip install -e .[docs,tests,extra]

Using Docker Images

If you are looking for docker images with stable-baselines already installed in it, we recommend using images from RL Baselines3 Zoo.

Otherwise, the following images contained all the dependencies for stable-baselines3 but not the stable-baselines3 package itself. They are made for development.

Use Built Images

GPU image (requires nvidia-docker):

docker pull stablebaselines/stable-baselines3

CPU only:

docker pull stablebaselines/stable-baselines3-cpu

Build the Docker Images

Build GPU image (with nvidia-docker):

make docker-gpu

Build CPU image:

make docker-cpu

Note: if you are using a proxy, you need to pass extra params during build and do some tweaks:

--network=host --build-arg HTTP_PROXY= --build-arg http_proxy= --build-arg HTTPS_PROXY= --build-arg https_proxy=

Run the images (CPU/GPU)

Run the nvidia-docker GPU image

docker run -it --runtime=nvidia --rm --network host --ipc=host --name test --mount src="$(pwd)",target=/root/code/stable-baselines3,type=bind stablebaselines/stable-baselines3 bash -c 'cd /root/code/stable-baselines3/ && pytest tests/'

Or, with the shell file:

./scripts/ pytest tests/

Run the docker CPU image

docker run -it --rm --network host --ipc=host --name test --mount src="$(pwd)",target=/root/code/stable-baselines3,type=bind stablebaselines/stable-baselines3-cpu bash -c 'cd /root/code/stable-baselines3/ && pytest tests/'

Or, with the shell file:

./scripts/ pytest tests/

Explanation of the docker command:

  • docker run -it create an instance of an image (=container), and run it interactively (so ctrl+c will work)

  • --rm option means to remove the container once it exits/stops (otherwise, you will have to use docker rm)

  • --network host don’t use network isolation, this allow to use tensorboard/visdom on host machine

  • --ipc=host Use the host system’s IPC namespace. IPC (POSIX/SysV IPC) namespace provides separation of named shared memory segments, semaphores and message queues.

  • --name test give explicitly the name test to the container, otherwise it will be assigned a random name

  • --mount src=... give access of the local directory (pwd command) to the container (it will be map to /root/code/stable-baselines), so all the logs created in the container in this folder will be kept

  • bash -c '...' Run command inside the docker image, here run the tests (pytest tests/)