The imitation library implements imitation learning algorithms on top of Stable-Baselines3, including:
DAgger with synthetic examples
Deep RL from Human Preferences (DRLHP)
You can install imitation with
pip install imitation. The imitation
documentation has more details
on how to use the library, including a quick start guide
for the impatient.