Air Hockey with Deep Reinforcement Learning

This is a participant repository for the Robot Air Hockey Challenge 2023.

drl_air_hockey_self_play.mp4

Overview

The implemented approach focuses on applying a model-based deep reinforcement learning algorithm DreamerV3 to acquire a policy capable of playing air hockey with continuous observations and actions.

• Additional details about the approach are presented in the report here.
• Models of pre-trained agents can be downloaded from here.

Instructions

Quick Test

As a quick test, you can try evaluating a pre-trained agent in a self-play mode by running .docker/run.bash directly via curl as shown below.

curl -sSfL "https://raw.githubusercontent.com/AndrejOrsula/drl_air_hockey/main/.docker/run.bash" | DOCKER_RUN_OPTS="--rm" bash -s -- drl_air_hockey/scripts/eval_dreamerv3.py -r

Python

Installation

Install air_hockey_challenge and drl_air_hockey Python modules with pip.

pip3 install git+https://github.com/AndrejOrsula/air_hockey_challenge.git
pip3 install git+https://github.com/AndrejOrsula/drl_air_hockey.git

Warning

Not all dependencies have their versions pinned, e.g. transitive dependencies of dreamerv3. Therefore, the functionality of this repository may be affected in future builds. Consider using the pre-built Docker image for a stable environment.

Training and Evaluation

To train a new agent, you can run the included train_dreamerv3.py script.

scripts/train_dreamerv3.py

To configure the training process, consider modifying any of these files directly:

• config.py for algorithm-specific parameters
• rewards.py and tournament_agent_strategies.py for the desired behavior of the agent
• agents for additional changes to the observation and action spaces of the agent

Once you are satisfied with the training progress, you can evaluate the agent by adjusting the included eval_dreamerv3.py script.

scripts/eval_dreamerv3.py

Docker

To install Docker on your system, you can run .docker/host/install_docker.bash to configure Docker with NVIDIA GPU support.

.docker/host/install_docker.bash

Build Image

To build a new Docker image from Dockerfile, you can run .docker/build.bash as shown below. Note that only the pre-built Docker image already contains models of pre-trained agents.

.docker/build.bash ${TAG:-latest} ${BUILD_ARGS}

Run Container

To run the Docker container, you can use .docker/run.bash as shown below.

.docker/run.bash ${TAG:-latest} ${CMD:-bash}

Run Dev Container

To run the Docker container in a development mode (source code mounted as a volume), you can use .docker/dev.bash as shown below.

.docker/dev.bash ${TAG:-latest} ${CMD:-bash}

As an alternative, VS Code users familiar with Dev Containers can modify the included .devcontainer/devcontainer.json to their needs. For convenience, .devcontainer/open.bash script is available to open this repository as a Dev Container in VS Code.

.devcontainer/open.bash

Join Container

To join a running Docker container from another terminal, you can use .docker/join.bash as shown below.

.docker/join.bash ${CMD:-bash}

Citation

@article{orsula2023learning,
  title   = {{Learning to Play Air Hockey with Model-Based Deep Reinforcement Learning}},
  author  = {Andrej Orsula},
  journal = {NeurIPS 2023 --- The Robot Air Hockey Challenge},
  year    = {2023},
}