public release v1.0

.gitignore

@@ -3,9 +3,6 @@ __pycache__/
 *.py[cod]
 *$py.class
 
-# C extensions
-*.so
-
 # Distribution / packaging
 .Python
 build/
@@ -127,3 +124,13 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
+# Vizdoom
+_vizdoom*
+
+# A2L
+data/*
+
+# Misc
+.vscode*
+

README.md

@@ -1,4 +1,130 @@
 # A2L - Active Affordance Learning
-Code coming soon, estimated release in March 2020.
 
-Please feel free to reach out via email if you have any questions in the meantime.
+#### Published at ICLR 2020 [[OpenReview]](https://openreview.net/forum?id=BJgMFxrYPB) [[Video]](https://iclr.cc/virtual/poster_BJgMFxrYPB.html) [[PDF]](https://arxiv.org/pdf/2001.02364.pdf)
+
+<img src='./docs/img/architecture.png'>
+
+This repo provides a reference implementation for active affordance learning, which can be employed to improve autonomous navigation performance in hazardous environments (demonstrated here using the VizDoom simulator). The repo also contains a variety of convenient utilities that can be re-used in other VizDoom-based projects to improve quality of life.
+
+## Setup
+
+This code has been tested on Ubuntu 16.04.
+
+### Requirements
+
+1. python >= 3.5
+2. keras >= 2.2.0
+3. [opencv-python](https://pypi.org/project/opencv-python/) >= 3.4.0
+
+### Installing Dependencies
+
+1. Install [VizDoom](https://github.com/mwydmuch/ViZDoom) simulator into local Python environment.
+
+    ``` bash
+    # Install ZDoom dependencies
+    sudo apt-get install build-essential zlib1g-dev libsdl2-dev libjpeg-dev \
+    nasm tar libbz2-dev libgtk2.0-dev cmake git libfluidsynth-dev libgme-dev \
+    libopenal-dev timidity libwildmidi-dev unzip
+
+    # Install Boost libraries
+    sudo apt-get install libboost-all-dev
+
+    # Install Python 3 dependencies
+    sudo apt-get install python3-dev python3-pip
+    pip3 install numpy
+    ```
+
+2. Install Keras-based [segmentation-models](https://github.com/qubvel/segmentation_models) library.
+
+    ``` bash
+    pip3 install segmentation-models
+    ```
+
+### Downloading Demo Data
+
+In order to download the demo data (which contains train/test maps, pre-computed train beacons, and a pre-trained A2L model), follow the steps below:
+
+1. Download the demo data into the root of the repo and uncompress using the following commands:
+
+    ``` bash
+    wget https://www.dropbox.com/s/0hn71njit81xiy7/demo_data.tar.gz
+    tar -xvzf demo_data.tar.gz
+    ```
+
+2. Check that the directory structure looks like the following:
+
+    ```bash
+    ├── data
+    │   ├── beacons
+    │   ├── configs
+    │   ├── experiments
+    │   ├── maps
+    │   ├── samples
+    │   └── models
+    ```
+
+## Usage
+
+Each executable script included in this repo is prefixed with *run* in its file name. For a detailed description of all possible configuration arguments, please run with the ```-h``` flag.
+
+### Generating Partially-Labeled Self-Supervised Samples
+
+The following sequence of commands are used to generate a configurable number of partially-labeled examples of navigability in a self-supervised manner. This should work with any set of  maps that are compatible with VizDoom.
+
+1. Generate a set of beacons for each map - which describe valid spawn points from which the agent can start a sampling episode.
+
+    ``` bash
+    python preprocess/run_beacon_generation.py --wad-dir ../data/maps/train/ --save-dir ../data/beacons/train/
+    ```
+
+2. Generate a configurable number of self-supervised examples per map.
+
+    ``` bash
+    python train/run_data_sampling.py --wad-dir ../data/maps/train/ --beacon-dir ../data/beacons/train/ --save-dir ../data/samples/train/ --samples-per-map 500
+    ```
+
+### Training Affordance Segmentation Models
+
+The following command is used to train a ResNet-18-based UNet segmentation model to predict pixel-wise navigability.
+
+1. Train UNet-based segmentation model.
+
+    ``` bash
+    python train/run_train_model.py --data-dir ../data/samples/train/ --save-dir ../data/models/ --epochs 50 --batch-size 40
+    ```
+
+### Training Segmentation Models with Active Affordance Learning
+
+The following command is used to train a ResNet-18-based UNet segmentation model using active affordance learning. The script alternates between data generation and model training, using trained seed models to actively seek out difficult examples.
+
+1. Train UNet-based segmentation model using active affordance learning.
+
+    ``` bash
+    python train/run_train_A2L.py --wad-dir ../data/maps/train --beacon-dir ../data/beacons/train --save-dir ../data/models/active --active-iterations 5 --samples-per-map 500 --epochs 50 --batch-size 40
+    ```
+
+### Evaluating Navigation Performance
+
+The following command is used to evaluate the performance of an affordance-based agent in a goal-directed navigation task. It can also be used to evaluate a geometry-based agent by dropping the ```--model-path``` argument.
+
+1. Run navigation experiments specified using JSON file.
+    ``` bash
+    python eval/run_eval_navigation.py --wad-dir ../data/maps/test --model-path ../data/models/seg_model.h5 --experiment-path ../data/experiments/navigation/demo.json --iterations 5
+    ```
+
+## Citing
+
+If you've found this code to be useful, please consider citing our paper!
+
+``` latex
+@inproceedings{qi2020learning,
+  title={Learning to Move with Affordance Maps},
+  author={Qi, William and Mullapudi, Ravi Teja and Gupta, Saurabh and Ramanan, Deva},
+  booktitle={International Conference on Learning Representations (ICLR)},
+  year={2020}
+}
+```
+
+## Questions
+
+If you have additional questions/concerns, please feel free to reach out to wq@cs.cmu.edu.

src/common/custom_game.py

@@ -0,0 +1,48 @@
+import numpy as np
+import vizdoom as vzd
+
+
+class CustomGame(vzd.DoomGame):
+    def __init__(self, localization_noise=0, pose_noise=0):
+        super().__init__()
+        self.localization_noise_sd = localization_noise
+        self.pose_noise_sd = pose_noise
+        self.last_true_location = None
+        self.last_noisy_location = None
+
+    def new_episode(self):
+        super().new_episode()
+        self.last_true_location = None
+        self.last_noisy_location = None
+
+    def get_agent_location(self):
+        # Get true location of agent
+        true_x = self.get_game_variable(vzd.GameVariable.POSITION_X)
+        true_y = self.get_game_variable(vzd.GameVariable.POSITION_Y)
+        true_angle = self.get_game_variable(vzd.GameVariable.ANGLE)
+
+        # Return true location if first time called
+        if self.last_true_location is None:
+            self.last_true_location = (true_x, true_y, true_angle)
+            self.last_noisy_location = (true_x, true_y, true_angle)
+            return true_x, true_y, true_angle
+
+        # Get change in location and angle
+        (last_true_x, last_true_y, last_true_angle) = self.last_true_location
+        diff_x = true_x - last_true_x
+        diff_y = true_y - last_true_y
+        diff_angle = true_angle - last_true_angle
+
+        # Generate localization noise for agent position
+        noise_localization = np.random.normal(1, self.localization_noise_sd)
+        noise_pose = np.random.normal(1, self.pose_noise_sd)
+
+        # Simulate agent position obtained by simulated noisy sensor
+        (last_x, last_y, last_angle) = self.last_noisy_location
+        agent_x = last_x + (noise_localization * diff_x)
+        agent_y = last_y + (noise_localization * diff_y)
+        agent_angle = last_angle + (noise_pose * diff_angle)
+
+        self.last_true_location = (true_x, true_y, true_angle)
+        self.last_noisy_location = (agent_x, agent_y, agent_angle)
+        return agent_x, agent_y, agent_angle

src/common/pyastar.py

@@ -0,0 +1,60 @@
+import ctypes
+import numpy as np
+
+import inspect
+from os.path import abspath, dirname, join
+
+fname = abspath(inspect.getfile(inspect.currentframe()))
+lib = ctypes.cdll.LoadLibrary(join(dirname(fname), 'astar.so'))
+
+astar = lib.astar
+ndmat_f_type = np.ctypeslib.ndpointer(
+    dtype=np.float32, ndim=1, flags='C_CONTIGUOUS')
+ndmat_i_type = np.ctypeslib.ndpointer(
+    dtype=np.int32, ndim=1, flags='C_CONTIGUOUS')
+astar.restype = ctypes.c_bool
+astar.argtypes = [ndmat_f_type, ctypes.c_int, ctypes.c_int,
+                  ctypes.c_int, ctypes.c_int, ctypes.c_bool,
+                  ndmat_i_type]
+
+
+def astar_path(weights, start, goal, allow_diagonal=False):
+    # For the heuristic to be valid, each move must cost at least 1.
+    if weights.min(axis=None) < 1.:
+        raise ValueError('Minimum cost to move must be 1, but got %f' % (
+            weights.min(axis=None)))
+    # Ensure start is within bounds.
+    if (start[0] < 0 or start[0] >= weights.shape[0] or
+            start[1] < 0 or start[1] >= weights.shape[1]):
+        raise ValueError('Start of (%d, %d) lies outside grid.' % (start))
+    # Ensure goal is within bounds.
+    if (goal[0] < 0 or goal[0] >= weights.shape[0] or
+            goal[1] < 0 or goal[1] >= weights.shape[1]):
+        raise ValueError('Goal of (%d, %d) lies outside grid.' % (goal))
+
+    height, width = weights.shape
+    start_idx = np.ravel_multi_index(start, (height, width))
+    goal_idx = np.ravel_multi_index(goal, (height, width))
+
+    # The C++ code writes the solution to the paths array
+    paths = np.full(height * width, -1, dtype=np.int32)
+    success = astar(
+        weights.flatten(), height, width, start_idx, goal_idx, allow_diagonal,
+        paths  # output parameter
+    )
+    if not success:
+        return np.array([])
+
+    coordinates = []
+    path_idx = goal_idx
+    while path_idx != start_idx:
+        pi, pj = np.unravel_index(path_idx, (height, width))
+        coordinates.append((pi, pj))
+
+        path_idx = paths[path_idx]
+
+    if coordinates:
+        coordinates.append(np.unravel_index(start_idx, (height, width)))
+        return np.vstack(coordinates[::-1])
+    else:
+        return np.array([])