How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)

This repository implements a demo of the networks described in "How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)" paper. Please visit our webpage or read bellow for instructions on how to run the code and access the dataset.

Training code: https://github.com/1adrianb/face-alignment-training

🔥 Python code available: https://github.com/1adrianb/face-alignment

Note: If you are interested in a binarized version, capable of running on devices with limited resources please also check https://github.com/1adrianb/binary-face-alignment for a demo.

Requirments

• Install the latest Torch7 version (for Windows, please follow the instructions available here)
• Install python 2.7.x

Lua packages

• cutorch
• nn
• cunn
• nngraph
• cudnn
• xlua
• image
• paths
• fb.python

Note: if you are having troubles compiling thpp (required for fb.python) against the latest version of torch7 please use the version available here: https://github.com/1adrianb/thpp

Python packages

• numpy
• matplotlib
• dlib (required for face detection, if no bbox are provided)

Please note that dlib performs poorly for faces found in challenging poses or difficult lighting conditions and it's provided only as a simple demo. For optimal performance we recommend using other deeplearning based face detection methods.

Setup

Clone the github repository and install all the dependencies mentiones above.

git  clone https://github.com/1adrianb/2D-and-3D-face-alignment
cd 2D-and-3D-face-alignment

Usage

In order to run the demo please download the required models available bellow and the associated data.

th main.lua

In order to see all the available options please run:

th main.lua --help

Docker image

For convenience, a Dockerfile is provided to build images with cuda support and cudnn v5.1:

docker build -t facealignment .

Alternatively you can use the image available on docker hub:

nvidia-docker run -it 1adrianb/facealignment-torch

In order to keep the image small the data and the models are not included and will have to be downloaded separately.

Docker image based on the cuda-torch by kaixhin.

Pretrained models

2D-FAN - trained on 300W-LP and finetuned on iBUG training set.

3D-FAN - trained on 300W-LP

2D-to-3D-FAN - trained on 300W-LP

3D-FAN-depth - trained on 300W-LP

Citation

@inproceedings{bulat2017far,
  title={How far are we from solving the 2D \& 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)},
  author={Bulat, Adrian and Tzimiropoulos, Georgios},
  booktitle={International Conference on Computer Vision},
  year={2017}
}

Dataset

You can download the annotations alongside the images used by visiting our page.