There are following main parts:
- Covariance Pooling of Convolution Features
- Temporal Pooling of Features
For pooling Convolution Features, I do not have exact hyperparameters to reproduce exact numbers in the paper. I at least obtained 86% with model1 (covpoolnet2.py and train2.sh). However, I have uploaded all the pretrained models from the paper.
You can download following models (2.5 GB total)
- models and run reproducepaper.sh (after uncommenting appropriate lines)
- For the code for inception-resnet-v1, I the used same implementation of inception-resnet in facenet
- For baseline, the network is same as included here except this code contains few additional (covariance pooling) layers.
Features extracted with CNN (model proposed in the paper) from AFEW dataset are placed in zip afew_features.zip. Extract the zip to afew_features in same folder. To classify result, simple run bash classify.sh
- python 2.7
- tensorflow
- numpy
- sklearn
- This code framework is mostly based on facenet
- Apply the patch suggested in tensorflow_patch.txt file. While computing gradient of eigen-decomposition, NaNs are returned by tensorflow when eigenvalues are identical. This throws error and cannot continue training. The patch only replaces NANs with zeros. This makes training easier.