MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask
Feature warping is a core technique in optical flow estimation; however, the ambiguity caused by occluded areas during warping is a major problem that remains unsolved. In this paper, we propose an asymmetric occlusionaware feature matching module, which can learn a rough occlusion mask that filters useless (occluded) areas immediately after feature warping without any explicit supervision. The proposed module can be easily integrated into end-to-end network architectures and enjoys performance gains while introducing negligible computational cost. The learned occlusion mask can be further fed into a subsequent network cascade with dual feature pyramids with which we achieve state-of-the-art performance. At the time of submission, our method, called MaskFlownet, surpasses all published optical flow methods on the MPI Sintel, KITTI 2012 and 2015 benchmarks. Code is available at https://github.com/microsoft/MaskFlownet.
| Models | Training datasets | Flying Chairs | Sintel (training) | KITTI2012 (training) | KITTI2015 (training) | Log | Config | Download | |
| clean | final | EPE | Fl-all | ||||||
|---|---|---|---|---|---|---|---|---|---|
| MaskFlowNet-S | Flying Chairs | 1.54 | - | - | - | - | log | Config | Model |
| MaskFlowNet-S | Flying Chairs + Flying Thing3d | - | 2.30 | 3.73 | 3.94 | 29.70% | log | Config | Model |
| MaskFlowNet | Flying Chairs | 1.37 | - | - | - | - | log | Config | Model |
| MaskFlowNet | Flying Chairs + Flying Thing3d subset | - | 2.23 | 3.70 | 3.82 | 29.26% | log | Config | Model |
@inproceedings{zhao2020maskflownet,
title={Maskflownet: Asymmetric feature matching with learnable occlusion mask},
author={Zhao, Shengyu and Sheng, Yilun and Dong, Yue and Chang, Eric I and Xu, Yan and others},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={6278--6287},
year={2020}
}