Skip to content
/ CMKD Public

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (ECCV 2022 Oral)

License

Notifications You must be signed in to change notification settings

Cc-Hy/CMKD

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CMKD: Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (ECCV 2022 Oral)

Paper

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (arXiv, Supplimentary Included)

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (ECCV Open Access)

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (Waymo Challenge Report)

If you find our papers helpful for your research, you may cite our paper as

@inproceedings{yuhong-CMKD-ECCV2022,
author = {Yu Hong and
Hang Dai and
Yong Ding},
title = {Cross-Modality Knowledge
Distillation Network for Monocular 3D Object
Detection},
booktitle = {{ECCV}},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
year = {2022}
}

Introduction

This is the official implementation of CMKD with OpenPCDet for KITTI and Waymo datasets.

We have also implemented another version with MMDetection3D for Nuscenes dataset.

News

[2023.3.25] We have several updates.

  • Release the multi-camera version CMKD for Nuscenes dataset, refer to this link for detail
  • Release more pre-trained models for KITTI dataset

[2023.2.14] We have several updates.

Notice: Due to the short schedule, instructions and pre-trained models will be released gradually in the near future, and there may be many issues and bugs, please feel free to let us know if you have any questions.

  • Support center head in response distillation
  • Support more teacher models in the framework (We now have SECOND, CenterPoint and PointPillar), more pre-trained models will be released later
  • Support to set different feature level as the distillation guidance
  • Add visualization utils to visualize the BEV feature maps and the detection results
  • Support for Nuscenes dataset will be released very soon

[2022.11.20] We release some instructions and pre-trained models covering the KITTI experiments.

This implementation has some differences from our paper, but the core idea is the same. Overall, the current version is faster to train, uses less memory, and has similar performance to the older version.

Waymo experiments and Nuscenes experiments are on the way.

[2022.7.9] Our paper has been accepted by ECCV 2022 as Oral presentation. 🔥🔥🔥

[2022.7.4] Our paper has been accepted by ECCV 2022. 🔥🔥

[2022.5.24] CMKD gets the 3rd place in the 2022 Waymo 3D camera-only detection challenge. 🔥

In the challenge, we simply extend our baseline model from single-camera version to multi-camera version without any challenge-specific skills and achieve good results. Specifically, we use a lightweight res-50 backbone with 20% of the total training samples, no previous frames, no data augmentation, and no training and testing tricks to rank 3rd in the challenge.

Framework Overview

image

BEV Features Generation

image

Use CMKD

Installation

Please follow INSTALL to install CMKD.

Getting Started

Please follow GETTING_START to train or evaluate the models.

Models

KITTI

Teacher Model Car Easy@R40 Car Moderate@R40 Car Hard@R40 Model Teacher Model
CMKD-R50 (kitti train + eigen clean) SECOND 33.36 21.61 17.97 model model
CMKD-R50 (kitti train) SECOND 24.02 15.80 13.22 model model
CMKD-R50 (kitti train + eigen clean) CenterPoint 29.78 21.17 18.41 model model
CMKD-R50 (kitti train) CenterPoint 22.56 16.02 13.52 model model
CMKD-R50 (kitti train + eigen clean) PointPillar 32.25 21.47 18.21 model model
CMKD-R50 (kitti train) PointPillar 23.84 16.44 13.58 model model

Waymo

Coming Soon

Nuscenes

mAP NDS Model
BEVDet-R50 30.7 38.2 -
BEVDet-R50 + CMKD 34.7 42.6 -

About

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (ECCV 2022 Oral)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published