Skip to content

[CVPR 2024] Learning Occupancy for Monocular 3D Object Detection

Notifications You must be signed in to change notification settings

SPengLiang/OccupancyM3D

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OccupancyM3D

Note that this is an initial version, this repository needs to be further cleaned and refactored.

paper link: [2305.15694] Learning Occupancy for Monocular 3D Object Detection

Installation

We employ the design and framework of OpenPCDet, and follows the base setup in CaDDN, thanks for their great work!

Therefore, please follow the installation steps in OpenPCDet.

Getting Started

First, please follow the KITTI data file generation in CaDDN. You can find "depth_caddn" in CaDDN, and "lidar_depth" in Google Drive.

We do not use the heavy deeplibV3 backbone. Instead, we use the pre-trained dla34 backbone from DD3D. Note that the authors slightly modify the model. We re-map the key and provide it at Google Drive.

Then, train the model:

cd OccupancyM3D/OpenPCDet/tools

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --rdzv_endpoint=localhost:6400 --nproc_per_node=4  train.py --launcher pytorch  --cfg_file ./cfgs/kitti_models/OccupancyM3D.yaml \
  --sync_bn --workers 4 \
  --num_epochs_to_eval 20  \
  --extra_tag OccupancyM3D

Eval the model using the pre-trained model:

CUDA_VISIBLE_DEVICES=0 python test.py --cfg_file ./cfgs/kitti_models/OccupancyM3D.yaml \
  --extra_tag val \
  --batch_size 2 --workers 4 \
  --ckpt $pre-trained-model-path

Pretrained Model

To ease the usage, we provide the pre-trained model at: Google Drive

Here we give the comparison.

Models Car@BEV IoU=0.7 Car@3D IoU=0.7
Easy Mod Hard Easy Mod Hard
original paper 35.72 26.60 23.68 26.87 19.96 17.15
this repo 36.26 26.25 23.22 28.64 19.84 17.77

Visualization Results

Some good cases and bad cases (marked using arrows)

KITTI results

Waymo results

About

[CVPR 2024] Learning Occupancy for Monocular 3D Object Detection

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published