MMDetection3D V1.1.0rc0 Release

Changelog of v1.1

v1.1.0rc0 (1/9/2022)

We are excited to announce the release of MMDetection3D 1.1.0rc0.
MMDet3D 1.1.0rc0 is the first version of MMDetection3D 1.1, a part of the OpenMMLab 2.0 projects.
Built upon the new training engine and MMDet 3.x,
MMDet3D 1.1 unifies the interfaces of dataset, models, evaluation, and visualization with faster training and testing speed.
It also provides a standard data protocol for different datasets, modalities, and tasks for 3D perception.
We will support more strong baselines in the future release, with our latest exploration on camera-only 3D detection from videos.

Highlights

1. New engines. MMDet3D 1.1 is based on MMEngine and MMDet 3.x, which provides a universal and powerful runner that allows more flexible customizations and significantly simplifies the entry points of high-level interfaces.

2. Unified interfaces. As a part of the OpenMMLab 2.0 projects, MMDet3D 1.1 unifies and refactors the interfaces and internal logics of train, testing, datasets, models, evaluation, and visualization. All the OpenMMLab 2.0 projects share the same de# those interfaces and logics to allow the emergence of multi-task/modality algorithms.

3. Standard data protocol for all the datasets, modalities, and tasks for 3D perception. Based on the unified base datasets inherited from MMEngine, we also design a standard data protocol that defines and unifies the common keys across different datasets, tasks, and modalities. It significantly simplifies the usage of multiple datasets and data modalities for multi-task frameworks and eases dataset customization. Please refer to the documentation of customized datasets for details.

4. Strong baselines. We will release strong baselines of many popular models to enable fair comparisons among state-of-the-art models.

5. More documentation and tutorials. We add a bunch of documentation and tutorials to help users get started more smoothly. Read it here.

Breaking Changes

MMDet3D 1.1 has undergone significant changes to have better design, higher efficiency, more flexibility, and more unified interfaces.
Besides the changes of API, we briefly list the major breaking changes in this section.
We will update the migration guide to provide complete details and migration instructions.
Users can also refer to the compatibility documentation and API doc for more details.

Dependencies

• MMDet3D 1.1 runs on PyTorch>=1.6. We have deprecated the support of PyTorch 1.5 to embrace the mixed precision training and other new features since PyTorch 1.6. Some models can still run on PyTorch 1.5, but the full functionality of MMDet3D 1.1 is not guaranteed.
• MMDet3D 1.1 relies on MMEngine to run. MMEngine is a new foundational library for training deep learning models of OpenMMLab and are widely depended by OpenMMLab 2.0 projects. The dependencies of file IO and training are migrated from MMCV 1.x to MMEngine.
• MMDet3D 1.1 relies on MMCV>=2.0.0rc0. Although MMCV no longer maintains the training functionalities since 2.0.0rc0, MMDet3D 1.1 relies on the data transforms, CUDA operators, and image processing interfaces in MMCV. Note that the package mmcv is the version that provides pre-built CUDA operators and mmcv-lite does not since MMCV 2.0.0rc0, while mmcv-full has been deprecated since 2.0.0rc0.
• MMDet3D 1.1 is based on MMDet 3.x, which is also a part of OpenMMLab 2.0 projects.

Training and testing

• MMDet3D 1.1 uses Runner in MMEngine rather than that in MMCV. The new Runner implements and unifies the building logic of dataset, model, evaluation, and visualizer. Therefore, MMDet3D 1.1 no longer relies on the building logics of those modules in mmdet3d.train.apis and tools/train.py. Those code have been migrated into MMEngine. Please refer to the migration guide of Runner in MMEngine for more details.
• The Runner in MMEngine also supports testing and validation. The testing scripts are also simplified, which has similar logic as that in training scripts to build the runner.
• The execution points of hooks in the new Runner have been enriched to allow more flexible customization. Please refer to the migration guide of Hook in MMEngine for more details.
• Learning rate and momentum scheduling has been migrated from Hook to Parameter Scheduler in MMEngine. Please refer to the migration guide of Parameter Scheduler in MMEngine for more details.

Configs

• The Runner in MMEngine uses a different config structure to ease the understanding of the components in runner. Users can read the config example of MMDet3D 1.1 or refer to the migration guide in MMEngine for migration details.
• The file names of configs and models are also refactored to follow the new rules unified across OpenMMLab 2.0 projects. The names of checkpoints are not updated for now as there is no BC-breaking of model weights between MMDet3D 1.1 and 1.0.x. We will progressively replace all the model weights by those trained in MMDet3D 1.1. Please refer to the user guides of config for more details.

Dataset

The Dataset classes implemented in MMDet3D 1.1 all inherits from the Det3DDataset and Seg3DDataset, which inherits from the BaseDataset in MMEngine. In addition to the changes of interfaces, there are several changes of Dataset in MMDet3D 1.1.

• All the datasets support to serialize the internal data list to reduce the memory when multiple workers are built for data loading.
• The internal data structure in the dataset is changed to be self-contained (without losing information like class names in MMDet3D 1.0.x) while keeping simplicity.
• Common keys across different datasets and data modalities are defined and all the info files are unified into a standard protocol.
• The evaluation functionality of each dataset has been removed from dataset so that some specific evaluation metrics like KITTI AP can be used to evaluate the prediction on other datasets.

Data Transforms

The data transforms in MMDet3D 1.1 all inherits from BaseTransform in MMCV>=2.0.0rc0, which defines a new convention in OpenMMLab 2.0 projects.
Besides the interface changes, there are several changes listed as below:

• The functionality of some data transforms (e.g., Resize) are decomposed into several transforms to simplify and clarify the usages.
• The format of data dict processed by each data transform is changed according to the new data structure of dataset.
• Some inefficient data transforms (e.g., normalization and padding) are moved into data preprocessor of model to improve data loading and training speed.
• The same data transforms in different OpenMMLab 2.0 libraries have the same augmentation implementation and the logic given the same arguments, i.e., Resize in MMDet 3.x and MMSeg 1.x will resize the image in the exact same manner given the same arguments.

Model

The models in MMDet3D 1.1 all inherits from BaseModel in MMEngine, which defines a new convention of models in OpenMMLeb 2.0 projects.
Users can refer to the tutorial of model in MMengine for more details.
Accordingly, there are several changes as the following:

• The model interfaces, including the input and output formats, are significantly simplified and unified following the new convention in MMDet3D 1.1.
  Specifically, all the input data in training and testing are packed into inputs and data_samples, where inputs contains model inputs like a dict contain a list of image tensors and the point cloud data, and data_samples contains other information of the current data sample such as ground truths, region proposals, and model predictions. In this way, different tasks in MMDet3D 1.1 can share the same input arguments, which makes the models more general and suitable for multi-task learning and some flexible training paradigms like semi-supervised learning.
• The model has a data preprocessor module, which are used to pre-process the input data of model. In MMDet3D 1.1, the data preprocessor usually does necessary steps to form the input images into a batch, such as padding. It can also serve as a place for some special data augmentations or more efficient data transformations like normalization.
• The internal logic of model have been changed. In MMDet3D 1.1, model uses forward_train, forward_test, simple_test, and aug_test to deal with different model forward logics. In MMDet3D 1.1 and OpenMMLab 2.0, the forward function has three modes: 'loss', 'predict', and 'tensor' for training, inference, and tracing or other purposes, respectively.
  The forward function calls self.loss, self.predict, and self._forward given the modes 'loss', 'predict', and 'tensor', respectively.

Evaluation

The evaluation in MMDet3D 1.0.x strictly binds with...

MMDetection3D V1.0.0rc4 Release

Highlights

• Support FCAF3D

New Features

• Support FCAF3D (#1547)
• Add the transformation to support multi-camera 3D object detection (#1580)
• Support lift-splat-shoot view transformer (#1598)

Improvements

• Remove the limitation of the maximum number of points during SUN RGB-D preprocessing (#1555)
• Support circle CI (#1647)
• Add mim to extras_require in setup.py (#1560, #1574)
• Update dockerfile package version (#1697)

Bug Fixes

• Flip yaw angle for DepthInstance3DBoxes.overlaps (#1548, #1556)
• Fix DGCNN configs (#1587)
• Fix bbox head not registered bug (#1625)
• Fix missing objects in S3DIS preprocessing (#1665)
• Fix spconv2.0 model loading bug (#1699)

Contributors

A total of 9 developers contributed to this release.

@Tai-Wang, @ZwwWayne, @filaPro, @lianqing11, @ZCMax, @HuangJunJie2017, @Xiangxu-0103, @ChonghaoSima, @VVsssssk

New Contributors

• @HuangJunJie2017 made their first contribution in #1580
• @ChonghaoSima made their first contribution in #1614

Full Changelog: v1.0.0rc3...v1.0.0rc4

MMDetection3D V1.0.0rc3 Release

Highlights

• Support SA-SSD

New Features

• Support SA-SSD (#1337)

Improvements

• Add Chinese documentation for vision-only 3D detection (#1438)
• Update CenterPoint pretrained models that are compatible with refactored coordinate systems (#1450)
• Configure myst-parser to parse anchor tag in the documentation (#1488)
• Replace markdownlint with mdformat for avoiding installing ruby (#1489)
• Add missing gt_names when getting annotation info in Custom3DDataset (#1519)
• Support S3DIS full ceph training (#1542)
• Rewrite the installation and FAQ documentation (#1545)

Bug Fixes

• Fix the incorrect registry name when building RoI extractors (#1460)
• Fix the potential problems caused by the registry scope update when composing pipelines (#1466) and using CocoDataset (#1536)
• Fix the missing selection with order in the box3d_nms introduced by #1403 (#1479)
• Update the PointPillars config to make it consistent with the log (#1486)
• Fix heading anchor in documentation (#1490)
• Fix the compatibility of mmcv in the dockerfile (#1508)
• Make overwrite_spconv packaged when building whl (#1516)
• Fix the requirement of mmcv and mmdet (#1537)
• Update configs of PartA2 and support its compatibility with spconv 2.0 (#1538)

Contributors

A total of 13 developers contributed to this release.

@Xiangxu-0103, @ZCMax, @jshilong, @filaPro, @atinfinity, @Tai-Wang, @wenbo-yu, @yi-chen-isuzu, @ZwwWayne, @wchen61, @VVsssssk, @AlexPasqua, @lianqing11

New Contributors

• @atinfinity made their first contribution in #1508
• @wenbo-yu made their first contribution in #1337
• @wchen61 made their first contribution in #1516
• @AlexPasqua made their first contribution in #1519
• @lianqing11 made their first contribution in #1545

Full Changelog: v1.0.0rc2...v1.0.0rc3

MMDetection3D V1.0.0rc2 Release

Highlights

• Support spconv 2.0
• Support MinkowskiEngine with MinkResNet
• Support training models on custom datasets with only point clouds
• Update Registry to distinguish the scope of built functions
• Replace mmcv.iou3d with a set of bird-eye-view (BEV) operators to unify the operations of rotated boxes

New Features

• Add loader arguments in the configuration files (#1388)
• Support spconv 2.0 when the package is installed. Users can still use spconv 1.x in MMCV with CUDA 9.0 (only cost more memory) without losing the compatibility of model weights between two versions (#1421)
• Support MinkowskiEngine with MinkResNet (#1422)

Improvements

• Add the documentation for model deployment (#1373, #1436)
• Add Chinese documentation of
  • Speed benchmark (#1379)
  • LiDAR-based 3D detection (#1368)
  • LiDAR 3D segmentation (#1420)
  • Coordinate system refactoring (#1384)
• Support training models on custom datasets with only point clouds (#1393)
• Replace mmcv.iou3d with a set of bird-eye-view (BEV) operators to unify the operations of rotated boxes (#1403, #1418)
• Update Registry to distinguish the scope of building functions (#1412, #1443)
• Replace recommonmark with myst_parser for documentation rendering (#1414)

Bug Fixes

• Fix the show pipeline in the browse_dataset.py (#1376)
• Fix missing init files after coordinate system refactoring (#1383)
• Fix the incorrect yaw in the visualization caused by coordinate system refactoring (#1407)
• Fix NaiveSyncBatchNorm1d and NaiveSyncBatchNorm2d to support non-distributed cases and more general inputs (#1435)

Contributors

A total of 11 developers contributed to this release.

@ZCMax, @ZwwWayne, @Tai-Wang, @VVsssssk, @HanaRo, @JoeyforJoy, @ansonlcy, @filaPro, @jshilong, @Xiangxu-0103, @deleomike

New Contributors

• @HanaRo made their first contribution in #1379
• @JoeyforJoy made their first contribution in #1368
• @ansonlcy made their first contribution in #1391
• @deleomike made their first contribution in #1383

Full Changelog: v1.0.0rc1...v1.0.0rc2

MMDetection3D V1.0.0rc1 Release

Compatibility

• We migrate all the mmdet3d ops to mmcv and do not need to compile them when installing mmdet3d.
• To fix the imprecise timestamp and optimize its saving method, we reformat the point cloud data during Waymo data conversion. The data conversion time is also optimized significantly by supporting parallel processing. Please re-generate KITTI format Waymo data if necessary. See more details in the compatibility documentation.
• We update some of the model checkpoints after the refactor of coordinate systems. Please stay tuned for the release of the remaining model checkpoints.

	Fully Updated	Partially Updated	In Progress	No Influcence
SECOND		✓
PointPillars		✓
FreeAnchor	✓
VoteNet	✓
H3DNet	✓
3DSSD		✓
Part-A2	✓
MVXNet	✓
CenterPoint			✓
SSN	✓
ImVoteNet	✓
FCOS3D				✓
PointNet++				✓
Group-Free-3D				✓
ImVoxelNet	✓
PAConv				✓
DGCNN				✓
SMOKE				✓
PGD				✓
MonoFlex				✓

Highlights

• Migrate all the mmdet3d ops to mmcv
• Support parallel waymo data converter
• Add ScanNet instance segmentation dataset with metrics
• Better compatibility for windows with CI support, op migration and bug fixes
• Support loading annotations from Ceph

New Features

• Add ScanNet instance segmentation dataset with metrics (#1230)
• Support different random seeds for different ranks (#1321)
• Support loading annotations from Ceph (#1325)
• Support resuming from the latest checkpoint automatically (#1329)
• Add windows CI (#1345)

Improvements

• Update the table format and OpenMMLab project orders in README.md (#1272, #1283)
• Migrate all the mmdet3d ops to mmcv (#1240, #1286, #1290, #1333)
• Add with_plane flag in the KITTI data conversion (#1278)
• Update instructions and links in the documentation (#1300, 1309, #1319)
• Support parallel Waymo dataset converter and ground truth database generator (#1327)
• Add quick installation commands to getting_started.md (#1366)

Bug Fixes

• Update nuimages configs to use new nms config style (#1258)
• Fix the usage of np.long for windows compatibility (#1270)
• Fix the incorrect indexing in BasePoints (#1274)
• Fix the incorrect indexing in the pillar_scatter.forward_single (#1280)
• Fix unit tests that use GPUs (#1301)
• Fix incorrect feature dimensions in DynamicPillarFeatureNet caused by previous upgrading of PillarFeatureNet (#1302)
• Remove the CameraPoints constraint in PointSample (#1314)
• Fix imprecise timestamps saving of Waymo dataset (#1327)

Contributors

A total of 10 developers contributed to this release.

@ZCMax, @ZwwWayne, @wHao-Wu, @Tai-Wang, @wangruohui, @zjwzcx, @Xiangxu-0103, @EdAyers, @hongye-dev, @zhanggefan

New Contributors

• @VVsssssk made their first contribution in #1275
• @Xiangxu-0103 made their first contribution in #1300
• @Subjectivist made their first contribution in #1298
• @EdAyers made their first contribution in #1258
• @hongye-dev made their first contribution in #1280
• @jshilong made their first contribution in #1366

Full Changelog: v1.0.0rc0...v1.0.0rc1

MMDetection3D V1.0.0rc0 Release

Compatibility

• We refactor our three coordinate systems to make their rotation directions and origins more consistent, and further remove unnecessary hacks in different datasets and models. Therefore, please re-generate data information or convert the old version to the new one with our provided scripts. We will also provide updated checkpoints in the next version. Please refer to the compatibility documentation for more details.
• Unify the camera keys for consistent transformation between coordinate systems on different datasets. The modification changes the key names to lidar2img, depth2img, cam2img, etc., for easier understanding. Customized codes using legacy keys may be influenced.
• The next release will begin to move files of CUDA ops to MMCV. It will influence the way to import related functions. We will not break the compatibility but will raise a warning first and please prepare to migrate it.

Highlights

• Support new monocular 3D detectors: PGD, SMOKE, MonoFlex
• Support a new LiDAR-based detector: PointRCNN
• Support a new backbone: DGCNN
• Support 3D object detection on the S3DIS dataset
• Support compilation on Windows
• Full benchmark for PAConv on S3DIS
• Further enhancement for documentation, especially on the Chinese documentation

New Features

• Support 3D object detection on the S3DIS dataset (#835)
• Support PointRCNN (#842, #843, #856, #974, #1022, #1109, #1125)
• Support DGCNN (#896)
• Support PGD (#938, #940, #948, #950, #964, #1014, #1065, #1070, #1157)
• Support SMOKE (#939, #955, #959, #975, #988, #999, #1029)
• Support MonoFlex (#1026, #1044, #1114, #1115, #1183)
• Support CPU Training (#1196)

Improvements

• Support point sampling based on distance metric (#667, #840)
• Refactor coordinate systems (#677, #774, #803, #899, #906, #912, #968, #1001)
• Unify camera keys in PointFusion and transformations between different systems (#791, #805)
• Refine documentation (#792, #827, #829, #836, #849, #854, #859, #1111, #1113, #1116, #1121, #1132, #1135, #1185, #1193, #1226)
• Add a script to support benchmark regression (#808)
• Benchmark PAConvCUDA on S3DIS (#847)
• Support to download pdf and epub documentation (#850)
• Change the repeat setting in Group-Free-3D configs to reduce training epochs (#855)
• Support KITTI AP40 evaluation metric (#927)
• Add the mmdet3d2torchserve tool for SECOND (#977)
• Add code-spell pre-commit hook and fix typos (#995)
• Support the latest numba version (#1043)
• Set a default seed to use when the random seed is not specified (#1072)
• Distribute mix-precision models to each algorithm folder (#1074)
• Add abstract and a representative figure for each algorithm (#1086)
• Upgrade pre-commit hook (#1088, #1217)
• Support augmented data and ground truth visualization (#1092)
• Add local yaw property for CameraInstance3DBoxes (#1130)
• Lock the required numba version to 0.53.0 (#1159)
• Support the usage of plane information for KITTI dataset (#1162)
• Deprecate the support for "python setup.py test" (#1164)
• Reduce the number of multi-process threads to accelerate training (#1168)
• Support 3D flip augmentation for semantic segmentation (#1181)
• Update README format for each model (#1195)

Bug Fixes

• Fix compiling errors on Windows (#766)
• Fix the deprecated nms setting in the ImVoteNet config (#828)
• Use the latest wrap_fp16_model import from mmcv (#861)
• Remove 2D annotations generation on Lyft (#867)
• Update index files for the Chinese documentation to be consistent with the English version (#873)
• Fix the nested list transpose in the CenterPoint head (#879)
• Fix deprecated pretrained model loading for RegNet (#889)
• Fix the incorrect dimension indices of rotations and testing config in the CenterPoint test time augmentation (#892)
• Fix and improve visualization tools (#956, #1066, #1073)
• Fix PointPillars FLOPs calculation error (#1075)
• Fix missing dimension information in the SUN RGB-D data generation (#1120)
• Fix incorrect anchor range settings in the PointPillars config for KITTI (#1163)
• Fix incorrect model information in the RegNet metafile (#1184)
• Fix bugs in non-distributed multi-gpu training and testing (#1197)
• Fix a potential assertion error when generating corners from an empty box (#1212)
• Upgrade bazel version according to the requirement of Waymo Devkit (#1223)

Contributors

A total of 12 developers contributed to this release.

@THU17cyz, @wHao-Wu, @wangruohui, @Wuziyi616, @filaPro, @ZwwWayne, @Tai-Wang, @DCNSW, @xieenze, @robin-karlsson0, @ZCMax, @Otteri

New Contributors

• @Otteri made their first contribution in #1070
• @zeyu-hello made their first contribution in #1225
• @maskjp made their first contribution in #1207

Full Changelog: v0.18.1...v1.0.0rc0

MMDetection3D V0.18.1 Release

Improvements

• Support Flip3D augmentation in semantic segmentation task (#1182)
• Update regnet metafile (#1184)
• Add point cloud annotation tools introduction in FAQ (#1185)
• Add missing explanations of cam_intrinsic in the nuScenes dataset doc (#1193)

Bug Fixes

• Deprecate the support for "python setup.py test" (#1164)
• Fix the rotation matrix while rotation axis=0 (#1182)
• Fix the bug in non-distributed multi-gpu training/testing (#1197)
• Fix a potential bug when generating corners of empty bounding boxes (#1212)

Contributors

A total of 4 developers contributed to this release.

@ZwwWayne, @ZCMax, @Tai-Wang, @wHao-Wu

Full Changelog: v0.18.0...v0.18.1

MMDetection3D V0.18.0 Release

Highlights

• Update the required minimum version of mmdet and mmseg

Improvements

• Use the official markdownlint hook and add codespell hook for pre-committing (#1088)
• Improve CI operation (#1095, #1102, #1103)
• Use shared menu content from OpenMMLab's theme and remove duplicated contents from config (#1111)
• Refactor the structure of documentation (#1113, #1121)
• Update the required minimum version of mmdet and mmseg (#1147)

Bug Fixes

• Fix symlink failure on Windows (#1096)
• Fix the upper bound of mmcv version in the mminstall requirements (#1104)
• Fix API documentation compilation and mmcv build errors (#1116)
• Fix figure links and pdf documentation compilation (#1132, #1135)

Contributors

A total of 4 developers contributed to this release.

@ZwwWayne, @ZCMax, @Tai-Wang, @wHao-Wu

Full Changelog: v0.17.3...v0.18.0

MMDetection3D V0.17.3 Release

What's Changed

• [Fix] Update mmcv version in dockerfile by @wHao-Wu in #1036
• [Fix] Fix the memory-leak problem in init_detector by @Tai-Wang in #1045
• [Fix] Fix default show value in show_result function and a typo in waymo_data_prep by @ZCMax in #1034
• [Fix] Fix incorrect velo indexing when formatting boxes on nuScenes by @Tai-Wang in #1049
• [Enhance] Clean unnecessary custom_imports in entrypoints by @ZCMax in #1068
• [Doc] Add MMFlow into README by @ZCMax in #1067
• Explicitly setting torch.cuda.device at init_model by @aldakata in #1056
• [Fix] Fix PointPillars FLOPs calculation error for master branch by @ZCMax in #1076
• [Enhance] Add mmFewShot in README by @ZCMax in #1085
• Label visualization by @MilkClouds in #1050
• [Enhance] add mmhuman3d in readme by @ZCMax in #1094
• [Enhance] fix mmhuman3d reference by @ZCMax in #1100
• Bump to v0.17.3 by @Tai-Wang in #1083

New Contributors

• @aldakata made their first contribution in #1056
• @MilkClouds made their first contribution in #1050

Full Changelog: v0.17.2...v0.17.3

MMDetection3D V0.17.2 Release

Improvements

• Update Group-Free-3D and FCOS3D bibtex (#985)
• Update the solutions for incompatibility of pycocotools in the FAQ (#993)
• Add Chinese documentation for the KITTI (#1003) and Lyft (#1010) dataset tutorial
• Add the H3DNet checkpoint converter for incompatible keys (#1007)

Bug Fixes

• Update mmdetection and mmsegmentation version in the Dockerfile (#992)
• Fix links in the Chinese documentation (#1015)

Contributors

A total of 4 developers contributed to this release.

@Tai-Wang, @wHao-Wu, @ZwwWayne, @ZCMax

Full Changelog: v0.17.1...v0.17.2