Introduction

English | 简体中文

MMHuman3D is an open-source PyTorch-based codebase for the use of 3D human parametric models in computer vision and computer graphics. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.7+.

If you are interested in multi-view motion capture, please refer to XRMoCap for more details.

mmhuman3d.demo.mp4

Major Features

• Reproducing popular methods with a modular framework

  MMHuman3D reimplements popular methods, allowing users to reproduce SOTAs with one line of code. The modular framework is convenient for rapid prototyping: the users may attempt various hyperparameter settings and even network architectures, without actually modifying the code.

• Supporting various datasets with a unified data convention

  With the help of a convention toolbox, a unified data format HumanData is used to align all supported datasets. Preprocessed data files are also available.

• Versatile visualization toolbox

  A suite of differential visualization tools for human parametric model rendering (including part segmentation, depth map and point clouds) and conventional 2D/3D keypoints are available.

News

• 2023-04-05: MMHuman3D v0.11.0 is released. Major updates include:
  • Add ExPose inference
  • Add PyMAF-X inference
  • Support CLIFF
• 2022-10-12: MMHuman3D v0.10.0 is released. Major updates include:
  • Add webcam demo and real-time renderer
  • Update dataloader to speed up training
  • Add balanced MSE loss for imbalanced HMR training
• 2022-07-08: MMHuman3D v0.9.0 is released. Major updates include:
  • Support SMPL-X estimation with ExPose for simultaneous recovery of face, hands and body
  • Support new body model STAR
  • Release of GTA-Human dataset with SPIN-FT (51.98 mm) and PARE-FT (46.84 mm) baselines! (Official)
  • Refactor registration and improve performance of SPIN to 57.54 mm

Benchmark and Model Zoo

More details can be found in model_zoo.md.

Supported body models:

(click to collapse)

• SMPL (SIGGRAPH Asia'2015)
• SMPL-X (CVPR'2019)
• MANO (SIGGRAPH ASIA'2017)
• FLAME (SIGGRAPH ASIA'2017)
• STAR (ECCV'2020)

Supported methods:

(click to collapse)

• SMPLify (ECCV'2016)
• SMPLify-X (CVPR'2019)
• HMR (CVPR'2018)
• SPIN (ICCV'2019)
• VIBE (CVPR'2020)
• HybrIK (CVPR'2021)
• PARE (ICCV'2021)
• DeciWatch (ECCV'2022)
• SmoothNet (ECCV'2022)
• ExPose (ECCV'2020)
• BalancedMSE (CVPR'2022)
• PyMAF-X (arXiv'2022)
• ExPose (ECCV'2020)
• PyMAF-X (arXiv'2022)
• CLIFF (ECCV'2022)

Supported datasets:

(click to collapse)

• 3DPW (ECCV'2018)
• AGORA (CVPR'2021)
• AMASS (ICCV'2019)
• COCO (ECCV'2014)
• COCO-WholeBody (ECCV'2020)
• CrowdPose (CVPR'2019)
• EFT (3DV'2021)
• GTA-Human (arXiv'2021)
• Human3.6M (TPAMI'2014)
• InstaVariety (CVPR'2019)
• LSP (BMVC'2010)
• LSP-Extended (CVPR'2011)
• MPI-INF-3DHP (3DC'2017)
• MPII (CVPR'2014)
• Penn Action (ICCV'2012)
• PoseTrack18 (CVPR'2018)
• SURREAL (CVPR'2017)
• UP3D (CVPR'2017)
• FreiHand (ICCV'2019)
• EHF (CVPR'2019)
• Stirling/ESRC-Face3D (FG'2018)

We will keep up with the latest progress of the community, and support more popular methods and frameworks.

If you have any feature requests, please feel free to leave a comment in the wishlist.

Get Started

Please see getting_started.md for the basic usage of MMHuman3D.

License

This project is released under the Apache 2.0 license. Some supported methods may carry additional licenses.

Citation

If you find this project useful in your research, please consider citing:

@misc{mmhuman3d,
    title={OpenMMLab 3D Human Parametric Model Toolbox and Benchmark},
    author={MMHuman3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmhuman3d}},
    year={2021}
}

Contributing

We appreciate all contributions to improve MMHuman3D. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMHuman3D is an open source project that is contributed by researchers and engineers from both the academia and the industry. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedback. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new models.

Projects in OpenMMLab

• MMCV: OpenMMLab foundational library for computer vision.
• MIM: MIM Installs OpenMMLab Packages.
• MMClassification: OpenMMLab image classification toolbox and benchmark.
• MMDetection: OpenMMLab detection toolbox and benchmark.
• MMDetection3D: OpenMMLab next-generation platform for general 3D object detection.
• MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
• MMAction2: OpenMMLab next-generation action understanding toolbox and benchmark.
• MMTracking: OpenMMLab video perception toolbox and benchmark.
• MMPose: OpenMMLab pose estimation toolbox and benchmark.
• MMEditing: OpenMMLab image and video editing toolbox.
• MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding.
• MMGeneration: OpenMMLab next-generation toolbox for generative models.
• MMFlow: OpenMMLab optical flow toolbox and benchmark.
• MMFewShot: OpenMMLab FewShot Learning Toolbox and Benchmark.
• MMHuman3D: OpenMMLab 3D Human Parametric Model Toolbox and Benchmark.
• MMSelfSup: OpenMMLab self-supervised learning toolbox and benchmark.
• MMRazor: OpenMMLab model compression toolbox and benchmark.
• MMDeploy: OpenMMLab model deployment framework.