Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Feature] Support STARLoss (CVPR'2023) #2644

Open
wants to merge 113 commits into
base: dev-1.x
Choose a base branch
from

Conversation

chg0901
Copy link

@chg0901 chg0901 commented Aug 22, 2023

Motivation

Support STAR loss in MMPose.

[Refered repo] https://github.com/ZhenglinZhou/STAR/blob/master/lib/loss/starLoss_v2.py
Train and test on WFLW dataset to reproduce the accuracy in paper
Train models on other datasets

Modification

Add files

  • configs/face_2d_keypoint/topdown_regression/wflw/resnet_starloss_wflw.md
  • configs/face_2d_keypoint/topdown_regression/wflw/resnet_starloss_wflw.yml
  • configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_starloss_8xb64-210e_wflw-256x256.py
  • mmpose/models/heads/heatmap_heads/star_head.py

Modify exist files

  • mmpose/models/heads/heatmap_heads/__init__.py
  • mmpose/models/losses/__init__.py
  • mmpose/models/losses/regression_loss.py

Use cases (Optional)

When we use the STAR Loss, we should change the model.neck and model.head ('STARHead' and 'STARLoss' in model.head)

# model settings
model = dict(
    # replace GlobalAveragePooling with FeatureMapProcessor
    # to obtain heatmap output
    # neck=dict(type='GlobalAveragePooling'),
    neck=dict(
        type='FeatureMapProcessor',
        concat=True,
    ),
    # using STARLoss with STARHead
    head=dict(
        type='STARHead',
        in_channels=2048,
        out_channels=98,
        deconv_out_channels=None,
        loss=dict(type='STARLoss', use_target_weight=True),
        decoder=codec_star),
    train_cfg=dict(),
    test_cfg=dict(
        flip_test=True,
        shift_coords=True,
    ))

Current process

  • main algorithm and data pipeline is adjusted with the wflw dataset and starloss algorithm

Problem to check and solve

  • the training is not converged, needs helps from the main contributors of MMPose

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • CLA has been signed and all committers have signed the CLA in this PR.

Tau-J and others added 30 commits July 6, 2023 11:51
Ben-Louis and others added 29 commits September 20, 2023 10:38
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.