-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
[Feature] Support using backbones from pytorch-image-models (timm) for TSN #880
Conversation
@kennymckormick typo, right? should be densenet161? |
Codecov Report
@@ Coverage Diff @@
## master #880 +/- ##
==========================================
- Coverage 83.57% 83.49% -0.09%
==========================================
Files 132 132
Lines 9966 9977 +11
Branches 1718 1720 +2
==========================================
+ Hits 8329 8330 +1
- Misses 1218 1226 +8
- Partials 419 421 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
Yes, that's a typo. Would you please help correct it in this PR? |
Thanks for the clarification, will fix the typo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, please ping me if checkpoints are ready.
tsn-swin-transformer ckpt/log/json could be found here. Results are quite good: top1/5 accuracy are 77.31/92.88 However, since our kinetics400 dataset is quite different(Now I'm using kinetics400 provided by cvdf), I did some tests with tsn-densenet161
So the accuracy of tsn-swin-transformer ckpt may be different between our datasets. Could you please test the tsn-swin-transformer on your kinetics400 and update accuracy in the TSN readme? If the accuracy is not good enough, maybe you have to help train this model... |
Great, I'm gonna test it on our validation dataset. Besides, I'm going to upload our Kinetics-400 validation dataset. |
Seems some updates need to be committed to pass CI. |
CI raises ImportError, mmcv-full is not properly installed, not sure why that happens |
Motivation
pytorch-image-models is one of the best pytorch classification projects, with a large number of backbones, including both cnns and transformers. It would be interesting if we can train tsn models (and other 2d recognizers) with various backbones.
Modification
Following #679 #720, add codes, docs, unittest.
Use cases (Optional)
Modify model settings, an example is listed as follows.
timm.create_model(backbone_type, **backbone)
https://github.com/rwightman/pytorch-image-models/blob/23c18a33e4168dc7cb11439c1f9acd38dc8e9824/timm/models/factory.py#L25
model['cls_head']['in_channels']
andimg_norm_cfg
accordingly.Test timm backbones
There are over 500 backbones in timm...... Write a script to test timm backbones.
check_timm_models.py.txt
Please note that
verify=True
are supported for MMAction2 TSN models.img_norm_cfg
, both std and mean should be multiplied by 255.TODO