[Feature] Support TSM-MobileNetV2

[irvingzhang0512]

TODO list

• mobilenetv2 backbone & unittest.
• tsm-mobilenetv2 backbone & unittest.
• covnert checkpoint from origin repo.
  • original repo: 30 test crops, 19520 samples, top1/5 accuracy is 69.54%/88.66%
  • mmaction2 convertion: 10 test crops, 18219 samples, top1/5 accuracy is 69.04%/88.23%.
• Refactor mobilenet with mmcls
• changelog
• training with mmaction2 & update model zoo.
  • I don't have enough gpus to train on kinetics400, maybe next week i can have a try...
  • 贫穷的泪水

training results of mobilenet-tsm with DenseSampleFrames1x1x8. (origin ckpt get 69.54%/88.66% top1/5 accuracy).

lr	epochs	gpus	weight decay	top1/5 acuracy
0.00875	50	7	0.0001	63.75%/85.52%
0.0025	50	4	0.0001	65.11%/85.99%
0.0025	100	4	0.0001	66.xx%/86.xx%
0.004	100	4	0.00004	68.31%/88.00%
0.0075	100	6	0.00004	68.41%/88.07%

[dreamerlin]

What a contribution ! We can help to train some config settings to share your gpu pressure : )

[codecov]

Codecov Report

Merging #415 (68b72ba) into master (02d5d9b) will increase coverage by 0.20%.
The diff coverage is 97.12%.

@@            Coverage Diff             @@
##           master     #415      +/-   ##
==========================================
+ Coverage   84.10%   84.31%   +0.20%     
==========================================
  Files         122      124       +2     
  Lines        8633     8771     +138     
  Branches     1435     1467      +32     
==========================================
+ Hits         7261     7395     +134     
- Misses       1018     1019       +1     
- Partials      354      357       +3     

Flag	Coverage Δ
unittests	84.30% <97.12%> (+0.20%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmaction/models/backbones/mobilenet_v2_tsm.py	94.44% <94.44%> (ø)
mmaction/models/backbones/mobilenet_v2.py	97.45% <97.45%> (ø)
mmaction/models/__init__.py	100.00% <100.00%> (ø)
mmaction/models/backbones/__init__.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02d5d9b...fd05748. Read the comment docs.

[irvingzhang0512]

What a contribution ! We can help to train some config settings to share your gpu pressure : )

The original repo doesn't provide training configs for tsm-mobilenetv2, so the training configs are copied from mmaction2-tsm-resnet50. Not sure if it is ok.

[irvingzhang0512]

trained mobilenet-tsm with DenseSampleFrames1x1x8/0.00875lr/7 gpus/50epochs, get top1/5 acuracy 63.75%/85.52%, far bellow origin ckpt 69.54%/88.66%.
May have a try in 2 weeks, a little busy these days.

[irvingzhang0512]

@kennymckormick next week I will refactor mobilenet with mmclassification

[kennymckormick]

@kennymckormick next week I will refactor mobilenet with mmclassification

Great! It would be the best case if we can directly use mmcls's mobilenet.

[irvingzhang0512]

trained mobilenet-tsm with DenseSampleFrames1x1x8/0.0025lr/4 gpus/50epochs, get top1/5 acuracy 65.11%/85.99%, far bellow origin ckpt 69.54%/88.66%(100 epochs). The best ckpt is from the 50th epoch, I guess more epochs would help.

@dreamerlin Maybe when I finish refactor mobilenetv2 with mmcls, you could kindly help to train this model with 8 gpus and 100 epochs, and upload the ckpt/json/log

[irvingzhang0512]

@dreamerlin ready to train and review.

Successfully convert origin ckpt to mmcls_mobilenetv2_tsm, get 69.5% top 1 val accuracy.

[irvingzhang0512]

trained mobilenet-tsm with DenseSampleFrames1x1x8/0.004lr/4 gpus/100epochs/0.00004 weight decay, get top1/5 acuracy 68.31%/88.00%, much better than before, not far away from origin ckpt 69.54%/88.66%(100 epochs).

Maybe the remaining 1% gap is caused by the lack of training samples(There are only 220k videos in my kinetics-400 dataset training set)

[dreamerlin]

trained mobilenet-tsm with DenseSampleFrames1x1x8/0.004lr/4 gpus/100epochs/0.00004 weight decay, get top1/5 acuracy 68.31%/88.00%, much better than before, not far away from origin ckpt 69.54%/88.66%(100 epochs).

Maybe the remaining 1% gap is caused by the lack of training samples(There are only 220k videos in my kinetics-400 dataset training set)

Sorry for my late respond due to personal affair. trained mobilenet-tsm with DenseSampleFrames1x1x8/0.01lr/8 gpus/100epochs/0.00004 weight decay, get top1/5 acuracy 68.33%/88.34%. Any suggestion ? @innerlee @irvingzhang0512

[dreamerlin]

@dreamerlin Have you trained TSM w. MobileNetv2 with the original repo on our datasets, maybe this number can be used for reference.

Yeah, I can help to train it :P

[dreamerlin]

@dreamerlin Have you trained TSM w. MobileNetv2 with the original repo on our datasets, maybe this number can be used for reference.

Yeah, I can help to train it :P

top1: 68.46, using 3 crop and full size (256)

[irvingzhang0512]

Don't know what to do. Maybe we should follow @kennymckormick 's suggestion, train tsm-mobilenetv2 with the original repo.

I'm a little busy these days, may have a try after spring festival..

[kennymckormick]

@innerlee I think we can merge this PR once the performance is the same or better than the paper. performance alignment with the released ckpt can be postponed, what's ur opinion?

[innerlee]

Yeah better than paper is good enough

[kennymckormick]

@dreamerlin Would you please help add the ckpts you trained? Maybe it's OK to merge it first and try to improve the performance later.

[dreamerlin]

related config files and links are updated now

[innerlee]

failing tests are related

[innerlee]

Thanks!