[Feature] Support GPU Normalize

[congee524]

fix #470

[codecov]

Codecov Report

Merging #586 (d69cbf9) into master (d8f6f34) will increase coverage by 0.02%.
The diff coverage is 92.00%.

@@            Coverage Diff             @@
##           master     #586      +/-   ##
==========================================
+ Coverage   84.31%   84.33%   +0.02%     
==========================================
  Files         124      125       +1     
  Lines        8774     8824      +50     
  Branches     1468     1479      +11     
==========================================
+ Hits         7398     7442      +44     
- Misses       1019     1022       +3     
- Partials      357      360       +3     

Flag	Coverage Δ
unittests	84.32% <92.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmaction/core/evaluation/eval_hooks.py	86.09% <ø> (ø)
mmaction/datasets/pipelines/formating.py	88.60% <0.00%> (-1.14%)	⬇️
mmaction/utils/module_hooks.py	95.74% <95.74%> (ø)
mmaction/utils/__init__.py	100.00% <100.00%> (ø)
mmaction/core/evaluation/accuracy.py	92.27% <0.00%> (-0.91%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8f6f34...a9a13f1. Read the comment docs.

[congee524]

@innerlee kindly ping :p

[innerlee]

I suppose there should be more reviewers? cc. @hellock

[congee524]

Test GPU Normalize on 1080TI, TITAN XP, and Tesla V100.
Only on V100, GPU Normalize can significantly increase the training speed and shorten the training time by nearly half.

[innerlee]

Okay, you can train a full model in V100, and upload the ckpt to modelzoo for future reference.