[Feature] Support LFB

[congee524]

Support Long-Term Feature Bank.

[codecov]

Codecov Report

Merging #553 (e611ec3) into master (ecf0208) will decrease coverage by 0.45%.
The diff coverage is 73.19%.

@@            Coverage Diff             @@
##           master     #553      +/-   ##
==========================================
- Coverage   84.75%   84.30%   -0.46%     
==========================================
  Files         127      130       +3     
  Lines        8972     9338     +366     
  Branches     1511     1572      +61     
==========================================
+ Hits         7604     7872     +268     
- Misses       1007     1071      +64     
- Partials      361      395      +34     

Flag	Coverage Δ
unittests	84.29% <73.19%> (-0.46%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmaction/models/heads/roi_head.py	26.41% <20.00%> (-1.85%)	⬇️
mmaction/models/heads/lfb_infer_head.py	47.50% <47.50%> (ø)
mmaction/models/heads/fbo_head.py	78.49% <78.49%> (ø)
mmaction/models/common/lfb.py	88.63% <88.63%> (ø)
mmaction/datasets/ava_dataset.py	90.60% <100.00%> (ø)
mmaction/models/__init__.py	100.00% <100.00%> (ø)
mmaction/models/backbones/resnet3d.py	90.66% <100.00%> (+0.05%)	⬆️
mmaction/models/common/__init__.py	100.00% <100.00%> (ø)
mmaction/models/heads/__init__.py	100.00% <100.00%> (ø)
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ecf0208...c75de61. Read the comment docs.

[congee524]

This PR and the ckpt have been almost ready. I will add unit tests next week. Maybe you could review it when you are free. :p

[innerlee]

The code looks good

[congee524]

We don't have a unit test for detectors like test_recognizers and test_localizers right now, so maybe we need a PR to fix it.

[congee524]

This PR is ready.