Support efficient video decoder and PIMS decoder

[dreamerlin]

No description provided.

[dreamerlin]

unittest and docstring will be updated soon

[codecov]

Codecov Report

Merging #946 (e473b8a) into master (e67405a) will increase coverage by 0.09%.
The diff coverage is 86.66%.

@@            Coverage Diff             @@
##           master     #946      +/-   ##
==========================================
+ Coverage   83.67%   83.77%   +0.09%     
==========================================
  Files         132      132              
  Lines        9983    10056      +73     
  Branches     1722     1733      +11     
==========================================
+ Hits         8353     8424      +71     
  Misses       1212     1212              
- Partials      418      420       +2     

Flag	Coverage Δ
unittests	83.77% <86.66%> (+0.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmaction/datasets/pipelines/__init__.py	100.00% <ø> (ø)
mmaction/datasets/pipelines/loading.py	89.18% <86.66%> (-0.43%)	⬇️
mmaction/core/evaluation/eval_hooks.py	11.17% <0.00%> (-0.59%)	⬇️
mmaction/models/heads/bbox_head.py	91.47% <0.00%> (ø)
mmaction/datasets/pipelines/pose_loading.py	94.38% <0.00%> (ø)
...tion/core/evaluation/ava_evaluation/np_box_list.py	47.72% <0.00%> (ø)
mmaction/models/backbones/resnet_audio.py	81.29% <0.00%> (+0.13%)	⬆️
...tion/ava_evaluation/object_detection_evaluation.py	70.83% <0.00%> (+0.17%)	⬆️
.../evaluation/ava_evaluation/per_image_evaluation.py	83.09% <0.00%> (+0.24%)	⬆️
mmaction/datasets/pipelines/augmentations.py	92.98% <0.00%> (+0.34%)	⬆️
... and 5 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e67405a...e473b8a. Read the comment docs.

[kennymckormick]

suggest to train one model with PIMS decoder (accurate / efficient) to validate its efficacy.

[dreamerlin]

For TSN 1x1x3 UCF-101:

Setting	Top-1	Top-5
Decord accurate	82.50	96.93
Decord efficient	75.97	92.84
PyAV accurate	82.50	96.93
PyAV efficient	75.50	93.10
PIMS accurate	82.50	96.93
PIMS efficient	82.50	96.93

For TSM 1x1x8 Diving48:

Setting	Top-1	Top-5
Decord accurate	77.41	97.06
Decord efficient	17.97	53.10
PyAV accurate	77.37	97.01
PyAV efficient	26.60	63.30
PIMS accurate	78.07	97.72
PIMS efficient	76.50	97.46

According to these results, I updated some descriptions in the docstring.

[innerlee]

Curious about the speed of PIMS

[dreamerlin]

Speed: Decord > PyAV ≈ PIMS

[whwu95]

@dreamerlin Hi Jintao, it’s better to state that this part of the code comes from MVFNet.

[jayleicn]

Speed: Decord > PyAV ≈ PIMS

Hi @dreamerlin, Do you have a numerical comparison of these video decoders? Like how many times faster can Decord be compared to PyAV? Meanwhile, is Decord used here the cpu version or the gpu version? How's speed comparison between the cpu and gpu verison of Decord? Thanks!