Skip to content

Commit aa536ec

Browse files
wutongshenqiuAllentDangrimoiregrimoirehanrui1sensetime
authoredApr 19, 2022
[Feature] Add mmrazor support (open-mmlab#220)
* Torchscript support (open-mmlab#159) * support torchscript * add nms * add torchscript configs and update deploy process and dump-info * typescript -> torchscript * add torchscript custom extension support * add ts custom ops again * support mmseg unet * [WIP] add optimizer for torchscript (open-mmlab#119) * add passes * add python api * Torchscript optimizer python api (open-mmlab#121) * add passes * add python api * use python api instead of executable * Merge Master, update optimizer (open-mmlab#151) * [Feature] add yolox ncnn (open-mmlab#29) * add yolox ncnn * add ncnn android performance of yolox * add ut * fix lint * fix None bugs for ncnn * test codecov * test codecov * add device * fix yapf * remove if-else for img shape * use channelshuffle optimize * change benchmark after channelshuffle * fix yapf * fix yapf * fuse continuous reshape * fix static shape deploy * fix code * drop pad * only static shape * fix static * fix docstring * Added mask overlay to output image, changed fprintf info messages to … (open-mmlab#55) * Added mask overlay to output image, changed fprintf info messages to stdout * Improved box filtering (filter area/score), make sure roi coordinates stay within bounds * clang-format * Support UNet in mmseg (open-mmlab#77) * Repeatdataset in train has no CLASSES & PALETTE * update result for unet * update docstring for mmdet * remove ppl for unet in docs * fix ort wrap about input type (open-mmlab#81) * Fix memleak (open-mmlab#86) * delete [] * fix build error when enble MMDEPLOY_ACTIVE_LEVEL * fix lint * [Doc] Nano benchmark and tutorial (open-mmlab#71) * add cls benchmark * add nano zh-cn benchmark and en tutorial * add device row * add doc path to index.rst * fix typo * [Fix] fix missing deploy_core (open-mmlab#80) * fix missing deploy_core * mv flag to demo * target link * [Docs] Fix links in Chinese doc (open-mmlab#84) * Fix docs in Chinese link * Fix links * Delete symbolic link and add links to html * delete files * Fix link * [Feature] Add docker files (open-mmlab#67) * add gpu and cpu dockerfile * fix lint * fix cpu docker and remove redundant * use pip instead * add build arg and readme * fix grammar * update readme * add chinese doc for dockerfile and add docker build to build.md * grammar * refine dockerfiles * add FAQs * update Dpplcv_DIR for SDK building * remove mmcls * add sdk demos * fix typo and lint * update FAQs * [Fix]fix check_env (open-mmlab#101) * fix check_env * update * Replace convert_syncbatchnorm in mmseg (open-mmlab#93) * replace convert_syncbatchnorm with revert_sync_batchnorm from mmcv * change logger * [Doc] Update FAQ for TensorRT (open-mmlab#96) * update FAQ * comment * [Docs]: Update doc for openvino installation (open-mmlab#102) * fix docs * fix docs * fix docs * fix mmcv version * fix docs * rm blank line * simplify non batch nms (open-mmlab#99) * [Enhacement] Allow test.py to save evaluation results (open-mmlab#108) * Add log file * Delete debug code * Rename logger * resolve comments * [Enhancement] Support mmocr v0.4+ (open-mmlab#115) * support mmocr v0.4+ * 0.4.0 -> 0.4.1 * fix onnxruntime wrapper for gpu inference (open-mmlab#123) * fix ncnn wrapper for ort-gpu * resolve comment * fix lint * Fix typo (open-mmlab#132) * lock mmcls version (open-mmlab#131) * [Enhancement] upgrade isort in pre-commit config (open-mmlab#141) * [Enhancement] upgrade isort in pre-commit config by refering to mmflow pr open-mmlab#87 * fix lint * remove .isort.cfg and put its known_third_party to setup.cfg * Fix ci for mmocr (open-mmlab#144) * fix mmocr unittests * remove useless * lock mmdet maximum version to 2.20 * pip install -U numpy * Fix capture_output (open-mmlab#125) Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: Johannes L <tehkillerbee@users.noreply.github.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com> Co-authored-by: lvhan028 <lvhan_028@163.com> Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com> Co-authored-by: Yifan Zhou <singlezombie@163.com> Co-authored-by: 杨培文 (Yang Peiwen) <915505626@qq.com> Co-authored-by: Semyon Bevzyuk <semen.bevzuk@gmail.com> * configs for all tasks * use torchvision roi align * remote unnecessary code * fix ut * fix ut * export * det dynamic * det dynamic * add ut * fix ut * add ut and docs * fix ut * skip torchscript ut if no ops available * add torchscript option to build.md * update benchmark and resolve comments * resolve conflicts * rename configs * fix mrcnn cuda test * remove useless * add version requirements to docs and comments to codes * enable empty image exporting for torchscript and accelerate ORT inference for MRCNN * rebase * update example for torchscript.md * update FAQs for torchscript.md * resolve comments * only use torchvision roi_align for torchscript * fix ut * use torchvision roi align when pool model is avg * resolve comments Co-authored-by: grimoire <streetyao@live.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: Johannes L <tehkillerbee@users.noreply.github.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com> Co-authored-by: lvhan028 <lvhan_028@163.com> Co-authored-by: Yifan Zhou <singlezombie@163.com> Co-authored-by: 杨培文 (Yang Peiwen) <915505626@qq.com> Co-authored-by: Semyon Bevzyuk <semen.bevzuk@gmail.com> * remove roi_align plugin for ORT (open-mmlab#258) * remove roi_align plugin * remove ut * skip single_roi_extractor UT for ORT in CI * move align to symbolic and update docs * recover UT * resolve comments * add mmcls example * add mmcls/mmdet/mmseg and their corresponding tests * add test data * simplify test data * add requirement in optional.txt * fix setup problem when adding mmrazor requirement * use get_codebase_config * change mmrazor requirement Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com> Co-authored-by: grimoire <streetyao@live.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: Johannes L <tehkillerbee@users.noreply.github.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com> Co-authored-by: lvhan028 <lvhan_028@163.com> Co-authored-by: Yifan Zhou <singlezombie@163.com> Co-authored-by: 杨培文 (Yang Peiwen) <915505626@qq.com> Co-authored-by: Semyon Bevzyuk <semen.bevzuk@gmail.com>
1 parent 9411fe0 commit aa536ec

File tree

13 files changed

+332
-10
lines changed

13 files changed

+332
-10
lines changed
 

‎mmdeploy/codebase/base/task.py

+20-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@
77
import torch
88
from torch.utils.data import DataLoader, Dataset
99

10-
from mmdeploy.utils import get_backend_config, get_codebase, get_root_logger
10+
from mmdeploy.utils import (get_backend_config, get_codebase,
11+
get_codebase_config, get_root_logger)
1112
from mmdeploy.utils.dataset import is_can_sort_dataset, sort_dataset
1213

1314

@@ -284,3 +285,21 @@ def get_model_name(self) -> str:
284285
str: the name of the model.
285286
"""
286287
pass
288+
289+
@property
290+
def from_mmrazor(self) -> bool:
291+
"""Whether the codebase from mmrazor.
292+
293+
Returns:
294+
bool: From mmrazor or not.
295+
296+
Raises:
297+
TypeError: An error when type of `from_mmrazor` is not boolean.
298+
"""
299+
codebase_config = get_codebase_config(self.deploy_cfg)
300+
from_mmrazor = codebase_config.get('from_mmrazor', False)
301+
if not isinstance(from_mmrazor, bool):
302+
raise TypeError('`from_mmrazor` attribute must be boolean type! '
303+
f'but got: {from_mmrazor}')
304+
305+
return from_mmrazor

‎mmdeploy/codebase/mmcls/deploy/classification.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,11 @@ def init_pytorch_model(self,
9595
nn.Module: An initialized torch model generated by OpenMMLab
9696
codebases.
9797
"""
98-
from mmcls.apis import init_model
98+
if self.from_mmrazor:
99+
from mmrazor.apis import init_mmcls_model as init_model
100+
else:
101+
from mmcls.apis import init_model
102+
99103
model = init_model(self.model_cfg, model_checkpoint, self.device,
100104
cfg_options)
101105

‎mmdeploy/codebase/mmdet/deploy/object_detection.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,11 @@ def init_pytorch_model(self,
8989
nn.Module: An initialized torch model generated by other OpenMMLab
9090
codebases.
9191
"""
92-
from mmdet.apis import init_detector
92+
if self.from_mmrazor:
93+
from mmrazor.apis import init_mmdet_model as init_detector
94+
else:
95+
from mmdet.apis import init_detector
96+
9397
model = init_detector(self.model_cfg, model_checkpoint, self.device,
9498
cfg_options)
9599
return model.eval()

‎mmdeploy/codebase/mmseg/deploy/segmentation.py

+5-1
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,11 @@ def init_pytorch_model(self,
8989
codebases.
9090
"""
9191
from mmcv.cnn.utils import revert_sync_batchnorm
92-
from mmseg.apis import init_segmentor
92+
if self.from_mmrazor:
93+
from mmrazor.apis import init_mmseg_model as init_segmentor
94+
else:
95+
from mmseg.apis import init_segmentor
96+
9397
model = init_segmentor(self.model_cfg, model_checkpoint, self.device)
9498
model = revert_sync_batchnorm(model)
9599
return model.eval()

‎requirements/optional.txt

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ mmdet>=2.19.0,<=2.20.0
33
mmedit
44
mmocr>=0.3.0,<=0.4.1
55
mmpose>=0.24.0
6+
mmrazor>=0.3.0
67
mmsegmentation
78
onnxruntime>=1.8.0
89
openvino-dev

‎tests/test_codebase/test_mmcls/data/mmrazor_model.py

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Copyright (c) OpenMMLab. All rights reserved.
2+
_base_ = 'model.py'
3+
4+
norm_cfg = dict(type='BN')
5+
6+
mutator = dict(
7+
type='OneShotMutator',
8+
placeholder_mapping=dict(
9+
all_blocks=dict(
10+
type='OneShotOP',
11+
choices=dict(
12+
shuffle_3x3=dict(
13+
type='ShuffleBlock', kernel_size=3, norm_cfg=norm_cfg),
14+
shuffle_5x5=dict(
15+
type='ShuffleBlock', kernel_size=5, norm_cfg=norm_cfg),
16+
shuffle_7x7=dict(
17+
type='ShuffleBlock', kernel_size=7, norm_cfg=norm_cfg),
18+
shuffle_xception=dict(
19+
type='ShuffleXception', norm_cfg=norm_cfg),
20+
))))
21+
22+
algorithm = dict(
23+
type='SPOS',
24+
architecture=dict(
25+
type='MMClsArchitecture',
26+
model={{_base_.model}},
27+
),
28+
mutator=mutator,
29+
distiller=None,
30+
mutable_cfg='tests/test_codebase/test_mmcls/data/mmrazor_mutable_cfg.yaml',
31+
retraining=True)

‎tests/test_codebase/test_mmcls/data/mmrazor_mutable_cfg.yaml

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
stage_0_block_0:
2+
chosen:
3+
- shuffle_7x7
4+
stage_0_block_1:
5+
chosen:
6+
- shuffle_5x5
7+
stage_0_block_2:
8+
chosen:
9+
- shuffle_3x3
10+
stage_0_block_3:
11+
chosen:
12+
- shuffle_5x5
13+
stage_1_block_0:
14+
chosen:
15+
- shuffle_7x7
16+
stage_1_block_1:
17+
chosen:
18+
- shuffle_3x3
19+
stage_1_block_2:
20+
chosen:
21+
- shuffle_7x7
22+
stage_1_block_3:
23+
chosen:
24+
- shuffle_3x3
25+
stage_2_block_0:
26+
chosen:
27+
- shuffle_7x7
28+
stage_2_block_1:
29+
chosen:
30+
- shuffle_3x3
31+
stage_2_block_2:
32+
chosen:
33+
- shuffle_7x7
34+
stage_2_block_3:
35+
chosen:
36+
- shuffle_xception
37+
stage_2_block_4:
38+
chosen:
39+
- shuffle_3x3
40+
stage_2_block_5:
41+
chosen:
42+
- shuffle_3x3
43+
stage_2_block_6:
44+
chosen:
45+
- shuffle_3x3
46+
stage_2_block_7:
47+
chosen:
48+
- shuffle_3x3
49+
stage_3_block_0:
50+
chosen:
51+
- shuffle_xception
52+
stage_3_block_1:
53+
chosen:
54+
- shuffle_7x7
55+
stage_3_block_2:
56+
chosen:
57+
- shuffle_xception
58+
stage_3_block_3:
59+
chosen:
60+
- shuffle_xception

‎tests/test_codebase/test_mmcls/test_classification.py

+28-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Copyright (c) OpenMMLab. All rights reserved.
2+
import copy
23
import os
34
from tempfile import NamedTemporaryFile, TemporaryDirectory
5+
from typing import Any
46

57
import mmcv
68
import numpy as np
@@ -37,9 +39,33 @@
3739
img = np.random.rand(*img_shape, 3)
3840

3941

40-
def test_init_pytorch_model():
42+
@pytest.mark.parametrize('from_mmrazor', [True, False, '123', 0])
43+
def test_init_pytorch_model(from_mmrazor: Any):
4144
from mmcls.models.classifiers.base import BaseClassifier
42-
model = task_processor.init_pytorch_model(None)
45+
if from_mmrazor is False:
46+
_task_processor = task_processor
47+
else:
48+
_model_cfg_path = 'tests/test_codebase/test_mmcls/data/' \
49+
'mmrazor_model.py'
50+
_model_cfg = load_config(_model_cfg_path)[0]
51+
_model_cfg.algorithm.architecture.model.type = 'mmcls.ImageClassifier'
52+
_model_cfg.algorithm.architecture.model.backbone = dict(
53+
type='SearchableShuffleNetV2', widen_factor=1.0)
54+
_deploy_cfg = copy.deepcopy(deploy_cfg)
55+
_deploy_cfg.codebase_config['from_mmrazor'] = from_mmrazor
56+
_task_processor = build_task_processor(_model_cfg, _deploy_cfg, 'cpu')
57+
58+
if not isinstance(from_mmrazor, bool):
59+
with pytest.raises(
60+
TypeError,
61+
match='`from_mmrazor` attribute must be '
62+
'boolean type! '
63+
f'but got: {from_mmrazor}'):
64+
_ = _task_processor.from_mmrazor
65+
return
66+
assert from_mmrazor == _task_processor.from_mmrazor
67+
68+
model = _task_processor.init_pytorch_model(None)
4369
assert isinstance(model, BaseClassifier)
4470

4571

‎tests/test_codebase/test_mmdet/data/mmrazor_model.py

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Copyright (c) OpenMMLab. All rights reserved.
2+
_base_ = 'model.py'
3+
4+
norm_cfg = dict(type='BN', requires_grad=True)
5+
mutator = dict(
6+
type='OneShotMutator',
7+
placeholder_mapping=dict(
8+
all_blocks=dict(
9+
type='OneShotOP',
10+
choices=dict(
11+
shuffle_3x3=dict(
12+
type='ShuffleBlock', norm_cfg=norm_cfg, kernel_size=3),
13+
shuffle_5x5=dict(
14+
type='ShuffleBlock', norm_cfg=norm_cfg, kernel_size=5),
15+
shuffle_7x7=dict(
16+
type='ShuffleBlock', norm_cfg=norm_cfg, kernel_size=7),
17+
shuffle_xception=dict(
18+
type='ShuffleXception',
19+
norm_cfg=norm_cfg,
20+
),
21+
))))
22+
23+
algorithm = dict(
24+
type='DetNAS',
25+
architecture=dict(
26+
type='MMDetArchitecture',
27+
model={{_base_.model}},
28+
),
29+
mutator=mutator,
30+
pruner=None,
31+
distiller=None,
32+
retraining=True,
33+
mutable_cfg='tests/test_codebase/test_mmdet/data/mmrazor_mutable_cfg.yaml',
34+
)

‎tests/test_codebase/test_mmdet/data/mmrazor_mutable_cfg.yaml

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
stage_0_block_0:
2+
chosen:
3+
- shuffle_7x7
4+
stage_0_block_1:
5+
chosen:
6+
- shuffle_5x5
7+
stage_0_block_2:
8+
chosen:
9+
- shuffle_7x7
10+
stage_0_block_3:
11+
chosen:
12+
- shuffle_3x3
13+
stage_1_block_0:
14+
chosen:
15+
- shuffle_7x7
16+
stage_1_block_1:
17+
chosen:
18+
- shuffle_5x5
19+
stage_1_block_2:
20+
chosen:
21+
- shuffle_5x5
22+
stage_1_block_3:
23+
chosen:
24+
- shuffle_7x7
25+
stage_2_block_0:
26+
chosen:
27+
- shuffle_xception
28+
stage_2_block_1:
29+
chosen:
30+
- shuffle_xception
31+
stage_2_block_2:
32+
chosen:
33+
- shuffle_5x5
34+
stage_2_block_3:
35+
chosen:
36+
- shuffle_xception
37+
stage_2_block_4:
38+
chosen:
39+
- shuffle_3x3
40+
stage_2_block_5:
41+
chosen:
42+
- shuffle_3x3
43+
stage_2_block_6:
44+
chosen:
45+
- shuffle_xception
46+
stage_2_block_7:
47+
chosen:
48+
- shuffle_5x5
49+
stage_3_block_0:
50+
chosen:
51+
- shuffle_xception
52+
stage_3_block_1:
53+
chosen:
54+
- shuffle_5x5
55+
stage_3_block_2:
56+
chosen:
57+
- shuffle_xception
58+
stage_3_block_3:
59+
chosen:
60+
- shuffle_7x7

‎tests/test_codebase/test_mmdet/test_object_detection.py

+28-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Copyright (c) OpenMMLab. All rights reserved.
2+
import copy
23
import os
34
from tempfile import NamedTemporaryFile, TemporaryDirectory
5+
from typing import Any
46

57
import mmcv
68
import numpy as np
@@ -48,9 +50,33 @@
4850
img = np.random.rand(*img_shape, 3)
4951

5052

51-
def test_init_pytorch_model():
53+
@pytest.mark.parametrize('from_mmrazor', [True, False, '123', 0])
54+
def test_init_pytorch_model(from_mmrazor: Any):
5255
from mmdet.models import BaseDetector
53-
model = task_processor.init_pytorch_model(None)
56+
if from_mmrazor is False:
57+
_task_processor = task_processor
58+
else:
59+
_model_cfg_path = 'tests/test_codebase/test_mmdet/data/' \
60+
'mmrazor_model.py'
61+
_model_cfg = load_config(_model_cfg_path)[0]
62+
_model_cfg.algorithm.architecture.model.type = 'mmdet.YOLOV3'
63+
_model_cfg.algorithm.architecture.model.backbone.type = \
64+
'mmcls.SearchableShuffleNetV2'
65+
_deploy_cfg = copy.deepcopy(deploy_cfg)
66+
_deploy_cfg.codebase_config['from_mmrazor'] = from_mmrazor
67+
_task_processor = build_task_processor(_model_cfg, _deploy_cfg, 'cpu')
68+
69+
if not isinstance(from_mmrazor, bool):
70+
with pytest.raises(
71+
TypeError,
72+
match='`from_mmrazor` attribute must be '
73+
'boolean type! '
74+
f'but got: {from_mmrazor}'):
75+
_ = _task_processor.from_mmrazor
76+
return
77+
assert from_mmrazor == _task_processor.from_mmrazor
78+
79+
model = _task_processor.init_pytorch_model(None)
5480
assert isinstance(model, BaseDetector)
5581

5682

‎tests/test_codebase/test_mmseg/data/mmrazor_model.py

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Copyright (c) OpenMMLab. All rights reserved.
2+
_base_ = 'model.py'
3+
4+
# algorithm setting
5+
algorithm = dict(
6+
type='GeneralDistill',
7+
architecture=dict(
8+
type='MMSegArchitecture',
9+
model={{_base_.model}},
10+
),
11+
distiller=dict(
12+
type='SingleTeacherDistiller',
13+
teacher={{_base_.model}},
14+
teacher_trainable=False,
15+
components=[
16+
dict(
17+
student_module='decode_head.conv_seg',
18+
teacher_module='decode_head.conv_seg',
19+
losses=[
20+
dict(
21+
type='ChannelWiseDivergence',
22+
name='loss_cwd_logits',
23+
tau=1,
24+
loss_weight=5,
25+
)
26+
])
27+
]),
28+
)

0 commit comments

Comments
 (0)