Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Why not recalibrate BN stats with autoslim? #75

Closed
twmht opened this issue Feb 10, 2022 · 1 comment
Closed

Why not recalibrate BN stats with autoslim? #75

twmht opened this issue Feb 10, 2022 · 1 comment

Comments

@twmht
Copy link
Contributor

twmht commented Feb 10, 2022 •
edited
Loading

Hi,

I found out the original implementation recalibrate BN stats after training.

https://github.com/JiahuiYu/slimmable_networks/blob/5dc14d0357ccfc596d706281acdc8a5b0b66c6d6/models/slimmable_ops.py#L204
https://github.com/JiahuiYu/slimmable_networks/blob/master/train.py#L758

But in mmrazor you don't do that but still get a good result

We do not recalibrating BN statistics after training

this is not consistent with the paper.

any idea? is recalibration not necessary?
@HIT-cwh
Copy link
Collaborator

HIT-cwh commented Feb 10, 2022 •
edited
Loading

In step 2, we use a large batch size and set batchnorm training mode to make sure the search is accurate.
In step 3, recalibration may be unnecessary when SwitchableBatchNorm is used.

@twmht twmht closed this as completed Feb 10, 2022
humu789 pushed a commit to humu789/mmrazor that referenced this issue Feb 13, 2023
@grimoire @SingleZombie
@AllentDan @hanrui1sensetime @RunningLeon @VVsssssk
[Refactor] Refactor config v1 (open-mmlab#80)
4c0b36b 
* [Refactor] Refactor configs according to new standard (open-mmlab#67)

* modify cfg and cfg_util

* modify tensorrt config

* fix bug

* lint

* Fix

1. Delete print
2. Modify the return value from "False, None" to "None" and related code
3. Rename 2 get functions

* modify apply_marks

* [Feature] Refactor ocr config (open-mmlab#71)

* add text detection config refactor

* add text recognition refactor

* add static exporting for mmocr

* fix lint

* set max space in child config

* use Sequence[int] instead

* add assert input_shape

* fix static bug and add ppl ort and trt static (open-mmlab#77)

* [Feature] Refine setup.py (open-mmlab#61)

* add setup.py and related files

* lint

* Edit requirements

* modify onnx version

* modify according to comments

* [Refactor] Refactor mmseg configs  (open-mmlab#73)

* refactor mmseg config

* change create_input

* fix lint

* fix lint

* fix lint

* fix yapf

* fix yapf

* update export

* remove Segmentation

* remove tast assert

* add onnx_config

* remove hardcode

* Inherit with static

* Remove blank line

* Add segmentation task enum

* add assert task

* mmocr version 0.3.0 (open-mmlab#79)

* add dump_info

* [Feature]: Refactor config in mmdet (open-mmlab#75)

* support onnxruntime

* add two stage

* test two-stage ort and ppl

* update fcos post_params

* fix calib

* test ok with maskrcnn dynamic

* add empty line

* add static into config filename

* add input_shape to create_input in mmdet

* add static to some configs

* remove todo codes

* remove partition config in base

* refactor create_input

* rename task name in mmdet

* return None if input_shape is None

* add size info into mmdet configs filenames

* reorganize mmdet configs

* add object detection task for mmdet

* rename get_mmdet_params

* keep naming style consistent

* update post_params for fcos

* fix typo in ncnn config

* [Refactor] Refactor mmedit static config (open-mmlab#78)

* add static cfg

* update create_input

* [Refactor]: Refactor mmcls configs (open-mmlab#74)

* refactor mmcls2.0

* fix classify_tensorrt_dynamic.py

* fix classify_tensorrt_dynmic.py

* classify_tensorrt_dynamic_int8.py

* fix file name

* fix ncnn ppl

* updata prepare_input.py

* update utils.py

* updata constant.py

* add

* fix prepare_input.py

* fix prepare_input.py

* add static config file

* add blank lines

* fix prepare_input.py(wait test)

* fix input_shape(wait test)

* Update prepare_input.py

* fix classification_tensorrt_dynamic(wait test)

* fix classification_tensorrt_dynamic_int8(wait test)

* fix classification_tensorrt_static_int8(wait test)

* Rename classification_tensorrt_dynamic.py to classification_tensorrt_dynamic-224x224-224x224.py

* Rename classification_tensorrt_dynamic_int8.py to classification_tensorrt_dynamic_int8-224x224-224x224.py

* Rename classification_tensorrt_dynamic_int8-224x224-224x224.py to classification_tensorrt_int8_dynamic_224x224-224x224.py

* Rename classification_tensorrt_dynamic-224x224-224x224.py to classification_tensorrt_dynamic_224x224-224x224.py

* Rename classification_tensorrt_static.py to classification_tensorrt_static_224x224.py

* Rename classification_tensorrt_static_int8.py to classification_tensorrt_int8_static_224x224.py

* Update prepare_input.py

* Rename classification_tensorrt_dynamic_224x224-224x224.py to classification_tensorrt_dynamic-224x224-224x224.py

* Rename classification_tensorrt_int8_dynamic_224x224-224x224.py to classification_tensorrt_int8-dynamic_224x224-224x224.py

* Rename classification_tensorrt_int8-dynamic_224x224-224x224.py to classification_tensorrt_int8_dynamic-224x224-224x224.py

* Rename classification_tensorrt_int8_static_224x224.py to classification_tensorrt_int8_static-224x224.py

* Rename classification_tensorrt_static_224x224.py to classification_tensorrt_static-224x224.py

* Update prepare_input.py

* Update prepare_input.py

* Update prepare_input.py

* Update prepare_input.py

* Update prepare_input.py

* Update prepare_input.py

* Update prepare_input.py

* change logging msg

Co-authored-by: maningsheng <mnsheng@yeah.net>

* fix

* fix else branch

* fix bug for trt in mmseg

* enable dump trt info

* fix trt static for mmdet

* remove two-stage_partition_tensorrt_static-800x1344 config

* fix wrong backend in ppl config

* fix partition calibration

Co-authored-by: Yifan Zhou <singlezombie@163.com>
Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com>
Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com>
Co-authored-by: RunningLeon <maningsheng@sensetime.com>
Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com>
Co-authored-by: maningsheng <mnsheng@yeah.net>
Co-authored-by: AllentDan <AllentDan@yeah.net>
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@twmht @HIT-cwh