Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Accuracy drop on Post-Training Quantization (PTQ) optimized model for maskrcnn-swimt #3593

Closed
yunchu opened this issue Jun 10, 2024 · 4 comments
Closed

Accuracy drop on Post-Training Quantization (PTQ) optimized model for maskrcnn-swimt #3593

yunchu opened this issue Jun 10, 2024 · 4 comments
Assignees
@sovrasov
Milestone
2.2.0

Comments

@yunchu
Copy link
Contributor

yunchu commented Jun 10, 2024

Post-Training Quantization (PTQ) optimization applied to maskrcnn_swimt in the instance segmentation task may result in significantly reduced accuracy.

How to run the ins_seg benchmark tests for OTX 2.0.0

pytest -vs tests/perf/test_instance_segmentation.py --output-root benchmark_otx_2.0.0_result --model-category all --data-root /path/to/val-data/v2 --eval-upto optimize --num-repeat 5

image-2024-06-05-16-29-29-929
@yunchu yunchu added this to the 2.1.0 milestone Jun 10, 2024
@chuneuny-emily
Copy link
Contributor

cc'ed @eugene123tw

@eugene123tw
Copy link
Contributor

Fixed in #3685

@chuneuny-emily
Copy link
Contributor

This issue is expected to be addressed with an upgrade to OpenVINO and NNCF in a future release.

@sovrasov sovrasov modified the milestones: 2.1.0, 2.2.0 Jul 24, 2024
@sovrasov
Copy link
Contributor

Fixed in #3900

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@sovrasov sovrasov

Labels
None yet
Projects
None yet
Milestone
2.2.0
Development

No branches or pull requests
5 participants
@yunchu @sovrasov @eugene123tw @harimkang @chuneuny-emily