Skip to content

[WIP]8205-Add qat support #8209

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Draft
wants to merge 5 commits into
base: dev
Choose a base branch
from

Conversation

binliunls
Copy link
Contributor

Fixes #8205 .

Description

Try to add quantization, calibration and pruning to MONAI to enable a better inference performance.
With these functions added, model inference is expected to be faster. And we expect an inconspicuous precision loss.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

root and others added 5 commits November 14, 2024 08:27
Signed-off-by: root <root@ipp1-1899.ipp1a1.colossus.nvidia.com>
Signed-off-by: binliu <binliu@nvidia.com>
Signed-off-by: binliu <binliu@nvidia.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for model quantization.
1 participant