Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Relieve memory usage criteria on batch size 2 during adaptive_bs for otx 2.3 #4023

Closed

Conversation

eunwoosh
Copy link
Contributor

@eunwoosh eunwoosh commented Oct 14, 2024

Summary

Same as #4009 but for otx 2.3
adpative_bs decides whether batch size is available conservatively. So even if some batch size doesn't make OOM, adaptive_bs can refuse to use it if it uses most of memory.
It's same to batch size 2, but it's better to try it if it doesn't make OOM because the only left thing to do is just raising an error if not.
So, this PR changes adaptive_bs to allow to return batch size 2 if it doesn't make OOM and use most of memory.

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have ran e2e tests and there is no issues.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

Copy link
Collaborator

@kprokofi kprokofi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to duplicate PRs from releases2.2 to releases2.3. We will perform one back merge PR after releases2.2 is stable.

@eunwoosh eunwoosh closed this Oct 14, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
OTX 2.0 TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants