Skip to content

fix: Address runtimes with 0D inputs #2188

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 22, 2023
Merged

fix: Address runtimes with 0D inputs #2188

merged 1 commit into from
Aug 22, 2023

Conversation

gs-olive
Copy link
Collaborator

Description

  • Remove dimension padding on inputs to TRT engines in runtime
  • Add regression testing in Dynamo path

Fixes #2187

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@github-actions github-actions bot added component: core Issues re: The core compiler component: runtime component: tests Issues re: Tests labels Aug 10, 2023
@github-actions github-actions bot requested a review from narendasan August 10, 2023 03:09
@gs-olive gs-olive force-pushed the zero_dim_tensor_fix branch from 298abac to 7838109 Compare August 10, 2023 03:10
@gs-olive gs-olive changed the title fix: Address compiling models with 0D inputs fix: Address runtimes with 0D inputs Aug 10, 2023
@gs-olive
fix: Address compiling models with 0D inputs
d177389 
- Remove dimension padding on inputs to TRT engines in runtime
- Reorder memory format checks to exempt 0D FakeTensors
- Add regression test
@gs-olive gs-olive force-pushed the zero_dim_tensor_fix branch from 7838109 to d177389 Compare August 14, 2023 22:51
@github-actions github-actions bot added the component: api [Python] Issues re: Python API label Aug 14, 2023
gs-olive
gs-olive commented Aug 14, 2023
View reviewed changes
py/torch_tensorrt/_Input.py
Comment on lines +340 to +341
disable_memory_format_check
or t.is_contiguous(memory_format=torch.contiguous_format)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certain 0D tensors do not have the attribute is_contiguous, specifically when fake-ified. This logic reordering preserves the original conditional expression but ensures we do not access that attribute when disable_memory_format_check is set.

@gs-olive gs-olive merged commit e58f319 into pytorch:main Aug 22, 2023
@gs-olive gs-olive deleted the zero_dim_tensor_fix branch August 22, 2023 22:32
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@narendasan narendasan Awaiting requested review from narendasan

Assignees
No one assigned
Labels
cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: runtime component: tests Issues re: Tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

🐛 [Bug] Compiling Models with Int-to-Tensor Casted Inputs
2 participants
@gs-olive @facebook-github-bot