Skip to content

perf: Add lowering passes to improve TRT runtime on SD #2351

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Merged
merged 2 commits into from
Sep 29, 2023
Merged

perf: Add lowering passes to improve TRT runtime on SD #2351

merged 2 commits into from
Sep 29, 2023

Conversation

gs-olive
Copy link
Collaborator

Description

  • Focus on variance and sum converters, reducing instances of extraneous layers from unnecessary reshapes
  • Add test cases to validate new additions

Fixes #2350

Type of change

  • Perf improvement

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@gs-olive gs-olive self-assigned this Sep 28, 2023
@github-actions github-actions bot added component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests component: torch_compile documentation Improvements or additions to documentation labels Sep 28, 2023
@github-actions github-actions bot requested a review from narendasan September 28, 2023 19:27
github-actions[bot]
github-actions bot approved these changes Sep 28, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to Python style guidelines

github-actions[bot]
github-actions bot approved these changes Sep 28, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to C++ style guidelines

@github-actions github-actions bot added component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters labels Sep 28, 2023
github-actions[bot]
github-actions bot approved these changes Sep 28, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to C++ style guidelines

github-actions[bot]
github-actions bot approved these changes Sep 28, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to Python style guidelines

@gs-olive gs-olive changed the title perf: Add lowering passes to improve TRT conversion perf: Add lowering passes to improve TRT runtime on SD Sep 28, 2023
narendasan
narendasan approved these changes Sep 29, 2023
View reviewed changes
Copy link
Collaborator

@narendasan narendasan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot requested a review from peri044 September 29, 2023 18:55
@gs-olive gs-olive force-pushed the sd_lowering_pass_optimizations branch from 4c3f4bf to 6cbb24e Compare September 29, 2023 21:13
github-actions[bot]
github-actions bot approved these changes Sep 29, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to Python style guidelines

github-actions[bot]
github-actions bot approved these changes Sep 29, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to C++ style guidelines

@gs-olive
perf: Add lowering passes to improve TRT conversion
e746243 
- Focus on variance and sum converters, reducing instances of extraneous
layers from unnecessary reshapes
- Add test cases to validate new additions
@gs-olive
perf: Add efficient attention lowering pass
b42203e
@gs-olive gs-olive force-pushed the sd_lowering_pass_optimizations branch from 6cbb24e to b42203e Compare September 29, 2023 22:33
github-actions[bot]
github-actions bot approved these changes Sep 29, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to C++ style guidelines

github-actions[bot]
github-actions bot approved these changes Sep 29, 2023
View reviewed changes
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code conforms to Python style guidelines

@gs-olive gs-olive merged commit 8ebf24d into main Sep 29, 2023
@gs-olive gs-olive deleted the sd_lowering_pass_optimizations branch September 29, 2023 23:39
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@narendasan narendasan narendasan approved these changes

@github-actions github-actions[bot] github-actions[bot] approved these changes

@peri044 peri044 Awaiting requested review from peri044

Assignees

@gs-olive gs-olive

Labels
cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests component: torch_compile documentation Improvements or additions to documentation priority: high
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement variance converter
3 participants
@gs-olive @narendasan @facebook-github-bot