feat: Module-Acceleration in Dynamo [5 / x] #1979

gs-olive · 2023-06-05T16:32:21Z

Description

Add support for excluding entire Torch modules from tracing in Dynamo using Torch custom operators
Develop new dataclass to store required replacement functions and operators in a streamlined way
Add new registry to store mapping between replacement operators and their corresponding dataclass
Add detailed tutorial descriptions to sample module substitution with step-by-step detailed instructions for creating a new module substitution

Fixes #1894

Type of change

New feature (non-breaking change which adds functionality)

Checklist:

[ x ] My code follows the style guidelines of this project (You can use the linters)
[ x ] I have performed a self-review of my own code
[ x ] I have commented my code, particularly in hard-to-understand areas and hacks
[ x ] I have made corresponding changes to the documentation
[ x ] I have added tests to verify my fix or my feature
[ x ] New and existing unit tests pass locally with my changes
[ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

py/torch_tensorrt/dynamo/backend/backends.py

narendasan · 2023-06-28T19:12:54Z

py/torch_tensorrt/dynamo/backend/lowering/module_substitutions/maxpool1d.py

+        "ceil_mode": False if len(args) < 6 else args[5],
+    }
+
+    return acc_ops_converters.acc_ops_max_pool1d(


I guess this would depend on the dynamo converters change but it would be good to use the impl directly, we can also add a TODO and comeback after

Instanced #1985 to add this in a subsequent PR

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

narendasan

LGTM, lets add a tutorial on this in the contributor docs

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

- Add support for excluding entire Torch modules from tracing in Dynamo using Torch custom operators - Develop new dataclass to store required replacement functions and operators in a streamlined way - Add new registry to store mapping between replacement operators and their corresponding dataclass - Add documentation for easy additions of new module-level exclusion operators - Add robust testing and address recent review comments

… [6 / x] (#1980)

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

gs-olive requested a review from narendasan June 5, 2023 16:32

gs-olive self-assigned this Jun 5, 2023

facebook-github-bot added the cla signed label Jun 5, 2023

github-actions bot added the component: api [Python] Issues re: Python API label Jun 5, 2023

gs-olive added component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path and removed component: api [Python] Issues re: Python API cla signed labels Jun 5, 2023

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive mentioned this pull request Jun 5, 2023

feat: Prototype Module-Acceleration in Dynamo #1921

Closed

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_module_level_acceleration branch from 668156b to 80b898a Compare June 5, 2023 16:36

facebook-github-bot added the cla signed label Jun 5, 2023

github-actions bot added the component: api [Python] Issues re: Python API label Jun 5, 2023

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive force-pushed the dynamo_module_level_acceleration branch from 80b898a to d235c0f Compare June 5, 2023 17:19

github-actions bot approved these changes Jun 5, 2023

View reviewed changes

gs-olive mentioned this pull request Jun 5, 2023

Centralize converters for MaxPool1D and einsum #1985

Closed

gs-olive force-pushed the torch_version_upgrade_jun23_nightly branch from 4ba253f to 1bf1ff7 Compare June 8, 2023 20:08

gs-olive force-pushed the dynamo_module_level_acceleration branch from d235c0f to a0ff354 Compare June 8, 2023 20:12

gs-olive changed the title ~~feat: Prototype Module-Acceleration in Dynamo [4 / x]~~ feat: Module-Acceleration in Dynamo [7 / x] Jun 8, 2023

github-actions bot approved these changes Jun 8, 2023

View reviewed changes

gs-olive force-pushed the torch_version_upgrade_jun23_nightly branch from 1bf1ff7 to 741364a Compare June 8, 2023 23:30

gs-olive force-pushed the dynamo_module_level_acceleration branch from a0ff354 to b16d17b Compare June 8, 2023 23:30

github-actions bot approved these changes Jun 8, 2023

View reviewed changes

gs-olive force-pushed the torch_version_upgrade_jun23_nightly branch 3 times, most recently from d0c39f3 to 3751e32 Compare June 20, 2023 19:53

gs-olive mentioned this pull request Jun 20, 2023

Add example scripts for module/function level acceleration #1984

Closed

gs-olive force-pushed the dynamo_module_level_acceleration branch from c0715f2 to 57215f1 Compare June 21, 2023 00:56

gs-olive changed the base branch from torch_version_upgrade_jun23_nightly to dynamo_compile_trt_module_next June 21, 2023 00:56

github-actions bot approved these changes Jun 21, 2023

View reviewed changes

gs-olive force-pushed the dynamo_module_level_acceleration branch from 57215f1 to fbe12e3 Compare June 23, 2023 03:19

gs-olive changed the base branch from dynamo_compile_trt_module_next to main June 23, 2023 03:19

github-actions bot approved these changes Jun 23, 2023

View reviewed changes

narendasan reviewed Jun 28, 2023

View reviewed changes

gs-olive force-pushed the dynamo_module_level_acceleration branch from fbe12e3 to 7e2388f Compare June 28, 2023 19:48

github-actions bot approved these changes Jun 28, 2023

View reviewed changes

gs-olive requested a review from narendasan June 28, 2023 21:11

github-actions bot approved these changes Jun 28, 2023

View reviewed changes

narendasan approved these changes Jun 28, 2023

View reviewed changes

github-actions bot approved these changes Jun 29, 2023

View reviewed changes

gs-olive added 2 commits June 28, 2023 17:53

feat: Add support for general-purpose function acceleration in Dynamo…

c9f06fc

… [6 / x] (#1980)

gs-olive force-pushed the dynamo_module_level_acceleration branch from f26ca80 to c9f06fc Compare June 29, 2023 00:53

github-actions bot approved these changes Jun 29, 2023

View reviewed changes

gs-olive merged commit 2844630 into main Jun 29, 2023

gs-olive deleted the dynamo_module_level_acceleration branch June 29, 2023 16:08

gs-olive mentioned this pull request Apr 5, 2024

✨[Feature] Preserve Higher-Level Operators in Torch-TRT Dynamo Paths #2729

Open

feat: Module-Acceleration in Dynamo [5 / x] #1979

feat: Module-Acceleration in Dynamo [5 / x] #1979

Uh oh!

Conversation

gs-olive commented Jun 5, 2023

Description

Type of change

Checklist:

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

narendasan Jun 28, 2023

Choose a reason for hiding this comment

Uh oh!

gs-olive Jun 28, 2023

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

narendasan left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment