Skip to content

fix: Add new TRT 8.6 features to Dynamo compile #1971

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed

Conversation

gs-olive
Copy link
Collaborator

@gs-olive gs-olive commented Jun 2, 2023

Description

  • Unify TRTInterpreter and other utilities to use the versions in the dynamo directory
  • Add support for new TRT 8.6 utilities, including auxiliary streams, version compatibility, and optimization levels, as per Upstream 3 features to fx_ts_compat: MS, VC, Optimization Level #1935
  • Add support for TRTModuleNext use during compilation with Dynamo compile
  • Improve documentation of features and version checking for TRT feature compatibility

New Options in this PR:

  • max_aux_streams
  • version_compatible
  • optimization_level
  • use_experimental_rt

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

@gs-olive gs-olive added component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Export/Compile Unification Issues relating to unification of Dynamo compile/export paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path labels Jun 2, 2023
@gs-olive gs-olive self-assigned this Jun 2, 2023
@gs-olive gs-olive requested a review from narendasan June 2, 2023 02:51
@github-actions github-actions bot added the component: api [Python] Issues re: Python API label Jun 2, 2023
@gs-olive gs-olive force-pushed the dynamo_compile_new_trt_features branch from cb2be86 to c46f614 Compare June 2, 2023 03:48
- Update argument naming for compatibility with `fx_ts_compat` utilities
- Add support for new TRT 8.6 utilities, including auxiliary streams,
version compatibility, and optimization levels
- Add support for TRTModuleNext use during compilation with Dynamo
compile
- Improve documentation of features and version checking for TRT feature
compatibility
@gs-olive gs-olive force-pushed the dynamo_compile_new_trt_features branch from c46f614 to a5eb7d4 Compare June 2, 2023 17:56
@gs-olive gs-olive closed this Jun 2, 2023
@gs-olive gs-olive deleted the dynamo_compile_new_trt_features branch June 2, 2023 20:52
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
cla signed component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path Story: Export/Compile Unification Issues relating to unification of Dynamo compile/export paths
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants