Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[WIP] Use DTensor-based tensor parallel #180

Open
wants to merge 1 commit into
base: gh/kwen2501/1/base
Choose a base branch
from
Open

[WIP] Use DTensor-based tensor parallel #180

wants to merge 1 commit into from

Conversation

kwen2501
Copy link

@kwen2501 kwen2501 commented Jun 12, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

Status:

  • Switched to DTensor based TP in regular tensor path
  • Result is correct, but there is a perf gap (seems to perform extra colls in the beginning, investigating)
  • TODO: switch to DTensor for quantized path too

kwen2501 added a commit that referenced this pull request Jun 12, 2024
@kwen2501
Use DTensor-based tensor parallel
eaae2f5 
ghstack-source-id: b55b264d20bd2c0054f7248435fd605a452e876b
Pull Request resolved: #180
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 12, 2024
@kwen2501 kwen2501 changed the title Use DTensor-based tensor parallel [WIP] Use DTensor-based tensor parallel Jun 12, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kwen2501 @facebook-github-bot