Skip to content
This repository was archived by the owner on Oct 19, 2024. It is now read-only.

[FEATURE] load balance in cross-mesh resharding #798

Merged
merged 16 commits into from
Dec 17, 2022
Merged

[FEATURE] load balance in cross-mesh resharding #798

merged 16 commits into from
Dec 17, 2022

Conversation

TarzanZhao
Copy link
Collaborator

@TarzanZhao TarzanZhao commented Dec 2, 2022 •
edited
Loading

In this PR,

  1. Merge the load balance code for cross mesh resharding. For more details, please refer to #486 . This feature is introduced in On Optimizing the Communication of Model Parallelism.
  2. Add code for microbenchmark in On Optimizing the Communication of Model Parallelism.

@merrymercy merrymercy requested a review from ZYHowell December 2, 2022 19:58
@ZYHowell ZYHowell changed the title Resharding Load balance feature and microbenchmarks [FEATURE] load balance in cross-mesh resharding Dec 2, 2022
ZYHowell
ZYHowell suggested changes Dec 10, 2022
View reviewed changes
@merrymercy
Copy link
Member

@ZYHowell Review the code and feel free to merge when you think it is ready

@TarzanZhao
Copy link
Collaborator Author

@merrymercy I will modify the code as Yonghao requested after I finish my phd application in two days.

@TarzanZhao TarzanZhao requested a review from ZYHowell December 16, 2022 13:23
@ZYHowell ZYHowell merged commit 6f1aab4 into main Dec 17, 2022
@ZYHowell
Copy link
Collaborator

Thanks for your efforts @TarzanZhao

@ZYHowell ZYHowell deleted the pr-resharding-microbenchmarks branch December 17, 2022 03:20
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Reviewers

@ZYHowell ZYHowell ZYHowell approved these changes

Assignees

@ZYHowell ZYHowell

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@TarzanZhao @merrymercy @ZYHowell