Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Unified Checkpoint] update non-merge checkpoint loading, move async_save_info.json location #9321

Merged
merged 25 commits into from
Oct 28, 2024

Conversation

DesmonDay
Copy link
Contributor

@DesmonDay DesmonDay commented Oct 28, 2024

PR types

Others

PR changes

Others

Description

  1. Update non-merge checkpoint loading to multi-threading load.
  2. Update async_save_info.json location.

DesmonDay and others added 23 commits August 21, 2024 14:56
Copy link

paddle-bot bot commented Oct 28, 2024

Thanks for your contribution!

@DesmonDay DesmonDay changed the title [Unified Checkpoint] [Unified Checkpoint] Update async_save_info.json location. Oct 28, 2024
Copy link

codecov bot commented Oct 28, 2024

Codecov Report

Attention: Patch coverage is 20.00000% with 8 lines in your changes missing coverage. Please review.

Project coverage is 52.86%. Comparing base (c9d5673) to head (fd5dea0).
Report is 257 commits behind head on develop.

Files with missing lines Patch % Lines
paddlenlp/trainer/trainer.py 20.00% 4 Missing ⚠️
...p/trainer/unified_checkpoint/unified_checkpoint.py 20.00% 4 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #9321   +/-   ##
========================================
  Coverage    52.86%   52.86%           
========================================
  Files          669      669           
  Lines       107240   107229   -11     
========================================
- Hits         56688    56686    -2     
+ Misses       50552    50543    -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DesmonDay DesmonDay changed the title [Unified Checkpoint] Update async_save_info.json location. [Unified Checkpoint] update non-merge checkpoint loading, move async_save_info.json location Oct 28, 2024
Copy link
Collaborator

@ZHUI ZHUI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ZHUI ZHUI merged commit ce3a1ce into PaddlePaddle:develop Oct 28, 2024
9 of 12 checks passed
DesmonDay added a commit to DesmonDay/PaddleNLP that referenced this pull request Oct 28, 2024
…save_info.json location (PaddlePaddle#9321)

* [Unified checkpoint] update optimizer async save signal

* update async_save_info.json file place
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants