Skip to content

Commit

Permalink
Add missing warmup_num_steps parameter.
Browse files Browse the repository at this point in the history
warmup_decay_num_steps is a valid user argument but is not added to the config.
This commit adds this argument to the configuration output.
  • Loading branch information
jonathanking authored Mar 7, 2023
1 parent 886494f commit 9b2a4f8
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion scripts/build_deepspeed_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,9 +248,10 @@
params["warmup_max_lr"] = args.warmup_max_lr
params["warmup_num_steps"] = args.warmup_num_steps
elif(args.scheduler == "WarmupDecayLR"):
params["total_num_steps"] = args.warmup_decay_total_num_steps
params["warmup_min_lr"] = args.warmup_decay_min_lr
params["warmup_max_lr"] = args.warmup_decay_max_lr
params["warmup_num_steps"] = args.warmup_decay_num_steps
params["total_num_steps"] = args.warmup_decay_total_num_steps
else:
raise ValueError("Invalid scheduler")

Expand Down

0 comments on commit 9b2a4f8

Please # to comment.