Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fixed code to use AdamWeightDecayOptimizer #146

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jinamshah
Copy link

Made changes to the code here.
This was done because when the code reaches the said line, the optimizer is not necessarily a AdamWeightDecayOptimizer object. It can be a CrossShardOptimizer object.
Thus, by change checking if the weight_decay flag is not equal to zero, we can conclusively say if the optimizer uses Adam weight decay or not.

Note: if FLAGS.weight_decay != 0 can be changed to if not FLAGS.weight_decay but that would hinder the code if someone has used None by mistake

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant