-
Notifications
You must be signed in to change notification settings - Fork 614
Rephrase documentation of adabelief #2704
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Conversation
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). For more information, open the CLA check for this pull request. |
You are owner of some files modified in this pull request. |
@@ -113,7 +113,7 @@ def __init__( | |||
rectify: boolean. Whether to apply learning rate rectification as | |||
from RAdam. | |||
total_steps: An integer. Total number of training steps. Enable | |||
warmup by setting a positive value. | |||
warmup by setting a value greater than zero. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that it was correct:
http://oeis.org/A000027
http://oeis.org/A001477
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice. However, "greater than zero" I think requires less expertise to be understood. I thought zero was a positive number for example, and I have a PhD.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There could be some confusion with signed zero but on integer I think it is hard to be confused. I will merge this as it is just an alternative.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
Looks great to me. The same issue is with
|
This PR fixes the documentation of
total_steps
, which enables the warmup only if the value is bigger than zero. However, the documentation says "positive value". Zero (the default value) is positive though