-
Notifications
You must be signed in to change notification settings - Fork 27k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
TF: XLA repetition penalty #16879
TF: XLA repetition penalty #16879
Conversation
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like a good port of the original numpy code!
(But I still think logit penalties should be additive rather than multiplicative)
Thinking about it more, a multiplicative logit penalty really doesn't work, right? Even if we use the reciprocal when the logit is negative, the scale of the penalty depends on the logit's distance from 0. For example, a logit in the range -0.1 to +0.1 will barely be moved by the penalty term, but such logits usually have quite a high probability of being chosen, because most logits are large and negative. |
(merging as the main goal was to port to XLA but, by all means, continue the discussion :) ) |
What does this PR do?
This PR adds our first XLA-compatible TF logit processor, as well as corresponding tests. Since this is the first of a series of small (but similar) PRs, I'd like to request a more thorough review, so the remaining ones are quick.
More specifically, this PR makes three changes: