Skip to content

Fix non-matching tensor with odd embedding size. #54

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zmy
Copy link

@zmy zmy commented Jan 26, 2019

If d_model integer parameter is odd rather than even, then tensor size is not matching (by a difference of 1) on the left and right sides of the assignment:

pe[:, 1::2] = torch.cos(position * div_term)

This pull request fixes the problem.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant