Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

🚀 perf(layer_seq): factorize by nbHeads #78

Merged
merged 35 commits into from
Mar 26, 2023
Merged

🚀 perf(layer_seq): factorize by nbHeads #78

merged 35 commits into from
Mar 26, 2023

Conversation

jean-francoisreboud
Copy link
Collaborator

@jean-francoisreboud jean-francoisreboud commented Mar 22, 2023 •
edited
Loading

Changes:

  • factorize by nbHeads in VisionTransformer
  • fix backward for ValueSeq
  • update QuerySeq, SoftmaxSeq, ValueSeq with nbHeads
  • use tanh alternative on GPU
  • rename GrAITorchTests

@jean-francoisreboud jean-francoisreboud merged commit b10763b into release_4 Mar 26, 2023
@jean-francoisreboud jean-francoisreboud deleted the jfr/heads branch March 26, 2023 11:52
@jean-francoisreboud jean-francoisreboud mentioned this pull request Aug 4, 2023
Merged
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jean-francoisreboud