New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

请教高手：关于./model/basic_var.py 实现的SelfAttention的两个问题 #140

Open

mswwd opened this issue Feb 26, 2025 · 0 comments

mswwd commented Feb 26, 2025

第一，我可以理解“attn_bias is None during inference”，但是kv cache和“attn_bias is None“之间的因果关系是什么？
第二，为什么单单令计算K的bias为0（即self.zero_k_bias）？
还请高手指点一二，万分感谢！

# for free to join this conversation on GitHub. Already have an account? # to comment