New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

series_patch_size和series_patch_num都是通过QK计算，并没有使用V #28

Open

wuyue5 opened this issue Dec 4, 2023 · 1 comment

wuyue5 commented Dec 4, 2023

请问您这边是使用V的效果不好吗，还是出于其他原因没有使用自注意力的一般计算过程？

Contributor

yyysjz1997 commented Dec 4, 2023

请问您这边是使用V的效果不好吗，还是出于其他原因没有使用自注意力的一般计算过程？

感谢关注，因为两个表征可以使用权重（KQ）的形式，也可以使用数值的形式（KQV）。增加计算V的过程一方面会增加算力/性能负担，另一方面与权重的形式无差别，所以选择了不计算数值的权重形式。

# for free to join this conversation on GitHub. Already have an account? # to comment