New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Naming: n_local_heads -> n_kv_heads #162

Open

ad8e opened this issue Apr 23, 2024 · 0 comments

ad8e commented Apr 23, 2024

n_local_heads refers to TP sharding, rather than GQA.

The text was updated successfully, but these errors were encountered:

ad8e changed the title ~~Naming: n_local_heads -> n_kv_heads~~ Naming: n_local_heads -> n_kv_head

ad8e changed the title ~~Naming: n_local_heads -> n_kv_head~~ Naming: n_local_heads -> n_kv_heads

# for free to join this conversation on GitHub. Already have an account? # to comment