New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Remove smaller limit for legacy bfloat16 serialization #505

Draft

borzunov wants to merge 1 commit into main from payload-size

Collaborator

borzunov commented Sep 5, 2023 •

edited

Loading

Revert #251 since it's not needed after #311. This may improve fine-tuning efficiency for medium-sized batches.

TODO:

Test it with increasingly larger batches. Watch that we switch from rpc_forward to rpc_forward_stream (can be distinguished using server logs) without errors.


          Remove smaller limit for legacy bfloat16 serialization

b945e38

# for free to join this conversation on GitHub. Already have an account? # to comment

Labels

None yet