New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

unreadable generation results #28

Open

Fonsifa opened this issue Nov 26, 2024 · 0 comments

Fonsifa commented Nov 26, 2024

Problems when use 2 A800 GPU serving LLaMA-3-8B

IIRC, no other change besides this. It's ok with one GPU. BTW what's the difference bwtween actual_gpu_num and gpu_num

The text was updated successfully, but these errors were encountered:

Fonsifa closed this as completed

Fonsifa reopened this

# for free to join this conversation on GitHub. Already have an account? # to comment