Serve Llama 3.2 models as downloaded from Meta #10934
Unanswered
corbsmartin
asked this question in
Q&A
Replies: 0 comments
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
-
Hi all,
Without modifying vllm code is it possible to deploy a Llama 3.2 model in the format Meta provides when you download via llama stack? For example when you download Llama 3.2 1b using llama stack the files are:
params.json content
vllm serve seems not to be able to load and serve these models without first running the HF transformer script.
Questions
Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions