Skip to content

Commit

Permalink
Update run.sh
Browse files Browse the repository at this point in the history
From the https://github.com/ggerganov/llama.cpp/blob/master/Makefile, it said "The 'quantize' binary is deprecated. Please use 'llama-quantize' instead." 

The command works after my testing using llama-quantize.
  • Loading branch information
melodyliu1986 authored Aug 8, 2024
1 parent b524ee6 commit cc3026d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion convert_models/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ python download_huggingface.py --model $hf_model_url --token $hf_token
python llama.cpp/examples/convert_legacy_llama.py /opt/app-root/src/converter/converted_models/$hf_model_url
python llama.cpp/convert_hf_to_gguf.py /opt/app-root/src/converter/converted_models/$hf_model_url
mkdir -p /opt/app-root/src/converter/converted_models/gguf/
llama.cpp/examples/quantize /opt/app-root/src/converter/converted_models/$hf_model_url/ggml-model-f16.gguf /opt/app-root/src/converter/converted_models/gguf/$model_org-$model_name-${QUANTIZATION}.gguf ${QUANTIZATION}
llama.cpp/llama-quantize /opt/app-root/src/converter/converted_models/$hf_model_url/ggml-model-f16.gguf /opt/app-root/src/converter/converted_models/gguf/$model_org-$model_name-${QUANTIZATION}.gguf ${QUANTIZATION}
rm -rf /opt/app-root/src/converter/converted_models/$model_org

if [ $keep_orgi = "False" ]; then
Expand Down

0 comments on commit cc3026d

Please # to comment.