Skip to content

Incorrect command for model quantization in README.md #1199

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
jayyaali95 opened this issue Apr 26, 2023 · 2 comments
Closed

Incorrect command for model quantization in README.md #1199

jayyaali95 opened this issue Apr 26, 2023 · 2 comments

Comments

@jayyaali95
Copy link

The correct command for quantizing the model is not reflected accurately in lines 206 and 207 of README.md (Prepare Data & Run). To perform model quantization, you should use the following command instead.

# quantize the model to 4-bits (using method 2 = q4_0)
./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
@slaren
Copy link
Member

slaren commented Apr 26, 2023

If this isn't working for you, you need to update to current master, check #1191 that was merged today.

@slaren slaren closed this as not planned Won't fix, can't repro, duplicate, stale Apr 26, 2023
@jayyaali95
Copy link
Author

Thanks @slaren for mentioning that!

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants