Skip to content

Add support for XTC and DRY samplers #1843

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add support for XTC and DRY samplers #1843

wants to merge 5 commits into from

Conversation

zpin
Copy link

@zpin zpin commented Nov 25, 2024

Seems to work, but someone who knows this project better please check the order in apply_func().

@oobabooga
Copy link
Contributor

This is a nice addition, as those parameters are present in llama.cpp but missing in the bindings. @abetlen

@SubatomicPlanets
Copy link

I would also love to see this merged!

@etemiz
Copy link

etemiz commented Jan 7, 2025

Patiently waiting for the merge

@zpin
Copy link
Author

zpin commented Mar 17, 2025

Updated for llama.cpp changes

@zpin
Copy link
Author

zpin commented Mar 19, 2025

The sampler order was wrong, causing crashes when using XTC. Fixed now.

@JustANyanCat
Copy link

JustANyanCat commented Apr 14, 2025

Hi, should dry_range be changed to dry_penalty_last_n to match llama.cpp?

Also, should the default value be changed from 0 to -1?
From llama.cpp/examples/main/README.md
--dry-penalty-last-n N: Set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size).

@zpin
Copy link
Author

zpin commented Apr 29, 2025

Good suggestions, thank you. Updated the PR.

@JustANyanCat
Copy link

@abetlen Hi, is it possible for you to merge this?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants