Skip to content

Stops talking mid sentence #988

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
Asory2010 opened this issue Apr 14, 2023 · 2 comments
Closed

Stops talking mid sentence #988

Asory2010 opened this issue Apr 14, 2023 · 2 comments

Comments

@Asory2010
Copy link

I am using the vicuna model with the DAN prompt, it works well but it sometimes stops mid sentence for example it would say: "example senten" then stop talking why does this happen?
here is my run.bat script:

title Vicuna!(;
:start
main -i -f prompts/DAN.txt --memory_f32 --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct --color -m ggml-vicuna-7b-4bit.bin -n 128
pause
goto start

@rabidcopy
Copy link
Contributor

The second -n 128 is the reason for that. Every 128 tokens it's going to stop. Set -n -1 and remove -n 128.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@prusnak @rabidcopy @Asory2010 and others