Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Add a model config option to disable system prompts #1432

Closed
nsarrazin opened this issue Aug 25, 2024 · 1 comment · Fixed by #1539
Closed

Add a model config option to disable system prompts #1432

nsarrazin opened this issue Aug 25, 2024 · 1 comment · Fixed by #1539
Labels
back This issue is related to the Svelte backend or the DB enhancement New feature or request models This issue is related to model performance/reliability

Comments

@nsarrazin
Copy link
Collaborator

Describe your feature request

Some models don't support system prompts and if you try to use the system role in their chat template they throw an error. We should add a config option in the model definition to specify that system prompts are not supported. In that case we

Implementation idea

add a systemPrompt: true|false in the modelConfig which will trigger logic to filter out role: 'system' messages and preprend the content of the first one to the first user message

Solves #1386
@nsarrazin nsarrazin added enhancement New feature or request back This issue is related to the Svelte backend or the DB models This issue is related to model performance/reliability labels Aug 25, 2024
@nsarrazin nsarrazin changed the title Add a config option to disable system prompts Add a model config option to disable system prompts Aug 25, 2024
@nsarrazin nsarrazin mentioned this issue Aug 25, 2024
Closed
@Mounayer
Copy link
Contributor

Hi there, I could give this a try!

@Mounayer Mounayer mentioned this issue Oct 22, 2024
Merged
barrensconsulting pushed a commit to barrensconsulting/llmindset-hug-ui that referenced this issue Nov 11, 2024
@evalstate
behaviour now matches huggingface#1432
7ca191c 
that is - put the preprompt in the first message rather than truncate
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
back This issue is related to the Svelte backend or the DB enhancement New feature or request models This issue is related to model performance/reliability
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Add a model config option to disable system prompts Mounayer/chat-ui
2 participants
@nsarrazin @Mounayer