Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

conversation template doc typo fix #771

Merged
merged 3 commits into from
Apr 17, 2024
Merged

conversation template doc typo fix #771

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2107da7
conversation template doc typo fix
wheresmyhair Apr 17, 2024
82a970a
supported template table update
wheresmyhair Apr 17, 2024
1aad07b
conversation template table update
wheresmyhair Apr 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions docs/source/examples/DATASETS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,15 +179,27 @@ Tips:

Conversations should be formatted before feeding into the model. As of now, we've preset the conversation template for following models:

| Model | Template Name | Actual Format & Filled Example |
| ----- | ------------- | ------------------------------ |
|Llama-2| `llama2` | [Link](./supported_conversation_template.md#llama-2) |
| Qwen-2 | `qwen2` | [Link](./supported_conversation_template.md#qwen-2-qwen-15) |
| Template Name | Filled Example | Detailed Template |
| ------------- | -------------- | ----------------- |
| `llama2` | `<s>[INST] <<SYS>>`<br>`You are a chatbot developed by LMFlow team.`<br>`<</SYS>>`<br><br>`Who are you? [/INST] I am a chatbot developed by LMFlow team.</s><s>[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.</s>` | [Link](./supported_conversation_template.md#llama-2) |
| `qwen2` | `<\|im_start\|>system`<br>`You are a Machine Learning expert.<\|im_end\|>`<br>`<\|im_start\|>user`<br>`Hi!<\|im_end\|>`<br>`<\|im_start\|>assistant`<br>`Hi, nice to meet you!<\|im_end\|>`<br>`<\|im_start\|>user`<br>`Who are you?<\|im_end\|>`<br>`<\|im_start\|>assistant`<br>`I'm a Machine Learning expert.<\|im_end\|>`<br> | [Link](./supported_conversation_template.md#qwen-2-qwen-1-5) |

Passing the template name to the `--conversation_template` argument to apply the corresponding conversation template:
```sh
# scripts/run_finetune.sh
# ...
deepspeed ${deepspeed_args} \
examples/finetune.py \
--model_name_or_path meta-llama/Llama-2-7b-chat-hf \
--dataset_path ${dataset_path} \
--conversation_template llama2 \
# ...
```

```{admonition} Formatted Dataset
:class: info

For dataset that system prompts, tool prompts and templates are already applied (like the one below), user can run the finetune shell by passing `empty` or `empty_no_special_tokens` to the `--conversation_template`` argument. `empty` template will add a bos token to the beginning of every round of conversation as well as a eos token to the end of every round of conversation. `empty_no_special_tokens` will not add any special tokens to the conversation, just concatenates the user and assistant messages.
For dataset that system prompts, tool prompts and templates are already applied (like the one below), user can run the finetune shell by passing `empty` or `empty_no_special_tokens` to the `--conversation_template` argument. `empty` template will add a bos token to the beginning of every round of conversation as well as a eos token to the end of every round of conversation. `empty_no_special_tokens` will not add any special tokens to the conversation, just concatenates the user and assistant messages.
```json
{
"type": "conversation",
Expand Down
Loading