Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Hi, I have a question about fine tuning. #41

Open
Yindy07 opened this issue Aug 5, 2024 · 0 comments
Open

Hi, I have a question about fine tuning. #41

Yindy07 opened this issue Aug 5, 2024 · 0 comments

Comments

@Yindy07
Copy link

Yindy07 commented Aug 5, 2024

  1. when inferring from a model to FIM, I want to fine-tune it with a dataset, but the format of the dataset is
    {"instruction" : "", "output": ""} should I use this format, or should I add tokens such as ?
    "<|fim▁hole|>" "<|fim▁begin|>", etc.

  2. when inferring the model, I am using a template like <|fim▁begin|>{prefix}<|fim▁hole|>{suffix}<|fim▁end|>,
    is this correct?
    For example :
    "<|fim▁begin|> handleMoveFocus = (<|fim▁hole|>) => {\n let index =<|fim▁end|>"
    Is this the right way to ask for reasoning in this form?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant