Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[New feature] Integrate DPO #762

Merged
merged 5 commits into from
Apr 12, 2024
Merged

[New feature] Integrate DPO #762

merged 5 commits into from
Apr 12, 2024

Conversation

gzliyu
Copy link
Contributor

@gzliyu gzliyu commented Apr 12, 2024 

./scripts/run_dpo_align.sh \
  --model_name_or_path /home/nlpintern1/liyu/models/0313_sft_llama_full \
  --dataset_path /home/nlpintern1/liyu/dataset/stack-exchange-paired/data \
  --output_lora_path output_models/dpo_lora
  • Built a dummy dataset at /home/nlpintern1/liyu/dataset/stack-exchange-paired-dummy by selecting 3k data from stack-exchange-500k. This dummy dataset is about 14M and can be used for debugging

@gzliyu gzliyu changed the title [New feature] Integrate DPO #759 [New feature] Integrate DPO Apr 12, 2024
research4pan
research4pan approved these changes Apr 12, 2024
View reviewed changes
Copy link
Contributor

@research4pan research4pan left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's do some tests first before merging into main

@research4pan research4pan merged commit 8a70f48 into OptimalScale:liyu-add-dpo Apr 12, 2024
0 of 2 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@research4pan research4pan research4pan approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gzliyu @research4pan