LLM-Toolkits

LLM 模型训练工具箱, 包括增量预训练、有监督微调和DPO。

环境准备

source ./conda_environment.torch_2.0.1.sh

MedicalGPT 大部分代码是基于 MedicalGPT 二次开发的。

Name		Name	Last commit message	Last commit date
Latest commit History 26 Commits
cluster		cluster
data		data
data_augmenting		data_augmenting
dpo		dpo
moe		moe
pretraining		pretraining
server		server
supervised_finetuning		supervised_finetuning
.gitignore		.gitignore
.gitmodules		.gitmodules
LICENSE		LICENSE
README.md		README.md
conda_environment.torch_2.0.1.sh		conda_environment.torch_2.0.1.sh
deepspeed_zero_stage3_config.json		deepspeed_zero_stage3_config.json
ds_config.json		ds_config.json
merge_peft_adapter.py		merge_peft_adapter.py
run_dpo.sh.sample		run_dpo.sh.sample
run_pt.sh.sample		run_pt.sh.sample
run_sft.sh.sample		run_sft.sh.sample