Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

finetune_cosmopedia.sh如何训练出来8B模型 #27

Open
RuipingWang1986 opened this issue May 20, 2024 · 1 comment
Open

finetune_cosmopedia.sh如何训练出来8B模型 #27

RuipingWang1986 opened this issue May 20, 2024 · 1 comment

Comments

@RuipingWang1986
Copy link

您好，根据您的指导，我目前已经可以跑通finetune_cosmopedia脚本的训练，但是我发现生成的模型和之前使用的llama2-7B模型规模基本是相同的，并没有增大；另外，按照您论文里提到的，应该是可以自己设计添加多少层，但我在训练脚本中也没有发现。
@season1blue
Copy link

我也是同样的问题，请问扩展参数的那部分代码是还没有放出来吗

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@season1blue @RuipingWang1986