⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)
-
Updated
Jun 25, 2024 - Python
#
continual-pre-training
Here are 4 public repositories matching this topic...
Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
llama
data-quality
mistral
pre-training
continual
neural-symbolic
data-centric-ai
llm
continual-pre-training
-
Updated
Oct 16, 2024 - Python
Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 Llama-3 的科学推理和中文能力
-
Updated
Aug 18, 2024
A open-source framework designed to adapt pre-trained Language Models (LLMs), such as Llama, Mistral, and Mixtral, to a wide array of domains and languages.
-
Updated
May 27, 2024 - Python
Improve this page
Add a description, image, and links to the continual-pre-training topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the continual-pre-training topic, visit your repo's landing page and select "manage topics."