Code

OLMo 2 (WIP)#1897

Code

Open

ysjprojects wants to merge 10 commits intoLightning-AI:mainLightning-AI/litgpt:mainfrom ysjprojects:olmo2ysjprojects/litgpt:olmo2Copy head branch name to clipboard

+276-8

Commits

Commits on Jan 4, 2025

• OLMo 2: implemented core

  ysjprojects

  committed
  b62eed2

  Copy full SHA for b62eed2

• minor fix

  ysjprojects

  committed
  f559763

  Copy full SHA for f559763

• fix vocab size

  ysjprojects

  committed
  276a8fc

  Copy full SHA for 276a8fc

• fix test_model

  ysjprojects

  committed
  1ac888f

  Copy full SHA for 1ac888f

Commits on Jan 8, 2025

• custom conversion fn for olmo2 due to new q_norm and k_norm components

  ysjprojects

  committed
  d3456e3

  Copy full SHA for d3456e3

• minor fix

  ysjprojects

  committed
  121f851

  Copy full SHA for 121f851

• minor fix on test_model.py

  ysjprojects

  committed
  3d34921

  Copy full SHA for 3d34921

• fix: post_feedforward_layernorm

  ysjprojects

  committed
  15f549d

  Copy full SHA for 15f549d

• minor fix

  ysjprojects

  committed
  ac3509f

  Copy full SHA for ac3509f

• input_norm

  ysjprojects

  committed
  852ca3e

  Copy full SHA for 852ca3e