Skip to content

Latest commit

 

History

History
9 lines (6 loc) · 110 Bytes

engine_inference_gap.md

File metadata and controls

9 lines (6 loc) · 110 Bytes
  1. rotary embedding
  2. layernorm
  3. LlamaMLP(12.2 -> 11.8)

Not important:

  • nn.Embedding

  • qkv_proj, o_proj