Code

Compilade/refactor kv cache#1

Code

Closed

ykhrustalev wants to merge 43 commits intomasterykhrustalev/llama.cpp:masterfrom compilade/refactor-kv-cacheykhrustalev/llama.cpp:compilade/refactor-kv-cacheCopy head branch name to clipboard

+2,592-784

Commits

Commits on Apr 4, 2024

• wip: llama : separate recurrent states from the KV cache

  Show description for 271104c
  compilade

  committed
  271104c

  Copy full SHA for 271104c

• llama : use std::find for seq_nodes in llama_rs_cache

  compilade

  committed
  8db1e4d

  Copy full SHA for 8db1e4d

Commits on Apr 8, 2024

• llama : state checkpoints for recurrent models

  compilade

  committed
  0028010

  Copy full SHA for 0028010

Commits on Apr 9, 2024

• llama : correctly handle more edge cases for the rs cache

  compilade

  committed
  0c8b3b2

  Copy full SHA for 0c8b3b2

Commits on Apr 10, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  d66849f

  Copy full SHA for d66849f

Commits on Apr 29, 2024

• llama : rename many llama_kv_cache_* functions

  compilade

  committed
  a09db95

  Copy full SHA for a09db95

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  c460ff1

  Copy full SHA for c460ff1

• llama : remove useless return value for some llama_cache_* functions

  compilade

  committed
  b6fafd1

  Copy full SHA for b6fafd1

Commits on May 12, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  b7ec12e

  Copy full SHA for b7ec12e

Commits on May 22, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  3b57b55

  Copy full SHA for 3b57b55

Commits on May 24, 2024

• llama : rethink recurrent state cell counts

  Show description for 7e13f19
  compilade

  committed
  7e13f19

  Copy full SHA for 7e13f19

• llama : support Jamba

  compilade

  committed
  cbc743e

  Copy full SHA for cbc743e

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  0fd13e9

  Copy full SHA for 0fd13e9

Commits on May 25, 2024

• llama : fix BERT inference without KV cache

  compilade

  committed
  61a88a1

  Copy full SHA for 61a88a1

• convert-hf : check for unprocessed Jamba experts

  compilade

  committed
  ea2e63e

  Copy full SHA for ea2e63e

• convert-hf : support Mini-Jamba conversion

  compilade

  committed
  fc59407

  Copy full SHA for fc59407

Commits on May 28, 2024

• llama : fix Jamba quantization sanity checks

  compilade

  committed
  181dadf

  Copy full SHA for 181dadf

• llama : sequence-length-aware batch splitting

  compilade

  committed
  3a414b0

  Copy full SHA for 3a414b0

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  4e4c41e

  Copy full SHA for 4e4c41e

Commits on Jun 1, 2024

• llama : use equal-sequence-length sub-batches for recurrent models

  Show description for 3587a94
  compilade

  committed
  3587a94

  Copy full SHA for 3587a94

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  5d3c7b9

  Copy full SHA for 5d3c7b9

• llama : fix batch split output count for embeddings

  compilade

  committed
  72eea49

  Copy full SHA for 72eea49

• llama : minimize swaps when reordering logits

  Show description for 18d1c14
  compilade

  committed
  18d1c14

  Copy full SHA for 18d1c14

• llama : fix edge case finding batch seq_id of split recurrent cell

  Show description for 61200ef
  compilade

  committed
  61200ef

  Copy full SHA for 61200ef

Commits on Jun 2, 2024

• llama : avoid copies for simple batch splits

  compilade

  committed
  eb589d5

  Copy full SHA for eb589d5

Commits on Jun 3, 2024

• llama : use im2col and mul_mat to perform convolution for Mamba

  Show description for 8fb57ac
  compilade

  committed
  8fb57ac

  Copy full SHA for 8fb57ac

• llama : fix .base() compilation error on Windows

  compilade

  committed
  17f6c1e

  Copy full SHA for 17f6c1e

• llama : allow doing the equivalent of SSM_CONV with SUM_ROWS and MUL

  Show description for fee3c1d
  compilade

  committed
  fee3c1d

  Copy full SHA for fee3c1d

Commits on Jun 8, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  6840ac0

  Copy full SHA for 6840ac0

• llama : rename llama_cache to llama_past

  Show description for 372482d
  compilade

  committed
  372482d

  Copy full SHA for 372482d

Commits on Jun 12, 2024

• examples : replace llama_kv_cache_seq_* with llama_past_seq_*

  compilade

  committed
  43d8d4b

  Copy full SHA for 43d8d4b

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  ff794f5

  Copy full SHA for ff794f5

• mamba : fix non-contiguous usage of ggml_silu

  compilade

  committed
  33425a7

  Copy full SHA for 33425a7

Commits on Jun 30, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  10c3c41

  Copy full SHA for 10c3c41

Commits on Jul 4, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  9b38f8b

  Copy full SHA for 9b38f8b

Commits on Sep 1, 2024

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  bc320ef

  Copy full SHA for bc320ef

Commits on Sep 2, 2024

• llama : session saving and reloading for hybrid models

  compilade

  committed
  fcb889c

  Copy full SHA for fcb889c

• Merge branch 'master' into compilade/refactor-kv-cache

  compilade

  committed
  a03e32a

  Copy full SHA for a03e32a

• convert_hf : fix Jamba conversion

  compilade

  committed
  9d3f44d

  Copy full SHA for 9d3f44d

• llama : fix mixed signedness comparison

  compilade

  committed
  5f62db7

  Copy full SHA for 5f62db7

• llama : use unused n_embd_k_gqa in k_shift

  Show description for 375de5b
  compilade

  committed
  375de5b

  Copy full SHA for 375de5b

Commits on Oct 11, 2024

• trigger build

  ykhrustalev

  committed
  d2b1d0e

  Copy full SHA for d2b1d0e

• trigger

  ykhrustalev

  committed
  0d6c36c

  Copy full SHA for 0d6c36c