MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference

The codebase for NAACL 2025 paper: MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference.

📃 [Paper] • 💻 [Github]

If you find our project helpful, please give us a star ⭐ on GitHub to stay updated.

Quick Step to Setup Environment

We recommend using Anaconda to create a new environment and install the required packages. You can create a new environment and install the required packages using the following commands:

conda create -n meda python=3.10
conda activate meda
pip install -r requirements.txt
pip install --upgrade pip  

Quick Step to Run the Code

You can run the inference code using the following command to run the Longbench sample:

cd Dynamic-MLLMs
bash download_dataset.sh
bash ./scripts/meda_eval.sh

For tasks related to video tasks of lm-evaluation-harness GitHub Repository,
we recommend using the latest version by running:

git clone https://github.com/EvolvingLMMs-Lab/lmms-eval.git

Then, follow the installation instructions provided in the repository.

Citation

@article{wan2025meda,
  title={MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference},
  author={Wan, Zhongwei and Shen, Hui and Wang, Xin and Liu, Che and Mai, Zheda and Zhang, Mi},
  journal={arXiv preprint arXiv:2502.17599},
  year={2025}
}