New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

add video support for Qwen2-VL #168

Open

davidkoski opened this issue Dec 18, 2024 · 1 comment

Collaborator

davidkoski commented Dec 18, 2024

https://github.com/awni/mlx-vlm/tree/video is a branch of mlx-vlm that has working video support.

I think there are a couple parts to it:

add video to UserInput
add video frame extraction to e.g. MediaProcessing
add video to LMInput
add the processing support + tokens to Qwen2-VL

I think it can follow a similar pattern to the image processing.

Contributor

nvnsho commented Dec 19, 2024

Will implement this.

# for free to join this conversation on GitHub. Already have an account? # to comment