[BOUNTY - $100] Add support for LLaVA

[AlexCheema]

Motivation: Vision models are useful.

What: exo supports different inference engines. Choose an inference engine (currently only MLX and tinygrad) for your implementation. You can probably find an existing LLaVA implementation somewhere (perhaps in https://github.com/ml-explore/mlx-examples for MLX). Modify that implementation to support sharded inference.

Reward: $100 Bounty paid out with USDC on Ethereum, email alex@exolabs.net.

[anirishrnbsinger]

taking a crack at this

[maciejjankowski]

taking a crack at this

Pair programming? (Keep the bounty)

[anirishrnbsinger]

What’s ur telegram? Discord?On 16 Jul 2024, at 17:38, Maciej Jankowski ***@***.***> wrote: taking a crack at this Pair programming? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[AlexCheema]

go for it guys. just leave a comment here if you need any help or run into issues :) @maciejjankowski @anirishrnbsinger

[anirishrnbsinger]

Didn't make as much progress as would've hoped. Will be backed up with work for a couple days. Anyone else welcome to take a crack before then

[INF800]

Interesting stuff. Wanted to read more about this but docs seem to be WIP.

https://github.com/exo-explore/exo?tab=readme-ov-file#documentation

[INF800]

Ain't this as straightforward as using the code from exo llama example and using it with mlx llava example?

[INF800]

I can do pair programming, planning to do it on weekend 28th :)

cc: @maciejjankowski

[INF800]

Ain't this as straightforward as using the code from exo llama example and using it with mlx llava example?

cc: @AlexCheema

And when can we expect the docs

[abhishek818]

attempting this..

[abhishek818]

@AlexCheema hey, seems exo is not yet supported for windows platform? Is there any plan to support it in future.

[khushgx]

taking a crack at this

if yall still need help, would love to get involved. (Also do not care abt bounty and u can keep it)

[AlexCheema]

taking a crack at this

if yall still need help, would love to get involved. (Also do not care abt bounty and u can keep it)

@khushgx looks like @varshith15 has started on it here: #88
You could jump in there to support!

[varshith15]

im almost done sharding the model and removing duplicate code
if anyone wants to work on it, please pick up the client part of the code (UI changes) and the code connecting it to the model @khushgx

[AshirMehmood]

is there any thing I can contribute in ? @varshith15

[AlexCheema]

This is completed. Thanks @varshith15 don’t forget to shoot me an email alex@exolabs.net to claim your bounty payout.

[varshith15]

@AlexCheema emailed you a couple of days ago, varshith15@gmail.com
Thanks!