Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Batch inference support (self-assigning) #48

Open
willccbb opened this issue Jul 2, 2024 · 2 comments
Open

Batch inference support (self-assigning) #48

willccbb opened this issue Jul 2, 2024 · 2 comments
Assignees
@willccbb

Comments

@willccbb
Copy link

willccbb commented Jul 2, 2024

Would be nice to have batch inference support similar to mlx_parallm, happy to try and add soon. @Blaizzy can you assign this to me?
@Blaizzy
Copy link
Owner

Blaizzy commented Jul 3, 2024

Hey Will,

Yes, that would be awesome!

I have assigned the task to you 😀

@Blaizzy
Copy link
Owner

Blaizzy commented Jul 3, 2024

Please comment #40 here so I can assign it to you and discuss details on a single issue.

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@willccbb willccbb

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@willccbb @Blaizzy