Skip to content

[User] How to specify which cuda device to use programmably #1622

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Closed
huichen opened this issue May 28, 2023 · 3 comments
Closed

[User] How to specify which cuda device to use programmably #1622

huichen opened this issue May 28, 2023 · 3 comments
Labels
stale

Comments

@huichen
Copy link

huichen commented May 28, 2023

Say I have four Nvidia cards and I want to run four models on each of the card in one program. The SDK doesn't provide parameter to specify which cuda device to run the model on?
@JohannesGaessler
Copy link
Collaborator

Fixed by #1607 .

@j-f1 j-f1 linked a pull request May 29, 2023 that will close this issue
Cuda refactor, multi GPU support #1607
Closed
@huichen
Copy link
Author

huichen commented May 29, 2023

Looks awesome!

@github-actions github-actions bot added the stale label Mar 25, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 9, 2024

This issue was closed because it has been inactive for 14 days since being marked as stale.

@github-actions github-actions bot closed this as completed Apr 9, 2024
@Bearsaerker Bearsaerker mentioned this issue Mar 12, 2025
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cuda refactor, multi GPU support JohannesGaessler/llama.cpp
2 participants
@huichen @JohannesGaessler