Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[FEA] write nvidia tool wrappers to allow old YARN versions to work with MIG #4305

Closed
tgravescs opened this issue Dec 6, 2021 · 0 comments · Fixed by NVIDIA/spark-rapids-examples#87
Assignees
Labels
feature request New feature or request P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Since many customers are slow to upgrade YARN versions and older versions of YARN (pre YARN 3.3.0) do not have a pluggable device framework, write some wrappers for nvidia-smi and nvidia-container-cli that return a format such that existing YARN versions work with MIG devices.

For instance nvidia-smi would return output such that the MIG devices look like normal GPUs, then corresonding the nvidia-container-cli which is used to launch docker would need to translate that device back into a MIG device and pass along the properly formatted id (ie 0:3 instead of just 3).

@tgravescs tgravescs added feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 6, 2021
@tgravescs tgravescs self-assigned this Dec 6, 2021
@Salonijain27 Salonijain27 removed the ? - Needs Triage Need team to review and classify label Dec 7, 2021
@sameerz sameerz added this to the Dec 13 - Jan 7 milestone Dec 10, 2021
@sameerz sameerz assigned gerashegalov and unassigned tgravescs Dec 10, 2021
@sameerz sameerz added the P0 Must have for release label Dec 14, 2021
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
feature request New feature or request P0 Must have for release
Projects
None yet
4 participants