bentoml serve GPU Memory usage #4107
Unanswered
bruce99kang
asked this question in
Q&A
Replies: 0 comments
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
-
While using bentoml serving an onnx model.

Using code below only, gpu mem usage was around 1500MB.
But, when I built this by
It seemed to be using all of my GPUs instead of using 1 GPU.
Even if I specified CUDA_VISIBLE_DEVICES=1, the GPU Mem usage took around 6000GB for a GPU.
Are there any files needs to be modified?
Beta Was this translation helpful? Give feedback.
All reactions