VRAM on prompt processing #1618
-
|
Hello there, does the size of VRAM have any effect on prompt processing? I see that in CuBlas the larger the batch size, the faster the prompt processing seems to get, however also consuming more VRAM. In that case, is it worth to get an RTX 2080ti 22G (which is available at ￥2800/$400USD in China) rather than a RTX 2080ti 11G (~￥1800/$250USD) just for the shake of loading larger models and faster prompt processing time? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
|
Having more vram shouldn't effect prompt processing speed too much but like you said it will allow you to load larger models. At that price it seems like it would be a good idea to get the 22G card. My only concern would be the issues with support from the 3rd party modder in case the card dies prematurely or has other issues. |
Beta Was this translation helpful? Give feedback.
-
Would you mind linking where this model is being sold? |
Beta Was this translation helpful? Give feedback.
-
|
Apparently, you can buy them in 闲鱼(not sure the English name of this) app, however it only ships to China. |
Beta Was this translation helpful? Give feedback.
-
|
can we sell them to the world? |
Beta Was this translation helpful? Give feedback.
Having more vram shouldn't effect prompt processing speed too much but like you said it will allow you to load larger models. At that price it seems like it would be a good idea to get the 22G card. My only concern would be the issues with support from the 3rd party modder in case the card dies prematurely or has other issues.