[9/N] torch.compile LLM usage #10552
Conversation
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
youkaichao
changed the title
| **kwargs: Arguments for :class:`~vllm.EngineArgs`. (See | ||
| :ref:`engine_args`) | ||
|
|
||
| Note: | ||
| This class is intended to be used for offline inference. For online | ||
| serving, use the :class:`~vllm.AsyncLLMEngine` class instead. | ||
| """ | ||
| """ # noqa |
There was a problem hiding this comment.
nit: is it because of the line-too-long error? If so, can you please fix L113?
There was a problem hiding this comment.
this is a triple quote, adding # noqa can ignore the line length limit for all lines inside the triple quote.
There was a problem hiding this comment.
Yes please don't use noqa in this case. Please adhere to the line limit as much as possible.
Signed-off-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: youkaichao <youkaichao@gmail.com> Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
No description provided.