Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[misc] add early error message for custom ops #11355

Merged
merged 2 commits into from
Dec 20, 2024
Merged

[misc] add early error message for custom ops #11355

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f5befcb
add clear error message
youkaichao Dec 20, 2024
09a1d0c
improve
youkaichao Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion vllm/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1577,8 +1577,18 @@ def direct_register_custom_op(
library object. If you want to bind the operator to a different library,
make sure the library object is alive when the operator is used.
"""
if is_in_doc_build() or not supports_custom_op():
if is_in_doc_build():
return

if not supports_custom_op():
assert not current_platform.is_cuda_alike(), (
"cuda platform needs torch>=2.4 to support custom op, "
"chances are you are using an old version of pytorch "
"or a custom build of pytorch. It is recommended to "
"use vLLM in a fresh new environment and let it install "
"the required dependencies.")
return

import torch.library
if hasattr(torch.library, "infer_schema"):
schema_str = torch.library.infer_schema(op_func,
Expand Down
Loading