You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The log of llama-box only shows the elapsed time. It's hard for troubleshooting. It will be better to show system time or UTC time like vLLM.
It will also be helpful to clearly show the log level such as INFO, DEBUG etc.
llama-box log
3.26.283.214 I srv main: initializing server
3.26.283.220 I srv init: initializing slots, n_slots = 4
3.26.287.499 I srv main: starting server
3.28.200.236 I srv log_server_request: rid 208045398 | GET /v1/models 127.0.0.1:51535
3.28.202.344 I srv log_server_response: rid 208045398 | GET /v1/models 127.0.0.1:51535 | status 200 | cost 0.00s
3.28.465.705 I srv log_server_request: rid 208310059 | GET /v1/models 10.93.231.34:57646
3.28.466.989 I srv log_server_response: rid 208310059 | GET /v1/models 10.93.231.34:57646 | status 200 | cost 0.00s
298.24.247.102 I srv log_server_request: rid 17903285910 | GET /v1/models 10.93.231.34:52258
298.24.247.367 I srv log_server_response: rid 17903285910 | GET /v1/models 10.93.231.34:52258 | status 200 | cost 0.00s
599.10.225.916 I srv log_server_request: rid 35948513440 | GET /v1/models 10.93.231.34:63461
599.10.226.249 I srv log_server_response: rid 35948513440 | GET /v1/models 10.93.231.34:63461 | status 200 | cost 0.00s
807.46.387.263 I srv log_server_request: rid 48464033366 | POST /v1/chat/completions 10.93.231.34:54006
807.46.390.982 I srv oaicompat_completions_req: rid 48464033366 | {"model":"DeepSeek-32B-Q6_K","messages":"[...]","stream":true,"temperature":0.6,"max_tokens":16384,"stream_options":{"include_usage":true}}
D:\a\llama-box\llama-box\llama.cpp\ggml\src\ggml-rpc\ggml-rpc.cpp:438: GGML_ASSERT(status) failed
vllm log
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/models, Methods: GET
INFO 03-04 14:59:48 launcher.py:29] Route: /version, Methods: GET
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/chat/completions, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/completions, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/embeddings, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /pooling, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /score, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/score, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v2/rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /invocations, Methods: POST
The text was updated successfully, but these errors were encountered:
llama-box log
vllm log
The text was updated successfully, but these errors were encountered: