Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Optimize the log output of llama-box #41

Open
pengjiang80 opened this issue Mar 5, 2025 · 1 comment
Open

Optimize the log output of llama-box #41

pengjiang80 opened this issue Mar 5, 2025 · 1 comment

Comments

@pengjiang80
Copy link

  1. The log of llama-box only shows the elapsed time. It's hard for troubleshooting. It will be better to show system time or UTC time like vLLM.
  2. It will also be helpful to clearly show the log level such as INFO, DEBUG etc.

llama-box log

3.26.283.214 I srv                      main: initializing server
3.26.283.220 I srv                      init: initializing slots, n_slots = 4
3.26.287.499 I srv                      main: starting server
3.28.200.236 I srv        log_server_request: rid 208045398 | GET /v1/models 127.0.0.1:51535
3.28.202.344 I srv       log_server_response: rid 208045398 | GET /v1/models 127.0.0.1:51535 | status 200 | cost 0.00s
3.28.465.705 I srv        log_server_request: rid 208310059 | GET /v1/models 10.93.231.34:57646
3.28.466.989 I srv       log_server_response: rid 208310059 | GET /v1/models 10.93.231.34:57646 | status 200 | cost 0.00s
298.24.247.102 I srv        log_server_request: rid 17903285910 | GET /v1/models 10.93.231.34:52258
298.24.247.367 I srv       log_server_response: rid 17903285910 | GET /v1/models 10.93.231.34:52258 | status 200 | cost 0.00s
599.10.225.916 I srv        log_server_request: rid 35948513440 | GET /v1/models 10.93.231.34:63461
599.10.226.249 I srv       log_server_response: rid 35948513440 | GET /v1/models 10.93.231.34:63461 | status 200 | cost 0.00s
807.46.387.263 I srv        log_server_request: rid 48464033366 | POST /v1/chat/completions 10.93.231.34:54006
807.46.390.982 I srv oaicompat_completions_req: rid 48464033366 | {"model":"DeepSeek-32B-Q6_K","messages":"[...]","stream":true,"temperature":0.6,"max_tokens":16384,"stream_options":{"include_usage":true}}
D:\a\llama-box\llama-box\llama.cpp\ggml\src\ggml-rpc\ggml-rpc.cpp:438: GGML_ASSERT(status) failed

vllm log

INFO 03-04 14:59:48 launcher.py:29] Route: /v1/models, Methods: GET
INFO 03-04 14:59:48 launcher.py:29] Route: /version, Methods: GET
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/chat/completions, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/completions, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/embeddings, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /pooling, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /score, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/score, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v1/rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /v2/rerank, Methods: POST
INFO 03-04 14:59:48 launcher.py:29] Route: /invocations, Methods: POST
@artemus717
Copy link

will llama-box support json format log output ?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants