.dll .so generation

[vaiju1981]

Hi I am currently trying to integrate llama.cpp with Java ( https://github.com/vaiju1981/java-llama.cpp/tree/b4689 ) this allows llamacpp server to run inside java.

I was wondering if there a way I can use llama-box instead of llama.cpp ( mostly due to fact that it has vision support ). How would I go about if that is possible.

[thxCode]

llama-box is an application that uses llama.cpp, so both the binder and application stand at the same level, one for the language-specific interface, and the other for the HTTP interface.

you can treat llama-box like Jetty or Tomcat in the Java domain.

I believe the binder can also implement the same VL logic, all you need is to get the right batch during llama_decode:

llama-box/llama-box/server.cpp

Lines 3868 to 3869 in 384ca12

	qwen2vl_text_token_batch_wrapper batch_txt = qwen2vl_text_token_batch_wrapper((tokens.data() + j), n_eval, batch_txt_mrope_pos.data(), slot.id);
	if (llama_decode(llm_ctx, batch_txt.batch)) {

.