Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Bug: Set ToolCallBehavior.allowAllKernelFunctions(false) doesn't work for getStreamingChatMessageContentsAsync #279

Open
thomaspeng90s opened this issue Dec 9, 2024 · 1 comment
Labels
bug Something isn't working triage Needs triage from engineering team

Comments

@thomaspeng90s
Copy link

Describe the bug
I try to invoke toolcall myself in a streaming chat.
I have learned that how to invoke a toolcall in an non-streaming chat. like

        Kernel kernel = Kernel.builder().withAIService(ChatCompletionService.class, chatCompletionService)
                .withPlugin(KernelPluginFactory.createFromObject(this, "time"))
                .build();

        chatCompletionService
                .getChatMessageContentsAsync("what time is it now?", kernel, InvocationContext.builder()
                        .withToolCallBehavior(ToolCallBehavior.allowAllKernelFunctions(false))
                        .build())
                .block()
                .stream().filter(it->it instanceof OpenAIChatMessageContent<?>)
                .map(it->((OpenAIChatMessageContent)it))
                .forEach(it->{
                    System.out.println(it.getToolCall().size());
                });

Image

As you can see, I can get the toolCall value;

but when I change to an streaming chat, like

        Kernel kernel = Kernel.builder().withAIService(ChatCompletionService.class, chatCompletionService)
                .withPlugin(KernelPluginFactory.createFromObject(this, "time"))
                .build();

        chatCompletionService
                .getStreamingChatMessageContentsAsync("what time is it now?", kernel, InvocationContext.builder()
                        .withToolCallBehavior(ToolCallBehavior.allowAllKernelFunctions(false))
                        .build())
                .filter(it->it instanceof OpenAIStreamingChatMessageContent<?>)
                .map(it->(OpenAIStreamingChatMessageContent)it)
                .doOnNext(it->{
                    System.out.println(it.getToolCall().size());
                }).subscribe();

Image
the toolCall will always be an empty array.

@thomaspeng90s thomaspeng90s added bug Something isn't working triage Needs triage from engineering team labels Dec 9, 2024
@johnoliver
Copy link
Member

Support for tool calling in a streaming chat call is a feature that is still to be implemented, we have had some requests for support, will update if we put this on the immediate roadmap.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working triage Needs triage from engineering team
Projects
None yet
Development

No branches or pull requests

2 participants