Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

OPIK-796: Fix issue with LLM setup and stream shutdown #1108

Merged
merged 32 commits into from
Jan 24, 2025

Conversation

thiagohora
Copy link
Contributor

@thiagohora thiagohora commented Jan 22, 2025

Details

Add tests to new logs endpoint and this PR also unveiled the following issues:

  1. The LLM providers' setup was based on components, not modules, which made it very hard to replace beans to mock them and avoid calling the actual LLM provider server.
  2. The new Redis stream consumer needs to follow the application lifecycle. For some reason, the consumer is not automatically deleted once the application's context is shut down, which caused side effects between tests. The solution was to make the consumer implement the Managed interface, which added the bean to the application lifecycle and allowed us to close the stream during the application shutdown.

Issues

OPIK-796

thiagohora and others added 28 commits January 16, 2025 12:57
@thiagohora thiagohora requested a review from a team as a code owner January 22, 2025 10:08
@thiagohora thiagohora self-assigned this Jan 22, 2025
@thiagohora thiagohora force-pushed the thiagohora/OPIK-796_implement_fix_current_issues branch from 62ad07f to a64dd61 Compare January 22, 2025 10:12
@thiagohora thiagohora force-pushed the thiagohora/OPIK-796_implement_fix_current_issues branch from 9f0bdf9 to 9d86e17 Compare January 22, 2025 10:17

@Provides
@Singleton
public LlmProviderFactory llmProviderFactory(LlmProviderApiKeyService llmProviderApiKeyService) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public LlmProviderFactory llmProviderFactory(LlmProviderApiKeyService llmProviderApiKeyService) {
public LlmProviderFactory llmProviderFactory(@NonNull LlmProviderApiKeyService llmProviderApiKeyService) {

return createInstance(llmProviderApiKeyService);
}

public LlmProviderFactory createInstance(LlmProviderApiKeyService llmProviderApiKeyService) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public LlmProviderFactory createInstance(LlmProviderApiKeyService llmProviderApiKeyService) {
public LlmProviderFactory createInstance(@NonNull LlmProviderApiKeyService llmProviderApiKeyService) {

}

@Override
public LlmProviderService getService(String apiKey) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public LlmProviderService getService(String apiKey) {
public LlmProviderService getService(@NonNull String apiKey) {

Other functions in this class as well

}

@Override
public LlmProviderService getService(String apiKey) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public LlmProviderService getService(String apiKey) {
public LlmProviderService getService(@NonNull String apiKey) {

Same for other functions in this class

@ParameterizedTest
@MethodSource("credentials")
@DisplayName("get logs per rule evaluators: when api key is present, then return proper response")
void getLogsPerRuleEvaluators__whenSessionTokenIsPresent__thenReturnProperResponse(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two getLogsPerRuleEvaluators__whenSessionTokenIsPresent__thenReturnProperResponse almost copypasted tests (only difference seems to be workspace). If you really need both it's worth incapsulating into function and reuse it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will see what can be reused and add the validations in the next iteration

@thiagohora thiagohora merged commit f485a56 into main Jan 24, 2025
8 checks passed
@thiagohora thiagohora deleted the thiagohora/OPIK-796_implement_fix_current_issues branch January 24, 2025 16:23
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants