Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[OPIK-354] Improve SDK robustness to connection issues #721

Conversation

alexkuzmik
Copy link
Collaborator

@alexkuzmik alexkuzmik commented Nov 25, 2024

Details

  1. Added retries to fern-generated clients. Retries are configured via tenacity lib, which was added to the dependencies.
  2. Updated debug messages format to include more information about the timestamp, process, thread, logger, line number.
  3. Updated cookbooks that were missing pandas dependency.
  4. Restricted tokenizers dependency for py3.8 since it's broken for py3.8

Issues

Resolves the issue when the connection is expired during the request processing, request is failing and there are no more attempts to send it. SDK is more robust now to connectivity issues.

Testing

Testing was performed with the cookbooks github action workflow. It was modified to install opik locally instead of downloading it from pypi. The amount of connection/protocol errors reduced significantly and became way more rare, but it is still possible to see such failures sometimes.

@alexkuzmik alexkuzmik requested review from a team as code owners November 25, 2024 17:32
@alexkuzmik alexkuzmik marked this pull request as draft November 25, 2024 17:32
@alexkuzmik alexkuzmik changed the title [NA] Add log level debug to cookbooks [NA] Cookbooks errors investigation Nov 25, 2024
@alexkuzmik alexkuzmik changed the title [NA] Cookbooks errors investigation [OPIK-354] Cookbooks errors investigation Nov 25, 2024
@alexkuzmik alexkuzmik force-pushed the OPIK-354-investigate-network-issues-in-daily-cookbook-test-execution branch from 8042036 to a1892e6 Compare November 25, 2024 21:00
@alexkuzmik alexkuzmik force-pushed the OPIK-354-investigate-network-issues-in-daily-cookbook-test-execution branch from 967838d to 9e6e9aa Compare November 27, 2024 10:34
@alexkuzmik alexkuzmik requested review from japdubengsub and removed request for japdubengsub November 27, 2024 11:28
@alexkuzmik alexkuzmik changed the title [OPIK-354] Cookbooks errors investigation [OPIK-354] Improve SDK robustness to connection issues Nov 27, 2024
@alexkuzmik alexkuzmik marked this pull request as ready for review November 27, 2024 12:00
@alexkuzmik alexkuzmik requested a review from a team as a code owner November 27, 2024 12:00
japdubengsub
japdubengsub previously approved these changes Nov 27, 2024
instance: Any, decorator: Callable[[Callable], Callable]
) -> None:
attr_name: str
for attr_name in instance.__class__.__dict__.keys():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach works fine. Though we might consider an alternative using a dir(instance) call. However, we need to verify whether it performs worse.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, dir works better since it also grabs names from parent classes, I updated the implementation.

@alexkuzmik alexkuzmik merged commit 01fb1a5 into main Nov 27, 2024
23 checks passed
@alexkuzmik alexkuzmik deleted the OPIK-354-investigate-network-issues-in-daily-cookbook-test-execution branch November 27, 2024 13:46
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants