Skip to content

chore: refactor testing to use mock HTTP responses #413

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 16 commits into from
Feb 13, 2024
Merged

Conversation

nirga
Copy link
Member

@nirga nirga commented Feb 10, 2024

Move tests for specific instrumentations to be under that instrumentation directory, and refactor to use VCR.py

  • OpenAI
  • Anthropic
  • Cohere
  • VertexAI
  • Replicate
  • Langchain
  • Haystack
  • Chroma
  • Pinecone

@nirga nirga force-pushed the openai-testing branch 11 times, most recently from dde81f7 to c2c5981 Compare February 12, 2024 14:08
Copy link
Contributor

@galkleinman galkleinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

While I believe mocking is a much better option here, it will be far more complex to implement, and the suggested solution will introduce tremendous improvement in the short term.

@@ -31,3 +32,9 @@ def joke_workflow():
"joke_creation.task",
"pirate_joke_generator.workflow",
]
open_ai_span = spans[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spans[0]? I would look for span by name.

Comment on lines +36 to +40
assert (
open_ai_span.attributes["llm.prompts.0.content"]
== "Tell me a joke about opentelemetry"
)
assert open_ai_span.attributes.get("llm.completions.0.content")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the order seems off, doesn't it make more sense to assert the the value exists first?

@nirga nirga merged commit 7be7df2 into main Feb 13, 2024
@nirga nirga deleted the openai-testing branch February 13, 2024 09:43
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants