RFC: automatically use litellm if possible #534
Conversation
| from ...models.interface import Model, ModelProvider | ||
| from .litellm_model import LitellmModel | ||
|
|
||
| DEFAULT_MODEL: str = "gpt-4.1" |
There was a problem hiding this comment.
Would importing https://github.com/openai/openai-agents-python/blob/v0.0.11/src/agents/models/openai_provider.py#L11 instead be better? Also, huge 👍 to switching from gtp-4o to gpt-4.1
There was a problem hiding this comment.
Feels like a minor breaking change. Though probably fine! Good call.
There was a problem hiding this comment.
@rm-openai Perhaps you're already aware of this, but switching to gpt-4.1 might break existing CUA apps (the tool is not yet available with 4.1 while web_search_preview works with 4.1), so indeed switching the default model could be a breaking change
|
|
||
| def _create_fallback_provider(self, prefix: str) -> ModelProvider: | ||
| if prefix == "litellm": | ||
| from ..extensions.models.litellm_provider import LitellmProvider |
There was a problem hiding this comment.
I like this dynamic import here!
nice-to-have: on the LitellmProvider side, having try/except clause for loading litellm module and raising a more user-friendly error message than exposing the missing litellm could make dev experience better
There was a problem hiding this comment.
@seratch - LitellmProvider will try to import LitellmModel, which will display the nice error message: https://github.com/openai/openai-agents-python/blob/main/src/agents/extensions/models/litellm_model.py#L16-L19
| from .openai_provider import OpenAIProvider | ||
|
|
||
|
|
||
| class MultiProviderMap: |
There was a problem hiding this comment.
do we need this class to encapsulate the simple operations on plain dict?
Co-authored-by: Steven Heidel <steven@heidel.ca>
Summary
This replaces the default model provider with a
MultiProvider, which has the logic:openai/or doesn't contain "/", use OpenAIlitellm/, use LiteLLM to use the appropriate model provider.It's also extensible, so users can create their own mappings. I also imagine that if we natively supported Anthropic/Gemini etc, we can add it to MultiProvider to make it work.
The goal is that it should be really easy to use any model provider. Today if you pass
model="gpt-4.1", it works great. Butmodel="claude-sonnet-3.7"doesn't. If we can make it that easy, it's a win for devx.I'm not entirely sure if this is a good idea - is it too magical? Is the API too reliant on litellm? Comments welcome.
Test plan
For now, the example. Will add unit tests if we agree its worth mergin.