AzureOpenAI

[supertech1234]

Hello Team,

Can we get Azure OpenAI added to LLM? All the API use by OpenAI is the same with Azure OpenAI. Azure is just cheaper to use and would be a good option to help the community with cost.

Example:

AZURE_OPENAI_KEY=Your Key
AZURE_OPENAI_ENDPOINT=https://yourinstancename-northcentralus.openai.azure.com/

northcentralus = Location (Location can be different depending on where the instance is deployed)

AZURE_OPENAI_DEPLOYMENT=gpt-35-turbo

gpt-35-turbo = Model can be different depending on user choices.

[footballqq]

in models.py:
add "base_url"
model_provider == ModelProvider.OPENAI:
# Get and validate API key
api_key = os.getenv("OPENAI_API_KEY")
#base_url = os.getenv("OPENAI_BASE_URL")
base_url = os.getenv("https://openrouter.ai/api/v1")
if not api_key:
# Print error to console
print(f"API Key Error: Please make sure OPENAI_API_KEY is set in your .env file.")
raise ValueError("OpenAI API key not found. Please make sure OPENAI_API_KEY is set in your .env file.")
kwargs = {"model": model_name, "api_key": api_key}
if base_url:
kwargs["base_url"] = base_url
return ChatOpenAI(**kwargs)

[supertech1234]

@footballqq

Updating models.py to the below.

class ModelProvider(str, Enum):
"""Enum for supported LLM providers"""
OPENAI = "OpenAI"
AZURE_OPENAI = "AzureOpenAI" #New entry
GROQ = "Groq"
ANTHROPIC = "Anthropic"

from langchain_openai import ChatOpenAI, AzureChatOpenAI #New entry

# Add Azure OpenAI models
LLMModel(
    display_name="[azure] gpt-4",
    model_name="gpt-4",  # This should match your Azure deployment name
    provider=ModelProvider.AZURE_OPENAI
),
LLMModel(
    display_name="[azure] gpt-4-turbo",
    model_name="gpt-4-turbo",  # This should match your Azure deployment name
    provider=ModelProvider.AZURE_OPENAI
),
LLMModel(
    display_name="[azure] gpt-35-turbo",
    model_name="gpt-35-turbo",  # This should match your Azure deployment name
    provider=ModelProvider.AZURE_OPENAI
),

]

elif model_provider == ModelProvider.AZURE_OPENAI:
# Get and validate API key and other Azure-specific settings
api_key = os.getenv("AZURE_OPENAI_API_KEY")
endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
api_version = os.getenv("AZURE_OPENAI_API_VERSION")

    if not api_key or not endpoint:
        print(f"API Key Error: Please make sure AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT are set in your .env file.")
        raise ValueError("Azure OpenAI settings not found. Please set AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT in your .env file.")
    
    # Default API version if not specified
    if not api_version:
        api_version = "2023-05-15"
        
    return AzureChatOpenAI(
        azure_deployment=model_name,
        openai_api_version=api_version,
        azure_endpoint=endpoint,
        api_key=api_key
    )

.env:

AZURE_OPENAI_API_KEY=your key
AZURE_OPENAI_ENDPOINT=https://yourendpoint.openai.azure.com/
AZURE_OPENAI_API_VERSION=2023-05-15 # or their preferred API version #example

• Will be testing it this weekend and see how this works out.

[supertech1234]

@footballqq

I just got it working and validated on my end. Azure OpenAi is now working. Will submit code change later.