-
-
Notifications
You must be signed in to change notification settings - Fork 102
Possibility of adding beta header in Claude #85
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Comments
Hey @luismiguelamado thank you so much! I'd love to see that added to the library! Honestly, that header should be the default and I would go without configuration on this one. Regarding the implementation: I did something similar - changing something only for a model - for the o1 and o3 series of models here: ruby_llm/lib/ruby_llm/provider.rb Lines 13 to 18 in b32a435
A similar implementation would be much appreciated! |
This change implements support for Anthropic's `output-128k-2025-02-19` beta header, specifically for the `claude-3-7-sonnet-20250219` model. Including this header allows users to leverage the increased 128k maximum output token limit available for this model. This implementation attempts to follow the suggested pattern for provider-specific capabilities, similar to the existing temperature normalization logic: - A new method `additional_headers_for_model` was added to `Anthropic::Capabilities` to encapsulate the provider-specific logic. - The shared `Provider#post` method was updated to conditionally call this capability method via `respond_to?` and merge the resulting headers before making the API request. - The necessary `model_id` context is now passed down through the relevant methods (`embed`, `paint`, `sync_response`, `stream_response`) to `post`. This approach avoids overriding shared methods within the provider module and keeps the specific logic contained within the capabilities definition. Closes crmne#85
Hello,
Thank you for such a great library! I'm really enjoying using it.
I'm encountering a limitation with Claude 3.7 regarding the output token limit of 8192 tokens.
There is a beta header that can be added to increase the output tokens. According to Anthropic's [documentation](https://docs.anthropic.com/en/docs/about-claude/models/all-models), "Include the beta header
output-128k-2025-02-19
in your API request to increase the maximum output token length to 128k tokens for Claude 3.7 Sonnet."Is there a possibility of adding this header? Perhaps during the configuration, we could have a flag to enable or disable the header.
Thank you in advance!
The text was updated successfully, but these errors were encountered: