Expanding Function calling

[ily-R]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

I wanted to use function calling with claude 3.5 sonnet, but going through another serverless api provider. I defined My model as follows:

  "models": [
    {
      "model": "claude-3-5-sonnet-20240620",
      "title": "Claude 3.5 Sonnet",
      "apiKey": "api-key"
      "provider": "anthropic"
    },
    {
      "title": "My model",
      "provider": "openai",
      "model": "assistant:ContinueModel-claude-3-5-sonnet",
      "apiKey": "api-key",
      "apiBase": "https://api.serverX.ai"
    },

Even though the model I am using under the hood through my api is anthropic.claude-3.5-sonnet-v2 from AWS Bedrock I still don't have the Tools option working because you seem to have a condition that the provider has to be anthropic.

Solution

It will be nice if any Anthropic model message should be passed through Anthropic.ts wether it's coming through the Anthropic api itself or any other providers like Bedrock or openai in my case.

[mmaist]

+1, same issue when using claude-3.5-sonnet through aws

[VikashLoomba]

+1, Would be nice to have tools configurable via the config. It would be a huge positive to expand tool use to compatible open source models as well.

[dseeker]

there's no reason to "take away" tools because they are not listed
models are being updated every week, the tools should be open to all models, no reason to gatekeep.

the extension could instead, like lm studio does, have an icon showing the model is known to work with tools, but still allow us to decide for ourselves to use it or not.
https://github.com/continuedev/continue/blob/main/core/llm/toolSupport.ts#L1

On Cline I use AWS Bedrock but also use LMStudio with many models that were trained and even others who weren't.
On Continue I can't even get tool use with LMStudio at all, makes no sense