Skip to content

Commit

Permalink
Add docu on custom request settings (#662)
Browse files Browse the repository at this point in the history
fixed #661

Signed-off-by: Jonas Helming <jhelming@eclipsesource.com>
  • Loading branch information
JonasHelming authored Nov 28, 2024
1 parent 42fd6c5 commit 12cec62
Showing 1 changed file with 31 additions and 0 deletions.
31 changes: 31 additions & 0 deletions src/docs/user_ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Theia AI features within the Theia IDE are currently disabled by default. See th
- [Hugging Face](#hugging-face)
- [LlamaFile](#llamafile-models)
- [Ollama](#ollama)
- [Custom Request Settings](#custom-request-settings)
- [Current Agents in the Theia IDE](#current-agents-in-the-theia-ide)
- [Universal (Chat Agent)](#universal-chat-agent)
- [Orchestrator (Chat Agent)](#orchestrator-chat-agent)
Expand Down Expand Up @@ -108,6 +109,36 @@ To connect to models hosted via [Ollama](https://ollama.com/), enter the corresp

<img src="../../ollama-setting.png" alt="Ollama configuration in the Theia IDE" style="max-width: 525px">

### Custom Request Settings

You can define **custom request settings** for specific language models in the Theia IDE to tailor how models handle requests, based on their provider.

Add the settings in `settings.json`:

```json
"ai-features.modelSettings.requestSettings": [
{
"modelId": "Qwen/Qwen2.5-Coder-32B-Instruct",
"requestSettings": { "max_new_tokens": 2048 },
"providerId": "huggingface"
},
{
"modelId": "gemma2",
"requestSettings": { "stop": ["<file_sep>"] },
"providerId": "ollama"
}
]
```

Or navigate in the settings view to **`ModelSettings` => `Request Settings`**.

#### Key Fields
- **`modelId`**: The unique identifier of the model.
- **`requestSettings`**: Provider-specific options, such as token limits or stopping criteria.
- **`providerId`**: *(Optional)* Specifies the provider for the settings (e.g., `huggingface`, `ollama`, `openai`). If omitted, settings apply to all providers that match the `modelId`.

Valid options for `requestSettings` depend on the model provider.

## Current Agents in the Theia IDE

This section provides an overview of the currently available agents in the Theia IDE. Agents marked as “Chat Agents” are available in the global chat, while others are directly integrated into UI elements, such as code completion. You can configure and deactivate agents in the AI Configuration view.
Expand Down

0 comments on commit 12cec62

Please # to comment.