Using the AI Features in the Theia IDE as an End User

This section documents how to use AI features in the Theia IDE (available since version 1.54, see also this introduction). These features are based on Theia AI, a framework for building AI assistance in tools and IDEs. Theia AI is part of the Theia platform. If you're interested in building your own custom tool or IDE with Theia AI, please refer to the corresponding documentation.

Please note that these features are in early access and experimental. This means they may be unstable, behave unexpectedly, or undergo significant changes. In particular, using your own LLM might incur costs that you need to monitor closely. We have not yet optimized the AI assistants in the Theia IDE for token usage. Use these features at your own risk, and we welcome any feedback, suggestions, and contributions!

Theia AI features within the Theia IDE are currently disabled by default. See the next section on how to enable them.

Table of Contents

Set-Up

To activate AI support in the Theia IDE, go to Preferences and enable the setting “AI-features => AI Enable.”

To use Theia AI within the Theia IDE, you need to provide access to at least one LLM. Theia IDE comes with preinstalled support for OpenAI API-compatible models, either hosted by OpenAI or self-hosted via VLLM. Additionally, Theia IDE supports connecting to models via Ollama. See the corresponding sections below on how to configure these providers.

Other LLM providers, including local models, can be added easily. If you would like to see support for a specific LLM, please provide feedback or consider contributing.

Each LLM provider offers a configurable list of available models (see the screenshot below for Hugging Face Models models). To use a model in your IDE, configure it on a per-agent basis in the AI Configuration view.

Note: Theia IDE enables connections to various models (e.g., HuggingFace, custom OpenAPI models, LlamaFile). However, not all models may work out of the box, as they may require specific customizations or optimizations. If you encounter issues, please provide feedback, keeping in mind this is an early-phase feature.

OpenAI (Hosted by OpenAI)

To enable the use of OpenAI, you need to create an API key in your OpenAI account and enter it in the settings AI-features => OpenAiOfficial (see the screenshot below). Please note: By using this preference the Open AI API key will be stored in clear text on the machine running Theia. Use the environment variable OPENAI_API_KEY to set the key securely. Please also note that creating an API key requires a paid subscription, and using these models may incur additional costs. Be sure to monitor your usage carefully to avoid unexpected charges. We have not yet optimized the AI assistants in the Theia IDE for token usage.

Open AI configuration in the Theia IDE

The OpenAI provider is preconfigured with a list of available models. You can easily add new models to this list, for example, if new options are released.

OpenAI Compatible Models (e.g. via VLLM)

As an alternative to using an official OpenAI account, Theia IDE also supports arbitrary models compatible with the OpenAI API (e.g., hosted via VLLM). This enables you to connect to self-hosted models with ease. To add a custom model, click on the link in the settings section and add your configuration like this:

{
   "ai-features.openAiCustom.customOpenAiModels": [
       {
           "model": "your-model-name",
           "url": "your-URL",
           "id": "your-unique-id", // Optional: if not provided, the model name will be used as the ID
           "apiKey": "your-api-key" // Optional: use 'true' to apply the global OpenAI API key
       }
   ]
}

Hugging Face

To enable Hugging Face as an AI provider, you need to create an API key in your Hugging Face account and enter it in the Theia IDE settings: AI-features => Hugging Face Please note: By using this preference the Hugging Face API key will be stored in clear text on the machine running Theia. Use the environment variable HUGGINGFACE_API_KEY to set the key securely. Note also that Hugging Face offers both paid and free-tier options (including "serverless"), and usage limits vary. Monitor your usage carefully to avoid unexpected costs, especially when using high-demand models. Add or remove the desired Hugging Face models from the list of available models (see screenshot below). Please note that there is a copy button in the Hugging face UI to copy model IDs to the clipboard.

Hugging Face configuration in the Theia IDE

LlamaFile Models

To configure a LlamaFile LLM in the Theia IDE, add the necessary settings to your configuration (see example below)

{
   "ai-features.llamafile.llamafiles": [
       {
           "name": "modelname", //you can choose a name for your model
           "uri": "file:///home/.../YourModel.llamafile",
           "port": 30000 //you can choose a port to be used by llamafile
       }
   ]
}

Replace "name", "uri", and "port" with your specific LlamaFile details.

The Theia IDE also offers convenience commands to start and stop your LlamaFiles:

  • Start a LlamaFile: Use the command "Start Llamafile", then select the model you want to start.
  • Stop a LlamaFile: Use the "Stop Llamafile" command, then select the running Llamafile which you want to terminate.

Please make sure that your LlamaFiles are executable. For more details on LlamaFiles, including a quickstart, see the official Mozilla LlamaFile documentation.

Ollama

To connect to models hosted via Ollama, enter the corresponding URL, along with the available models, in the settings (as shown below).

Ollama configuration in the Theia IDE

Custom Request Settings

You can define custom request settings for specific language models in the Theia IDE to tailor how models handle requests, based on their provider.

Add the settings in settings.json:

"ai-features.modelSettings.requestSettings": [
    {
        "modelId": "Qwen/Qwen2.5-Coder-32B-Instruct",
        "requestSettings": { "max_new_tokens": 2048 },
        "providerId": "huggingface"
    },
    {
        "modelId": "gemma2",
        "requestSettings": { "stop": ["<file_sep>"] },
        "providerId": "ollama"
    }
]

Or navigate in the settings view to ModelSettings => Request Settings.

Key Fields

  • modelId: The unique identifier of the model.
  • requestSettings: Provider-specific options, such as token limits or stopping criteria.
  • providerId: (Optional) Specifies the provider for the settings (e.g., huggingface, ollama, openai). If omitted, settings apply to all providers that match the modelId.

Valid options for requestSettings depend on the model provider.

Current Agents in the Theia IDE

This section provides an overview of the currently available agents in the Theia IDE. Agents marked as “Chat Agents” are available in the global chat, while others are directly integrated into UI elements, such as code completion. You can configure and deactivate agents in the AI Configuration view.

Universal (Chat Agent)

This agent helps developers by providing concise and accurate answers to general programming and software development questions. It also serves as a fallback for generic user questions. By default, this agent does not have access to the current user context or workspace. However, you can add variables, such as #selectedText, to your requests to provide additional context.

Orchestrator (Chat Agent)

This agent analyzes user requests against the descriptions of all available chat agents and selects the best-fitting agent to respond (using AI). The user's request is delegated to the selected agent without further confirmation. The Orchestrator is currently the default agent in the Theia IDE for all chat requests. You can deactivate it in the AI Configuration View.

Command (Chat Agent)

This agent is aware of all commands available in the Theia IDE and the current tool the user is working with. Based on the user request, it can find the appropriate command and let the user execute it.

Workspace (Chat Agent)

This agent can access the user's workspace, retrieve a list of all available files, and view their content. It can answer questions about the current project, including project files and source code in the workspace, such as how to build the project, where to place source code, or where to find specific files or configurations.

Code Completion (Agent)

This agent provides inline code completion within the Theia IDE's code editor. By default, automatic inline completion is disabled to give users greater control over how AI code suggestions are presented. Users can manually trigger inline completion via the default key binding SHIFT+SPACE (adaptable). Requests are canceled when moving the cursor.

Users who prefer continuous suggestions can enable 'Automatic Code Completion' in the settings ('AIFeatures'=>'CodeCompletion'). This agent makes continuous requests to the underlying LLM while coding if automatic suggestions are enabled.

In the settings, you can also specify 'Excluded File Extensions' for which the AI-powered code completion will be deactivated.

Finally, the setting 'Max Context Lines' allows you to configure the maximum number of lines used for AI code completion context. This setting can be adjusted to customize the size of the context provided to the model, which is especially useful when using smaller models with limited token capacity.

Terminal Assistance (Agent)

This agent assists with writing and executing terminal commands. Based on the user's request, it suggests commands and allows them to be directly pasted and executed in the terminal. It can access the current directory, environment, and recent terminal output to provide context-aware assistance. You can open the terminal assistance agent via Ctrl+I in the terminal view.

Chat

The Theia IDE provides a global chat interface where users can interact with all chat agents. The Orchestrator automatically delegates user requests to the most appropriate agent. To send a request directly to a specific agent, mention the agent's name using '@', for example, '@Command'. Press '@' in the chat to see a list of available chat agents.

General AI Chat in the Theia IDE

Some agents produce special results, such as buttons (shown in the screenshot above) or code that can be directly inserted. You can augment your requests in the chat with context by using variables. For example, to refer to the currently selected text, use #selectedText in your request. Pressing '#' in the chat will show a list of available variables.

AI Configuration

The AI Configuration View allows you to review and adapt agent-specific settings. Select an agent on the left side and review its properties on the right:

  • Enable Agent: Disabled agents will no longer be available in the chat or UI elements. Disabled agents also won't make any requests to LLMs.
  • Edit Prompts: Click "Edit" to open the prompt template editor, where you can customize the agent's prompts (see the section below). "Reset" will revert the prompt to its default.
  • Language Model: Select which language model the agent sends its requests to. Some agents have multiple "purposes," allowing you to select a model for each purpose.
  • Variables and Functions: Review the variables and functions used by an agent. Global variables are shared across agents, and they are listed in the second tab of the AI Configuration View. Agent-specific variables are declared and used exclusively by one agent.
AI Configuration View in the Theia IDE

View and Modify Prompts

In the Theia IDE, you can open and edit prompts for all agents from the AI Configuration View. Prompts are shown in a text editor (see the screenshot below). Changes saved in the prompt editor will take effect with the next request made to the corresponding agent. You can reset a prompt to its default using the "Reset" button in the AI configuration view or the "Revert" toolbar item in the prompt editor (top-right corner).

Prompt Editor in the Theia IDE

Variables and functions can be used in prompts. Variables are replaced with context-specific information at the time of the request (e.g., the currently selected text), while functions can trigger actions or retrieve additional information. You can find an overview of all global variables in the "Variables" tab of the AI Configuration View and agent-specific variables in the agent's configuration.

Variables are used with the following syntax:

{{variableName}}

Tool functions are used with the following syntax:

~{functionName}

Custom Agents

Custom agents enable users to define new chat agents with custom prompts on the fly, allowing the creation of custom workflows and extending the Theia IDE with new capabilities. These agents are immediately available in the default chat.

To define a new custom agent, navigate to the AI Configuration View and click on "Add Custom Agent".

Add a custom Agents in the Theia IDE

This action opens a YAML file where all available custom agents are defined. Below is an example configuration:

- id: obfuscator
  name: Obfuscator
  description: This is an example agent. Please adapt the properties to fit your needs.
  prompt: Obfuscate the following code so that no human can understand it anymore. Preserve the functionality.
  defaultLLM: openai/gpt-4o
  • id: A unique identifier for the agent.
  • name: The display name of the agent.
  • description: A brief explanation of what the agent does.
  • prompt: The default prompt that the agent will use for processing requests.
  • defaultLLM: The language model used by default.

Custom agents can be configured in the AI Configuration View just like other chat agents. You can enable/disable them, modify their prompt templates, and integrate variables and functions within these templates to enhance functionality.

AI History

The AI History view allows you to review all communications between agents and underlying LLMs. Select the corresponding agent at the top to see all its requests in the section below.

AI History View in the Theia IDE

Learn more

If want to learn more about the AI support in the Theia AI, please see this introduction, our article on the vision of Theia AI and the demonstrations in Sneak Preview Series about Theia AI

Go to previous Page : Installing VS Code ExtensionsGo to next page : Using the dynamic Toolbar