# Bring your Model In Toolhouse, you can choose what AI model and inference provider you want your agents to use. Specifying a model gives you more flexibility: * It gives the agent agent advanced reasoning capabilities, or the ability to perform a specific task using a more specialized model. * When used with fast inference providers like Groq, it allows your agent to decrease its latency. * It allows you to use the API Keys of your preferred service. {% hint style="info" %} Not all models work in the same way. Some models, including preview models and specialized models, may behave inconsistently. If the performance of your agent degrades, try the following: * Switch to a production model. * Disable MCP servers by creating a Bundle with no MCP servers. Note that `@groq/groq/compound` and `@groq/groq/compound-mini` do not support MCP servers. When you select these models, MCP servers and bundles will be automatically disabled, even if you pass them in your th file. If you need help, you can [talk to our team on Discord](https://discord.toolhouse.ai/). {% endhint %} In Toolhouse, you can specify a model in your th file in this format: `@provider/model-name`, where `model-name` is the same model name you will find from your provider. Here are some examples of valid model names: * `@groq/llama-3.3-70b-versatile` * `@openai/gpt-4.1-2025-04-14` * `@anthropic/claude-sonnet-4-20250514` Toolhouse supports these providers, and we will add more regularly. | Provider | Identifier | | ----------- | ------------- | | Groq | `@groq` | | OpenAI | `@openai` | | Anthropic | `@anthropic` | | Together.ai | `@togetherai` | | OpenRouter | `@openrouter` | ## Specifying a model for your agent {% stepper %} {% step %} ### Obtain the API Key for the model provider You will need to obtain an API Key using the developer console of the provider you choose. * [Groq Developer Console](https://console.groq.com/?utm_source=toolhouse) * [OpenAI Developer Console](https://platform.openai.com/?urm_source=toolhouse) * [Anthropic Developer Console](https://console.anthropic.com/?utm_source=toolhouse) * [Together.ai Developer Console](https://api.together.ai/signin) * [OpenRouter Developer Console](https://openrouter.ai/settings/keys) {% endstep %} {% step %} ### Store your API Key in Toolhouse You need to store the corresponding API key in Toolhouse. Toolhouse provides a secure way to store these keys. To do so, use the `th secrets add` command. For more information about storing API keys, refer to [managing-secrets](https://docs.toolhouse.ai/toolhouse/advanced-concepts/managing-secrets "mention"). {% tabs %} {% tab title="Groq" %} ```bash th secrets set GROQ_API_KEY=your_api_key ``` {% endtab %} {% tab title="OpenAI" %} ```bash th secrets set OPENAI_API_KEY=your_api_key ``` {% endtab %} {% tab title="Anthropic" %} ```bash th secrets set ANTHROPIC_API_KEY=your_api_key ``` {% endtab %} {% tab title="Together.ai" %} ```bash th secrets set TOGETHER_API_KEY=your_api_key ``` {% endtab %} {% tab title="OpenRouter" %} ```bash th secrets set OPENROUTER_API_KEY=your_api_key ``` {% endtab %} {% endtabs %} {% endstep %} {% step %} ### Specify the model in your th file To use a specific model with your agent, you need to specify it in your `th` file. The model is defined using a string that includes the provider and the model name. The format for specifying the model is: `@provider/model-name`. For example: ```yaml model: @groq/llama-3.3-70b-versatile ``` or ```yaml model: @openai/gpt-4o-mini ``` {% endstep %} {% step %} ### Test your agent Testing your agent ensures that your configuration is valid and that the agent is working as intended. To test your agent, simply run: ```bash th run ``` {% endstep %} {% step %} ### Deploy your agent If the agent is running, you can deploy it to start using the model you chose. ```bash th deploy ``` {% endstep %} {% endstepper %} ## Supported models This list shows the models supported by Toolhouse. Use the value in Toolhouse model identifier in your th file to reference the model in your configuration. {% hint style="info" %} `@groq/groq/compound` and `@groq/groq/compound-mini` do not support Tools or MCP servers. When you select these models, MCP servers and bundles will be automatically disabled, even if you pass them in your th file. {% endhint %}
ProviderModelToolhouse model identifier
Groqmeta-llama/llama-4-maverick-17b-128e-instruct
@groq/meta-llama/llama-4-maverick-17b-128e-instruct
Groqmeta-llama/llama-4-scout-17b-16e-instruct
@groq/meta-llama/llama-4-scout-17b-16e-instruct
Groqllama-3.3-70b-versatile
@groq/llama-3.3-70b-versatile
Groqllama-3.1-8b-instant
@groq/llama-3.1-8b-instant
Groqcompound
@groq/groq/compound
Groqcompound-mini
@groq/groq/compound-mini
Groqqwen/qwen3-32b
@groq/qwen/qwen3-32b
OpenAIgpt-4.1
@openai/gpt-4.1
OpenAIgpt-4.1-mini
@openai/gpt-4.1-mini
OpenAIgpt-4.1-nano
@openai/gpt-4.1-nano
OpenAIgpt-4o
@openai/gpt-4o
OpenAIgpt-4o-mini
@openai/gpt-4o-mini
OpenAIgpt-4-turbo
@openai/gpt-4-turbo
OpenAIgpt-4
@openai/gpt-4
OpenAIo3-mini
@openai/o3-mini
OpenAIo3
@openai/o3
OpenAIo4-mini
@openai/o4-mini
OpenAIo1
@openai/o1
OpenAIo1-mini
@openai/o1-mini
Anthropicclaude-opus-4-20250514
@anthropic/claude-opus-4-20250514
Anthropicclaude-sonnet-4-20250514
@anthropic/claude-sonnet-4-20250514
Anthropicclaude-3-7-sonnet-20250219
@anthropic/claude-3-7-sonnet-20250219
Anthropicclaude-3-5-sonnet-20241022
@anthropic/claude-3-5-sonnet-20241022
Anthropicclaude-3-5-sonnet-20240620
@anthropic/claude-3-5-sonnet-20240620
Anthropicclaude-3-5-haiku-20241022
@anthropic/claude-3-5-haiku-20241022
Together.aimoonshotai/Kimi-K2-Instruct
@togetherai/moonshotai/Kimi-K2-Instruct
Together.aimeta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
@togetherai/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
Together.aimeta-llama/Llama-4-Scout-17B-16E-Instruct
@togetherai/meta-llama/Llama-4-Scout-17B-16E-Instruct
Together.aimeta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
@togetherai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
Together.aimeta-llama/Meta-Llama-3.1-70B-Instruct-Turbo
@togetherai/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo
Together.aimeta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
@togetherai/meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
Together.aimeta-llama/Llama-3.3-70B-Instruct-Turbo
@togetherai/meta-llama/Llama-3.3-70B-Instruct-Turbo
Together.aimeta-llama/Llama-3.2-3B-Instruct-Turbo
@togetherai/meta-llama/Llama-3.2-3B-Instruct-Turbo
Together.aiQwen/Qwen2.5-7B-Instruct-Turbo
@togetherai/Qwen/Qwen2.5-7B-Instruct-Turbo
Together.aiQwen/Qwen2.5-72B-Instruct-Turbo
@togetherai/Qwen/Qwen2.5-72B-Instruct-Turbo
Together.aiQwen/Qwen3-235B-A22B-fp8-tput
@togetherai/Qwen/Qwen3-235B-A22B-fp8-tput
Together.aideepseek-ai/DeepSeek-V3
@togetherai/deepseek-ai/DeepSeek-V3
Together.aimistralai/Mistral-Small-24B-Instruct-2501
@togetherai/mistralai/Mistral-Small-24B-Instruct-2501
Together.aiarcee-ai/virtuoso-large
@togetherai/arcee-ai/virtuoso-large
Together.aiarcee-ai/virtuoso-medium-v2
@togetherai/arcee-ai/virtuoso-medium-v2
Together.aiarcee-ai/caller
@togetherai/arcee-ai/caller
--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.toolhouse.ai/toolhouse/advanced-concepts/bring-your-model.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.