Bring your Model

In Toolhouse, you can choose what AI model and inference provider you want your agents to use.

Specifying a model gives you more flexibility:

• It gives the agent agent advanced reasoning capabilities, or the ability to perform a specific task using a more specialized model.

• When used with fast inference providers like Groq, it allows your agent to decrease its latency.

• It allows you to use the API Keys of your preferred service.

Not all models work in the same way. Some models, including preview models and specialized models, may behave inconsistently. If the performance of your agent degrades, try the following:

• Switch to a production model.

• Disable MCP servers by creating a Bundle with no MCP servers.

Note that @groq/compound-beta and @groq/compound-beta-mini do not support MCP servers. When you select these models, MCP servers and bundles will be automatically disabled, even if you pass them in your th file.

If you need help, you can talk to our team on Discord.

In Toolhouse, you can specify a model in your th file in this format: @provider/model-name, where model-name is the same model name you will find from your provider. Here are some examples of valid model names:

• @groq/llama-3.3-70b-versatile

• @openai/gpt-4.1-2025-04-14

• @anthropic/claude-sonnet-4-20250514

Toolhouse supports these providers, and we will add more regularly.

Provider	Identifier
Groq	@groq
OpenAI	@openai
Anthropic	@anthropic
Together.ai	@togetherai

Specifying a model for your agent

1

Obtain the API Key for the model provider

You will need to obtain an API Key using the developer console of the provider you choose.

• Groq Developer Console

• OpenAI Developer Console

• Anthropic Developer Console

• Together.ai Developer Console

2

Store your API Key in Toolhouse

You need to store the corresponding API key in Toolhouse. Toolhouse provides a secure way to store these keys. To do so, use the th secrets add command.

For more information about storing API keys, refer to Managing secrets.

Groq

th secrets set GROQ_API_KEY=your_api_key

OpenAI

Anthropic

Together.ai

3

Specify the model in your th file

To use a specific model with your agent, you need to specify it in your th file. The model is defined using a string that includes the provider and the model name.

The format for specifying the model is: @provider/model-name.

For example:

model: @groq/llama-3.3-70b-versatile

or

model: @openai/gpt-4o-mini

4

Test your agent

Testing your agent ensures that your configuration is valid and that the agent is working as intended.

To test your agent, simply run:

th run

5

Deploy your agent

If the agent is running, you can deploy it to start using the model you chose.

th deploy

Supported models

This list shows the models supported by Toolhouse. Use the value in Toolhouse model identifier in your th file to reference the model in your configuration.

@groq/compound-beta and @groq/compound-beta-mini do not support MCP servers. When you select these models, MCP servers and bundles will be automatically disabled, even if you pass them in your th file.

Provider	Model	Toolhouse model identifier
Groq	meta-llama/llama-4-maverick-17b-128e-instruct	@groq/meta-llama/llama-4-maverick-17b-128e-instruct
Groq	meta-llama/llama-4-scout-17b-16e-instruct	@groq/meta-llama/llama-4-scout-17b-16e-instruct
Groq	llama-3.3-70b-versatile	@groq/llama-3.3-70b-versatile
Groq	llama-3.1-8b-instant	@groq/llama-3.1-8b-instant
Groq	compound-beta	@groq/compound-beta
Groq	compound-beta-mini	@groq/compound-beta-mini
Groq	qwen/qwen3-32b	@groq/qwen/qwen3-32b
OpenAI	gpt-4.1	@openai/gpt-4.1
OpenAI	gpt-4.1-mini	@openai/gpt-4.1-mini
OpenAI	gpt-4.1-nano	@openai/gpt-4.1-nano
OpenAI	gpt-4o	@openai/gpt-4o
OpenAI	gpt-4o-mini	@openai/gpt-4o-mini
OpenAI	gpt-4-turbo	@openai/gpt-4-turbo
OpenAI	gpt-4	@openai/gpt-4
OpenAI	o3-mini	@openai/o3-mini
OpenAI	o3	@openai/o3
OpenAI	o4-mini	@openai/o4-mini
OpenAI	o1	@openai/o1
OpenAI	o1-mini	@openai/o1-mini
Anthropic	claude-opus-4-20250514	@anthropic/claude-opus-4-20250514
Anthropic	claude-sonnet-4-20250514	@anthropic/claude-sonnet-4-20250514
Anthropic	claude-3-7-sonnet-20250219	@anthropic/claude-3-7-sonnet-20250219
Anthropic	claude-3-5-sonnet-20241022	@anthropic/claude-3-5-sonnet-20241022
Anthropic	claude-3-5-sonnet-20240620	@anthropic/claude-3-5-sonnet-20240620
Anthropic	claude-3-5-haiku-20241022	@anthropic/claude-3-5-haiku-20241022
Together.ai	moonshotai/Kimi-K2-Instruct	@togetherai/moonshotai/Kimi-K2-Instruct
Together.ai	meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8	@togetherai/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
Together.ai	meta-llama/Llama-4-Scout-17B-16E-Instruct	@togetherai/meta-llama/Llama-4-Scout-17B-16E-Instruct
Together.ai	meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo	@togetherai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
Together.ai	meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo	@togetherai/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo
Together.ai	meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo	@togetherai/meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
Together.ai	meta-llama/Llama-3.3-70B-Instruct-Turbo	@togetherai/meta-llama/Llama-3.3-70B-Instruct-Turbo
Together.ai	meta-llama/Llama-3.2-3B-Instruct-Turbo	@togetherai/meta-llama/Llama-3.2-3B-Instruct-Turbo
Together.ai	Qwen/Qwen2.5-7B-Instruct-Turbo	@togetherai/Qwen/Qwen2.5-7B-Instruct-Turbo
Together.ai	Qwen/Qwen2.5-72B-Instruct-Turbo	@togetherai/Qwen/Qwen2.5-72B-Instruct-Turbo
Together.ai	Qwen/Qwen3-235B-A22B-fp8-tput	@togetherai/Qwen/Qwen3-235B-A22B-fp8-tput
Together.ai	deepseek-ai/DeepSeek-V3	@togetherai/deepseek-ai/DeepSeek-V3
Together.ai	mistralai/Mistral-Small-24B-Instruct-2501	@togetherai/mistralai/Mistral-Small-24B-Instruct-2501
Together.ai	arcee-ai/virtuoso-large	@togetherai/arcee-ai/virtuoso-large
Together.ai	arcee-ai/virtuoso-medium-v2	@togetherai/arcee-ai/virtuoso-medium-v2
Together.ai	arcee-ai/caller	@togetherai/arcee-ai/caller