Providers and Models

Perstack supports multiple LLM providers. Configure via CLI options, environment variables, or perstack.toml.

Default Model

When no model is specified, Perstack resolves it automatically using a tier-based system:

1. Expert’s defaultModelTier — if the expert definition sets a defaultModelTier (e.g. high, middle, low), the runtime picks the corresponding model from the provider
2. Provider’s middle tier — if no tier is set, falls back to the provider’s “middle” tier (e.g. claude-sonnet-4-5 for Anthropic)

This ensures cost-efficient defaults while letting experts request more capable models when needed. You can always override the resolved model with --model via the CLI or model = in perstack.toml.

To override the default, specify in perstack.toml:

model = "gemini-2.5-pro"

[provider]
providerName = "google"

Or via CLI:

npx perstack run my-expert "query" --provider google --model gemini-2.5-pro

Supported Providers

Provider	Key	Description
Anthropic	anthropic	Claude models
Google	google	Gemini models
OpenAI	openai	GPT and reasoning models
Fireworks	fireworks	Open-weight models (Kimi, DeepSeek)
DeepSeek	deepseek	DeepSeek models
Ollama	ollama	Local model hosting
Azure OpenAI	azure-openai	Azure-hosted OpenAI models
Amazon Bedrock	amazon-bedrock	AWS Bedrock-hosted models
Google Vertex AI	google-vertex	Google Cloud Vertex AI models

Anthropic

Environment variables:

Variable	Required	Description
ANTHROPIC_API_KEY	Yes	API key
ANTHROPIC_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "anthropic"
[provider.setting]
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional

Setting	Type	Description
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers

Native reasoning: Supported via extended thinking.

Models:

Model	Context	Max Output
claude-opus-4-5	200K	32K
claude-opus-4-1	200K	32K
claude-opus-4-20250514	200K	32K
claude-sonnet-4-5	200K	64K
claude-sonnet-4-20250514	200K	64K
claude-3-7-sonnet-20250219	200K	64K
claude-haiku-4-5	200K	8K
claude-3-5-haiku-latest	200K	8K

export ANTHROPIC_API_KEY=sk-ant-...
npx perstack run my-expert "query" --provider anthropic --model claude-sonnet-4-5

Google

Environment variables:

Variable	Required	Description
GOOGLE_GENERATIVE_AI_API_KEY	Yes	API key
GOOGLE_GENERATIVE_AI_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "google"
[provider.setting]
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional

Setting	Type	Description
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers

Models:

Model	Context	Max Output
gemini-3-pro-preview	1M	64K
gemini-2.5-pro	1M	64K
gemini-2.5-flash	1M	64K
gemini-2.5-flash-lite	1M	64K

export GOOGLE_GENERATIVE_AI_API_KEY=AIza...
npx perstack run my-expert "query" --provider google --model gemini-2.5-pro

OpenAI

Environment variables:

Variable	Required	Description
OPENAI_API_KEY	Yes	API key
OPENAI_BASE_URL	No	Custom endpoint (OpenAI-compatible)
OPENAI_ORGANIZATION	No	Organization ID
OPENAI_PROJECT	No	Project ID

perstack.toml settings:

[provider]
providerName = "openai"
[provider.setting]
baseUrl = "https://custom-endpoint.example.com"  # Optional
organization = "org-xxx"                          # Optional
project = "proj-xxx"                              # Optional
name = "custom-openai"                            # Optional: custom provider name
headers = { "X-Custom-Header" = "value" }         # Optional

Setting	Type	Description
baseUrl	string	Custom API endpoint
organization	string	OpenAI organization ID
project	string	OpenAI project ID
name	string	Custom provider name
headers	object	Custom HTTP headers

Native reasoning: Supported via reasoningEffort. Works with o-series models (o1, o3, o4-mini).

Models:

Model	Context	Max Output
gpt-5	400K	128K
gpt-5-mini	400K	128K
gpt-5-nano	400K	128K
gpt-5-chat-latest	128K	16K
o4-mini	200K	100K
o3	200K	10K
o3-mini	200K	10K
gpt-4.1	1M	32K

export OPENAI_API_KEY=sk-proj-...
npx perstack run my-expert "query" --provider openai --model gpt-5

DeepSeek

Environment variables:

Variable	Required	Description
DEEPSEEK_API_KEY	Yes	API key
DEEPSEEK_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "deepseek"
[provider.setting]
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional

Setting	Type	Description
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers

Models:

Model	Context	Max Output
deepseek-chat	128K	8K
deepseek-reasoner	128K	64K

export DEEPSEEK_API_KEY=sk-...
npx perstack run my-expert "query" --provider deepseek --model deepseek-chat

Fireworks

Environment variables:

Variable	Required	Description
FIREWORKS_API_KEY	Yes	API key
FIREWORKS_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "fireworks"
[provider.setting]
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional

Setting	Type	Description
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers

Models:

Model	Context	Max Output
accounts/fireworks/models/kimi-k2p5	262K	262K
accounts/fireworks/models/deepseek-v3p2	164K	164K
accounts/fireworks/models/glm-5	203K	203K

export FIREWORKS_API_KEY=fw_...
npx perstack run my-expert "query" --provider fireworks --model accounts/fireworks/models/kimi-k2p5

Ollama

Environment variables:

Variable	Required	Description
OLLAMA_BASE_URL	No	Server URL (default: http://localhost:11434)

perstack.toml settings:

[provider]
providerName = "ollama"
[provider.setting]
baseUrl = "http://localhost:11434"        # Optional
headers = { "X-Custom-Header" = "value" } # Optional

Setting	Type	Description
baseUrl	string	Ollama server URL
headers	object	Custom HTTP headers

Models:

Model	Context	Max Output
gpt-oss:20b	128K	128K
gpt-oss:120b	128K	128K
gemma3:1b	32K	32K
gemma3:4b	128K	128K
gemma3:12b	128K	128K
gemma3:27b	128K	128K

export OLLAMA_BASE_URL=http://localhost:11434
npx perstack run my-expert "query" --provider ollama --model gpt-oss:20b

Azure OpenAI

Environment variables:

Variable	Required	Description
AZURE_API_KEY	Yes	API key
AZURE_RESOURCE_NAME	Yes	Resource name
AZURE_API_VERSION	No	API version
AZURE_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "azure-openai"
[provider.setting]
resourceName = "your-resource-name"              # Optional (env fallback)
apiVersion = "2024-02-15-preview"                # Optional
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional
useDeploymentBasedUrls = true                    # Optional

Setting	Type	Description
resourceName	string	Azure resource name
apiVersion	string	Azure API version
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers
useDeploymentBasedUrls	boolean	Use deployment-based URLs

export AZURE_API_KEY=your_azure_key
export AZURE_RESOURCE_NAME=your_resource_name
npx perstack run my-expert "query" --provider azure-openai --model your-deployment-name

Amazon Bedrock

Environment variables:

Variable	Required	Description
AWS_ACCESS_KEY_ID	Yes	Access key ID
AWS_SECRET_ACCESS_KEY	Yes	Secret access key
AWS_REGION	Yes	Region (e.g., us-east-1)
AWS_SESSION_TOKEN	No	Session token (temporary credentials)

perstack.toml settings:

[provider]
providerName = "amazon-bedrock"
[provider.setting]
region = "us-east-1"  # Optional (env fallback)

Setting	Type	Description
region	string	AWS region

export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=us-east-1
npx perstack run my-expert "query" --provider amazon-bedrock --model anthropic.claude-v2

Google Vertex AI

Environment variables:

Variable	Required	Description
GOOGLE_VERTEX_PROJECT	No	GCP project ID
GOOGLE_VERTEX_LOCATION	No	GCP location (e.g., us-central1)
GOOGLE_VERTEX_BASE_URL	No	Custom endpoint

perstack.toml settings:

[provider]
providerName = "google-vertex"
[provider.setting]
project = "my-gcp-project"                       # Optional
location = "us-central1"                         # Optional
baseUrl = "https://custom-endpoint.example.com"  # Optional
headers = { "X-Custom-Header" = "value" }        # Optional

Setting	Type	Description
project	string	GCP project ID
location	string	GCP location
baseUrl	string	Custom API endpoint
headers	object	Custom HTTP headers

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
npx perstack run my-expert "query" --provider google-vertex --model gemini-1.5-pro