Skip to main content

Supported LLM Providers

goose is compatible with a wide range of LLM providers, allowing you to choose and integrate your preferred model.

Model Selection

Goose relies heavily on tool calling capabilities and currently works best with Claude 4 models.

Berkeley Function-Calling Leaderboard can be a good guide for selecting models.

Available Providers​

ProviderDescriptionParameters
Amazon BedrockOffers a variety of foundation models, including Claude, Jurassic-2, and others. AWS environment variables must be set in advance, not configured through goose configureAWS_PROFILE, or AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION
Amazon SageMaker TGIRun Text Generation Inference models through Amazon SageMaker endpoints. AWS credentials must be configured in advance.SAGEMAKER_ENDPOINT_NAME, AWS_REGION (optional), AWS_PROFILE (optional)
AnthropicOffers Claude, an advanced AI model for natural language tasks.ANTHROPIC_API_KEY, ANTHROPIC_HOST (optional)
Azure OpenAIAccess Azure-hosted OpenAI models, including GPT-4 and GPT-3.5. Supports both API key and Azure credential chain authentication.AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_DEPLOYMENT_NAME, AZURE_OPENAI_API_KEY (optional)
DatabricksUnified data analytics and AI platform for building and deploying models.DATABRICKS_HOST, DATABRICKS_TOKEN
Docker Model RunnerLocal models running in Docker Desktop or Docker CE with OpenAI-compatible API endpoints. Because this provider runs locally, you must first download a model.OPENAI_HOST, OPENAI_BASE_PATH
GeminiAdvanced LLMs by Google with multimodal capabilities (text, images).GOOGLE_API_KEY
GCP Vertex AIGoogle Cloud's Vertex AI platform, supporting Gemini and Claude models. Credentials must be configured in advance.GCP_PROJECT_ID, GCP_LOCATION and optionally GCP_MAX_RATE_LIMIT_RETRIES (5), GCP_MAX_OVERLOADED_RETRIES (5), GCP_INITIAL_RETRY_INTERVAL_MS (5000), GCP_BACKOFF_MULTIPLIER (2.0), GCP_MAX_RETRY_INTERVAL_MS (320_000).
GitHub CopilotAccess to AI models from OpenAI, Anthropic, Google, and other providers through GitHub's Copilot infrastructure. GitHub account with Copilot access required.No manual key. Must configure through the CLI using the GitHub authentication flow to enable both CLI and Desktop access.
GroqHigh-performance inference hardware and tools for LLMs.GROQ_API_KEY
LiteLLMLiteLLM proxy supporting multiple models with automatic prompt caching and unified API access.LITELLM_HOST, LITELLM_BASE_PATH (optional), LITELLM_API_KEY (optional), LITELLM_CUSTOM_HEADERS (optional), LITELLM_TIMEOUT (optional)
Mistral AIProvides access to Mistral models including general-purpose models, specialized coding models (Codestral), and multimodal models (Pixtral).MISTRAL_API_KEY
OllamaLocal model runner supporting Qwen, Llama, DeepSeek, and other open-source models. Because this provider runs locally, you must first download and run a model.OLLAMA_HOST
RamalamaLocal model using native OCI container runtimes, CNCF tools, and supporting models as OCI artifacts. Ramalama API an compatible alternative to Ollama and can be used with the goose Ollama provider. Supports Qwen, Llama, DeepSeek, and other open-source models. Because this provider runs locally, you must first download and run a model.OLLAMA_HOST
OpenAIProvides gpt-4o, o1, and other advanced language models. Also supports OpenAI-compatible endpoints (e.g., self-hosted LLaMA, vLLM, KServe). o1-mini and o1-preview are not supported because goose uses tool calling.OPENAI_API_KEY, OPENAI_HOST (optional), OPENAI_ORGANIZATION (optional), OPENAI_PROJECT (optional), OPENAI_CUSTOM_HEADERS (optional)
OpenRouterAPI gateway for unified access to various models with features like rate-limiting management.OPENROUTER_API_KEY
SnowflakeAccess the latest models using Snowflake Cortex services, including Claude models. Requires a Snowflake account and programmatic access token (PAT).SNOWFLAKE_HOST, SNOWFLAKE_TOKEN
Tetrate Agent Router ServiceUnified API gateway for AI models including Claude, Gemini, GPT, open-weight models, and others. Supports PKCE authentication flow for secure API key generation.TETRATE_API_KEY, TETRATE_HOST (optional)
Venice AIProvides access to open source models like Llama, Mistral, and Qwen while prioritizing user privacy. Requires an account and an API key.VENICE_API_KEY, VENICE_HOST (optional), VENICE_BASE_PATH (optional), VENICE_MODELS_PATH (optional)
xAIAccess to xAI's Grok models including grok-3, grok-3-mini, and grok-3-fast with 131,072 token context window.XAI_API_KEY, XAI_HOST (optional)

CLI Providers​

goose also supports special "pass-through" providers that work with existing CLI tools, allowing you to use your subscriptions instead of paying per token:

ProviderDescriptionRequirements
Claude Code (claude-code)Uses Anthropic's Claude CLI tool with your Claude Code subscription. Provides access to Claude with 200K context limit.Claude CLI installed and authenticated, active Claude Code subscription
Cursor Agent (cursor-agent)Uses Cursor's AI CLI tool with your Cursor subscription. Provides access to GPT-5, Claude 4, and other models through the cursor-agent command-line interface.cursor-agent CLI installed and authenticated
Gemini CLI (gemini-cli)Uses Google's Gemini CLI tool with your Google AI subscription. Provides access to Gemini with 1M context limit.Gemini CLI installed and authenticated
CLI Providers

CLI providers are cost-effective alternatives that use your existing subscriptions. They work differently from API providers as they execute CLI commands and integrate with the tools' native capabilities. See the CLI Providers guide for detailed setup instructions.

Configure Provider and Model​

To configure your chosen provider, see available options, or select a model, visit the Models tab in goose Desktop or run goose configure in the CLI.

First-time users:

On the welcome screen the first time you open goose, you have three options:

We recommend starting with Tetrate Agent Router. Tetrate provides access to multiple AI models with built-in rate limiting and automatic failover.

Free Credits Offer

You'll receive $10 in free credits the first time you automatically authenticate with Tetrate through goose. This offer is available to both new and existing Tetrate users.

  1. Choose Automatic setup with Tetrate Agent Router.
  2. goose will open a browser window for you to authenticate with Tetrate, or create a new account if you don't have one already.
  3. When you return to the goose desktop app, you're ready to begin your first session.

To update your LLM provider and API key:

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Configure providers
  5. Click your provider in the list
  6. Add your API key and other required configurations, then click Submit

To change your current model:

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Switch models
  5. Choose from your configured providers in the dropdown, or select Use other provider to configure a new one
  6. Select a model from the available options, or choose Use custom model to enter a specific model name
  7. Click Select model to confirm your choice
Shortcut

For faster access, click your current model name at the bottom of the app and choose Change Model.

To start over with provider and model configuration:

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Reset Provider and Model to clear your current settings and return to the welcome screen

Using Custom OpenAI Endpoints​

The built-in OpenAI provider can connect to OpenAI's official API (api.openai.com) or any OpenAI-compatible endpoint, such as:

  • Self-hosted LLMs (e.g., LLaMA, Mistral) using vLLM or KServe
  • Private OpenAI-compatible API servers
  • Enterprise deployments requiring data governance and security compliance
  • OpenAI API proxies or gateways
Custom Provider Option

Need to connect to multiple OpenAI-compatible endpoints? Configure custom providers instead for easier switching and better organization, as well as custom naming and shareable configurations.

Configuration Parameters​

ParameterRequiredDescription
OPENAI_API_KEYYesAuthentication key for the API
OPENAI_HOSTNoCustom endpoint URL (defaults to api.openai.com)
OPENAI_ORGANIZATIONNoOrganization ID for usage tracking and governance
OPENAI_PROJECTNoProject identifier for resource management
OPENAI_CUSTOM_HEADERSNoAdditional headers to include in the request. Can be set via environment variable, configuration file, or CLI, in the format HEADER_A=VALUE_A,HEADER_B=VALUE_B.

Example Configurations​

If you're running LLaMA or other models using vLLM with OpenAI compatibility:

OPENAI_HOST=https://your-vllm-endpoint.internal
OPENAI_API_KEY=your-internal-api-key

Setup Instructions​

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Configure providers
  5. Click OpenAI in the provider list
  6. Fill in your configuration details:
    • API Key (required)
    • Host URL (for custom endpoints)
    • Organization ID (for usage tracking)
    • Project (for resource management)
  7. Click Submit
Enterprise Deployment

For enterprise deployments, you can pre-configure these values using environment variables or configuration files to ensure consistent governance across your organization.

Configure Custom Provider​

Custom providers let you connect to services that aren't in the available providers list. They appear in goose's provider list and can be selected like any other provider.

Benefits:

  • Multiple endpoints: Switch between different services (e.g., vLLM, corporate proxy, OpenAI)
  • Pre-configured models: Store a list of preferred models
  • Shareable configuration: JSON files can be shared across teams or checked into repos
  • Custom naming: Show "Corporate API" instead of "OpenAI" in the UI
  • Separate credentials: Assign each provider its own API key

Custom providers must use OpenAI, Anthropic, or Ollama compatible API formats. OpenAI-compatible providers can include custom headers for additional authentication, API keys, tokens, or tenant identifiers. Each custom provider maps to a JSON configuration file.

To add a custom provider:

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Configure providers
  5. Click Add Custom Provider at the bottom of the window
  6. Fill in the provider details:
    • Provider Type:
      • OpenAI Compatible (most common)
      • Anthropic Compatible
      • Ollama Compatible
    • Display Name: A friendly name for the provider
    • API URL: The base URL of the API endpoint
    • API Key: The API key, which is accessed using a custom environment variable and stored in the keychain (or secrets.yaml if the keyring is disabled)
      • For Ollama Compatible providers, click This is a local model (no auth required)
    • Available Models: Comma-separated list of available model names
    • Streaming Support: Whether the API supports streaming responses (click to toggle)
  7. Click Create Provider
Custom Headers

Currently, custom headers for OpenAI compatible providers can't be defined in goose Desktop. As a workaround, configure the provider using goose CLI or edit the provider configuration file directly.

To update a custom provider:

  1. Click the button in the top-left to open the sidebar
  2. Click the Settings button on the sidebar
  3. Click the Models tab
  4. Click Configure providers
  5. Click on your custom provider in the list
  6. Update the fields you want to change
    Important: Verify that Provider Type shows the correct value before saving. Otherwise, it may default to OpenAI Compatible regardless of the original setting.
  7. Click Update Provider

Your changes are available in your next goose session.

To remove a custom provider:

Currently you cannot remove custom providers using goose Desktop.

Using goose for Free​

goose is a free and open source AI agent that you can start using right away, but not all supported LLM Providers provide a free tier.

Below, we outline a couple of free options and how to get started with them.

Limitations

These free options are a great way to get started with goose and explore its capabilities. However, you may need to upgrade your LLM for better performance.

Groq​

Groq provides free access to open source models with high-speed inference. To use Groq with goose, you need an API key from Groq Console.

Groq offers several open source models that support tool calling:

  • moonshotai/kimi-k2-instruct - Mixture-of-Experts model with 1 trillion parameters, optimized for agentic intelligence and tool use
  • qwen/qwen3-32b - 32.8 billion parameter model with advanced reasoning and multilingual capabilities
  • gemma2-9b-it - Google's Gemma 2 model with instruction tuning
  • llama-3.3-70b-versatile - Meta's Llama 3.3 model for versatile applications

To set up Groq with goose, follow these steps:

To update your LLM provider and API key:

  1. Click the button in the top-left to open the sidebar.
  2. Click the Settings button on the sidebar.
  3. Click the Models tab.
  4. Click Configure Providers
  5. Choose Groq as provider from the list.
  6. Click Configure, enter your API key, and click Submit.

Google Gemini​

Google Gemini provides a free tier. To start using the Gemini API with goose, you need an API Key from Google AI studio.

To set up Google Gemini with goose, follow these steps:

To update your LLM provider and API key:

  1. Click the button in the top-left to open the sidebar.
  2. Click the Settings button on the sidebar.
  3. Click the Models tab.
  4. Click Configure Providers
  5. Choose Google Gemini as provider from the list.
  6. Click Configure, enter your API key, and click Submit.

Local LLMs​

goose is a local AI agent, and by using a local LLM, you keep your data private, maintain full control over your environment, and can work entirely offline without relying on cloud access. However, please note that local LLMs require a bit more set up before you can use one of them with goose.

Limited Support for models without tool calling

goose extensively uses tool calling, so models without it can only do chat completion. If using models without tool calling, all goose extensions must be disabled.

Here are some local providers we support:

  1. Download Ollama.
  2. In a terminal, run any model supporting tool-calling

Example:

ollama run qwen2.5
  1. In a separate terminal window, configure with goose:
goose configure
  1. Choose to Configure Providers
┌   goose-configure 
│
◆  What would you like to configure?
│  ● Configure Providers (Change provider or update credentials)
│  ○ Toggle Extensions 
│  ○ Add Extension 
└
  1. Choose Ollama as the model provider
┌   goose-configure 
│
◇  What would you like to configure?
│  Configure Providers 
│
◆  Which model provider should we use?
│  ○ Anthropic 
│  ○ Databricks 
│  ○ Google Gemini 
│  ○ Groq 
│  ● Ollama (Local open source models)
│  ○ OpenAI 
│  ○ OpenRouter 
└
  1. Enter the host where your model is running
Endpoint

For Ollama, if you don't provide a host, we set it to localhost:11434. When constructing the URL, we prepend http:// if the scheme is not http or https. If you're running Ollama on a different server, you'll have to set OLLAMA_HOST=http://{host}:{port}.

┌   goose-configure 
│
◇  What would you like to configure?
│  Configure Providers 
│
◇  Which model provider should we use?
│  Ollama 
│
◆  Provider Ollama requires OLLAMA_HOST, please enter a value
│  http://localhost:11434
└
  1. Enter the model you have running
┌   goose-configure 
│
◇  What would you like to configure?
│  Configure Providers 
│
◇  Which model provider should we use?
│  Ollama 
│
◇  Provider Ollama requires OLLAMA_HOST, please enter a value
│  http://localhost:11434
│
◇  Enter a model from that provider:
│  qwen2.5
│
◇  Welcome! You're all set to explore and utilize my capabilities. Let's get started on solving your problems together!
│
└  Configuration saved successfully
Context Length

If you notice that goose is having trouble using extensions or is ignoring .goosehints, it is likely that the model's default context length of 4096 tokens is too low. Set the OLLAMA_CONTEXT_LENGTH environment variable to a higher value.

Azure OpenAI Credential Chain​

goose supports two authentication methods for Azure OpenAI:

  1. API Key Authentication - Uses the AZURE_OPENAI_API_KEY for direct authentication
  2. Azure Credential Chain - Uses Azure CLI credentials automatically without requiring an API key

To use the Azure Credential Chain:

  • Ensure you're logged in with az login
  • Have appropriate Azure role assignments for the Azure OpenAI service
  • Configure with goose configure and select Azure OpenAI, leaving the API key field empty

This method simplifies authentication and enhances security for enterprise environments.

Multi-Model Configuration​

Beyond single-model setups, goose supports multi-model configurations that can use different models and providers for specialized tasks:

  • Lead/Worker Model - Automatic switching between a lead model for initial turns and a worker model for execution tasks
  • Planning Mode - Manual planning phase using a dedicated model to create detailed project breakdowns before execution

If you have any questions or need help with a specific provider, feel free to reach out to us on Discord or on the goose repo.