Skip to main content

AI Providers Setup Guide

Welcome to the TruvaG3 AI providers guide! This document explains how to configure AI providers for your agents and tools, from simple single-provider setups to production-ready multi-provider failover systems. Think of this as your complete reference for doing AI integration the right way.

Table of Contents​

Why This Guide Exists​

In a production system, AI integration is rarely as simple as "call OpenAI and hope for the best." You need to handle:

  • Provider outages: What happens when OpenAI is down?
  • Budget exhaustion: What happens when your API quota or spending limit is reached?
  • Cost management: How do you use cheaper models in development?
  • API key rotation: How do you change keys without redeploying?
  • Regional routing: How do you route traffic to regional endpoints (e.g., EU data residency)?

Without a clear strategy, you end up with:

  • Hardcoded API keys (security nightmare)
  • No failover (single point of failure)
  • Different code paths for different environments (maintenance nightmare)

This guide ensures every TruvaG3 deployment handles AI providers in a consistent, production-ready way.

The Two Types of AI Clients​

TruvaG3 provides two ways to connect to AI providers. Understanding when to use each is the first decision you'll make.

Single Client: The Simple Path​

A Single Client connects directly to one provider. It's the simplest approach and works great when you don't need failover.

import (
    "github.com/truvaagents/truva-g3/ai"
    _ "github.com/truvaagents/truva-g3/ai/providers/openai"
)

// The simplest possible setup - auto-detects from environment
client, err := ai.NewClient()

// Or explicitly choose a provider
client, err := ai.NewClient(
    ai.WithProviderAlias("openai.groq"),
    ai.WithModel("smart"),
)

Behind the scenes, when you call ai.NewClient() without arguments:

  1. The module checks registered providers in priority order
  2. Each provider's DetectEnvironment() method checks for API keys
  3. The first available provider wins
  4. You get a configured client without writing any configuration

When Single Client makes sense:

  • Development and testing
  • Simple applications where downtime is acceptable
  • When you're locked into one provider (e.g., enterprise agreement)
  • Background jobs where latency isn't critical

Chain Client: Production-Grade Resilience​

A Chain Client tries multiple providers in order until one succeeds. It's the production-ready approach for systems that can't afford downtime.

import (
    "github.com/truvaagents/truva-g3/ai"
    _ "github.com/truvaagents/truva-g3/ai/providers/openai"
    _ "github.com/truvaagents/truva-g3/ai/providers/anthropic"
)

// Option 1: Explicit chain — you control the order
client, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic", "openai.groq"),
)

// Option 2: Auto-detect — discovers providers from environment variables
// Builds chain in priority order from whatever API keys are available
client, err := ai.NewChainClient(
    ai.WithChainLogger(logger),
)
// With OPENAI_API_KEY + ANTHROPIC_API_KEY + OPENROUTER_API_KEY +
// GROQ_API_KEY set:
// → chain = ["openai" (1000), "anthropic" (900),
//            "openai.openrouter" (850), "openai.groq" (700)]

// Use it exactly like a single client
response, err := client.GenerateResponse(ctx, "Analyze this data...", nil)

Behind the scenes, when you make a request:

  1. Chain Client tries Provider 1 (OpenAI)
  2. If it fails with a retryable error, it tries Provider 2 (Anthropic)
  3. If that fails too, it tries Provider 3 (OpenRouter)
  4. It then tries Provider 4 (Groq)
  5. Returns the first successful response
  6. If all fail, returns one joined error containing every annotated entry failure (entry name and attempt number)

Auto-detect provider priorities (when no explicit chain is specified):

ProviderAliasPriority
OpenAIopenai1000
Anthropicanthropic900
OpenRouteropenai.openrouter850
Geminigemini800
Groqopenai.groq700
DeepSeekopenai.deepseek600
xAIopenai.xai500
Mistralopenai.mistral450
Qwenopenai.qwen400
Togetheropenai.together300
Bedrockbedrock200
Ollamaopenai.ollama100

Bedrock participates only in binaries built with -tags bedrock. Ollama auto-detection requires an explicitly configured loopback OLLAMA_BASE_URL whose /models endpoint responds; use an explicit alias for a remote Ollama endpoint.

The key insight: Each provider in the chain resolves model aliases independently. When you pass Model: "smart", OpenAI resolves it to gpt-5.6-sol, Anthropic resolves it to claude-opus-5-5, and Groq resolves it to openai/gpt-oss-120b. You don't need different code for different providers.

When to Use Which​

SituationRecommendedWhy
Local developmentSingle ClientSimpler, faster iteration
Staging/testingChain Client (auto-detect)Mirrors production env vars, tests failover
Production APIChain ClientHigh availability is essential
Production (deterministic order)Chain Client (explicit)Locks failover order regardless of env changes
Production (ops-driven)Chain Client (auto-detect)Ops controls chain via which API keys are deployed
Background processingEitherDepends on retry strategy
Cost-sensitive batch jobsChain Client (explicit)Try cheap providers first in specific order
Compliance-restrictedSingle ClientMay not be allowed to send data to multiple providers

The golden rule: If you'd lose money or users when AI is down, use Chain Client.

Provider Aliases: The Clean Way to Configure​

What Problem Do They Solve?​

Before provider aliases, configuring an OpenAI-compatible service looked like this:

// The old, messy way
client, _ := ai.NewClient(
    ai.WithProvider("openai"),
    ai.WithBaseURL("https://api.groq.com/openai/v1"),  // Have to remember this
    ai.WithAPIKey(os.Getenv("GROQ_API_KEY")),          // Different env var
    ai.WithModel("openai/gpt-oss-120b"),               // Provider-specific model
)

Every new provider meant remembering URLs, env vars, and model names. And if Groq changed their URL? You'd have to update every project.

With provider aliases, it's one line:

// The clean way
client, _ := ai.NewClient(ai.WithProviderAlias("openai.groq"))

The framework knows that openai.groq means:

  • Use GROQ_API_KEY for authentication
  • Connect to https://api.groq.com/openai/v1
  • Use Groq's model naming conventions

Complete Provider Alias Reference​

AliasTypeServiceCredential configurationBase URL / route configurationDefault URL
openaiNativeOpenAIOPENAI_API_KEYOPENAI_BASE_URLhttps://api.openai.com/v1
anthropicNativeAnthropic ClaudeANTHROPIC_API_KEYANTHROPIC_BASE_URLhttps://api.anthropic.com/v1
geminiNative request-aware GenerateContentGoogle GeminiGOOGLE_API_KEY then GEMINI_API_KEYGEMINI_BASE_URL or EndpointResolverhttps://generativelanguage.googleapis.com/v1beta
azureopenai.v1Hosted request-aware profileAzure OpenAI v1CredentialSource, WithAuthHeader, or Azure WithAPIKeyRequired EndpointResolverRoute-owned
azureopenai.classicHosted request-aware profileAzure OpenAI classicCredentialSource, WithAuthHeader, or Azure WithAPIKeyRequired EndpointResolverRoute-owned
anthropic.vertexHosted request-aware profileClaude on Vertex AIGoogle CredentialSource or WithAuthHeaderRequired EndpointResolverRoute-owned
bedrockNative SDKAWS BedrockAWS SDK default configuration or WithAWSCredentialsAWS region/configuration plus optional SDK-destination EndpointResolverRegion-owned
openai.groqOpenAI-compatibleGroqGROQ_API_KEYGROQ_BASE_URLhttps://api.groq.com/openai/v1
openai.openrouterOpenAI-compatibleOpenRouterOPENROUTER_API_KEYOPENROUTER_BASE_URLhttps://openrouter.ai/api/v1
openai.deepseekOpenAI-compatibleDeepSeekDEEPSEEK_API_KEYDEEPSEEK_BASE_URLhttps://api.deepseek.com
openai.xaiOpenAI-compatiblexAI GrokXAI_API_KEYXAI_BASE_URLhttps://api.x.ai/v1
openai.mistralOpenAI-compatibleMistralMISTRAL_API_KEYMISTRAL_BASE_URLhttps://api.mistral.ai/v1
openai.qwenOpenAI-compatibleAlibaba QwenQWEN_API_KEYQWEN_BASE_URLhttps://dashscope-intl.aliyuncs.com/compatible-mode/v1
openai.togetherOpenAI-compatibleTogether AITOGETHER_API_KEYTOGETHER_BASE_URLhttps://api.together.ai/v1
openai.ollamaOpenAI-compatibleOllama (local)(none)OLLAMA_BASE_URLhttp://localhost:11434/v1

Gemini credentials are sent in the x-goog-api-key header and never in the request URL. Use an AI Studio auth key: Google rejects unrestricted Standard keys and has announced that all Standard keys stop working in September 2026. See Google's Gemini API-key guide.

The Azure and Vertex aliases are request-aware-only, never auto-detected, and do not accept a static base URL. Their resolver and credential recipes are in CUSTOM_AI_PROVIDER_GUIDE.md. Bedrock is build-tagged and SDK-native: its optional resolver selects an AWS modelId, not an HTTP endpoint. See AWS Bedrock SDK-Native Routing.

OpenRouter configuration​

For auto-detection, a .env file needs only the key:

OPENROUTER_API_KEY=...
# Optional; omit to use https://openrouter.ai/api/v1
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1

OpenRouter has priority 850, immediately after Anthropic. If both OPENROUTER_API_KEY and TOGETHER_API_KEY are present, an auto-detected single client or chain tries OpenRouter first. Use an explicit provider chain when environment presence should not determine order.

Adding OPENROUTER_API_KEY can change an unpinned ai.NewClient() or ai.NewRequestClient() selection. When neither OpenAI nor Anthropic is available, OpenRouter is selected ahead of Gemini and every remaining provider. Use ai.WithProviderAlias("openai.openrouter") when provider identity must be stable.

An OpenRouter model selection may be either a framework alias or an exact OpenRouter model slug. The built-in mappings and their independent environment overrides are:

Framework selectionBuilt-in OpenRouter modelEnvironment override
default or an unspecified modelopenrouter/autoTRUVAG3_OPENROUTER_MODEL_DEFAULT
smartopenrouter/autoTRUVAG3_OPENROUTER_MODEL_SMART
fastopenai/gpt-6-lunaTRUVAG3_OPENROUTER_MODEL_FAST
codeopenrouter/pareto-codeTRUVAG3_OPENROUTER_MODEL_CODE

Set only the aliases you want to change. For example, this .env pins GPT as the application default, uses Kimi K3 for smart and code, and uses Gemini for fast:

OPENROUTER_API_KEY=...
TRUVAG3_OPENROUTER_MODEL_DEFAULT=openai/gpt-5.6-sol
TRUVAG3_OPENROUTER_MODEL_SMART=moonshotai/kimi-k3
TRUVAG3_OPENROUTER_MODEL_FAST=google/gemini-2.5-flash
TRUVAG3_OPENROUTER_MODEL_CODE=moonshotai/kimi-k3

Model mappings are non-secret configuration: the same variables may live in a local .env, a Kubernetes ConfigMap, or a Deployment env stanza. Keep only OPENROUTER_API_KEY in a Secret.

These variables alter OpenRouter's model aliases; they do not force provider selection when higher-priority provider keys are also present. Pin openai.openrouter in code when the provider itself must be deterministic.

To make one exact model the constructor default for every call that omits a model:

client, err := ai.NewClient(
    ai.WithProviderAlias("openai.openrouter"),
    ai.WithModel("moonshotai/kimi-k3"),
)

To select an alias in code while retaining its .env override and built-in fallback:

fastClient, err := ai.NewClient(
    ai.WithProviderAlias("openai.openrouter"),
    ai.WithModel("fast"), // _FAST override, otherwise openai/gpt-6-luna
)

To override the constructor default for one legacy-style call, supply an exact model or alias in AIOptions:

response, err := client.GenerateResponse(ctx, prompt, &core.AIOptions{
    Model: "google/gemini-2.5-flash",
})

The request-aware equivalent sets Generation.Model:

client, err := ai.NewRequestClient(
    ai.WithProviderAlias("openai.openrouter"),
    ai.WithModel("default"),
)
request := core.NewAIRequest(prompt, "plan_generation")
request.Generation.Model = "moonshotai/kimi-k3"
result, err := client.Generate(ctx, request)

Selection precedence is per-request model, then constructor WithModel, then the constructor's implicit default alias. After an alias is selected, its non-empty TRUVAG3_OPENROUTER_MODEL_{ALIAS} value wins over the built-in mapping; an exact model slug passes through unchanged. Model availability is provider-owned and can change, so confirm production slugs in OpenRouter's model browser. The concrete Kimi and Gemini examples above correspond to OpenRouter's current Kimi K3 and Gemini 2.5 Flash entries.

Every OpenRouter request protects provider.data_collection="deny" and provider.zdr=true; portable reasoning or JSON output also protects provider.require_parameters=true. A caller or request policy that conflicts with or weakens those values is rejected before HTTP. Request-policy paths use the documented canonical casing: variants such as /provider/ZDR or /Models are rejected rather than emitted beside canonical fields. OpenRouter's top-level request-body session_id controls upstream sticky routing; it is unrelated to framework session_id baggage used for application correlation.

The adapter uses reviewed capability rows, not live model discovery, to validate portable options. The current exact OpenRouter rows are:

Exact model IDJSON objectJSON schemaPortable reasoning
deepseek/deepseek-v4.1-flashYesNot declaredNot declared
z-ai/glm-5.3-flashYesNot declaredNot declared
openai/gpt-5.6-lunaYesYesOpenAI-style
openai/gpt-6-lunaYesYesOpenAI-style
openai/gpt-6.1-solYesYesOpenAI-style

An undeclared feature is not automatically granted to other model IDs or endpoints. Explicit unsupported portable options fail preparation; protected require_parameters filters upstream endpoint eligibility when used. The exact GPT-6 Luna and GPT-6.1 Sol routes each have a built-in temperature-removal rule: default requestpolicy.CompatibilityCompatible permits the adjustment (including explicit values) and reports it; requestpolicy.CompatibilityStrict requires application acknowledgement for an explicit value. This reuses the policy engine rather than changing caller prompts.

For exact openai/gpt-6.1-sol, choose low, medium, high, xhigh or max, or omit effort through the presence-aware API to use the model default. none is rejected, including an inherited chain default. Its routed final validator also rejects unsupported sampling/logprob controls and noncanonical parameter names after policy. Shared codec validation still runs before policy, so explicit sampling with active effort can fail before the removal rule; use explicit temperature omission when constructing a presence-aware request. The routed contract is separate from native OpenAI's tool restrictions. OpenRouter owns upstream tool/API translation; this support does not add a native Responses adapter. See the AI module's GPT-6.1 Sol setup and verification scope.

For this routed Sol contract, reasoning.enabled:false is also rejected. enabled and exclude require booleans; exclude:true hides reasoning output without disabling reasoning. A standalone positive-integer reasoning.max_tokens is allowed for OpenRouter's budget-to-effort mapping, but combining it with effort is rejected after policy. This includes inherited effort: explicitly omit effort or remove the conflicting field through a request patch when using the budget form. Values are validated without silently changing caller data.

For models without a declared OpenAI-style reasoning capability, effort inherited only from a client/chain default is omitted with a recorded adjustment and Debug diagnostic; explicit unsupported requests retain their error/warning contract. The capability table is an adapter contract, not a guarantee of current endpoint availability or live certification.

The built-in default, smart, and code aliases resolve to routers, so requests using them are not AI-output-cache eligible. fast resolves to the concrete openai/gpt-6-luna model and can be cache-eligible when no native models fallback list is present. In a failover chain, one unstable OpenRouter entry makes the entire ordered chain cache-ineligible.

The framework does not advertise a built-in free alias because the protected privacy live probe failed for the tested free routes. Exact :free model IDs and TRUVAG3_OPENROUTER_MODEL_FREE remain experimental pass-through options; verify that the chosen route works with the protected privacy values before depending on it, and consult OpenRouter's current free-model and account limits rather than assuming a fixed quota. When OpenRouter's native top-level models field is present, it must be an array on every route. A free primary route additionally requires every fallback to be an exact :free route; malformed or paid fallbacks are rejected before transport.

How Auto-Configuration Works​

When you use a provider alias, the framework resolves configuration in this order:

  1. Explicit code configuration (highest priority)

    ai.WithAPIKey("sk-explicit-key")  // This wins

  2. Provider-specific environment variables

    GROQ_API_KEY=gsk-from-env  # Used if no explicit key

  3. Base URL overrides (for a provider-documented endpoint or an internal proxy)

    GROQ_BASE_URL=https://ai-proxy.company.internal/groq/openai/v1

  4. Default values (built into the framework)

    • Default URL for the provider
    • Default model aliases

Practical example: Your code uses openai.groq. In production, you set:

GROQ_API_KEY=gsk-prod-key
GROQ_BASE_URL=https://ai-proxy.company.internal/groq  # Route through internal proxy

No code changes needed. The proxy gets all Groq traffic for logging/monitoring.

Model Aliases: Portable Model Names​

The Model Name Problem​

Every AI provider has different model names:

  • OpenAI: gpt-5.6-terra, gpt-5.6-luna, gpt-5.6-sol; native GPT-6 Astra/Sol/Luna and GPT-6.1 Sol are also supported explicitly
  • Anthropic: claude-sonnet-5, claude-opus-5-5, claude-haiku-5-5
  • Groq: openai/gpt-oss-120b, llama-3.3-70b-versatile, llama-3.1-8b-instant

If you hardcode model names, switching providers means changing code everywhere. And when providers release new models? More code changes.

OpenAI's general default alias now selects GPT-6.1 Sol. Its fast alias selects GPT-6 Luna, as does OpenRouter's fast alias (with the openai/ prefix). Other alias mappings are unchanged. See the AI module's family compatibility and override guide for effort/sampling restrictions, explicit model selection and verification scope. Native OpenAI and OpenRouter are separate capability routes.

Anthropic's cost-focused refresh selects Haiku 5.5 for fast and Opus 5.5 for smart/code/vision, preserving Sonnet 5 for default and Fable 5 for premium. These new selections use adaptive thinking; budget and native-extra migration requirements are documented in the Anthropic alias refresh. Existing environment overrides still win; Vertex publisher mappings remain application-owned. This does not change OpenRouter or Bedrock model selection.

The native OpenAI mapping is built in; the first assignment below is optional. OpenRouter still needs an explicit override to select Sol rather than its router:

# Native OpenAI (provider alias: openai; already the built-in default)
TRUVAG3_OPENAI_MODEL_DEFAULT=gpt-6.1-sol
# OpenRouter (provider alias: openai.openrouter)
TRUVAG3_OPENROUTER_MODEL_DEFAULT=openai/gpt-6.1-sol

Alternatively, supply the corresponding concrete ID with ai.WithModel or a per-request model. Native hyphen-suffixed Sol variants/snapshots are recognized; the OpenRouter row applies only to its exact reviewed ID. Alias selection does not clear an inherited reasoning effort: replace an incompatible none with a supported effort, or explicitly omit it through AIRequest.Generation. Native reasoning budgets retain the configured multiplier; OpenRouter budgets remain exact. No automatic model-output-limit clamp is applied. This default is shared by any call that inherits it, including planning and synthesis. Existing .env pins remain in force; set TRUVAG3_OPENAI_MODEL_DEFAULT=gpt-5.6-terra to retain the previous model. Azure/shared-catalog resolvers also receive gpt-6.1-sol for default; update their semantic-model-to-deployment mapping or retain an explicit model pin. Suffix matching includes names such as gpt-6.1-sol-pro; this selects family wire rules, not verified support for that native model ID. OpenRouter's separate openai/gpt-6.1-sol-pro listing is not covered by the exact base-model row.

Model aliases solve this by providing portable names:

// This code works across providers that define this portable alias.
client, _ := ai.NewClient(
    ai.WithProviderAlias("openai"),  // or "anthropic", or "openai.groq"
    ai.WithModel("smart"),           // Resolves to the right model for each provider
)

Standard Model Aliases​

AliasPurposeOpenAIAnthropicOpenRouterGeminiGroqDeepSeekTogether
defaultGeneral catalog choicegpt-6.1-solclaude-sonnet-5openrouter/autogemini-2.5-flashopenai/gpt-oss-120bdeepseek-chatdeepseek-ai/DeepSeek-V4-Flash-0731
fastLatency-oriented catalog choicegpt-6-lunaclaude-haiku-5-5openai/gpt-6-lunagemini-3.5-flash-litellama-3.1-8b-instantdeepseek-chatgoogle/gemma-4-31B-it
smartReasoning-oriented catalog choicegpt-5.6-solclaude-opus-5-5openrouter/autogemini-3.1-pro-previewopenai/gpt-oss-120bdeepseek-reasonermoonshotai/Kimi-K3
premiumHighest-tier catalog choicegpt-5.6-solclaude-fable-5(N/A)gemini-3.1-pro-preview(N/A)(N/A)(N/A)
codeCode-oriented catalog choicegpt-5.6-solclaude-opus-5-5openrouter/pareto-codegemini-3.1-pro-previewopenai/gpt-oss-120bdeepseek-chatmoonshotai/Kimi-K3
visionImage-capable catalog choicegpt-4.1claude-opus-5-5(N/A)gemini-2.5-flash(N/A)(N/A)(N/A)

Note: The premium alias is available for OpenAI, Anthropic, and Gemini. Other built-in catalogs use smart for their reasoning-oriented choice.

The table reports the aliases compiled into this branch, not a guarantee that every model is enabled in every account or region. Provider lifecycle and availability can change independently; verify the current provider docs and override an alias with TRUVAG3_<PROVIDER>_MODEL_<ALIAS> when needed.

Provider lifecycle alert (verified July 22, 2026): This branch still maps the DeepSeek aliases to deepseek-chat and deepseek-reasoner. DeepSeek will retire both model names after July 24, 2026 at 15:59 UTC and replaces them with deepseek-v4-flash and deepseek-v4-pro. Do not treat a model override as a complete migration: the framework capability catalog does not yet model V4's thinking object or its reasoning-effort contract. Update and unit-test the model catalog, capability rows, and request translation before using the V4 reasoning controls in production. See DeepSeek's V4 announcement and thinking-mode contract.

The Gemini premium catalog entry names gemini-3.1-pro-preview directly. Google shut down its predecessor, gemini-3-pro-preview, on March 9, 2026; the catalog does not rely on Google's redirect from that retired ID. See the official Gemini release notes. Live GenerateContent validation on August 17, 2026 found that Google rejects gemini-2.5-flash-lite and gemini-2.5-pro for new users and names gemini-3.5-flash-lite and gemini-3.1-pro-preview as their replacements. The portable fast, smart, and code aliases use those current targets; exact Gemini 2.5 IDs remain available to grandfathered projects where Google still permits them. The provider's dated GenerateContent capability snapshot also covers current in-scope models independently of the portable alias map. A model is removed only when Google's same-surface schedule gives an authoritative shutdown less than 45 calendar days away; refresh the snapshot and tests together.

Portable aliases are catalog-backed, not universal. The built-in OpenAI, Anthropic, Gemini, and registered OpenAI-compatible aliases above resolve them; Bedrock and an arbitrary custom provider do not unless that provider implements equivalent resolution. In a failover chain, use an alias only when every entry can resolve it.

Environment Variable Overrides​

Here's where it gets powerful. You can override any alias at runtime without changing code:

# Pattern: TRUVAG3_{PROVIDER}_MODEL_{ALIAS}=actual-model-name

# Override OpenAI's "smart" alias
export TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-terra

# Override Anthropic's "fast" alias
export TRUVAG3_ANTHROPIC_MODEL_FAST=claude-haiku-5-5

# For OpenAI-compatible providers, strip the "openai." prefix
export TRUVAG3_GROQ_MODEL_DEFAULT=llama-3.1-8b-instant

# Use one concrete OpenRouter model as the default instead of its auto router
export TRUVAG3_OPENROUTER_MODEL_DEFAULT=openai/gpt-5.6-sol

# Do not override DeepSeek to V4 until the framework's V4 capability and
# thinking-mode translation has been updated and tested; see the alert above.

Why this matters for ops:

  • Cost control: Set TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-luna in dev to use the efficient tier
  • A/B testing: Route traffic to different models without code changes
  • Rollback: If a new model has issues, switch back via env var

How Model Resolution Works​

When you call client.GenerateResponse(ctx, prompt, &core.AIOptions{Model: "smart"}), here's the resolution order:

  1. Environment variable (highest priority)

    TRUVAG3_OPENAI_MODEL_SMART=gpt-4.1  # If set, use this

  2. Hardcoded alias mapping

    modelAliases["openai"]["smart"] = "gpt-5.6-sol"  // Built-in default

  3. Pass-through (lowest priority)

    // If "smart" isn't recognized, use it literally
    // This lets you use explicit model names when needed

Example flow:

Request: Model="smart", Provider="openai"
  ↓
Check: TRUVAG3_OPENAI_MODEL_SMART env var?
  → Not set
  ↓
Check: modelAliases["openai"]["smart"]?
  → Returns "o3"
  ↓
Result: Use model "o3"

Understanding Failover Behavior​

How Chain Client Decides to Failover​

Not all errors should trigger failover. If your request is malformed, trying another provider won't help—you'll just get the same error three times.

Chain Client classifies completed provider failures using the core.ProviderError interface, which carries structured metadata (HTTP status code, provider name, model, transient flag, and provider-retryable flag) from the AI provider layer. Provider-owned route-resolution or invocation-viability errors may additionally implement ai.AIRequestFailureReasoner and return the typed ai.AIRequestFailureReasonRoute constant. This replaces fragile string matching with type-safe classification via errors.As().

Errors that ALLOW failover (tries next provider):

Error Typecore.ProviderError ClassificationWhy Failover Makes Sense
Authentication (401)StatusCode() == 401Different providers have different keys
Authorization (403)StatusCode() == 403Permission scopes differ per provider
Server errors (5xx)StatusCode() >= 500Provider-specific outage
Rate limits (429)StatusCode() == 429Limits are per-provider
Network errorsNon-ProviderError without cancellation/deadline identityMight be routing/DNS issue
Transient proxy errorsIsTransient() == trueProxy/CDN issue (e.g., Cloudflare HTML 400), not a request problem
Provider-specific terminal errorsIsRetryable() == trueBilling, hard quota, or account state may differ on another provider
Provider-classified HTTP timeoutIsTransient() or IsRetryable(), or an eligible 5xx statusA different provider may remain healthy; eligible OpenRouter HTTP/in-body 408 is classified as a retryable timeout
Provider route unavailableAIRequestFailureReason() == ai.AIRequestFailureReasonRouteAnother independently routed entry may remain viable

Errors that STOP failover (fails immediately):

Error Typecore.ProviderError ClassificationWhy Failover Won't Help
Bad request (400)StatusCode() == 400 && !IsTransient() && !IsRetryable()Same input fails everywhere
Content policyStatusCode() == 400 with policy messageSame content fails everywhere
Malformed inputStatusCode() == 400 with parse errorStructural issue in your code
Returned cancellation or deadline identityerrors.Is(err, context.Canceled) or context.DeadlineExceededStops the chain, including a provider-call deadline even while the outer caller remains active

A provider-returned timeout response with an active attempt context differs from an expired call context. The former can permit bounded recovery; returned context cancellation/deadline identity stops recovery. Buffered OpenAI-wire decoding preserves that identity through the error chain.

Transient proxy errors: When a request passes through a CDN or reverse proxy (e.g., Cloudflare) and the proxy itself returns an error (typically an HTML page with a 400 status), the framework marks this as IsTransient() == true. These are infrastructure issues, not request problems, so failover to the next provider is appropriate. The core.ProviderError interface distinguishes these from genuine API 400 errors.

Why Authentication Errors Allow Failover​

This is a design decision that confuses some people. Traditionally, a 401 error means "your credentials are wrong, stop trying." But in a multi-provider chain, each provider has its own API key.

Consider this scenario:

Chain: ["openai", "anthropic", "openai.groq"]

Environment:
  OPENAI_API_KEY=sk-expired-key     # Oops, forgot to rotate
  ANTHROPIC_API_KEY=sk-ant-valid    # Works fine
  GROQ_API_KEY=gsk-valid            # Works fine

With traditional error handling:

Request → OpenAI → 401 "invalid key" → ERROR (stop)
User gets an error even though two providers would work

With Chain Client:

Request → OpenAI → 401 "invalid key" → Try next
        → Anthropic → Success!
User gets their response, ops gets alerted about OpenAI key

The tradeoff: You might make extra API calls before finding a working provider. But in production, uptime usually matters more than a few extra milliseconds.

The Options Isolation Problem (And How We Solved It)​

This is a subtle bug that caused real production issues before we fixed it. Here's what happened:

The problem: When Provider 1 fails, Provider 2 receives Provider 1's resolved model name.

Step 1: Request with Model="smart"
        Chain Client tries OpenAI
        OpenAI resolves "smart" → "gpt-5.6-sol"
        OpenAI fails with 401

Step 2: Chain Client tries Anthropic
        Options still has Model="gpt-5.6-sol" (from OpenAI!)
        Anthropic doesn't know "gpt-5.6-sol"
        Anthropic uses default model instead of resolving "smart"

The fix: Chain Client now clones options for each provider and resets the model to the original value:

// Inside Chain Client (simplified)
originalModel := options.Model  // Save "smart"

for _, provider := range providers {
    providerOpts := cloneOptions(options)
    providerOpts.Model = originalModel  // Reset to "smart"

    response, err := provider.GenerateResponse(ctx, prompt, providerOpts)
    // Now each provider resolves "smart" independently
}

What this means for you: Model aliases work correctly during failover. You don't need to do anything special.

How Errors Propagate Through the Stack​

Understanding the error propagation path is critical for ensuring ChainClient failover works correctly. Here's the full lifecycle of an AI provider error:

1. Provider HTTP call fails (e.g., OpenAI returns 429)
   ↓
2. ai/providers/base.go HandleError() wraps it as *providerError
   (implements core.ProviderError: StatusCode=429, Provider="openai", Model="o3")
   ↓
3. ChainClient.isClientError() inspects via errors.As(err, &pe)
   → 429 is excluded from client errors → FAILOVER to next provider
   ↓
4. If ALL providers fail, ChainClient annotates every failure with its entry
   name and attempt, joins them with errors.Join(), and wraps the joined error
   with fmt.Errorf("...: %w", joined)
   ↓
5. Agent handler receives the error from orch.ProcessRequest()
   → errors.As(err, &pe) can find a matching ProviderError through the joined
     wrapping tree (do not assume that match is the final attempt)

The key rule: Every layer in the stack uses %w verb when wrapping errors:

// CORRECT — preserves core.ProviderError for upstream callers
return fmt.Errorf("orchestration failed: %w", err)

// WRONG — destroys the ProviderError type, ChainClient can't classify it
return fmt.Errorf("orchestration failed: %s", err)     // String conversion, type lost
return errors.New(err.Error())                          // New error, type lost
return fmt.Errorf("orchestration failed: %v", err)      // Value format, type lost

If any layer breaks the %w chain:

  • ChainClient can't classify the error → defaults to failover (safe but wasteful for true client errors)
  • Agent handler can't extract the status code → falls back to 500 (misleading to callers)
  • LLM debug store can't extract provider/model → error records have empty metadata

What this means for agent developers: When you wrap errors from orchestrator or AI client calls, always use %w. The framework's own code already follows this rule — you just need to maintain it in your handler code.

See also: AGENT_DEVELOPMENT_GUIDE.md for the complete handler pattern showing core.ProviderError extraction.

Operational Scenarios​

This section covers real-world deployment scenarios. Find the one that matches your situation.

Scenario 1: Local Development with Ollama​

Goal: Develop without cloud API costs, test offline

Setup on your laptop:

# Install and start Ollama
ollama serve

# Pull a model
ollama pull llama3.2

# Your .env file
OLLAMA_BASE_URL=http://localhost:11434/v1
TRUVAG3_OLLAMA_MODEL_DEFAULT=llama3.2
TRUVAG3_OLLAMA_MODEL_SMART=llama3.2

Code:

// Single client is fine for local dev
client, err := ai.NewClient(ai.WithProviderAlias("openai.ollama"))

Setup from Kind / OrbStack on your laptop:

# Pods need to reach the Ollama server running on your host
OLLAMA_BASE_URL=http://host.docker.internal:11434/v1
TRUVAG3_OLLAMA_MODEL_DEFAULT=llama3.2

If you want Ollama to win over cloud providers:

client, err := ai.NewChainClient(
    ai.WithProviderChain("openai.ollama", "openai"),
)

If you rely on auto-detect and also set OPENAI_API_KEY, ANTHROPIC_API_KEY, or other higher-priority providers, Ollama will usually be used only as a fallback.

Useful notes:

  • openai.ollama does not require an API key.
  • Use the exact model tag installed in your Ollama instance. If you use a different tag, set the alias override to that exact value.
  • openai.ollama can now accept reasoning controls through TruvaG3, but support is still model-dependent.

Pro tip: Create a make dev target that starts Ollama:

dev:
    ollama serve &
    OLLAMA_BASE_URL=http://localhost:11434/v1 go run .

Scenario 2: Development with Cloud Providers​

Goal: Use cloud AI for development, but minimize costs

Setup:

# .env.development
GROQ_API_KEY=gsk-dev-key

# Use smaller/cheaper models in dev
TRUVAG3_GROQ_MODEL_DEFAULT=llama-3.1-8b-instant
TRUVAG3_GROQ_MODEL_SMART=llama-3.1-8b-instant

Code:

// Use the provider selected for your development latency and cost target.
client, err := ai.NewClient(ai.WithProviderAlias("openai.groq"))

Groq exposes an OpenAI-compatible API and is often selected for low-latency development. Quotas and prices are provider- and account-specific and change independently of TruvaG3; confirm the current terms before choosing it for cost.

Scenario 3: Staging Environment​

Goal: Mirror production setup, test failover, validate before deployment

Setup:

# .env.staging
# Use same providers as production
OPENAI_API_KEY=sk-staging-key
ANTHROPIC_API_KEY=sk-ant-staging-key
GROQ_API_KEY=gsk-staging-key

# But use mid-tier models to save costs
TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-terra
TRUVAG3_ANTHROPIC_MODEL_SMART=claude-haiku-5-5

Code:

// Same chain as production
client, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic", "openai.groq"),
)

Testing failover in staging:

# Temporarily break OpenAI to test failover
export OPENAI_API_KEY=sk-invalid-key

# Make a request - should succeed via Anthropic
curl -X POST http://staging:8080/api/analyze

# Check context-aware framework/ai logs for:
# operation=ai_chain_provider_failed entry_name=openai error_type=credential
# operation=ai_chain_failover_success successful_entry=anthropic

Scenario 4: Production with High Availability​

Goal: Maximum uptime, automatic failover, production-approved models

Setup:

# .env.production
OPENAI_API_KEY=sk-prod-key
ANTHROPIC_API_KEY=sk-ant-prod-key
GROQ_API_KEY=gsk-prod-key

# Use the models approved for this production workload
TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-sol
TRUVAG3_ANTHROPIC_MODEL_SMART=claude-opus-5-5
TRUVAG3_GROQ_MODEL_SMART=openai/gpt-oss-120b

Code:

client, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic", "openai.groq"),
)

Monitoring production failover:

# Alert on sustained increases in the exported ai.chain.failover counter.
# Exported Prometheus names depend on your OpenTelemetry collector/exporter
# normalization. The framework metric has bounded module/status/reason labels;
# identify the affected entry from correlated logs or spans, not a metric label.

Scenario 5: Cost-Optimized Production​

Goal: Minimize AI costs while maintaining quality

Setup:

# .env.production-cost-optimized
# Supply credentials for providers ordered below by your verified current cost
GROQ_API_KEY=gsk-key
OPENAI_API_KEY=sk-key
ANTHROPIC_API_KEY=sk-ant-key

# Use smaller models where acceptable
TRUVAG3_GROQ_MODEL_DEFAULT=llama-3.1-8b-instant

Code:

// Example cost-oriented chain. Validate this order against current contracts,
// regions, quotas, and measured output quality before deployment.
client, err := ai.NewChainClient(
    ai.WithProviderChain("openai.groq", "openai", "anthropic"),
)

Cost monitoring:

// Log token usage per request for cost analysis
response, err := client.GenerateResponse(ctx, prompt, nil)
if err != nil {
    return fmt.Errorf("generate AI response: %w", err)
}
logger.Info("Request completed", map[string]interface{}{
    "model":  response.Model,           // Which model was used
    "tokens": response.Usage.TotalTokens,
})
// Chain entry tracking is available on ai.chain.provider_attempt spans and
// on the parent ai.chain.generate span's ai.chain.successful_entry attribute.

TruvaG3 records normalized token usage, not an estimated USD amount. For authoritative spend reporting, join provider/model/token telemetry with the provider's billing export and your negotiated pricing.

Scenario 6: Privacy-First Deployment​

Goal: Keep sensitive data local, use cloud only as fallback

Setup:

# .env.production-privacy
# Local model is primary
OLLAMA_BASE_URL=http://gpu-server.internal:11434/v1

# Cloud fallback for when local is overloaded
OPENAI_API_KEY=sk-key

# Use capable local model
TRUVAG3_OLLAMA_MODEL_DEFAULT=your-approved-model:tag
TRUVAG3_OLLAMA_MODEL_SMART=your-approved-model:tag

Code:

// Privacy-first: local → cloud
client, err := ai.NewChainClient(
    ai.WithProviderChain("openai.ollama", "openai"),
)

Privacy monitoring (for compliance):

// Track route use via sanitized framework/ai logs and traces.
// ai.chain.generate carries ai.chain.successful_entry; each
// ai.chain.provider_attempt span carries ai.chain.entry_name and status.
// Entry/provider names are intentionally not metric labels because they are
// operator-controlled and would create unbounded time series.

Scenario 7: Multi-Region Deployment​

Goal: Route to nearest provider, handle regional outages

Setup:

# .env.production-us
OPENAI_API_KEY=internal-us-token
OPENAI_BASE_URL=https://ai-gateway.us.company.internal/openai/v1
ANTHROPIC_API_KEY=sk-ant-backup

# .env.production-eu
OPENAI_API_KEY=internal-eu-token
OPENAI_BASE_URL=https://ai-gateway.eu.company.internal/openai/v1
ANTHROPIC_API_KEY=sk-ant-backup

Code (same for all regions):

client, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic"),
)

The key insight: Same code, different environment variables per region. Use only endpoints documented by the provider or owned by your organization; do not infer a regional vendor hostname. For Azure OpenAI, Google Cloud OpenAI compatibility, and Vertex-hosted Claude, use the validated resolver recipes in CUSTOM_AI_PROVIDER_GUIDE.md.

Kubernetes Deployment​

Managing API Keys with Secrets​

Never put API keys in ConfigMaps or environment variables directly. Use Kubernetes Secrets:

# secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: ai-api-keys
  namespace: production
type: Opaque
stringData:
  OPENAI_API_KEY: "sk-prod-..."
  ANTHROPIC_API_KEY: "sk-ant-prod-..."
  OPENROUTER_API_KEY: "..."
  GROQ_API_KEY: "gsk-prod-..."

Apply it:

kubectl apply -f secrets.yaml

Reference in deployment:

spec:
  containers:
  - name: app
    envFrom:
    - secretRef:
        name: ai-api-keys

The repository example setup scripts normally copy every configured provider credential from .env into the example's Kubernetes Secret, enabling ordered failover. For an isolated provider test, select one credential group for that setup invocation without editing .env:

TRUVAG3_SETUP_AI_PROVIDER=together ./setup.sh rollout

TRUVAG3_SETUP_AI_PROVIDER is consumed only by examples/k8-deployment/setup-env-lib.sh and is excluded from the runtime ConfigMap. Omit it on the next rollout to restore all configured providers.

Managing Model Aliases with ConfigMaps​

Model aliases aren't secrets—they can go in ConfigMaps:

# configmap-dev.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ai-model-config
  namespace: development
data:
  TRUVAG3_OPENAI_MODEL_SMART: "gpt-5.6-terra"
  TRUVAG3_ANTHROPIC_MODEL_SMART: "claude-haiku-5-5"
  TRUVAG3_OPENROUTER_MODEL_DEFAULT: "google/gemini-2.5-flash"
  TRUVAG3_OPENROUTER_MODEL_SMART: "google/gemini-2.5-flash"
  TRUVAG3_GROQ_MODEL_DEFAULT: "llama-3.1-8b-instant"

---
# configmap-prod.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ai-model-config
  namespace: production
data:
  TRUVAG3_OPENAI_MODEL_SMART: "gpt-5.6-sol"
  TRUVAG3_ANTHROPIC_MODEL_SMART: "claude-opus-5-5"
  TRUVAG3_OPENROUTER_MODEL_DEFAULT: "moonshotai/kimi-k3"
  TRUVAG3_OPENROUTER_MODEL_SMART: "moonshotai/kimi-k3"
  TRUVAG3_GROQ_MODEL_DEFAULT: "openai/gpt-oss-120b"

Same Image, Different Behavior​

The power of this setup: one container image works in all environments.

# deployment.yaml (same for dev, staging, prod)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-ai-service
spec:
  template:
    spec:
      containers:
      - name: app
        image: my-registry/my-ai-service:v1.2.3  # Same image everywhere
        envFrom:
        - secretRef:
            name: ai-api-keys      # Different per namespace
        - configMapRef:
            name: ai-model-config  # Different per namespace

Promotion workflow:

# Dev → Staging (same image, different config)
kubectl apply -f configmap-staging.yaml -n staging

# Staging → Prod (same image, different config)
kubectl apply -f configmap-prod.yaml -n production

Rolling Updates Without Downtime​

When you need to change models or API keys:

# Update the ConfigMap
kubectl edit configmap ai-model-config -n production
# Change TRUVAG3_OPENAI_MODEL_SMART from "o3" to "gpt-4.1"

# Trigger rolling restart
kubectl rollout restart deployment/my-ai-service -n production

# Watch the rollout
kubectl rollout status deployment/my-ai-service -n production

Pods restart one at a time, picking up the new environment variables. Zero downtime.

Troubleshooting Common Issues​

Issue 1: "No Providers Available" Error​

Symptom: Auto-detected NewChainClient returns "no providers detected (check API keys)", or an explicit chain cannot construct any entry.

Cause: Auto-detection found no configured provider. In an explicit chain, construction fails only when entries cannot be materialized (for example, an unregistered alias or invalid endpoint configuration). Most direct HTTP providers do not preflight a missing API key during construction; that error is returned when the entry is first called and can then fail over.

Diagnosis:

# Check which env vars are set
echo "OPENAI_API_KEY: ${OPENAI_API_KEY:-(not set)}"
echo "ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-(not set)}"
echo "GROQ_API_KEY: ${GROQ_API_KEY:-(not set)}"

Fix: For auto-detection, set at least one provider credential (or configure and start loopback Ollama):

export OPENAI_API_KEY=sk-your-key

In Kubernetes:

# Check if secret is mounted
kubectl exec -it pod/my-ai-service-xxx -- env | grep API_KEY

Issue 2: Wrong Model Being Used​

Symptom: Logs show gpt-5.6-luna but you expected gpt-5.6-sol

Cause: Environment variable override is taking precedence

Diagnosis:

# Check for overrides
env | grep TRUVAG3_

# Look for:
TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-luna  # This overrides the default!

Fix: Clear the override or set it to the model you want:

unset TRUVAG3_OPENAI_MODEL_SMART
# or
export TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-sol

Issue 3: Failover Not Working​

Symptom: Request fails immediately instead of trying next provider

Cause: Error is classified as "client error" (non-retryable)

Diagnosis: Check logs for error classification:

{"message": "Provider failed", "error": "bad request: invalid prompt", "is_client_error": true}

Understanding: is_client_error: true means the error is in your request, not the provider. Failover won't help because the same request will fail everywhere.

Fix: Fix the underlying request issue (malformed JSON, invalid parameters, etc.)

Issue 4: Model Not Found After Failover​

Symptom: First provider fails, second provider says "model not found"

Cause: You might be running an old version with the options mutation bug

Diagnosis: Check the logs:

{"provider": "anthropic", "model": "o3"}  // Anthropic shouldn't see "o3"!

Fix: Update to the latest TruvaG3 version. The options cloning fix was added in December 2025.

Issue 5: Unexpected Provider Being Used​

Symptom: You expected OpenAI but Groq handled the request

Cause: Auto-detection picked a different provider based on available env vars

Diagnosis:

# If OPENAI_API_KEY isn't set but GROQ_API_KEY is...
# Auto-detection will use Groq

echo "OPENAI_API_KEY: ${OPENAI_API_KEY:-(not set)}"
echo "GROQ_API_KEY: ${GROQ_API_KEY:-(not set)}"

Fix: Either set the expected API key, or use explicit provider alias:

// Force OpenAI specifically
client, err := ai.NewClient(ai.WithProviderAlias("openai"))

Issue 6: Ollama Is Configured but Cloud Providers Still Win​

Symptom: OLLAMA_BASE_URL is set, but requests still use OpenAI or Anthropic.

Cause: Auto-detect uses provider priority. Ollama is lower priority than cloud providers.

Fix: Use an explicit chain if you want local models first:

client, err := ai.NewChainClient(
    ai.WithProviderChain("openai.ollama", "openai", "anthropic"),
)

Issue 7: Concrete Model Name Breaks Chain Failover​

Symptom: The first provider fails and the second fails with 404 model not found.

Cause: A provider-specific concrete model name was used with ChainClient.

Fix: Use portable aliases like default, fast, or smart only when every entry in the failover chain defines that alias. Bedrock and arbitrary custom providers require an explicit strategy because they do not inherit the built-in OpenAI/Anthropic/Gemini catalogs.

Issue 8: Streaming Text Arrives but the Request Does Not Finish Immediately​

Symptom: The user sees streamed text, but the final done metadata takes longer.

Cause: Post-synthesis hooks such as user-memory extraction still run before the streaming request is considered complete.

What to know:

  • The streamed text is already on screen
  • The remaining delay is usually post-processing work, not the main synthesis call
  • Token usage reported at the end may include these late hook calls

Debugging and Observability​

Enabling Debug Logging​

For detailed provider resolution logs, set the debug environment variable:

# Enable debug logging for all TruvaG3 components
export TRUVAG3_DEBUG=true

# Or set log level directly
export TRUVAG3_LOG_LEVEL=debug

Alternatively, use a custom logger with your chain client:

import "github.com/truvaagents/truva-g3/core"

// Create a production logger with debug config
logger := core.NewProductionLogger(
    core.LoggingConfig{Level: "debug", Format: "json"},
    core.DevelopmentConfig{DebugLogging: true},
    "my-service",
)

// Pass to chain client
client, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic"),
    ai.WithChainLogger(logger),
)

Understanding AI Module Logs​

AI module logs use the component prefix framework/ai. Here's what to look for:

Successful provider response:

{
  "component": "framework/ai",
  "level": "INFO",
  "message": "AI response received",
  "operation": "ai_response",
  "provider": "openai",
  "provider_alias": "openai",
  "model": "o3",
  "prompt_tokens": 150,
  "completion_tokens": 200,
  "total_tokens": 350,
  "status": "success",
  "duration_ms": 1250
}

Failover event:

{
  "component": "framework/ai",
  "level": "WARN",
  "message": "Provider failed in chain, trying next",
  "operation": "ai_chain_provider_failed",
  "entry_name": "openai",
  "attempt": 1,
  "remaining": 1,
  "failover_reason": "auth",
  "error": "AI provider request failed: provider_client",
  "error_type": "provider_client"
}

All providers exhausted:

{
  "component": "framework/ai",
  "level": "ERROR",
  "message": "All chain providers exhausted",
  "operation": "ai_chain_exhausted",
  "entries_tried": 3,
  "failed_entries": ["openai", "anthropic", "openai.groq"],
  "failover_reason": "rate_limit",
  "error": "AI provider request failed: provider_rate_limit",
  "error_type": "provider_rate_limit"
}

These observation errors are deliberately sanitized; the original typed error is still returned to the caller. request_id is included when the supplied context carries it (framework handlers and orchestration normally seed it). For billing, hard-quota, and account-action alerts, match the bounded failover_reason=provider_retryable field across generate and stream failover and exhaustion operations. A provider may report insufficient_quota as HTTP 429; the provider-retryable classification intentionally takes precedence over the generic rate_limit failover reason in that case.

Tracing AI Requests in Jaeger​

If you have telemetry enabled, AI requests create spans:

  1. Open Jaeger: http://localhost:16686
  2. Select your service
  3. Find traces with ai.generate or ai.stream spans
  4. Expand to see:
    • ai.provider: Which provider handled the request
    • ai.model: Resolved model name
    • ai.prompt_tokens, ai.completion_tokens: Token usage
    • ai.request.prepared: Sanitized request-policy preparation event
    • ai.generate_response / ai.stream_response: Provider execution
    • ai.http_attempt: Each HTTP retry attempt

Advanced Configuration​

This section covers advanced configuration options for fine-tuning AI client behavior, particularly for reasoning models and long-running requests.

Portable Fields vs Provider-Specific Escape Hatches​

Most requests should use the portable core.AIOptions fields:

  • Model
  • Temperature
  • MaxTokens
  • SystemPrompt
  • ReasoningEffort
  • ResponseFormat

When a provider exposes a feature TruvaG3 does not model directly yet, use:

  • Extra map[string]interface{}
  • Headers map[string]string

Example:

resp, err := client.GenerateResponse(ctx, prompt, &core.AIOptions{
    Model:          "smart",
    ResponseFormat: "json",
    Extra: map[string]interface{}{
        "top_p": 0.9,
    },
    Headers: map[string]string{
        "anthropic-beta": "context-1m-2025-08-07",
    },
})

Precedence rules:

  • Framework-managed request fields win over everything else
  • Per-request Extra / Headers override client defaults
  • Protected headers such as auth and content type are not user-overridable
  • The same resolved custom headers are applied to sync and streaming requests

Use escape hatches sparingly. If a field is portable, prefer the typed option.

Request-Aware and Custom Integrations​

There are two separate decisions here:

  1. How much request intent must TruvaG3 preserve? The ordinary API carries portable values such as model, temperature, and max tokens. The request-aware API can also say "send zero," "omit this field," or "apply this named policy."
  2. Who owns the provider protocol? A built-in provider can still be used with enterprise credentials and routing. You need a custom provider only when TruvaG3 does not already own the correct provider identity, wire format, SDK integration, or validation rules.

Request-aware does not automatically mean custom:

Ordinary built-in call
Prompt + AIOptions
        └── built-in provider

Advanced built-in call
AIRequest + policy/routing/credentials
        └── the same built-in provider

New provider integration
AIRequest + policy/routing/credentials
        └── your provider factory + adapter

Most applications should stay with the first path. Move to the second or third only when the request or integration has behavior that the ordinary setup cannot preserve:

RequirementUse
Built-in provider, portable generation options, or homogeneous failoverNewClient or NewChainClient; continue using this guide
Explicit zero or omission, request policy, rotating credentials, or per-request routingNewRequestClient
Independently configured or caller-owned failover entriesNewChain
New provider identity, custom wire contract, or SDK-native adapterA custom request-aware provider factory

Here are three concrete examples:

  • You use OpenAI with a normal API key: stay with NewClient.
  • Your company routes OpenAI through regional gateways and issues a new token for every attempt: keep the built-in OpenAI provider and use NewRequestClient with routing and credential hooks.
  • Your company has an internal model platform with its own SDK and response format: implement a custom request-aware provider.

NewRequestClient accepts existing options such as WithProvider, WithModel, and WithTimeout because every AIOption also satisfies ClientOption:

client, err := ai.NewRequestClient(
    ai.WithProvider("openai"),
    ai.WithModel("smart"),
)

request := core.NewAIRequest("Extract the entities", "entity_extraction")
request.Generation.Temperature = core.SetAIParameter(float32(0))
request.Generation.TopP = core.OmitAIParameter[float32]()
request.Generation.ResponseFormat = core.SetAIParameter("json")

result, err := core.GenerateAI(ctx, client, request)

The zero value of AIParameter means inherit, SetAIParameter means send the value even when it is zero, and OmitAIParameter means the provider field must be absent. If a provider or a legacy fallback cannot preserve that intent, the call returns core.ErrAIRequestFeatureUnsupported; it never silently drops the field.

Request-aware built-in construction currently supports direct OpenAI, Anthropic, and Gemini, Azure OpenAI (azureopenai.v1 and azureopenai.classic), Google-hosted Claude (anthropic.vertex), and Bedrock when built with -tags bedrock. The Azure and Vertex profiles are request-aware-only and require endpoint and credential sources; they do not auto-detect. The legacy NewClient, GenerateResponse, and NewChainClient APIs remain supported for existing and simple portable calls.

The Custom AI Providers and Enterprise Integration Guide starts with the second example and shows when it becomes necessary to move to the third. It explains policy and middleware, enterprise credential and route hooks, heterogeneous chains, custom factories, reusable OpenAI-compatible codecs, retry-body requirements, and semantic cache fingerprints.

AWS Bedrock SDK-Native Routing​

Build with -tags bedrock and import the provider package so its factory is registered. With no model option, Bedrock uses the direct anthropic.claude-sonnet-5 model in the default us-east-1 region. Because AWS currently documents that bare model for direct in-region use only there, request preparation in another region rejects an invocation that would actually use the implicit default. Client construction remains valid when the deployment supplies a model on every request. Supply an explicit supported client model, per-request model/profile ID, or endpoint resolver:

import (
    "context"
    "fmt"

    "github.com/truvaagents/truva-g3/ai"
    "github.com/truvaagents/truva-g3/ai/providers/bedrock"
)

client, err := ai.NewClient(
    ai.WithProvider("bedrock"),
    ai.WithRegion("us-east-1"),
    ai.WithModel(bedrock.ModelClaudeSonnet5),
)

For direct package construction with bedrock.NewClient, call client.SetDefaultModel(modelID) before concurrent use. That method records that the default was an explicit application routing choice; assigning the embedded DefaultModel field alone does not.

TruvaG3 never silently changes that direct ID to us., global., or another inference profile. AWS documents in-region, geographic cross-region, and global cross-region routing as different choices. Select one explicitly when your region, throughput, residency, IAM/SCP, or pricing requirements call for it.

For request-aware routing, ResolvedEndpoint.Deployment is the exact AWS modelId supplied to Converse and RouteIdentity is a stable, non-secret cache identity. Resolver maps are keyed by the post-default semantic model ID:

type bedrockRoute struct {
    modelID  string
    identity string
}

type bedrockRoutes map[string]bedrockRoute

func (routes bedrockRoutes) ResolveEndpoint(
    _ context.Context,
    request ai.EndpointRequest,
) (ai.ResolvedEndpoint, error) {
    route, ok := routes[request.ResolvedModel]
    if !ok {
        return ai.ResolvedEndpoint{}, fmt.Errorf(
            "no Bedrock route for semantic model %q",
            request.ResolvedModel,
        )
    }
    return ai.ResolvedEndpoint{
        Deployment:    route.modelID,
        RouteIdentity: route.identity,
    }, nil
}

client, err := ai.NewRequestClient(
    ai.WithProvider("bedrock"),
    ai.WithRegion("us-east-1"),
    ai.WithModel(bedrock.ModelClaudeSonnet5),
    ai.WithEndpointResolver(bedrockRoutes{
        bedrock.ModelClaudeSonnet5: {
            modelID:  "us.anthropic.claude-sonnet-5",
            identity: "bedrock-us-sonnet-primary-v1",
        },
    }),
)

A Bedrock resolver must return no URL, query, or credential scope. AWS SDK configuration continues to own the region, credentials, SigV4, service endpoint, and HTTP transport. Accordingly, Bedrock rejects WithCredentialSource, WithHTTPClient, and request headers. The resolver's deployment must remain semantically equivalent to request.ResolvedModel: request policy uses the semantic model and never reclassifies an opaque deployment/profile ARN.

Current Claude compatibility is provider-owned and shared by sync and stream:

  • Sonnet 5 and Opus 4.7/4.8 omit modified temperature, top_p, and top_k.
  • Fable 5 omits incompatible inherited temperature and top-k, preserves temperature 1 or omission and top-p from 0.99 inclusive to 1 exclusive or omission, and rejects other values.
  • Unique legacy Fable temperature/top_p spellings remain in additionalModelRequestFields while policy runs. Application rules, middleware, or per-request patches may remove them, or set the canonical inferenceConfig field and remove the legacy copy. Case-insensitive duplicates and unremediated wrong-container fields fail locally.
  • Numeric values decoded with JSON UseNumber are accepted and range-checked in common inference fields and remain numeric recursively in model-specific additional fields. The full additional document is validated before its fingerprint becomes stable; empty or malformed numbers, structs, uintptr, non-string map keys, cycles, and non-finite floats fail locally. Named signed and unsigned Go numeric values are also accepted.
  • Other model families retain ordinary sampling behavior.
  • Current Mythos model cards expose the Messages surface rather than Converse, so Mythos is not included in this Converse policy table.

WithMaxRetries(n) means n retries after the first call; the Bedrock adapter passes n+1 total attempts to each AWS SDK operation. Explicitly selected standalone Bedrock clients and direct bedrock.NewClient values default to 60 minutes. Auto-detected clients and framework-managed chain entries default to 300 seconds. Positive WithTimeout and WithChainTimeout values override those defaults; zero and negative values mean unset rather than unbounded.

GetEmbeddings now defaults to 1024-dimensional Titan V2. An application with an existing 1536-dimensional Titan V1 store must pin bedrock.ModelTitanEmbedV1 through bedrock.WithEmbeddingModel. A per-call V1 pin omits V2-only dimensions/normalization inherited from client defaults, but explicit V2 controls on that same call are rejected. Keep the pin until the vector store is migrated or rebuilt. bedrock.WithoutEmbeddingNormalization() explicitly omits an inherited V2 normalization field for one call.

For the complete resolver and Titan V2 helper contracts, see the Bedrock recipe. Confirm changing catalog details against AWS's Claude Sonnet 5 model card, Claude Opus 4.7 model card, Claude Opus 4.8 model card, Converse API reference, and cross-region inference documentation.

Anthropic Sampling Compatibility​

Some Anthropic model families reject explicit sampling controls. The Anthropic adapter resolves the model alias or environment override first, then applies a model-family policy shared by sync and streaming requests. For the restricted families currently listed in ai/providers/anthropic/request_policy.go, it removes temperature, top_p, and top_k and records only fields that were actually present as adjustments. Matching uses an exact-or-hyphen boundary, so a similarly prefixed model name is not classified accidentally.

Known compatible model families retain sampling fields. An unknown model also preserves the legacy behavior instead of guessing that sampling is forbidden. This protects older and private models, but a newly released restricted family must be added to the built-in list. Until the framework is updated, a request-aware client can apply an explicit removal rule:

client, err := ai.NewRequestClient(
    ai.WithProvider("anthropic"),
    ai.WithRequestRules(core.AIProviderPatch{
        Name:    "new-claude-sampling-compatibility",
        Version: "1",
        Selector: core.AIProviderSelector{
            Provider: "anthropic",
            Surface:  "messages",
            Model:    "claude-next-*",
        },
        Remove: []string{"/temperature", "/top_p", "/top_k"},
    }),
)

Use this only after confirming the provider's model contract. Prefer upgrading to a framework release with the family in its built-in policy so every caller gets the same sync/stream behavior.

Request Timeouts​

Most clients constructed through TruvaG3's built-in provider factories default to a 300-second (5 minute) timeout. An explicitly selected standalone SDK-native Bedrock provider declares a 60-minute default for long-running inference; direct bedrock.NewClient does the same. Auto-detected clients and framework-managed failover entries keep 300 seconds so one provider cannot block selection or failover for an hour. Positive WithTimeout/WithChainTimeout wins over TRUVAG3_AI_TIMEOUT, then the applicable default. The environment value must be a positive Go duration; invalid present values fail construction. Application-defined clients and injected chain entries retain their own timeout behavior.

The environment is read by root AI constructors and managed chain entries, not direct provider constructors/factory calls or core.Config.LoadFromEnv. This is a total call deadline including retries/streaming, not an idle timeout. Earlier caller/phase/auxiliary-stage or injected HTTP-client deadlines still win. Output budgets are ceilings, not latency guarantees; set a longer bounded timeout explicitly when measurements justify it. Deadline errors do not cause automatic chain failover, even when the parent context is still live.

Single Client Timeout​

import "time"

// Auto-detection defaults to 300s, even if it selects Bedrock.
client, err := ai.NewClient()

// Explicit standalone Bedrock defaults to 60m.
bedrockClient, err := ai.NewClient(
    ai.WithProvider("bedrock"),
)

// Custom timeout for very complex tasks
client, err := ai.NewClient(
    ai.WithTimeout(300 * time.Second),  // 5 minutes
)

// Shorter timeout for latency-sensitive applications
client, err := ai.NewClient(
    ai.WithTimeout(60 * time.Second),   // 1 minute
)

Chain Client Timeout​

// Chain client with custom timeout applied to all providers
chainClient, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic"),
    ai.WithChainTimeout(240 * time.Second),  // 4 minutes
)

Without WithChainTimeout, every framework-managed chain entry uses 300 seconds, including Bedrock. ProviderEntry follows the same rule. A caller-owned ClientEntry is never mutated.

Chain clients default each entry to 0 in-provider retries because moving to the next entry is already the retry mechanism. NewClient defaults to 3 transport retries. Use ai.WithChainMaxRetries(n) only when you intentionally want each entry to retry before the chain advances.

If code does not supply an explicit retry option, a positive TRUVAG3_AI_RETRY_ATTEMPTS value overrides the relevant default for both client types. Explicit WithMaxRetries(n) or WithChainMaxRetries(n) wins over the environment. Zero, negative, and non-integer environment values are ignored; disable single-client retries explicitly with WithMaxRetries(0).

When to Adjust Timeout​

ScenarioRecommended Timeout
Simple queries with standard models (GPT-4, Claude)60s
Complex prompts with standard models120s
Reasoning models (GPT-5, GPT-6, o1, o3, o4)300s (default)
Orchestration/plan generation240-300s
Very complex multi-step reasoning300s+

Reasoning Model Support​

Reasoning controls are supported through ReasoningEffort, but support is provider- and model-dependent.

Reasoning Effort​

resp, err := client.GenerateResponse(ctx, prompt, &core.AIOptions{
    Model:           "gpt-5.4",
    ReasoningEffort: "none",   // or low, medium, high, xhigh
    MaxTokens:       2000,
})

Chain-wide default:

chainClient, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic"),
    ai.WithChainReasoningEffort("none"),
)

Notes:

  • Native openai supports ReasoningEffort directly.
  • openai.ollama now allows reasoning controls to pass through as well.
  • For unsupported OpenAI-compatible aliases, advanced reasoning fields may be stripped conservatively.
  • Anthropic does not expose this portable control. A request-aware ReasoningEffort set returns core.ErrAIRequestFeatureUnsupported instead of being silently dropped.
  • There is currently no env-var shortcut for ReasoningEffort; use code or orchestration overrides.

OpenAI reasoning profiles (GPT-5, GPT-6, o1, o3 and o4) require model-specific translation:

  1. Completion budget — The OpenAI reasoning wire profile uses max_completion_tokens.
  2. Sampling controls — Active/default reasoning restricts sampling; support with effort none depends on the model. Original GPT-6 Sol and GPT-6 Luna support none; GPT-6 Astra and GPT-6.1 Sol do not. Native GPT-6 validation and the exact OpenRouter Luna/Sol adaptation rules have separate contracts.
  3. Reasoning consumption — Internal reasoning can consume the completion budget. Available provider-reported reasoning is returned separately as ReasoningContent; it is not automatically substituted for final answer text.

The adapter selects the wire profile and validates supported controls. On a scaling profile, the token multiplier reserves additional budget when the effective effort is nonempty and not none, including an inherited client/chain effort. Reviewed exact native GPT-6 Astra/Sol/Luna/GPT-6.1 Sol and routed Luna/Sol also receive headroom with absent effort because their defaults reason; the adapter does not inject effort. Unknown defaults, explicit omission, none and native-extra-only reasoning do not multiply it. This is a budget allocation, not a guarantee of visible answer length or provider acceptance.

The shared openaiwire.IsReasoningModel helper now recognizes names exactly at the GPT-6 family boundary (gpt-6 or gpt-6-*) plus gpt-6.1-sol and its hyphen-suffixed variants/snapshots. Matching is case-insensitive; unrelated gpt-6.1-*, gpt-6.10-*, gpt-60 and vendor/gpt-6-sol do not match. Direct Azure adapters use it on the resolved semantic model. Azure v1 selects its reasoning completion/sampling profile; Azure classic rejects reasoning-model requests because it has no verified reasoning surface contract. Changing only the route's deployment name does not necessarily change the semantic model classification. This recognition does not certify that a deployment exists or supports a particular effort.

Reasoning Shares the Output Budget​

Reasoning and visible text consume the same completion allowance. For example, if reasoning consumes 25000 tokens from a 30000-token cap, at most 5000 remain for the answer. Neither portion is reserved; even a larger cap can be exhausted before visible output. The default multiplier is 3× on scaling profiles; explicit 1× keeps the configured budget exact. A multiplier adds headroom only on scaling profiles; it does not guarantee a complete response.

Configuring Token Multiplier​

// Default 3x multiplier on supported active-reasoning profiles
client, err := ai.NewClient()

// Explicit opt-in to additional headroom on scaling profiles
client, err := ai.NewClient(
    ai.WithReasoningTokenMultiplier(3),  // 3x multiplier
)

// Higher multiplier for very complex reasoning tasks
client, err := ai.NewClient(
    ai.WithReasoningTokenMultiplier(8),  // 8x multiplier
)

// Chain client with custom multiplier
chainClient, err := ai.NewChainClient(
    ai.WithProviderChain("openai", "anthropic"),
    ai.WithChainReasoningTokenMultiplier(4),  // 4x multiplier
)

When to Adjust Token Multiplier​

ScenarioRecommended Multiplier
Predictable configured output capExplicit 1x
General active-reasoning headroom3x (default)
Preserve the previous implicit scalingExplicit 5x; check the resulting model output limit
Truncated reasoning-heavy responsesRaise the exact output cap first; evaluate effort, latency and cost

Note: Scaling profiles apply the multiplier for active effort (including client/chain defaults) or the reviewed native/routed defaults listed above. Unknown defaults, explicit omission, none, native-extra-only controls and non-scaling profiles stay exact. Reviewed exact native GPT-5.6 Sol/GPT-6 Astra/Sol/Luna/GPT-6.1 Sol and routed GPT-5.6 Luna/GPT-6 Luna/GPT-6.1 Sol have a 128000 output capacity. The GPT-5.6 entries add capacity only. Headroom for a valid base is bounded in both modes (50000 × 3 → 128000); only over-cap base budgets require a built-in clamp in Compatible mode or application acknowledgement in Strict mode. Final capacity checks still cover patches. Other model/route limits remain the application's responsibility; metadata support is separate work. Configure the multiplier in code or with TRUVAG3_AI_REASONING_TOKEN_MULTIPLIER; invalid explicit settings fail construction.

Orchestration Model Overrides​

The orchestration module makes several types of internal LLM calls. Each can be routed to a different model for cost/latency optimization:

Call TypeConfig FieldFactory FunctionEnv VarDefault
Plan generationPlanAIOptions.ModelWithPlanAIOptions()TRUVAG3_PLAN_MODELAIClient default
Synthesis (streaming + non-streaming)SynthesisAIOptions.ModelWithSynthesisAIOptions()TRUVAG3_SYNTHESIS_MODELAIClient default
Micro-resolutionMicroResolutionAIOptions.ModelWithMicroResolutionAIOptions()TRUVAG3_MICRO_RESOLUTION_MODELAIClient default
Tiered selectionTieredSelectionAIOptions.ModelWithTieredSelectionAIOptions()(code only today)AIClient default
Error analysisErrorAnalysisAIOptions.ModelWithErrorAnalysisAIOptions()(code only today)AIClient default
Result distillationResultDistillAIOptions.ModelWithResultDistillAIOptions()TRUVAG3_RESULT_DISTILL_MODELfast alias

Important: Use portable aliases with ChainClient.

When your orchestrator uses a ChainClient (multi-provider failover), model override values must be portable aliases ("fast", "default", "smart"), not concrete model names.

Why: The ChainClient tries providers in order. If the model string is a concrete name like "gpt-4o-mini", a non-matching provider (e.g., Anthropic) returns 404. The ChainClient classifies 404 as a non-retryable client error and stops immediately — it does not try the next provider. This silently breaks failover.

Portable aliases work because each provider resolves them independently:

  • "fast" → Anthropic resolves to Haiku 5.5, OpenAI resolves to GPT-6 Luna
  • Failover works correctly since each provider gets a model it recognizes

Example — route each orchestration phase to an appropriate model tier:

# Set orchestration model overrides to portable aliases
TRUVAG3_PLAN_MODEL=smart
TRUVAG3_SYNTHESIS_MODEL=default
TRUVAG3_MICRO_RESOLUTION_MODEL=fast
TRUVAG3_RESULT_DISTILL_MODEL=fast

# Optionally control which concrete model each alias maps to per-provider
TRUVAG3_ANTHROPIC_MODEL_SMART=claude-opus-5-5
TRUVAG3_OPENAI_MODEL_SMART=gpt-5.6-sol
TRUVAG3_ANTHROPIC_MODEL_FAST=claude-haiku-5-5
TRUVAG3_OPENAI_MODEL_FAST=gpt-5.6-luna

Or programmatically:

orch, _ := orchestration.CreateOrchestratorWithOptions(deps,
    orchestration.WithPlanAIOptions(&orchestration.AIOptionsOverride{
        Model: orchestration.StringPtr("smart"),
    }),
    orchestration.WithSynthesisAIOptions(&orchestration.AIOptionsOverride{
        Model:           orchestration.StringPtr("default"),
        ReasoningEffort: orchestration.StringPtr("none"),
        Temperature:     orchestration.Float32Ptr(0.7),
    }),
    orchestration.WithMicroResolutionAIOptions(&orchestration.AIOptionsOverride{
        Model: orchestration.StringPtr("fast"),
    }),
    orchestration.WithTieredSelectionAIOptions(&orchestration.AIOptionsOverride{
        Model: orchestration.StringPtr("fast"),
    }),
)

Concrete model names are normally appropriate for a single-provider ai.NewClient(). In a chain they are safe only when every entry accepts that same literal model ID:

// Safe — single provider, no failover chain
client, _ := ai.NewClient(ai.WithProviderAlias("openai"))
orch, _ := orchestration.CreateOrchestratorWithOptions(deps,
    orchestration.WithPlanAIOptions(&orchestration.AIOptionsOverride{Model: orchestration.StringPtr("o3")}),
    orchestration.WithSynthesisAIOptions(&orchestration.AIOptionsOverride{Model: orchestration.StringPtr("gpt-4.1")}),
    orchestration.WithMicroResolutionAIOptions(&orchestration.AIOptionsOverride{Model: orchestration.StringPtr("gpt-4.1-mini")}),  // OK, no chain
)

Quick Reference​

Environment Variable Cheat Sheet​

API Keys:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=...          # wins over GEMINI_API_KEY when both are set
DEEPSEEK_API_KEY=sk-...
GROQ_API_KEY=gsk-...
XAI_API_KEY=xai-...
MISTRAL_API_KEY=...
QWEN_API_KEY=...
TOGETHER_API_KEY=...
OPENROUTER_API_KEY=...

Base URL Overrides:

OPENAI_BASE_URL=https://...
ANTHROPIC_BASE_URL=https://...
GEMINI_BASE_URL=https://...
DEEPSEEK_BASE_URL=https://...
GROQ_BASE_URL=https://...
XAI_BASE_URL=https://...
MISTRAL_BASE_URL=https://...
QWEN_BASE_URL=https://...
TOGETHER_BASE_URL=https://...
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
OLLAMA_BASE_URL=http://...

Model Alias Overrides:

# Pattern: TRUVAG3_{PROVIDER}_MODEL_{ALIAS}
TRUVAG3_OPENAI_MODEL_SMART=gpt-4.1
TRUVAG3_ANTHROPIC_MODEL_FAST=claude-haiku-5-5
TRUVAG3_GROQ_MODEL_DEFAULT=openai/gpt-oss-120b
TRUVAG3_OLLAMA_MODEL_DEFAULT=llama3.2
# For openai.deepseek, strip prefix → TRUVAG3_DEEPSEEK_MODEL_*

Decision Tree: Which Client Type?​

Do you need 99.9%+ uptime for AI features?
├── YES → Use Chain Client
│         └── Do you need deterministic provider order?
│             ├── YES → WithProviderChain("openai", "anthropic", ...)
│             └── NO  → Auto-detect (no WithProviderChain)
│                       └── Chain built from available API keys, sorted by priority
│
└── NO → Use Single Client
         └── Is cost a concern?
             ├── YES → Compare current provider pricing, quotas, and measured quality
             └── NO → Use your preferred provider

Error Classification Reference​

Error classification uses the core.ProviderError interface for structured, type-safe decisions. The Chain Client calls errors.As(err, &pe) to extract the provider error and inspects its fields:

core.ProviderError FieldFailover?Rationale
StatusCode() == 401YesDifferent keys per provider
StatusCode() == 403YesPermission scopes differ per provider
StatusCode() >= 500YesProvider outage
StatusCode() == 429YesPer-provider rate limits
IsTransient() == true (any status)YesProxy/CDN issue, not a request problem
IsRetryable() == true (any status)YesProvider-specific billing, quota or other classified condition may succeed elsewhere
Non-ProviderError without context cancellation/deadlineYesConnection/DNS issue
errors.Is(err, context.Canceled)NoReturned cancellation identity takes precedence over provider flags
errors.Is(err, context.DeadlineExceeded)NoReturned deadline identity, including the provider-call timeout, stops the chain
StatusCode() == 400 && !IsTransient() && !IsRetryable()NoSame input fails everywhere
Other 4xx, excluding 401/403/429, with both flags falseNoFix your request

Implementation (ai/chain_request.go and ai/chain_client.go, simplified):

func shouldFailOver(err error) bool {
    if err == nil || errors.Is(err, context.Canceled) ||
        errors.Is(err, context.DeadlineExceeded) {
        return false
    }
    return !isClientError(err)
}

func isClientError(err error) bool {
    var pe core.ProviderError
    if errors.As(err, &pe) {
        if pe.IsTransient() { return false }          // Proxy errors → failover
        if pe.IsRetryable() { return false }          // Provider-specific conditions → failover
        status := pe.StatusCode()
        return status >= 400 && status < 500 &&
            status != 401 && status != 403 && status != 429  // Auth/rate-limit → failover
    }
    return false // Unstructured/network error → failover
}

See Also​