Docs › LLM Gateway

LLM Gateway (OpenAI-compatible /v1)

Payment wires: every paid endpoint accepts x402 and MPP (Machine Payments Protocol) on the same 402 - see Paying with x402 and Paying with MPP. Agent402 is the applied layer of Agentic Finance: agents that pay and get paid on their own.

Acceptable use. Hosted-instance traffic is governed by the Terms of Service - including a generative-content acceptable-use policy - and by the upstream model providers' usage policies. Wallets used for prohibited content are blocked before settlement. Outputs are generated by third-party models from your inputs; you are responsible for how you use them.

Pay-per-call inference and embeddings at the OpenAI wire paths - any OpenAI SDK, agent framework, or plain HTTP client adopts the gateway by changing one setting:

from openai import OpenAI
client = OpenAI(base_url="https://agent402.tools/v1", api_key="unused")
# pay per call with USDC over x402 - no API key, no signup, no account

Ordering, because it decides who eats an upstream failure: the handler runs first, and @x402/express settles afterwards, and only for a response below 400. A 502 from a provider therefore cancels settlement and you are not charged; the failover chains below exist so a provider outage becomes a retry rather than your 502. A 200 is charged once settlement succeeds, and streaming starts only after that (see Streaming below). Upstream is OpenRouter for chat, images and speech, and OpenAI for embeddings; per-tier model allowlists and input/output caps keep worst-case upstream cost well below the flat x402 price. GET /v1/models (free) lists every model with its tier, price, and caps.

Because tiers are flat-priced while upstream bills per token, every request is also priced server-side before it goes upstream: input tokens are counted exactly (including tool schemas and images), and max_tokens is automatically tightened when an expensive model plus a large input would otherwise approach the tier price. Cheap and mid-priced models never hit this bound. A request whose input alone exceeds the budget returns a 400 explaining the fix (shrink the input, lower n, or pick a cheaper model). n is capped at 4.

Privacy routing: send zdr: true (or provider: {zdr: true}) on any chat tier to route only to providers with a zero-data-retention policy. Same price, same models where available; a model with no ZDR provider errors upstream and the failover chain walks to one that has it. zdr is the only provider preference a buyer can set - pricing bounds stay server-owned.

Tiers

Endpoint	Price	Serves	Input cap	Output cap
`POST /v1/nano/chat/completions`	$0.003	nano models (gpt-5.6-luna, gpt-4.1-nano, gemini flash-lite, small llama/ministral/qwen, deepseek-chat, laguna) - priced for high-frequency agent loops	12k chars	768 tokens
`POST /v1/auto/chat/completions`	$0.01	model optional - deterministic eval-ranked routing (see below)	16k chars	1,024 tokens
`POST /v1/chat/completions`	$0.02	budget/mid models (gpt-4o-mini, claude haiku, gemini flash, deepseek, llama, mistral, qwen)	32k chars	2,048 tokens
`POST /v1/pro/chat/completions`	$0.10	mid-frontier (gpt-4o, gpt-4.1, claude sonnet incl. sonnet-5, gemini pro and 3.x flash, grok)	48k chars	4,096 tokens
`POST /v1/premium/chat/completions`	$0.50	frontier (gpt-5, o3/o4, claude opus)	85k chars	8,192 tokens
`POST /v1/embeddings`	$0.002	text-embedding-3-small (default), 3-large, ada-002 - batch up to 64 inputs	16k chars	-
`POST /v1/images/generations`	$0.08	Gemini 2.5 Flash Image (nano banana) - one image per call, inline base64 out	4k-char prompt	1 image
`POST /v1/audio/speech`	$0.06	a five-model failover chain on OpenRouter's audio API; raw mp3/pcm bytes out, the 11 OpenAI voice names plus each model's native voices	2k-char input	-

Bare OpenAI-style names (gpt-4o-mini) are accepted and mapped; requesting a model on the wrong tier returns a self-correcting 400 naming the right endpoint and price. All tiers are wallet-only - every call burns real upstream credit, so there is no proof-of-work free tier (see Security Model). The pro and premium chat tiers also accept three upstream server tools under a server-owned bound (openrouter:web_search, openrouter:web_fetch, openrouter:datetime, each with a hard use cap that GET /v1/models lists per tier); other server tools are refused by name.

Metered tier - pay per request, quoted first

POST /v1/metered/chat/completions (v1-chat-metered) serves every explicit model the flat chat tiers serve (the same allowlists; GET /v1/models names it beside each chat model as meteredEndpoint, with meteredMaxInputChars 200k and meteredMaxTokens 8,192), but the price is not flat: the 402 quotes this exact request from its own body. The quote is the request's worst case (input counted exactly, plus your max_tokens, at the model's list price) times 1.15, never below the $0.001 settlement floor, rounded up to a micro-dollar; a request whose quote would exceed $2 is refused with a 400 naming the cap (a 4xx cancels settlement, so nothing is charged). Because the price is derived from the body actually served, a payment authorized for a small quote cannot ride a bigger request. Set max_tokens to what you need: it is what you pay for.

Two ways to pay it:

Exact clients (most stock x402 clients) pay the quote. A short call costs a fraction of a cent; a long one pays for what it asks.
upto clients (an x402 client with the upto scheme registered, which on Base needs a one-time USDC approval to Permit2) authorize the quote as a ceiling and settle actual usage x 1.15 under it. Prepaid-credits buyers get the same treatment automatically: the quote is held, and only actual usage x 1.15 is debited on a 200.

The catalog lists the tier at its $0.001 floor ("from"), and every chat model on GET /v1/models carries meteredFromUsd. Use the flat tiers when you want a known price per call regardless of length; use the metered tier when calls vary a lot in size and you want to pay for the size you send.

OpenClaw provider plugin (`agent402-openclaw`)

OpenClaw talks to any OpenAI-compatible provider through one block in openclaw.json, and the agent402-openclaw npm plugin writes that block for this gateway: AGENT402_CREDITS_KEY=a402_... npx agent402-openclaw setup --write stores a prepaid credits key (bought by card at /credits) and starts a loopback proxy that carries it, so OpenClaw itself never holds a payment credential. auto (routed per prompt, flat $0.01 per call) is offered beside every id on GET /v1/models; explicit models ride the metered route by default (--flat keeps them on their flat tiers), and setup picks the cheapest preferred metered model whose input cap holds OpenClaw's own system prompt as the primary. A wallet can pay instead of a credits key (AGENT402_WALLET_KEY, an EVM key holding USDC on Base): exact by default, or upto after a one-time agent402-openclaw permit2-approve, so the wallet settles actual usage; agent402-openclaw doctor reports which mode it is in. Full guide: agent402.tools/guides/openclaw-model-provider.

Image generation

POST /v1/images/generations speaks the OpenAI images wire - any OpenAI SDK's images.generate() works by changing base_url. Send {"prompt": "..."} (up to 4,000 chars) and get {created, model, data: [{b64_json, media_type}]} back - one image per call at a flat $0.08, n locked to 1, response_format is always inline b64_json (nothing is hosted). zdr: true works here too. Upstream is Gemini 2.5 Flash Image via OpenRouter with server-owned price bounds, same margin discipline as the chat tiers.

Text-to-speech

POST /v1/audio/speech speaks the OpenAI TTS wire, so any OpenAI SDK's audio.speech.create() works by changing base_url. Send {"input": "...", "voice": "alloy"} (up to 2,000 chars) and raw audio bytes come back, mp3 by default and pcm on request, at a flat $0.06 per call. zdr: true works here too.

There is no OpenAI TTS model on the upstream, so the tier serves a five-model failover chain over OpenRouter's audio API instead, walking to the next link on any upstream failure including an empty audio body. The 11 OpenAI voice names (alloy, ash, ballad, coral, echo, fable, onyx, nova, sage, shimmer, verse) are accepted on every link and mapped to that model's closest native voice; each model's native voice ids (e.g. en_paul_cheerful) are also accepted directly and are listed per model on GET /v1/models. Pinning model moves that link to the front of the chain rather than disabling the rest.

Two request fields differ from OpenAI's:

instructions is rejected with a self-explaining 400: no serving model supports it. Pick an expressive native voice instead.
speed is accepted anywhere in 0.25–4. Upstream bills per input character, so speed is cost-neutral here; most serving models ignore it.

Because billing is per input character, the 2,000-char cap makes the worst case deterministic: every link in the chain lands under the $0.06 price.

The auto tier - routing without picking a model

Send messages with no model field (or model: "auto") and the gateway routes the prompt to the top-ranked model for its task type. Routing is fully deterministic - lexical classification (code / reasoning / long-context / general) against a fixed, eval-derived ranking table; no LLM in the routing path, identical requests always route identically.

An optional quality field picks the ranking band at the same flat price (a per-request price cannot exist under x402's fixed per-route quote):

"fast" - cheapest/snappiest serving, right for loop turns
"balanced" - the default
"best" - the strongest models the price still covers

The response discloses the decision alongside the standard model field:

"agent402_router": { "category": "code", "quality": "balanced", "served": "deepseek/deepseek-chat" }

Each ranking doubles as a failover chain: if a provider returns 502/503/504, the gateway walks down the list instead of charging you for an upstream error - every chain ends in the model the daily paid canary proves alive.

Streaming

Add stream: true on any chat tier for standard OpenAI SSE framing (data: {chunk} … data: [DONE]). Payment settles before the first byte; max_tokens is clamped server-side so the stream stops at the tier cap. Streamed responses are not idempotency-replayable and never cached.

Response caching

Two policies over one cache (10-minute TTL, served before the paywall - a hit costs nothing):

Chat tiers - opt-in. LLM output is sampled, and a resend usually wants a fresh sample, so nothing is cached unless the request carries cache: true. A byte-identical opted-in repeat returns the stored response free with X-Cache: hit.
Embeddings - default-on. Embeddings are deterministic per model, so identical repeats are free automatically; opt out with cache: false.

Keys are computed over the normalized body (model aliases and field order collapse; every sampling-relevant field is included), so equivalent requests share one entry.

POST /api/route/execute ($0.01; execute-plus $0.05, execute-max $0.55 and execute-pro $3.30 for pricier tools) - resolve a task description to the best-matching tool and run it in the same call, returning {result, receipt}. With include:"external" it can pay a proven external x402 seller on your behalf and relay the result. See x402 Index and Router.
POST /api/my-usage ($0.005) - your wallet's own purchase history (totals, per-tool counts, per-chain breakdown, receipts with settle txs). No wallet parameter: the x402 payment that buys the report determines whose report it is - nobody can read another wallet's profile.
The older custom-JSON proxies remain available: LLM Proxy Gateway (/api/llm*) and Text Embeddings (/api/embed*).

Verified daily

The paid canary buys from the gateway every day with real USDC: a nano completion (exercising the failover chain), a model-less auto completion (asserting the router disclosure), a live SSE stream, an embeddings vector, and both cache behaviors (paid once → identical repeat served free). If any of it breaks, an alarm issue opens on the repo.

Upstream service tiers

Where the upstream offers it (Gemini 2.5/3.x families, gpt-5-nano, gpt-5.6-*, and the image model), the gateway asks for OpenRouter's flex service tier first (lower price, higher latency, lower availability) and retries the same model on the default tier if flex has no capacity, before moving to the next failover link. Buyers see the same price either way; the response's service_tier field says which tier served.

Prompt caching

Every chat call asks the upstream to cache the prompt prefix (OpenRouter's top-level cache_control: {type:"ephemeral"}, 5-minute TTL) and pins your turns to one provider (session_id), so a multi-turn agent conversation is served from the provider cache on repeated prefixes. Same flat price to you either way. Send cache_control: false to opt out. The budget tiers (nano, auto) additionally pick the cheapest provider under their price cap; pro and premium keep OpenRouter's default provider balancing.

Rerank

POST /v1/rerank ($0.002) speaks the Cohere rerank wire: {query, documents[], top_n} in, results[{index, relevance_score, document}] out, served by cohere/rerank-v3.5. Up to 50 documents (1,600 chars each, 40k total) and a 500-char query per call. Deterministic, so a byte-identical repeat within 10 minutes is served free from cache (cache:false opts out). Pair it with /v1/embeddings: embed, recall your top candidates, rerank them.

Anthropic Messages API

The same five tiers also speak the Anthropic Messages wire: POST /v1/nano/messages, /v1/auto/messages, /v1/messages, /v1/pro/messages, /v1/premium/messages (same prices, models, caps and failover as each tier's chat route). Point the Anthropic SDK, Claude Code or the Agent SDK at https://agent402.tools/v1 (or /v1/pro, /v1/premium) with an x402-paying fetch and call messages.create as usual: system, content blocks (text, image, tool_use, tool_result), client tools, thinking, stop_sequences, streaming (message_start … message_stop). Any model on the tier is served through this wire (Claude natively, others translated upstream). max_tokens is required by the wire and clamped to the tier cap; a reply that spends its whole cap thinking and says nothing is never served (the failover chain walks on, and an exhausted chain is a 502 you are not charged for).

OpenAI Responses API

The same five tiers also speak the OpenAI Responses wire: POST /v1/nano/responses, /v1/auto/responses, /v1/responses, /v1/pro/responses, /v1/premium/responses. Point responses.create() (or the OpenAI Agents SDK) at https://agent402.tools/v1 (or a tier prefix) through an x402-paying fetch. input as a string or item list, instructions, function tools, text.format, reasoning, streaming (response.created … response.completed) all work; server-side tools (web search, file search, computer use, MCP) are not served, and there is no stored conversation state (store is always false, previous_response_id is refused) - send the full input each call.

Grounded answers (web search)

POST /v1/grounded/chat/completions ($0.03) is the auto router plus a live Exa web search on every call (up to 5 results): the model answers from the results and the reply carries url_citation annotations. Omit model (or send auto) and the gateway picks the model by task type; the response adds agent402_router. This is the one place web search is offered - :online model variants are refused on the other tiers because search is billed per request on top of tokens, and here that fee is part of the flat price. Never cached.

Other hosts (Claude Code, Cursor, Continue, ElizaOS, AgentCore, any OpenAI SDK): https://agent402.tools/guides/agent-hosts