Using an AI Gateway

Create a gateway in the dashboard or with the API, then call it through the OpenAI-compatible chat/completions endpoint by passing its ID or slug as model.

Create and manage gateways #

An edit request may contain only the fields to change; omitted fields keep their current values.

Call the gateway #

JSON

{
    "model": "support:50c3",
    "messages": [
        {
            "role": "user",
            "content": "How do I reset my password?"
        }
    ]
}

The request carries only the conversation. The gateway adds its instructions, retrieved documents, and tools before the model runs. To understand how this pre-configuration changes the request, see Inference with AI Gateways.

Override gateway values #

Requests sent with a private key can override supported gateway values for that request only. The gateway itself is not changed.

Request field	Overrides
`temperature`, `top_p`, `seed`, `presence_penalty`	Sampling parameters.
`reasoning_effort`, `verbosity`	Reasoning and verbosity settings, when the model supports them.
`max_completion_tokens`, `stop`	Output length and stop sequences.
`tools`	Adds client-side tools to the gateway tools.
`builtin_tools`	Enables additional built-in tools.
`response_schema`, `response_format`	Structured output.
`time_zone`	The gateway time zone.
`multimodal_resolver`	The multimodal pre-processing engines.
`tool_invocation_explanations`	Tool explanations.
`routing_options`	Provider routing.

frequency_penalty is configured only on the gateway. Instructions, RAG, MCP sources, protocol functions, skills, workers, and moderation cannot be overridden per request.

Public API keys accept only a small set of request fields and reject the others; see Limits.

Test incrementally #

When a gateway behaves unexpectedly, test a plain prompt first, then add RAG, tools, media, or structured output one at a time. This isolates the resource that causes the problem. See Troubleshooting.

Use a gateway from MCP clients #

To expose an integrated model or an AI Gateway as a tool for an external MCP client, see Inference MCP.