AIVAX

Documentation

Introduction

RAG and collections

Inference

AI Gateway Concepts AI Gateway concepts An AI Gateway stores a reusable inference configuration. Pass the gateway’s ID or slug in the request’s … Models and Gateway Names Models and gateway names The model field of /v1/chat/completions selects what runs the request. It accepts three kinds of value: Value … Using an AI Gateway Using an AI Gateway Create a gateway in the dashboard or with the API, then call it through the OpenAI-compatible chat/completions endpoint … Using AI Gateways via SDKs Using AI Gateways via SDKs The inference endpoint follows the OpenAI chat completions shape, so existing OpenAI-compatible SDKs work by … Taking an AI Gateway to Production Taking an AI Gateway to production Start with a conservative configuration: clear system instructions, a model that supports the required … Inference Concepts Inference concepts AIVAX exposes an OpenAI-compatible chat/completions API with additional AIVAX parameters. The additions are optional and … Server-side Context Server-side context chat/completions is stateless: each request carries the conversation it needs, and AIVAX does not keep a message history … Session Reuse Session reuse Session reuse lets a stateless chat/completions client keep the server-side context of a conversation between requests. Your … Inference with AI Gateways Inference with AI Gateways An AI Gateway pre-configures an inference. When model is a gateway ID or slug, the request you send is only part … Processing Pipeline Processing pipeline When a request uses an AI Gateway, AIVAX runs a fixed sequence of steps before, during, and after each model generation. … Input and Multimodality Input and multimodality Send the conversation in messages, using the OpenAI roles system, user, assistant, and tool. With an AI Gateway, the … Caching Caching Several caches affect an inference request. Prompt caching lowers the cost of input the provider has already processed. Other caches … Tools Tools Tools let the model request actions and data. AIVAX supports two execution models, and one request can use both: Client-side tools … MCP and Protocol Functions in Inference MCP and protocol functions in inference MCP tools and protocol functions are server-side tools that call your systems. AIVAX lists or loads … Structured Output Structured output Use structured responses when your application needs JSON output that matches a schema. AIVAX supports them through … On-demand Built-in Tools On-demand built-in tools Use builtin_tools to enable AIVAX built-in tools for a direct request without creating a gateway: JSON { … Streaming Streaming Set stream: true to receive the response as server-sent events (SSE) while it is generated. Each event is a data: line with a JSON … Custom Provider Request Body Custom provider request body Use extra_body to send provider-specific options. When a gateway uses your own provider API key and an … Multimodal Pre-processing Multimodal pre-processing Use multimodal_resolver when the main model should receive a textual description of media instead of the original … Context Truncation Context truncation Context truncation keeps long conversations within a maximum input size. An AI Gateway can limit the input with … Moderation Moderation Moderation enforces safety policy on user requests before the main model answers them. Use it to block unsafe or out-of-policy … Function Interpreter (Deprecated) Function interpreter (deprecated) Deprecated. The function interpreter is kept only for existing gateways. Do not use it in new gateways. … Inference Limits Inference limits This page lists the limits that apply to chat/completions requests. Account-wide request rates and plan limits are … Troubleshooting Inference Troubleshooting inference AIVAX uses two response styles: OpenAI-compatible endpoints return an OpenAI-style error object. Account and … Setup Provider Routing Setup provider routing Some integrated models are available through more than one provider. Provider routing lets AIVAX choose among those … Inline Routing Inline routing in the model name An integrated model name accepts a routing suffix after #. Use it when your client can only change the … Zero Data Retention Providers Zero data retention providers only Enable Zero data retention providers only in the dashboard account settings to route text inference with … Special Providers Special providers Some integrated models are also offered by special providers, which serve them at discounted prices under conditions that … Structured Responses Concepts Structured responses concepts AIVAX can produce structured JSON through two paths: response_schema: AIVAX validates the final model output … JSON Healing JSON Healing JSON Healing is how AIVAX makes any model return JSON that matches a schema, including models without native structured output. … Structured Responses with Reasoning and Tools Reasoning and tools A structured request can use a reasoning model and tools. The model reasons and calls tools as in any other request, and … JSON-Only Responses JSON-only responses Set json_only: true when the client should receive only the generated JSON: JSON { "model": … Chat Presentation Guidance Chat presentation guidance A streamed assistant turn can contain reasoning, server-side tool activity, partial answers, and the final … Raw Multi-turn Example Raw multi-turn example The example below shows the shape of a streamed response when server-side reasoning is visible to the client, … Tool Explanations Tool explanations Use tool explanations when your client shows users why a server-side tool is running. The setting can also be stored on an … Chat Clients Overview Chat Clients Overview A chat client publishes an AI Gateway to end users. It gives them a place to talk to the assistant—a hosted web chat, … Managing Chat Client Sessions Managing Chat Client Sessions A session is one conversation between a chat client and an end user. It stores the message history, the … Web Chat Client Web Chat Client The web client is the browser-facing side of a chat client. You can use it in two ways: Hosted web chat: send the user to … Chat Client Integrations Chat Client Integrations Integrations connect a chat client to messaging apps, so users can talk to the assistant where they already are. … Subscription Models Overview Subscription models Subscription models are text models included in the Pro, Max, and Ultimate plans. Their names start with … Harness Integration Harness integration Coding agents and other harnesses reach subscription models through the same [OI]-compatible Chat Completions API used … Agentic Tests Agentic Tests Agentic Tests evaluate how an AI Gateway behaves across a complete, goal-oriented conversation instead of grading one isolated … Voice and Realtime Voice and Realtime Voice Session is AIVAX’s low-latency, stateful voice API. Use it for a live spoken conversation. Your client opens … Gateway Workers Gateway Workers Gateway Workers are HTTP hooks that let your service control an AI Gateway while a request is running. Use one when a rule … Skills Skills Skills (also known as abilities) are instructions that your agent loads on demand to perform specific tasks better. Use them for … Batch Processing Batch Processing Batch runs the same AI workflow over many independent items. It turns a list of inputs into a background queue with fixed …

Web Foundation

Media generation

Data processing

Tools

MCP Utilities

Type to search the documentation.