# AIVAX Documentation

> Build, operate, and evaluate AI applications with AIVAX.

Language: English. Embedded API references are linked, not fetched.

# AIVAX Documentation

AIVAX is an AI orchestration platform for building, operating, and evaluating AI applications through one account and API surface. Use hosted or bring-your-own-key (BYOK) models, then add reusable instructions, knowledge, tools, media, user channels, and background processing as your product grows.

## Choose where to start

- **Make your first model call:** follow [Getting Started](https://docs.aivax.net/docs/getting-started.md) for a minimal OpenAI-compatible chat completion.
- **Understand the platform:** read the [Overview](https://docs.aivax.net/docs/overview.md) to choose between direct inference, AI Gateways, RAG, generations, Batch, and other products.
- **Prepare a production integration:** review [Authentication](https://docs.aivax.net/docs/authentication.md), [Pricing](https://docs.aivax.net/docs/pricing.md), and [Plans and limits](https://docs.aivax.net/docs/limits.md).

## Build an AI application

- [Inference](https://docs.aivax.net/docs/inference/inference.md) — generate responses with hosted or BYOK models through an OpenAI-compatible API.
- [AI Gateways](https://docs.aivax.net/docs/inference/ai-gateway.md) — reuse a model, instructions, RAG, skills, tools, moderation, and inference settings as one assistant runtime.
- [RAG collections](https://docs.aivax.net/docs/rag/collections.md) — index your own knowledge for semantic search and grounded answers.
- [Rerankers](https://docs.aivax.net/docs/rag/reranking.md) — reorder candidate documents by relevance, with or without a managed collection.
- [Skills](https://docs.aivax.net/docs/features/skills.md) — package reusable instructions and operating knowledge for AI Gateways.
- [Tools](https://docs.aivax.net/docs/tools/builtin-tools.md) and [MCP](https://docs.aivax.net/docs/tools/mcp.md) — connect assistants to AIVAX capabilities and external systems.
- [Chat clients](https://docs.aivax.net/docs/features/chat-clients.md) — publish a gateway through web chat or supported messaging integrations.

## Process text and media

- [Text classification](https://docs.aivax.net/docs/rag/classification.md) and [text segmentation](https://docs.aivax.net/docs/rag/text-segmentation.md) — prepare documents for routing, analysis, and retrieval.
- [Image generation](https://docs.aivax.net/docs/generations/images.md) — create or edit images from text and reference images.
- [Speech generation](https://docs.aivax.net/docs/generations/speech.md) and [audio transcription](https://docs.aivax.net/docs/generations/audio-transcriptions.md) — convert between text and audio.
- [Media descriptions](https://docs.aivax.net/docs/generations/media-descriptions.md) — turn images, audio, video, or files into text for downstream processing.
- [Voice Sessions](https://docs.aivax.net/docs/inference/voice-session.md) — build low-latency, two-way voice experiences.

## Operate at scale and improve quality

- [Batch](https://docs.aivax.net/docs/features/batch.md) — run the same AI workflow over many independent items in the background.
- [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md) — evaluate complete, goal-oriented conversations and track repeatable gateway regressions.
- [Structured responses](https://docs.aivax.net/docs/inference/structured-responses.md) — validate generated JSON against an application contract.
- [MCP utilities](https://docs.aivax.net/docs/mcp-utilities/account-management-mcp.md) — expose account, collection, documentation, web, and inference capabilities to compatible agents.

For endpoint schemas and generated request details, use the [AIVAX API reference](https://inference.aivax.net/apidocs).

---

Source: https://docs.aivax.net/docs/overview.html

# Overview

AIVAX is an AI orchestration platform for building, operating, and evaluating AI applications through one account, API surface, and billing wallet. It combines hosted and bring-your-own-key (BYOK) models with reusable assistant configuration, knowledge retrieval, tools, text and media processing, user-facing channels, background jobs, and conversational evaluation.

You do not need every product for every application. Start with direct inference for one response, then add the products that solve a specific reuse, knowledge, integration, scale, or quality requirement.

## Choose the right starting point

| Goal | Start with | Why |
| --- | --- | --- |
| Generate or analyze text in one request | [Inference](https://docs.aivax.net/docs/inference/inference.md) | Call a hosted or BYOK model through the OpenAI-compatible API without creating reusable assistant configuration. |
| Reuse instructions, knowledge, tools, and model settings | [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) | Give your application one stable assistant runtime that can evolve without rebuilding every request. |
| Search your own documents or generate grounded answers | [RAG collections](https://docs.aivax.net/docs/rag/collections.md) | Store and index knowledge for semantic retrieval, citations, and gateway context. |
| Reorder candidates your application already retrieved | [Rerankers](https://docs.aivax.net/docs/rag/reranking.md) | Improve relevance without requiring a managed AIVAX collection. |
| Publish an assistant to end users | [Chat clients](https://docs.aivax.net/docs/features/chat-clients.md) | Connect a gateway to web chat or supported messaging integrations with session and channel controls. |
| Process many independent records | [Batch](https://docs.aivax.net/docs/features/batch.md) | Run one repeatable workflow asynchronously with per-item state, validation, retries, cost, and export. |
| Test a complete assistant conversation | [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md) | Simulate a goal-oriented user and judge the gateway across multiple turns. |
| Build a low-latency two-way voice experience | [Voice Sessions](https://docs.aivax.net/docs/inference/voice-session.md) | Stream user and assistant audio in an interactive session instead of combining separate audio jobs. |

## Build the assistant runtime

### Inference and AI Gateways

AIVAX exposes OpenAI-compatible model listing and chat completion endpoints. Use a **direct model call** for exploration, one-off generation, or configuration that does not need to be reused. Use an **AI Gateway** when the same model, instructions, RAG collections, skills, tools, moderation, or output behavior should serve multiple calls or users.

Most production assistants use a gateway because the application can keep calling one identifier while the assistant configuration changes independently. Gateways can use integrated AIVAX models or external OpenAI-compatible providers.

Production API base URL:

```text
https://inference.aivax.net
```

OpenAI-compatible SDK base URL:

```text
https://inference.aivax.net/v1
```

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Inference%20(chat%20completions))

### Knowledge, retrieval, and reranking

A [RAG collection](https://docs.aivax.net/docs/rag/collections.md) is a semantic knowledge library. Add documents, test them with [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md), and then attach the collection to an AI Gateway when the assistant should answer from that knowledge. AIVAX can also generate grounded answers directly from collections and expose collection search through [Collections MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md).

Reranking is a separate step: it receives a query and candidate documents, then returns the candidates in a more relevant order. Use a collection for managed storage and retrieval; use the standalone [reranking](https://docs.aivax.net/docs/rag/reranking.md) generation when your application already owns the candidates.

### Skills and tools

[Skills](https://docs.aivax.net/docs/features/skills.md) package reusable instructions and operating knowledge. Use a skill when the assistant needs to know **how** to perform a task. Use RAG when it needs to retrieve **facts or source material** that may grow or change independently.

Tools let the assistant take action or retrieve live information. Choose among:

- [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) for capabilities provided by AIVAX.
- [MCP](https://docs.aivax.net/docs/tools/mcp.md) for Model Context Protocol servers and reusable tool ecosystems.
- [Protocol functions](https://docs.aivax.net/docs/tools/protocol-functions.md) for HTTP functions defined by your application.
- [Shell](https://docs.aivax.net/docs/tools/shell.md) for controlled command execution when the use case requires it.

Keep the tool surface as small as the assistant's job allows. Each additional tool expands cost, latency, permissions, and failure paths.

## Process text, documents, and media

AIVAX includes focused generation products for work that does not need a full chat conversation:

- [Text classification](https://docs.aivax.net/docs/rag/classification.md) assigns labels to one or more documents.
- [Text segmentation](https://docs.aivax.net/docs/rag/text-segmentation.md) splits long content into useful chunks for indexing or downstream processing.
- [Media descriptions](https://docs.aivax.net/docs/generations/media-descriptions.md) convert images, audio, video, and files into text that another model or workflow can use.
- [Image generation](https://docs.aivax.net/docs/generations/images.md) creates or edits images.
- [Speech generation](https://docs.aivax.net/docs/generations/speech.md) turns text into audio.
- [Audio transcription](https://docs.aivax.net/docs/generations/audio-transcriptions.md) turns audio into text.

Use direct multimodal inference when the selected chat model supports the input and should reason over it in the same request. Use a focused generation endpoint when you need a reusable artifact, a transcript, a description, or a preprocessing stage. For large independent input sets, run the appropriate operation through [Batch](https://docs.aivax.net/docs/features/batch.md).

For interactive two-way audio, use [Voice Sessions](https://docs.aivax.net/docs/inference/voice-session.md) instead of manually chaining transcription, text inference, and speech generation.

## Deliver, scale, and evaluate

### Chat clients

A [chat client](https://docs.aivax.net/docs/features/chat-clients.md) connects an AI Gateway to an end-user channel. It owns presentation, session behavior, allowed origins, uploads, audio replies, channel integrations, and user-facing limits. The gateway continues to own assistant behavior such as the model, instructions, RAG, and tools.

Use a chat client for a browser widget or supported messaging integration. Use the inference API directly when your own backend or interface already manages users, conversation state, and delivery.

### Batch

[Batch](https://docs.aivax.net/docs/features/batch.md) applies one workflow to dozens or thousands of independent records. A workflow defines the instruction, model or gateway, structured output, validation, tools, and retry policy. A job imports items, processes them in the background, exposes per-item progress and cost, and exports results.

Do not use Batch when one item depends on another or when a user needs an immediate answer. Use direct inference for one synchronous result and RAG for searchable knowledge.

### Agentic Tests

[Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md) evaluates the configured behavior of an AI Gateway across a bounded conversation. A simulated user pursues a goal while an independent judge evaluates progress. Use persisted tests for reusable, scheduled regression coverage or an ephemeral evaluation for one immediate run.

A completed test run is not automatically a successful behavior result. Review the run outcome, judge result, retained conversation, usage, and cost together.

## Operate and connect AIVAX

AIVAX records conversations and usage so you can trace behavior, attribute cost, and diagnose failures. The dashboard and account APIs expose account balance, usage, conversations, gateway resources, collection transactions, Batch items, and Agentic Test runs. Start with [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md) before enabling a high-volume or media-heavy workflow.

AIVAX also provides MCP utilities for compatible agents:

- [Account management MCP](https://docs.aivax.net/docs/mcp-utilities/account-management-mcp.md)
- [Collections MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md)
- [Documentation MCP](https://docs.aivax.net/docs/mcp-utilities/documentation-mcp.md)
- [Web utilities MCP](https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.md)
- [Media generation MCP](https://docs.aivax.net/docs/mcp-utilities/media-generation-mcp.md)
- [Inference MCP](https://docs.aivax.net/docs/mcp-utilities/inference-mcp.md)

These utilities expose existing AIVAX capabilities through MCP; they do not replace the underlying account, collection, or inference products.

## Next steps

1. Follow [Getting Started](https://docs.aivax.net/docs/getting-started.md) to make and verify your first chat completion.
2. Read [Authentication](https://docs.aivax.net/docs/authentication.md) before choosing private keys, public keys, or chat sessions for an application boundary.
3. Review [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md) before increasing traffic or processing large collections, media, tests, or Batch jobs.
4. Move reusable assistant behavior into an [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md), then add RAG, skills, tools, and a chat client only when the use case requires them.

---

Source: https://docs.aivax.net/docs/getting-started.html

# Getting Started

This guide takes you from an AIVAX account to a verified OpenAI-compatible chat completion. The example uses Python and a private API key from a server-side environment.

By the end, you will have confirmed that your key and selected model or AI Gateway can complete a request.

## Before you begin

You need:

- An AIVAX account with dashboard access and permission to create a private API key.
- Python 3.8 or later with `pip` available.

For pricing and operational limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md).

Production API base URL:

```text
https://inference.aivax.net
```

OpenAI-compatible SDK base URL:

```text
https://inference.aivax.net/v1
```

## 1. Create a private API key

Create a **private** key from the API Keys area of the AIVAX dashboard. Copy the key when it is shown and store it as a secret; do not paste the real value into the code below.

Private keys are intended for trusted server-side applications. Public keys are restricted credentials for intentionally exposed client-side routes and are not a substitute for a backend key.

If you are building a public web widget or messaging experience, review [Chat clients](https://docs.aivax.net/docs/features/chat-clients.md) before exposing any credential. Chat sessions provide a clearer boundary for user identity, conversation history, and attachments.

See [Authentication](https://docs.aivax.net/docs/authentication.md) for supported authentication schemes, private and public key behavior, and secret-handling guidance.

## 2. Install the OpenAI SDK

Install the SDK in the Python environment you will use for this example:

```bash
python -m pip install openai
```

Keep the key outside your source file. For example, set an environment variable named `AIVAX_API_KEY` using the secret-management method appropriate for your shell or deployment platform.

## 3. Choose a model or AI Gateway

The `model` field can identify:

- A hosted model returned by the model listing endpoint.
- An AI Gateway available to your account.

Use a **hosted model** for a direct, one-off call or early experiment. Use an **AI Gateway** when you want to reuse the same model, instructions, RAG collections, skills, tools, moderation, and output settings across requests or users.

Gateway slugs are supported with private keys. Public-key chat completions must use the full gateway UUID and cannot call integrated models directly.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Model%20listing)

Copy one model name or gateway identifier that is available to your account. You will use it as `<MODEL_OR_GATEWAY_ID>` in the next step.

## 4. Make the first request

Create a file named `quickstart.py` with the following code:

```python
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.aivax.net/v1",
    api_key=os.environ["AIVAX_API_KEY"],
)

response = client.chat.completions.create(
    model="<MODEL_OR_GATEWAY_ID>",
    messages=[
        {"role": "user", "content": "Write a one-sentence welcome message."}
    ],
)

print(response.choices[0].message.content)
```

Replace `<MODEL_OR_GATEWAY_ID>` with the exact hosted model name or gateway identifier selected in the previous step. Do not replace `AIVAX_API_KEY` with the key itself; the code reads the secret from the environment.

Run the file:

```bash
python quickstart.py
```

A successful request prints one generated sentence and exits without an API error.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Inference%20(chat%20completions))

## 5. Confirm the integration

Confirm that the generated response matches the prompt and comes from the model or AI Gateway selected in the previous step. This verifies the endpoint, credential, and model selection used by your application.

Before increasing traffic or processing large inputs, review [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md).

## Troubleshoot the first request

AIVAX uses two response styles:

- OpenAI-compatible endpoints return an OpenAI-style `error` object.
- Account and administrative endpoints return an AIVAX response envelope with an error or a successful `data` value.

| Status | What to check |
| --- | --- |
| `400 Bad Request` | Confirm the model or gateway identifier and remove unsupported parameters from the request. |
| `401 Unauthorized` | Confirm that the private key is present, complete, active, and sent through the SDK configuration. |
| `402 Payment Required` | Review [Pricing](https://docs.aivax.net/docs/pricing.md) and confirm that the account is ready for a billable request. |
| `403 Forbidden` | Confirm that the key type, model, or selected resource allows this operation. |
| `429 Too Many Requests` | Retry later and review [Plans and limits](https://docs.aivax.net/docs/limits.md) before increasing request volume. |
| `500 Internal Server Error` | An unexpected AIVAX failure occurred. Retry later; the response does not include internal details. |
| `503 Service Unavailable` | A service AIVAX depends on is temporarily unavailable. Retry after the interval in the `Retry-After` header. |

If the request still fails, verify in this order:

1. `base_url` is `https://inference.aivax.net/v1`.
2. `AIVAX_API_KEY` is available to the Python process and contains a private key.
3. The selected model or gateway exists and is available to the account.
4. For a gateway, test a plain prompt before adding RAG, tools, media, or structured output so you can isolate configuration problems.

## Long-running inference

If a request ends with HTTP `524` or a proxy timeout while AIVAX is still processing it, use the direct inference host:

```text
https://direct.inference.aivax.net/v1
```

The request remains synchronous, not a background job: keep the client connection open until the completion finishes, and configure a client timeout that covers the expected processing time.

Use the same private API key, model or AI Gateway identifier, messages, and request parameters. Change the SDK base URL and timeout:

```python
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://direct.inference.aivax.net/v1",
    api_key=os.environ["AIVAX_API_KEY"],
    timeout=300.0,
)

response = client.chat.completions.create(
    model="<MODEL_OR_GATEWAY_ID>",
    messages=[
        {
            "role": "user",
            "content": "Analyze this case carefully and provide a detailed recommendation.",
        }
    ],
)

print(response.choices[0].message.content)
```

## Choose the next product

Once the minimal request works, add one capability at a time:

- [AI Gateways](https://docs.aivax.net/docs/inference/ai-gateway.md) — make the assistant configuration reusable across requests and users.
- [Structured responses](https://docs.aivax.net/docs/inference/structured-responses.md) — require generated JSON to follow an application schema.
- [RAG collections](https://docs.aivax.net/docs/rag/collections.md) — index your documents, test retrieval, and attach grounded knowledge to a gateway.
- [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md), [MCP](https://docs.aivax.net/docs/tools/mcp.md), or [Protocol functions](https://docs.aivax.net/docs/tools/protocol-functions.md) — let the assistant retrieve live information or take action.
- [Chat clients](https://docs.aivax.net/docs/features/chat-clients.md) — deliver a gateway through web chat or supported messaging channels.
- [Text and media products](https://docs.aivax.net/docs/overview.md#process-text-documents-and-media) — classify or segment documents, generate images or speech, transcribe audio, and describe media.
- [Batch](https://docs.aivax.net/docs/features/batch.md) — apply the same workflow to many independent records asynchronously.
- [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md) — evaluate a complete gateway conversation before and after configuration changes.

Before increasing traffic or processing large inputs, review [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md).

---

Source: https://docs.aivax.net/docs/authentication.html

# Authentication

AIVAX authenticates API requests with account API keys. AIVAX accepts API keys via:

- `Authorization: Bearer <API_KEY>`
- `Authorization: Basic <BASE64_USERNAME_COLON_API_KEY>`
- `?api-key=<API_KEY>`

Prefer the `Authorization` header for server-to-server calls. Use the query parameter only when a client or integration cannot send headers, because URLs can be logged by proxies, browsers, and monitoring tools.

## API key types

AIVAX has two key families because browser-facing and server-side use cases have different risk profiles. If your code runs on your server, use a private key and keep it out of client bundles, logs, and public repositories. If your code runs in a browser or another environment where the key can be inspected by the end user, use a public key and keep the workflow limited to routes designed for public access.

| Type | Prefix | Intended use | Access |
| --- | --- | --- | --- |
| Private key | `sk-aiv-acc` | Server-side integrations and administrative API calls. | Authenticated account APIs and OpenAI-compatible inference. |
| Public key | `pk-aiv-` | Restricted client-side calls to explicitly public routes. | Public RAG query/answer routes and restricted chat-completion calls. |

Public keys can be used for RAG semantic search, RAG answer generation, speech generation, media descriptions, image generation, and chat completions. When a public key calls chat completions:

- The `model` must be a full AI Gateway UUID; direct integrated-model calls and gateway slug lookup are disabled.
- Gateway slug lookup is disabled.
- MCP sources, protocol functions, built-in tools, Bash, skills, and sentinel options are stripped from the request.
- Only these request parameters are accepted: `model`, `messages`, `prompt`, `temperature`, `top_p`, `top_k`, `seed`, `tools`, `reasoning_effort`, `max_completion_tokens`, `idempotency_key`, and `stream`.
- Request and token rate limits are applied both globally per key and per remote address.

Use private keys for backend services, account management, model listing, collection management, batch operations, and any workflow that needs the full gateway tool surface.

For a first server-side request, continue with [Getting Started](https://docs.aivax.net/docs/getting-started.md). If you are exposing a browser or widget experience to end users, review [Chat Clients](https://docs.aivax.net/docs/features/chat-clients.md) before deciding whether a public key is the right boundary.

## Create and list keys

API keys belong to an account and can have a label, expiration, and type. A key with a negative duration does not expire; expired keys are rejected by authentication and are later removed by cleanup jobs.

Create separate keys for separate applications. That makes rotation safer: if one integration is compromised, you can revoke only that key instead of breaking every service tied to the account. Labels and expiration dates are operational tools, not decoration; use them to identify who owns the key and when it should be reviewed.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20API%20Key)

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=List%20API%20Keys)

## Send a key with Bearer auth

For OpenAI-compatible SDKs, pass the AIVAX key as the SDK API key and set the base URL to `https://inference.aivax.net/v1`.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.aivax.net/v1",
    api_key="<AIVAX_API_KEY>"
)
```

## Hook authentication

AIVAX can authenticate outbound requests to your services, such as AI Gateway workers and server-side protocol functions.

This is the reverse direction of normal API authentication. In a normal API call, your application proves it is allowed to call AIVAX by sending an API key. In a worker or protocol-function callback, AIVAX is calling your service, so your service needs a way to verify that the request really came from the account configuration you control. That is what `X-Request-Nonce` is for.

If your account has a hook key, AIVAX sends:

```text
X-Request-Nonce: <BCrypt hash>
```

The nonce is a BCrypt hash derived from the account hook key. Validate the header by verifying the stored plain-text hook key against the hash. If the account has no hook key, the header is not sent.

Rolling the hook key invalidates existing worker and integration validation secrets.

### C# example

```csharp
using BCrypt.Net;

var nonce = request.Headers["X-Request-Nonce"];
var hookKey = Environment.GetEnvironmentVariable("AIVAX_HOOK_SECRET");

if (nonce is null || hookKey is null)
{
    return Results.Unauthorized();
}

if (!BCrypt.Net.BCrypt.Verify(hookKey, nonce, enhancedEntropy: false))
{
    return Results.Forbid();
}
```

### Python example

```python
import os
import bcrypt
from flask import abort, request

nonce = request.headers.get("X-Request-Nonce")
hook_key = os.getenv("AIVAX_HOOK_SECRET")

if nonce is None or hook_key is None:
    abort(401)

if not bcrypt.checkpw(hook_key.encode("utf-8"), nonce.encode("utf-8")):
    abort(403)
```

### JavaScript example

```javascript
import bcrypt from "bcrypt";

const nonce = req.header("X-Request-Nonce");
const hookKey = process.env.AIVAX_HOOK_SECRET;

if (!nonce || !hookKey) {
  return res.sendStatus(401);
}

if (!(await bcrypt.compare(hookKey, nonce))) {
  return res.sendStatus(403);
}
```

---

Source: https://docs.aivax.net/docs/pricing.html

# Pricing

Service usage prices are listed below in USD. **M** means one million tokens; **1k** means one thousand units. Approximate prices (`~`) vary with the model used and the work performed.

See [subscription pricing](https://aivax.net/pricing) for monthly plan prices and [Plans and limits](https://docs.aivax.net/docs/limits.md) for quotas. Usage rates are subject to the plan multiplier:
- Free: **+25%** on inference taxes;
- Pro: **+5%** on inference taxes;
- Max: **0%** on inference taxes.

BYOK are not affected by inference taxes.

Free, Pro, and Max include separate daily allowances for eligible RAG embeddings, Reflex reranking, Julia-1 semantic decisions, and Fetch/OCR extraction. The rates below apply when a metered item is not covered. Coverage is all-or-nothing per item, not necessarily per complete request: an item that cannot fit within the remaining allowance and its permitted margin is billed in full. Compare the allowances and check exclusions in [Plans and limits](https://docs.aivax.net/docs/limits.md#included-daily-subscription-allowances). LLM subscription coverage is currently disabled.

## Inference and Moderation

Inference rates depend on the selected model, provider, input size, and media type. Moderation is charged separately in Processing Units (PUs), covering input, cached input, and output usage; its PU price varies with the model and provider used.

| Description | Pricing |
| --- | ---: |
| AI model and AI Gateway inference | Selected model and provider rates |
| Input moderation | Variable price per PU; separate from the main inference charge |

## Semantic decisions

The decision-model rates below are base USD prices per million input tokens, before account and plan adjustments. Output tokens have no charge in the current decision-model catalog. Julia-1 is eligible for the daily allowance described in [Plans and limits](https://docs.aivax.net/docs/limits.md#included-daily-subscription-allowances); other decision models are billed normally.

| Model | Input price / million tokens |
| --- | ---: |
| `@supersonic-labs/julia-1` | **$0.008** |
| `@typesafe/jev-1.13` | **$0.042** |
| `@respan/span-01` | **$0.020** |
| `@respan/span-01-lite` | **$0.000** |
| `@jaredpalmer/kev-4b` | **$0.042** |

See [Semantic decisions](https://docs.aivax.net/docs/generations/decisions.md) for model selection and how input usage is measured.

## Agentic Tests

Each test includes the selected model or AI Gateway's inference charges, plus simulated-user and judge usage at the selected profile's rates.

| Description | Pricing |
| --- | ---: |
| Model or AI Gateway under test | Regular inference rates |
| Low profile - simulated user | Input **$0.25/M tokens**; cache **$0.025/M tokens**; output **$1.50/M tokens** |
| Low profile - judge | Input **$0.30/M tokens**; cache **$0.03/M tokens**; output **$2.50/M tokens** |
| Medium profile - simulated user | Input **$0.75/M tokens**; cache **$0.075/M tokens**; output **$3.75/M tokens** |
| Medium profile - judge | Input **$0.75/M tokens**; cache **$0.075/M tokens**; output **$3.75/M tokens** |
| High profile - simulated user | Input **$0.75/M tokens**; cache **$0.075/M tokens**; output **$3.75/M tokens** |
| High profile - judge | Input **$1.25/M tokens**; cache **$0.15/M tokens**; output **$4.25/M tokens** |

## RAG and Collections

Indexing and search are billed by token usage. Generated RAG responses are charged separately from query embedding, and their price varies with the summarization model.

| Description | Pricing |
| --- | ---: |
| Collection text embedding | **$0.10/M tokens** |
| Semantic search - query cache miss | **$0.10/M tokens** |
| Semantic search - query cache hit | Zero |
| RAG response generation | **~$0.50/M tokens**, excluding query rates |
| Reflex - cache miss | **$0.015/M tokens** |
| Reflex - cache hit | **$0.003/M tokens** |

## Media Injector

Converting media into RAG documents is billed for input, cached input, output, and media usage. The source file, optional context, and generated content affect the total. Rates depend on media type and input-token volume.

| Description | Pricing |
| --- | ---: |
| PDFs and images - up to 272K input tokens | Input **$0.30/M tokens**; cache **$0.03/M tokens**; output **$1.80/M tokens** |
| PDFs and images - above 272K input tokens | Input **$0.60/M tokens**; cache **$0.06/M tokens**; output **$3.60/M tokens** |
| Audio - up to 256K input tokens | Input/media **$0.60/M tokens**; cache **$0.12/M tokens**; output **$3.00/M tokens** |
| Audio - above 256K input tokens | Input/media **$1.20/M tokens**; cache **$0.24/M tokens**; output **$6.00/M tokens** |
| Video | Input/media **$0.45/M tokens**; cache **$0.045/M tokens**; output **$3.75/M tokens** |

## Text Tools

Text segmentation and classification are billed by token usage.

| Description | Pricing |
| --- | ---: |
| Text segmentation | **$0.30/M tokens** |
| Text classification | **$0.10/M tokens** |

## Voice and Media

Generation and transcription rates depend on the selected model. Media description pricing is approximate and depends on the available processing model.

| Description | Pricing |
| --- | ---: |
| Voice Sessions | Selected realtime model rates |
| Speech-to-text | Varies by model |
| Text-to-speech | Varies by model |
| Image generation | Fixed output and reference-image tariffs by model |
| Media descriptions | **~$1.50/M tokens** |

Image generation charges each delivered output at the selected model's fixed output price, plus its per-reference price for every reference sent with that output. Prompt processing is included. Token- and megapixel-priced providers use rounded-up estimates, not exact provider-cost pass-through. No additional AIVAX image-generation markup or account and plan multiplier applies. Current tariffs are listed in the Models catalog; see [Image generation](https://docs.aivax.net/docs/generations/images.md).

## Web Search, OCR and Fetch

Web and X searches are billed per search. Advanced web search is billed by token usage and varies with the model and number of interactions. Fetch and OCR extraction use Processing Units (PUs), with a daily free allowance by plan. Optional schema-guided JSON conversion is charged separately. The extraction allowances and PU rates do not apply to JSON conversion or moderation.

| Description | Pricing |
| --- | ---: |
| Web search | **$5/1k searches** |
| X (Twitter) search | **$5/1k searches** |
| Advanced web search | **~$0.75/M tokens** |
| Fetch and OCR extraction - Free | Base daily allowance; uncovered items **$0.15/1k PUs** |
| Fetch and OCR extraction - Pro | **10× Free** daily allowance; uncovered items **$0.05/1k PUs** |
| Fetch and OCR extraction - Max | **5× Pro** daily allowance; uncovered items **$0.02/1k PUs** |
| Fetch JSON conversion (`responseSchema`) | Variable inference-based price per PU; charged separately, with no daily extraction allowance |

For the [Fetch API](https://docs.aivax.net/docs/web-foundation/fetch-and-ocr.md), `processingUnits` reports text/OCR extraction usage and `jsonProcessingUnits` reports the additional schema-guided JSON conversion usage. JSON PUs account for input, cached input, and output token usage at the processing model and provider's rates; they are not priced at the plan's OCR rate. The plan's inference multiplier applies to JSON conversion. Omitting `responseSchema` or setting it to `null` disables conversion, reports `jsonProcessingUnits: 0`, and incurs no JSON conversion charge.

## Storage

Each plan includes storage. Pro and Max overages are billed hourly at the monthly rates below; Free storage cannot be expanded.

| Description | Pricing |
| --- | ---: |
| Free storage | **30 MB included**; no expansion |
| Pro storage | **2 GB included**; excess **$0.50/GB/month** |
| Max storage | **20 GB included**; excess **$0.20/GB/month** |

## Other Tools

The following tools have no separate tool charge. Model inference used to invoke them is still billed at its regular rate.

| Description | Pricing |
| --- | ---: |
| Memory and calendar | No separate charge |
| Advanced requests | No separate charge |
| Document generation | No separate charge |
| Web page generation | No separate charge |

---

Source: https://docs.aivax.net/docs/limits.html

# Plans and Limits

AIVAX has three account plans: **Free**, **Pro**, and **Max**. The current plan is stored on the account and controls model access, commissions, rate limits, RAG quotas, tool limits, storage quota, conversation retention, and included daily service allowances.

For commercial subscription prices and plan packaging, use the [AIVAX pricing page](https://aivax.net/pricing). This page documents the technical limits of the API.

## How limits are enforced

Limits are enforced at different layers:

- Authentication rejects missing, expired, or unknown API keys.
- Public API keys are restricted to public routes and have key-level and per-IP request and token limits.
- Balance middleware rejects billable requests when the account balance is below the required minimum.
- Storage middleware rejects requests when account storage exceeds the plan quota.
- Inference checks model access, request rate, input-token rate, BYOK rate, and Free-plan context size.
- RAG checks collection count, search rate, insertion rate, and JSONL import size.
- Built-in tools check daily service limits.
- Batch processing checks how many workflow items can be processed per day.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Get%20Account%20Balance)

## Plan limits

An em dash (`—`) means the plan does not impose a limit. Model, gateway, provider, or endpoint-specific limits may still apply.

| Feature | Free | Pro | Max |
| --- | --- | --- | --- |
| **Inference** |  |  |  |
| Model access | Low-price models | All models | All models |
| Inference commission multiplier | 1.25x | 1.05x | 1.00x |
| Integrated model requests | 20/min and 500/day | 200/min | — |
| Integrated model input tokens | 1,000,000/min | 20,000,000/min | — |
| BYOK requests | 30/min | 200/min | — |
| Maximum context | 65,536 input tokens | — | — |
| LLM subscription coverage | Currently disabled | Currently disabled | Currently disabled |
| Standalone text-to-speech requests | 3/min and 40/hour | 30/min | 300/min |
| Standalone audio-transcription requests | 3/min and 40/hour | 30/min | 300/min |
| Semantic decision requests | 10/min | 50/min | — |
| **RAG and collections** |  |  |  |
| Collections | 5 | — | — |
| Semantic searches | 20/min | 500/min | 3,000/min |
| Text-classification documents | 30/min and 300/day | 1,000/min | 10,000/min |
| Text-segmentation documents | 10/min and 100/day | 300/min | 2,500/min |
| Reranking searches | 30/min | 1,000/min | — |
| Reflex processing time | 30 minutes/day | 6 hours/day | — |
| Document insertions | 500/day | 10,000/day | — |
| JSONL documents per import request | 1,000 | 10,000 | 1,000,000 |
| Media Injector | 2 files/day | 30 files/day | 1,000 files/day |
| **Built-in tools** |  |  |  |
| Web search | 15/day | 1,000/day | 10,000/day |
| X/Twitter search | Not available | 1,000/day | 10,000/day |
| Advanced web search | Not available | 100/day | 1,000/day |
| Document and web page generation | 5/day | 1,000/day | 50,000/day |
| Image generation and editing | 5/day | 500/day | 5,000/day |
| General service actions | 30/day | 5,000/day | 100,000/day |
| Bash commands | 300/hour | 30,000/hour | — |
| **Agentic tests** |  |  |  |
| New runs per account | 5/min | 30/min | — |
| Concurrent runs per account | 1 | 4 | 8 |
| **Batch processing** |  |  |  |
| Workflow items processed | 500/day | 100,000/day | — |
| Files per import request | 1,000 | 1,000 | 1,000 |
| Total import size | 100 MiB/request | 100 MiB/request | 100 MiB/request |
| Single imported file size | 10 MiB | 10 MiB | 10 MiB |
| **Account and support** |  |  |  |
| Storage quota | 30 MB | 2 GB | 20 GB |
| Cost per excess GB | — | $0.50/GB/month | $0.20/GB/month |
| Conversation retention | 2 hours | 2 days | 30 days |
| Support level | Email | Priority | Dedicated |

### Semantic decision and Agentic Test rate limits

These per-minute limits are shared across API keys belonging to the same account. They are independent of subscription allowances and billing: included usage still consumes the applicable request or run quota.

- **Semantic decisions:** each request consumes one unit, regardless of how many questions it contains or which decision model it selects. A request exceeding the account's limit returns `429 Too Many Requests` before evaluation. See [Semantic decisions](https://docs.aivax.net/docs/generations/decisions.md).
- **Agentic Tests:** manual runs, scheduled runs, and direct evaluations share one new-run quota. A persisted run consumes its unit when it is queued, not again when execution starts; individual conversation turns do not consume additional run units. Excess manual run requests and direct evaluations return `429 Too Many Requests`. A scheduled test without available quota waits for a later scheduling check rather than creating an extra run. Existing runs remain subject to their separate concurrency and inference limits. See [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md).

Pace requests across the account and use bounded retries with backoff after a 429. An immediate retry still encounters the active rate-limit window. Max has no plan-imposed limit for these two quotas, but other applicable limits remain in effect.

### Semantic decision model limits

These model-specific limits apply in addition to the account-level request quotas above. An unspecified context limit does not imply unlimited input.

| Model | Context |
| --- | --- |
| `@supersonic-labs/julia-1` | 1,024 tokens per question |
| `@typesafe/jev-1.13` | 32,768 tokens |
| `@respan/span-01` | Not specified in the current catalog |
| `@respan/span-01-lite` | Not specified in the current catalog |
| `@jaredpalmer/kev-4b` | 8,192 tokens |

Julia-1 has additional serving limits:

| Limit | Value |
| --- | --- |
| Questions per request | 1–32 |
| Choices or score levels per question | 2–20 |
| Boolean options | Exactly two: false and true |
| Combined context per question | 1,024 tokens, including state, question, options, and special tokens |
| Question and options budget | 256 tokens within the combined context |
| Description of an individual option | At most 48 tokens |
| Decision payload limit | 256 KiB |

These limits interact: twenty options can exceed the combined question/options budget even if each description fits its individual limit. The current AIVAX Julia-1 serving limits apply even if an upstream model card lists a larger context. See [Semantic decisions](https://docs.aivax.net/docs/generations/decisions.md) for usage and error guidance.

### Included daily subscription allowances

Free, Pro, and Max include separate daily allowances for the services below. Each comparison refers to the same service on the named plan, not to a shared credit balance or a guaranteed number of requests. Unused allowance from one service cannot cover another. Reseller accounts do not receive subscription allowances.

| Included service | Free | Pro | Max |
| --- | --- | --- | --- |
| RAG search and insertion embeddings | Base allowance | 25× Free | 4× Pro |
| Reranking with Reflex | Base allowance | 5× Free | 10× Pro |
| Semantic decisions with Julia-1 | Base allowance | 2.5× Free | 2× Pro |
| Fetch and OCR extraction | Base allowance | 10× Free | 5× Pro |

RAG searches and document insertions share the embedding allowance. It does not cover answer generation, media processing, text classification, or segmentation. A query embedding served from cache does not consume it. Reflex uses a separate reranking allowance that includes both cached and uncached input. Julia-1 is currently the only decision model covered by the semantic decision allowance; other decision models are billed normally. Optional Fetch JSON conversion is separate from the extraction allowance.

Coverage is evaluated for each metered service item: a document's embedding, an individual query-term embedding, a reranking call, a decision call's input usage, or an extraction operation. Each item is either fully included or billed in full at normal rates. Included items are tracked in subscription consumption, not as zero-cost entries in billing history. The current allowances permit a 10% margin above their base capacity. An item that would exceed that margin leaves the allowance unchanged and is billed normally. One request can contain several items, so some may be included while others are charged.

Daily allowances reset at midnight in the server's local time. Check the account's subscription usage indicators for consumption and reset status; usage can exceed 100% while within the margin. LLM subscription coverage is currently disabled, so text-model inference and RAG answer generation remain metered separately. Allowances do not bypass balance requirements, rate limits, or Reflex's separate processing-time cap. See [Pricing](https://docs.aivax.net/docs/pricing.md) for charges when an item is not covered.

Reseller accounts support 8 concurrent agentic test runs per account.

Integrated model requests are limited by both request count and input tokens. Model rate-limit groups adjust the request-count thresholds:

| Rate-limit group | Threshold multiplier |
| --- | --- |
| Common | 1.0x |
| Discounted | 0.5x |
| Low | 0.3x |
| Free | 0.1x |

For example, a Pro account normally has 200 integrated-model requests per minute. With a `Discounted` model group, the adjusted threshold is 100 requests per minute.

BYOK uses a provider key configured on the gateway instead of an integrated AIVAX model, but requests still pass through AIVAX infrastructure and use the plan's BYOK limit.

Text-classification and text-segmentation quotas each count every item in the request's `documents` array, not each HTTP request. A request that would exceed any active window returns `429 Too Many Requests`. Text classification uses the default embedding model and is billed for the embedding work performed.

Standalone text-to-speech and audio transcription each use their own plan request quota. Voice Sessions use the selected realtime model and are subject to applicable model access, balance, and inference limits instead of these standalone request quotas. Input transcription is not currently supported inside Voice Sessions.

The JSONL import endpoint rejects a request when it reaches the plan's per-request document limit. The reranking limit applies to the autonomous reranking endpoint and to RAG searches that use a reranker, including searches performed through AI Gateways and MCP tools.

The Reflex limit counts the time spent processing Reflex requests. It applies to the autonomous reranking endpoint and RAG searches that use Reflex; cached input does not consume the quota separately. Requests that exceed the plan limit return `429 Too Many Requests`. See [Reflex](https://docs.aivax.net/docs/rag/reflex.md) for request limits, cache behavior, and pricing.

General service actions share the service-action quota shown above. Batch processing is asynchronous; if processing is paused or fails because of quota, retry after the quota window resets or upgrade the account.

## Public API keys

Public keys have additional limits independent of the account plan.

| Scope | Request limits |
| --- | --- |
| Per remote address | 3/5s, 20/min, 300/hour, 1,000/day |
| Global per key | 10/5s, 60/min, 1,500/hour, 10,000/day |

| Scope | Token limits |
| --- | --- |
| Per remote address | 100,000/5min, 500,000/30min, 2,000,000/6h, 5,000,000/day |
| Global per key | 500,000/5min, 2,000,000/30min, 10,000,000/6h, 25,000,000/day |

Public keys can be used for RAG semantic search, RAG answer generation, speech generation, media descriptions, image generation, and chat completions. For chat completions, public keys also require a full AI Gateway UUID, restrict request parameters, and omit server-side tool surfaces. See [Authentication](https://docs.aivax.net/docs/authentication.md).

---

Source: https://docs.aivax.net/docs/data-collecting.html

# Data Collecting

AIVAX offers an optional semantic data collection program for accounts that choose to contribute eligible RAG and reranking data to model development. Document indexing and storage are outside this program. All collected RAG and reranking records are anonymized before they are written to the training dataset. The setting is disabled by default and must be enabled by an authorized Account Manager — the person using the AIVAX account, not a role in the API.

## What changes when collection is enabled

While the setting is enabled:

- eligible RAG query-embedding usage receives a 10% discount;
- eligible reranking operations receive a 10% discount; and
- document indexing, storage, unrelated inference, tools, and other services keep their regular prices.

The discount applies only to eligible operations performed while collection is enabled. Disabling collection removes the discount from future operations.

## Data included

Depending on the operation, AIVAX may collect:

| Operation | Collected data |
| --- | --- |
| RAG semantic search | Query terms, selected reranker, and the contents and relevance scores of returned documents. |
| Reranking | Query, submitted documents, selected reranker, and ranking results. Unknown response metadata and usage or billing information are not included. |

The anonymized dataset does not store the account ID, API key, request ID, collection or document IDs, document names, billing data, or collection timestamp. The account is consulted transiently only to verify that collection is enabled and to apply the eligible operation discount. Document indexing and stored collections are never copied into the training dataset by this program; document content is included only when it is submitted for reranking or returned by a RAG search.

Enabling this setting does not by itself include unrelated chat completions, conversations, tool calls, or other account resources in the training dataset.

## Purpose and use

AIVAX may use collected data to develop, train, fine-tune, evaluate, test, and improve models and systems related to embeddings, retrieval, ranking, reranking, and other semantic processing. This may include preparing datasets, annotating or transforming records, measuring quality, and producing aggregated or derived artifacts.

Access is limited to authorized personnel and service providers that support these purposes under applicable confidentiality and data-protection obligations. AIVAX does not sell collected semantic data or maintain an account-to-record mapping for this dataset.

## Account Manager responsibilities

In AIVAX legal terms, the Account Manager is the person using the AIVAX account. Before enabling collection, the Account Manager must:

- have authority to accept these conditions for the account;
- have an appropriate legal basis for AIVAX to use the submitted data for the purposes above;
- provide any notices and obtain any permissions or consents required from end users or other data subjects; and
- avoid submitting credentials, secrets, regulated data, or sensitive personal data unless its collection and use are legally permitted and necessary.

## Enabling, disabling, and deletion

The Account Manager can control collection in **Dashboard > My account > Semantic data collection**.

- The setting is disabled by default.
- Enabling it authorizes anonymized collection of future eligible RAG and reranking operations.
- Disabling it stops new collection and ends the discount for future operations.
- Disabling it does not automatically delete records collected while consent was active or reverse training already completed.

Because account identifiers and account-to-record mappings are not stored, AIVAX cannot retrieve or delete training records merely from an account ID. Requests concerning personal data present inside submitted semantic content can be sent to **privacy@aivax.net** or **wm@aivax.net** and must include enough information to locate the content where applicable. Source records are retained only for as long as reasonably necessary for the documented purposes, legal obligations, security, and audit requirements, and may then be deleted. Deletion of source records does not require AIVAX to retrain or destroy models or aggregated artifacts that no longer identify a person, except where required by applicable law.

See the [Privacy Policy](https://docs.aivax.net/docs/legal/privacy-policy.md) and [Terms of Use](https://docs.aivax.net/docs/legal/terms-of-service.md) for the governing legal terms.

---

Source: https://docs.aivax.net/docs/changelogs.html

# Changelogs

Technical changes that affect AIVAX products, services, or the public API. Dates identify when entries were added or updated, not confirmed production rollout dates. Each item identifies the affected product or service; maintenance with no user-facing effect is omitted.

## Saturday, October 3rd, 2026

Changes:

- **Documentation — Updated reading and search experience.** Documentation adds language-specific search, light and dark themes, mobile navigation, and a page menu for viewing or copying Markdown. AI agents can use the documentation index and full-text files. Existing documentation URLs continue to resolve; some older guides redirect to their current product documentation.

## Friday, October 2nd, 2026

Changes:

- **Models — Claude Sonnet 5.5 and GPT-6.1 Sol added.** Adds `@anthropic/claude-5.5-sonnet` (Claude Sonnet 5.5), the direct successor to Claude Sonnet 5, and `@openai/gpt-6.1-sol` (GPT-6.1 Sol), an upgrade to GPT-6 Sol, to the text model catalog. Both support thinking, image and file input, and tool calling; GPT-6.1 Sol also supports structured output. The `@model-router/claude:mid` and `@model-router/openai:mid` aliases now select Claude Sonnet 5.5 and GPT-6.1 Sol, replacing Claude Sonnet 5 and GPT-6 Sol, respectively. Applications using these aliases may see changes in response quality, latency, and cost. Existing explicit model identifiers remain unchanged.

## Thursday, October 1st, 2026

Breaking changes:

- **API — Error status codes reflect the failure source.** Failures are no longer all returned as `400 Bad Request` by account, RAG, web, and generation endpoints. Invalid requests still return `400`, and authentication, balance, and permission failures that previously surfaced as `400` now return `401`, `402`, or `403`. Unexpected AIVAX failures return `500 Internal Server Error` with a generic message, and unavailable external services, including model providers, return `503 Service Unavailable` with a `Retry-After` header. On OpenAI-compatible endpoints, the error `code` is now `invalid_request_error`, `service_unavailable`, or `server_error` instead of always `server_error`. Clients that treat every non-2xx response as a request error should retry `500` and `503` responses, honoring `Retry-After`. See [Troubleshoot the first request](https://docs.aivax.net/docs/getting-started.md#troubleshoot-the-first-request).

Fixes:

- **Chat completions — Malformed tool declarations rejected as invalid requests.** Requests whose `tools` entries have missing or wrongly typed fields now return `400 Bad Request` instead of a server error.

- **Gateways — Bash tool reports invalid options to the model.** Unknown options or values that do not match a tool's parameters now return a command error that the model can correct, instead of failing the tool call.

Changes:

- **MCP utilities — Media generation MCP.** A new hosted MCP server at `https://inference.aivax.net/v1/mcp/media-generation` exposes `list_models`, `generate_image`, and `generate_speech` to MCP-compatible clients. `list_models` takes a `type` of `image` or `audio`; generated images and MP3 speech are returned as public URLs. Use the optional `X-Mcp-Enabled-Tools` header to choose which tools a client sees. Generations use the same pricing and limits as the Image Generation and Speech Generation APIs and require a positive balance. See [Media generation MCP](https://docs.aivax.net/docs/mcp-utilities/media-generation-mcp.md).

## Monday, September 28th, 2026

Fixes:

- **Gateways — Bash tool help accepts nullable parameters.** Requesting help for tools whose parameters accept multiple types, including `null`, no longer fails while listing their arguments. Help preserves the accepted types and nested parameter paths. No tool schema changes are required.

- **Chat integrations — Failure notifications restored.** Streaming and non-streaming conversations again attempt to send “System: something went wrong. Please, try again later.” after an unrecoverable generation failure, exhausted recovery attempts, or a turn that sends no message. A final failure is reported even if an earlier partial reply was delivered. Notification delivery still depends on the messaging service being available.

Changes:

- **Gateways / MCP — Server instructions and remote skills.** MCP sources can include server instructions and root skill documents alongside tools, including sources added by workers. Both options default to enabled; set `allowClientInstructions` or `allowRemoteSkills` to `false` to exclude that content. Skill discovery requires the remote server to advertise compatible capabilities; account skills remain available. Remote skill content is checked against its advertised size, digest, and frontmatter. Dynamic skills, supporting files, scripts, and servers requiring the newer discovery protocol are not supported. See [MCP functions](https://docs.aivax.net/docs/tools/mcp.md#server-instructions-and-remote-skills) for compatibility and trust limits.

- **Gateways — Time zone selection.** The current date and time setting now offers a time zone dropdown grouped by region, including UTC. Existing saved time zones are preserved when reopening the setting.

- **Models — Seven new text models.** Adds `@cohere/command-a-plus`, `@upstage/solar-mini4`, `@aion-labs/aion-3.5`, `@aion-labs/aion-3.5-mini`, `@qwen/qwen3.8-max-prime`, `@z-ai/glm-5.3-prime`, and `@fireworks/ember-1` as selectable text models, billed per provider at published token rates with existing account and plan adjustments still applying. Upstage and Fireworks models now display their provider icons instead of the generic fallback. Existing model identifiers remain unchanged.

## Sunday, September 27th, 2026

Changes:

- **Telegram — Compact tool progress.** Streamed replies show only the latest tool preamble in the thinking indicator when tool-call visibility is enabled, instead of accumulating tool-name blocks in the response. The final reply contains no tool-progress blocks. Other messaging channels and non-streamed replies are unchanged.

- **Gateways — Bash tool selection.** The Bash tool list now includes a shortcut for `get_date_time` (Current date and time). Inclusion and exclusion lists accept case-insensitive wildcard patterns: `*` matches any number of characters, as in `something_*`, and `?` matches one character. Exact tool names remain supported.

- **Semantic decisions and Agentic Tests — Account rate limits.** Per-minute account limits apply to semantic decision requests and new Agentic Test runs; manual, scheduled, and direct evaluations share the test-run limit. Requests above the limit return HTTP 429, while scheduled tests wait for a later scheduling check. Clients should pace requests and retry after the rate-limit window clears. Existing inference limits still apply. See [Plans and Limits](https://docs.aivax.net/docs/limits.md#semantic-decision-and-agentic-test-rate-limits), [Semantic decisions](https://docs.aivax.net/docs/generations/decisions.md#account-rate-limits), and [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md#run-and-inspect-a-test).

- **Models — Consistent speech synthesis prices.** Text-to-speech catalog prices now derive from the same per-character rates used to calculate synthesis usage, displayed per 1,000 characters. Existing model identifiers and billing rates remain unchanged.

- **Models — Comparable transcription prices.** Speech-to-text model prices are displayed consistently in USD per minute, converting hourly and per-second rates for comparison without changing billing rates or duration measurement.

- **Image generation — Fixed output and reference prices.** Image generation uses a fixed price per delivered output plus a per-reference price for each output. Models whose providers bill tokens or megapixels now use rounded-up estimates instead of metered token charges. Prompt processing is included in the output estimate; models without a separate reference charge list a zero reference rate. These are fixed tariffs, not receipts for the provider's actual consumption. Image charges no longer include the AIVAX image-generation markup or account and plan pricing multipliers. See [Image generation](https://docs.aivax.net/docs/generations/images.md).
- **Models — Subscription coverage.** The Models page now includes a “Subscription Usage” column for rerankers and semantic decision models. “Included” identifies models eligible for daily plan allowances; “-” identifies models without coverage. Eligibility does not indicate an account's remaining allowance. Both service catalogs expose this eligibility as `subscriptionUsage`.

- **Subscriptions — Included daily usage allowances.** Free, Pro, and Max subscriptions include separate daily allowances for RAG search and insertion embeddings, Reflex input (including cached input), and Julia-1 semantic decisions. Each metered item is either fully included or billed at normal rates; a request with multiple items can combine included and paid usage. This also applies to OCR extraction, replacing partial coverage. Included items appear in subscription consumption without zero-cost billing-history entries; usage indicators may exceed the base allowance within the permitted margin. Uncovered items retain normal billing records. `includesSubscriptionModels` is false while inference subscriptions are disabled; LLM subscription coverage remains disabled. Reflex's daily processing-time cap remains separate, and reseller accounts do not receive subscription allowances. See [Plans and Limits](https://docs.aivax.net/docs/limits.md#included-daily-subscription-allowances) and [Pricing](https://docs.aivax.net/docs/pricing.md).

## Saturday, September 26th, 2026

Breaking changes:

- **Image generation — Deprecated models removed.** Removes `majicMIX-realistic`, `AbsoluteReality`, `CyberRealistic`, `CyberRealistic-Pony`, `RealCartoon-Realistic`, `Hassaku-XL`, and `Meina-Mix` from the available models. Direct API requests using these identifiers now fail; select an active model instead. Built-in image generation uses `flux-schnell` when no valid model is configured, replacing the deprecated `AbsoluteReality` default. Review saved configurations; output style and pricing differ.

Changes:

- **Image generation — Additional Pollinations models.** Adds 16 official raster-image models, including FLUX 1.1 Pro and FLUX 2 variants, MAI Image variants, GPT Image 2.5 Flare and Sunburst, Qwen Image 2.1 and 3, Grok Imagine Image 2.0, Recraft V4.1 Flash, Krea 2 Medium, DreamShaper 8 LCM, and Seedream 5 Pro, with model-generated previews in the image picker. Community and SVG models are excluded. The catalog displays the billing units. See the September 27th entry for the subsequent fixed-price billing change. Failed generations are not counted as delivered images. See [Image generation](https://docs.aivax.net/docs/generations/images.md).

- **Models — Service model catalogs.** The Models page now includes tables for image generation, speech-to-text, text-to-speech, reranking, and semantic decisions, with backend-provided descriptions, release dates, base USD pricing with billing units, and an Actions dropdown in every service-model table for copying model names and opening integration documentation. Model names show a friendly label when available while copying the identifier accepted by the API. Pricing uses compact input, cached-input, output, image, character, and duration units separated by arrows where applicable, with full rates and units available in the tooltip. Catalogs are ordered newest first. OpenRouter catalog dates and friendly names supplement missing launch metadata; catalog dates are explicitly labeled rather than presented as manufacturer release dates. Models without either date remain last. Failed catalog requests can be retried independently. The public information catalogs expose these details, including the new `GET /api/v1/information/speech-models.json` catalog. Account and plan pricing adjustments still apply. See [Pricing](https://docs.aivax.net/docs/pricing.md) and [Semantic decisions](https://docs.aivax.net/docs/generations/decisions.md).

- **Privacy — Judicial disclosure and retention clarified.** The [Privacy Policy](https://docs.aivax.net/docs/legal/privacy-policy.md) and [Terms of Use](https://docs.aivax.net/docs/legal/terms-of-service.md) specify Brazilian court orders for disclosure, foreign requests to preserve existing logs for up to 1 year, and up to 1 year of technical logs and metadata. They clarify that conversation content is collected only when Conversations is enabled for the request or account, and that available account resources and backups dating back up to 3 months may be disclosed under a Brazilian court order. The content license in the Terms is expressly subject to these limits.

- **Generations — Additional decision models.** The [Semantic decisions guide](https://docs.aivax.net/docs/generations/decisions.md) explains question types and response interpretation. The public `GET /api/v1/information/decisions-models.json` endpoint lists canonical names, aliases, supported question types, context lengths, release dates, and base token prices. Adds `@respan/span-01`, `@respan/span-01-lite`, `@jaredpalmer/kev-4b`, and `@supersonic-labs/julia-1` as model choices for semantic decisions. Julia-1 supports `choice`, `score`, and `noul`; its input usage includes the state repeated for each question. Existing account and plan pricing adjustments apply, and model identifiers and request formats remain unchanged. See [Plans and Limits](https://docs.aivax.net/docs/limits.md#semantic-decision-model-limits) and [Pricing](https://docs.aivax.net/docs/pricing.md#semantic-decisions) for current limits and rates.

## Tuesday, September 22nd, 2026

Changes:

- **Models — GPT-6 Sol and Luna added.** Adds `@openai/gpt-6-sol` and `@openai/gpt-6-luna`, including their `:pro` reasoning variants. The `@model-router/openai:mid` and `@model-router/openai:budget` aliases now select GPT-6 Sol and GPT-6 Luna, respectively. Applications using these aliases may see changes in response quality, latency, and cost. Existing explicit model identifiers remain unchanged.

## Monday, September 21st, 2026

Breaking changes:

- **Gateways — Off-topic moderation is being removed.** The dedicated off-topic threshold will no longer block requests that stray from the conversation's purpose. If your application relies on this check, review its topic restrictions before adopting this change; the remaining moderation categories are not an equivalent replacement.
- **Built-in tools — Advanced web research is being disabled.** The `AdvancedWebUsage` tool will return an unavailable response instead of performing research. Remove reliance on this tool from gateway instructions and workflows. Standard [Web Search](https://docs.aivax.net/docs/web-foundation/web-search.md) and URL extraction remain separate alternatives, not equivalent replacements for multi-step research.
- **Models — Mercury 2.5 model identifier changed.** Replace `@inception/mercury-2.5-preview` with `@inception/mercury-2.5` in requests and gateway settings. The preview identifier is no longer listed, and the replacement is no longer marked as preview.
- **Gateways / MCP — MCP tool names are source-qualified.** Tools from different MCP sources no longer share an unqualified name in the gateway. Review gateway instructions, tool-selection rules, and workers that match exact tool names. The original tool name at the connected MCP server is unchanged. See [MCP functions](https://docs.aivax.net/docs/tools/mcp.md).

Fixes:

- **Collections, Gateways, and Generations — Service model availability.** Collection answer generation, gateway routing, chat utilities, and media-processing services avoid selecting temporarily unavailable models. Teach Skill and multimodal preprocessing can try another available model after a retryable failure; success still depends on service availability.
- **Teach Skill — Usage calculation.** Teach Skill processing charges use the model pricing associated with the completed request, including when a retry changes the model used. See [Teach Skill](https://docs.aivax.net/docs/generations/teach-skill.md) and [Pricing](https://docs.aivax.net/docs/pricing.md).
- **Fetch and OCR — More reliable page extraction.** Cancelled or timed-out page-content requests no longer leave page extraction running indefinitely. This addresses cases where web content extraction could stall.
- **Chat clients — Final reply and message history.** `completionText` now selects the final generated assistant reply rather than combining it with earlier assistant text during tool use. The added `createdMessages` response field preserves newly generated messages in order, including tool interactions; submitted messages are not repeated. Use this field when you need the complete generated turn.
- **Gateways / MCP — Structured MCP results are retained.** Assistants receive structured result content in addition to supported content blocks, avoiding missing information when an MCP tool returns structured output.
- **Fetch and OCR — X post extraction.** Improved readable-text extraction from public X post links. Content availability and access restrictions still apply.
- **Fetch and OCR — Plain-text normalization changed.** Plain-text conversion no longer collapses internal whitespace or normalizes Unicode characters. Surrounding whitespace may still be trimmed in extraction results. Applications that compare extracted text exactly or require normalized spacing should perform that normalization themselves.

Changes:

- **Fetch and OCR — Optional structured extraction.** Supply `responseSchema` to convert extracted content into JSON. Results add `extractedObject` and `jsonProcessingUnits` while retaining `extractedText` on success. Without a schema, the new fields are null and zero respectively. JSON conversion is billed separately from extraction and is not covered by the daily extraction allowance. See [Fetch and OCR](https://docs.aivax.net/docs/web-foundation/fetch-and-ocr.md).
- **Generations — Semantic decisions.** Evaluate multiple named questions against a shared JSON state using `noul` (true/false criteria), `choice`, or `score`. Responses include named answers, token usage, and cost. The service requires a positive balance, and its token usage contributes to account usage totals.
- **Built-in tools — Current date and time.** The `DateTime` option lets assistants request the current date, time, weekday, time zone, and UTC offset. Set `dateTimeTimeZone` to the desired time zone; the default is `America/Los_Angeles`, with daylight-saving adjustments, independent of the browser's time zone. Invalid time zones are rejected.
- **Models — Additional model choices.** Adds GLM-5.3-FlashX, Fugu Max, Pareto, Ling 3.0 Flash VL, MiMo-V2.6-Pro, MiMo-V2.6-Flash, MiMo-V2.6-Pro-UltraSpeed, and Grok 4.7 to the inference catalog. The Xiaomi frontier, mid, and budget aliases now select MiMo V2.6 models, while `@model-router/grok:latest` selects Grok 4.7. Alias users may see different response quality, latency, and cost; availability and supported capabilities depend on the selected model and account plan.
- **Inference — Transient-failure recovery.** Inference requests can make additional recovery attempts when a provider is temporarily unavailable. This may avoid some failed requests, but can also increase response time before an error is returned.
- **Avi Assistant — Updated default model.** The AIVAX console assistant changes its default model, which may change response style, latency, and usage cost. This does not change the model selected in your own gateways.
- **Documentation — Updated service guidance.** The API documentation overview and Avi Assistant guidance cover voice sessions, agentic tests, classification, segmentation, reranking, and web extraction, with current documentation links and service-specific billing guidance. This is a guidance update, not the introduction of those services.
- **Models — DeepSeek V4.1 Flash added.** The inference catalog includes `@deepseek/deepseek-v4.1-flash` with tool-calling support. Check model availability and plan eligibility before selecting it.
- **Models — Router selections updated.** `@model-router/deepseek:latest` and `@model-router/deepseek:budget` now select DeepSeek V4.1 Flash. Applications using these aliases may see different response quality, latency, and cost without changing the alias. Adds `@model-router/claude:frontier-mythos` and `@model-router/mercury:latest` as additional choices.
- **Chat clients — Structured prompt input.** Synchronous chat-client prompts accept plain text, a single message, or an ordered array of messages, including assistant tool calls and matching tool results. Existing single-message input remains supported. Optional `instructions` adds context for that request without replacing saved session context. See [Chat clients](https://docs.aivax.net/docs/features/chat-clients.md).
- **Chat clients — Turns without session persistence.** Set `commit` to false to generate a reply without saving the submitted and generated messages to session history. The default remains true. This is not a free preview: inference and tool actions still run. To continue an uncommitted tool interaction, send the assistant tool-call message together with its tool results.
- **Agentic Tests — Optional testing notifications.** Account notification preferences can enable weekly test summaries and alerts when a test reaches three consecutive failures. These complement existing failure and recovery notifications. See [Agentic Tests](https://docs.aivax.net/docs/inference/agentic-tests.md).
- **Fetch and OCR — Rendered web content.** HTML extraction supports rendered page content, improving coverage of pages whose readable text depends on scripts. Rendering is metered in processing units; this does not guarantee access to every website or restricted page. See [Fetch and OCR](https://docs.aivax.net/docs/web-foundation/fetch-and-ocr.md).

---

Source: https://docs.aivax.net/docs/rag/collections.html

# Collections and Documents

AIVAX provides a RAG (Retrieval-Augmented Generation) service for storing documents and retrieving them later through semantic search. A collection is an account-owned group of documents. Each document stores text, optional tags, an optional reference, optional metadata, and the vectors generated by the indexing job.

Collections can be searched directly through the RAG API or attached to an AI Gateway so retrieved documents can be injected into the model context.

Compare vector storage with the surrounding ingestion and retrieval pipeline in [RAG vs vector database](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/).

## Collections

Use collections to group documents that belong to the same knowledge base, product, tenant, language, or operational purpose.

A collection is the container you create before adding searchable knowledge. Think of it as the boundary for a knowledge base: a support collection can hold help-center answers, a legal collection can hold contract clauses, and a product collection can hold descriptions, policies, and troubleshooting notes. Later, you can search the collection directly with the [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md) API, expose it through [Collections MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md), or attach it to an [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) so retrieved documents are placed into the model context automatically.

Each collection has:

- A unique collection ID.
- A name.
- Optional context and contextual tags.
- A set of documents.
- Usage statistics based on RAG transactions.

Collection availability and account limits depend on the current account configuration. See [Plans and limits](https://docs.aivax.net/docs/limits.md) before creating collections for production use.

## Documents

A document is the unit that gets indexed and retrieved. It should be small enough to match a specific question and complete enough to be useful on its own.

This is the part that most affects RAG quality. A document should not be "everything you know" about a source; it should be one piece of knowledge that can stand alone when the model reads it later. If a user asks about cancellation fees, the retrieved document should already contain the relevant rule, product, condition, and exception. If the answer only makes sense when the model also sees the previous page, the document is probably too dependent on surrounding context.

A good document usually has:

- A stable name.
- Focused text.
- Optional tags for filtering or maintenance.
- Optional metadata for application-specific data.
- An optional reference ID when the document is one chunk of a larger logical item.

For example, a car manual should not be indexed as one document. Index separate documents for topics such as starting the vehicle, checking tire pressure, pairing Bluetooth, and replacing a headlight. Each document should include enough context to be read independently. For broader chunking guidance, see [Best Practices for RAG](https://docs.aivax.net/docs/rag/best-practices.md); for query behavior after indexing, see [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md).

## Document Fields

When you import documents in JSONL, each line represents one document that can be created or updated. The important field is `docid`: it is the stable name AIVAX uses to recognize the same document on future imports. If you send the same `docid` again with different text, the existing document is updated and reindexed. If you only need to preserve extra application data, use `__meta` instead of mixing that data into the searchable text.

The JSONL import endpoint accepts one JSON object per line:

| Property | Type | Required | Description |
| --- | --- | --- | --- |
| `docid` | `string` | Yes | Stable document name. Existing documents are matched by this value. |
| `text` | `string` | Yes | Text content to index semantically. |
| `__ref` | `string` | No | Reference ID used to group related chunks. Maximum stored length is 64 characters. |
| `__tags` | `string[]` | No | Tags for filtering, browsing, and maintenance. |
| `__meta` | `object` | No | Metadata returned with document details and search results. Metadata is not the semantic text used for embeddings. |

The document name must be non-empty and is limited by the API to 256 characters. The stored document content is required and cannot be empty.

## Upserts and Reindexing

Documents are matched by name (`docid` in JSONL, `Name` in the single-document API).

When a document is created, it is queued for indexing. When existing document text changes, the document is queued again and its vectors are regenerated by the background indexer. When only `__meta` changes, metadata is updated without reindexing the document text.

Reference and tag values are stored with the document. In the single-document API, changed reference, tags, or metadata can update an existing document without reindexing when the text is unchanged. In the JSONL import endpoint, changed text queues reindexing, metadata-only changes update metadata without reindexing, and changed text can also update reference, tags, and metadata. A JSONL entry that changes only the reference or tags is skipped.

## References

Use `__ref` when multiple documents represent parts of the same logical source, such as:

- Sections of the same contract.
- Clauses from the same policy.
- Chunks from the same PDF.
- Product fragments that should be shown together.

When search reference expansion is enabled, if one chunk matches, other documents in the same collection with the same reference can be included in the response.

## Media File Import

The AIVAX dashboard can upload a source file and process it into RAG documents with [Media Injector](https://docs.aivax.net/docs/rag/media-injector.md). Use it when you have a source file but do not already have focused, self-contained document text prepared for direct or JSONL import.

The original file name is normalized to Unicode NFC and preserved during upload, including accented letters, non-Latin scripts, typographic punctuation, and other Unicode characters. You do not need to rename a file to an ASCII-only name before importing it.

A Media Injector job is created only after every file chunk has uploaded and the dashboard successfully completes the upload. You can then follow it under **Batch > Media Processing**. If no job appears, the upload did not reach its completion step; retry the upload and check the error shown by the dashboard.

## Batch Import Limits

Batch import is sent as a JSONL file in the `documents` multipart field.

Use batch import when you already have many documents prepared outside AIVAX, such as chunks generated from PDFs, product catalogs, policies, or help-center articles. If you are creating or updating one document from an application flow, the single-document endpoint below is usually easier. If you are preparing a large knowledge base, import in batches, wait for indexing, and then test retrieval through [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md) before attaching the collection to a production gateway.

Per-request JSONL line limits and daily RAG insertion limits vary by plan; see [Plans and limits](https://docs.aivax.net/docs/limits.md#plan-limits). If your import exceeds the request limit, split it into multiple files. If your account reaches the daily insertion limit, wait for the rate window to reset or upgrade the plan.

> [!WARNING]
> Indexing incurs cost based on document text tokens when documents are created or when their text changes.

The embedded API reference is the source of truth for the import request and response contract.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Index%20Documents%20(JSONL))

## Document Management

### Create or update document

This endpoint is useful when your application manages documents one at a time. For example, an admin screen can save one FAQ entry, one policy clause, or one product note directly into a collection. AIVAX matches the document by name: changed text queues reindexing, while metadata-only changes update metadata without reindexing the content.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20or%20Update%20Document)

### List documents

The browse endpoint helps you inspect what is already inside a collection. Use it when you need to verify an import, find a document by name, review queued versus indexed documents, or filter content before deciding whether to update, delete, or reimport part of the knowledge base.

Supported filters:

- `-t "tag"`: documents containing the tag.
- `-r "reference"`: documents with the exact reference ID.
- `-c "content"`: documents whose content contains the text snippet.
- `-n "name"`: documents whose name contains the text snippet.
- `-i "id"`: documents whose ID contains the supplied text.

Supported states:

- `queued`: documents waiting for indexing.
- `indexed`: documents already indexed.

Supported sort values:

- `created_at_asce`
- `created_at_desc`
- `updated_at_asce`
- `updated_at_desc`
- `indexed_at_asce`
- `indexed_at_desc`

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Browse%20Documents)

---

Source: https://docs.aivax.net/docs/rag/media-injector.html

# Media Injector

Media Injector turns a source file into focused, self-contained documents inside an AIVAX RAG collection. It examines the source, identifies materially useful knowledge, writes concise factual documents in the source's predominant language, and queues those documents for semantic indexing.

Use Media Injector when you have a file whose useful knowledge has not already been split into retrieval-ready text. If you already have clean document strings, use [Create or Update Document or JSONL import](https://docs.aivax.net/docs/rag/collections.md#document-fields) instead; those paths are more predictable and avoid the additional processing needed to interpret a source file.

## When to use it

Media Injector is useful for:

- PDFs such as reports, manuals, and policies that contain several independent topics.
- Images or scanned pages whose visible content should become searchable knowledge.
- Audio and video whose material facts should be available through RAG.

It is not a general file-storage feature and does not preserve the source as one searchable document. The output is a set of generated RAG documents. Review those documents after processing when wording, coverage, legal fidelity, or sensitive-data handling is important.

Use direct document import when you need exact source wording, deterministic boundaries, stable document names, or application-controlled metadata. Use [Text Segmentation](https://docs.aivax.net/docs/rag/text-segmentation.md) when you only need cohesive source-text segments returned to your application without creating collection documents.

Use this [RAG responsibility checklist](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/) to decide which preparation steps to manage.

## How ingestion works

In the AIVAX dashboard:

1. Open the target collection and choose **Import from files**.
2. Select one or more source files.
3. Optionally provide processing context. The same context is applied to every selected file.
4. Confirm the import. The dashboard uploads the files sequentially, and a separate job is created for each file after all of its chunks have reached AIVAX.
5. Follow the jobs under **Batch > Media Processing**.
6. After each job completes, review the generated documents and wait for their indexing state before testing [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md).

A job can be `queued`, `processing`, `completed`, `failed`, or `cancelled`. The dashboard reports the source file, elapsed time, number of documents produced, and current cost. Failed or cancelled jobs may be retried when their recoverable uploaded data is still available.

Audio and video can be divided into time-based segments for processing. Segmentation is automatic and does not change the original file name shown for the job. See [Plans and limits](https://docs.aivax.net/docs/limits.md) for current upload limits.

## Define processing context

Processing context is an optional instruction that helps Media Injector decide which facts are most valuable for your knowledge base. It is considered together with the source, but it is not treated as a factual source and cannot add facts that are absent from the file.

Good context describes:

- The source's identity and purpose.
- The audience that will search the collection.
- The topics, products, jurisdictions, periods, or fields that matter.
- Ambiguous labels or internal terminology that the source itself establishes.
- Content that should be deprioritized, such as repeated headers or administrative boilerplate.

Example:

```text
This is the 2026 support policy for Acme Cloud customers in Brazil. Prioritize eligibility rules, deadlines, plan differences, exceptions, and the steps a support agent must communicate. Ignore repeated page headers and signature blocks.
```

Avoid asking the mechanism to infer conclusions, supply missing information, or use outside knowledge. For example, do not instruct it to decide whether a contract is legally enforceable or to calculate values the source does not report.

Processing context is different from a collection's context. Processing context guides this import only. Collection context describes the knowledge base to an AI Gateway when the collection is used later. Put source-specific ingestion guidance here; keep durable collection-wide guidance in the collection settings.

## Accepted source types

The dashboard accepts four source groups for Media Injector:

| Source group | Processing behavior |
| --- | --- |
| PDF documents | Reads document structure, text, and relevant visual content. |
| Images | Interprets visible text and content to produce textual RAG documents. |
| Audio | Interprets spoken and other relevant audible content; large files are segmented automatically. |
| Video | Interprets relevant visual and audible content; large files are segmented automatically. |

Use the original, accurate file extension because AIVAX uses it to identify the media type. A mislabeled extension can select the wrong processing path or make the job fail. Container and codec support can vary; if an audio or video file fails, convert it to a common format and retry.

## Generated documents

Each generated item is designed to be a useful knowledge unit rather than a page-by-page transcription. Media Injector:

- Prioritizes source identity, scope, principal facts, relationships, exceptions, and material distinctions.
- Combines closely related facts instead of creating one document per label, table cell, or repeated value.
- Ignores decorative text, pagination, repeated summaries, and incidental metadata unless they change meaning.
- Preserves the source's language, terminology, displayed dates, and number formats.
- Stops when the source has no materially new knowledge to add.

Generated documents are tagged so they can be identified as automatically produced content. They are then indexed like other collection documents and incur the collection's normal text-embedding cost in addition to Media Injector processing.

For retrieval-quality guidance after ingestion, see [Best Practices for RAG](https://docs.aivax.net/docs/rag/best-practices.md). In particular, inspect documents generated from tables, scans, and sources with repeated layouts before relying on them in production.

## Usage, pricing, and limits

Media Injector usage depends on the source, optional context, generated questions and answers, cache reuse, and media tokens when applicable. Billing aggregates input, cached input, output, and media usage for the processing job without exposing the underlying processing model. See [Pricing](https://docs.aivax.net/docs/pricing.md#media-injector) for the final rates.

Media Injector availability and operational limits depend on the account configuration. See [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md) before uploading files in production.

---

Source: https://docs.aivax.net/docs/rag/semantic-search.html

# Semantic Search

The semantic search API searches one or more collections and returns the most relevant indexed documents for the supplied search terms.

If your application already owns the candidate document strings, consider [Reflex](https://docs.aivax.net/docs/rag/reflex.md): a collection-less RAG search that ranks supplied documents without indexing or storage. Use managed semantic search when AIVAX should store and search a persistent corpus or when the corpus is too large to submit as candidates with every request.

Compare [vector search with the complete RAG pipeline](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/) before deciding what to build.

After creating a collection, search it with complete terms that reflect the question a user would ask. The response can include the matched documents and their associated collection data for use in your application or AI Gateway flow.

For the supported request, response, authentication, and error contract, use the API Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Semantic%20search)

## Reranking

A reranker can adjust the order of candidates returned by semantic search. It does not search additional documents or recover text that the retrieval stage did not select. See [Rerankers](https://docs.aivax.net/docs/rag/reranking.md) for selection guidance.

## Multiple Terms

Multiple terms cover alternative retrieval paths rather than requiring every term to match the same document. Use them for synonyms, alternative phrasings, or several acceptable ways to find an answer.

If the user intent is one composite idea, send that idea as one complete term. For example, prefer:

```text
How do I cancel an annual subscription without a penalty?
```

Over disconnected keywords:

```text
cancellation
annual subscription
penalty
```

## Search Quality

A complete query usually performs better than a list of disconnected keywords because it preserves the relationship between concepts.

If search returns poor results:

1. Confirm that the documents are indexed.
2. Query the collection directly before testing through an AI Gateway.
3. Compare complete questions with alternative phrasings.
4. Check whether the relevant document is too short, too long, or not self-contained.
5. Check whether the query language matches the document language.
6. If the gateway rewrites questions before searching, test with the plain query path to isolate rewriting issues.

## Collections MCP

To expose AIVAX collections as tools for an external MCP client, see [Collections MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md).

For current service availability and account limits, see [Plans and Limits](https://docs.aivax.net/docs/limits.md).

---

Source: https://docs.aivax.net/docs/rag/text-segmentation.html

# Text Segmentation

Text segmentation divides source documents into semantically cohesive strings that can be embedded or indexed in a RAG collection. It returns segments to your application; it does not create embeddings or store submitted documents.

Use it when a source document needs reviewable, retrieval-ready boundaries before you create or update collection documents. If you already have focused, self-contained text, you can import it directly. If you want AIVAX to process source files into collection documents, see [Media Injector](https://docs.aivax.net/docs/rag/media-injector.md).

See [where segmentation fits in a RAG pipeline](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/), alongside storage, updates, and retrieval.

## Skip segmentation when the text is already focused

Segmentation earns its keep on long, multi-topic sources — manuals, articles, transcripts — where one embedding per page would blur distinct subjects together. Do not segment text that is already one idea per unit: FAQ answers, product descriptions, short policies, or pre-chunked passages can go straight into the collection. Each unnecessary split adds indexing work and risks separating statements that only make sense together.

## What makes a good segment

A good segment is the smallest span that still answers a question on its own: a complete statement or a tight group of statements about one subtopic, typically a paragraph or a short section. Segments are most useful when the source has clear headings, paragraphs, and complete statements — the segmenter preserves those boundaries instead of cutting mid-thought.

Watch the failure modes of messy sources. Tables lose their headers, OCR drops line structure, transcripts ramble across topics, and exported documents repeat running headers on every page. Review results from those sources before indexing, and prefer cleaning the source (fixing headings, removing boilerplate) over asking the segmenter to guess around it.

## Prepare source text

Supply complete source text whenever possible. Segments are most useful when the source has clear headings, paragraphs, and complete statements. Review results from tables, OCR, transcripts, or documents with repeated headers before indexing them.

Use sanitization only when omitted content is genuinely irrelevant to retrieval. When exact source wording, legal fidelity, or full traceability matters, retain and review the source text instead.

For the supported request, response, authentication, and error contract, use the API Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Segment%20text)

For current service availability and account limits, see [Plans and Limits](https://docs.aivax.net/docs/limits.md).

---

Source: https://docs.aivax.net/docs/rag/classification.html

# Text Classification

Use text classification to rank a fixed set of labels for one or more documents without training a custom classifier. AIVAX embeds every document and label with the default embedding model, compares their vectors using cosine similarity, and returns every label from the most similar to the least similar for each document.

Before calling this endpoint, [create an API key](https://docs.aivax.net/docs/authentication.md) and make sure the account has a positive balance.

## Endpoint

<div class="request-item post">
    <span>POST</span>
    <span>/api/v1/generations/classify</span>
</div>

## Request behavior

| Property | Type | Required | Description |
| --- | --- | --- | --- |
| `documents` | `string[]` | Yes | One or more non-empty documents to classify. Results preserve this order and each document's zero-based index. |
| `labels` | `string[]` | Yes | One or more non-empty labels. Every label is scored for every document. |

Duplicate documents and labels are preserved. The endpoint always uses the current default embedding model and does not accept a model parameter, score threshold, or result limit.

Example request:

```json
{
  "documents": [
    "Compare the total cost of two loans for a principal of $10,000 over 5 years at different annual rates",
    "Erklären Sie die Unterschiede zwischen Merge-Sort und Quicksort-Algorithmen in Bezug auf Zeitkomplexität, Platzkomplexität und Leistung in der Praxis.",
    "Write a poem about the beauty of nature and its healing power on the human soul"
  ],
  "labels": [
    "Creative writing",
    "Complex problem",
    "Simple task"
  ]
}
```

## Read the response

`results` contains one item for each input document. Each `scores` array contains every supplied label, ordered by descending cosine similarity. Labels with equal scores preserve their original order.

```json
{
  "results": [
    {
      "index": 0,
      "document": "Compare the total cost of two loans for a principal of $10,000 over 5 years at different annual rates",
      "scores": [
        {
          "label": "Complex problem",
          "score": 0.98828
        },
        {
          "label": "Simple task",
          "score": 0.45272
        },
        {
          "label": "Creative writing",
          "score": 0.06823
        }
      ]
    }
  ]
}
```

A score measures vector similarity, not a calibrated probability. Compare scores within the same request and embedding model rather than interpreting a value as a percentage confidence. Negative scores are valid and remain in the response because the endpoint does not filter labels.

Embedding usage is billed for text that requires inference and is associated with the authenticated API key. Repeated text may be served from an internal cache, reducing latency and costs. Because the endpoint returns every document-label pair, response size and comparison work grow with `documents × labels`.

The embedded API reference contains the server-maintained request, response, authentication, and error details:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Classify%20documents)

---

Source: https://docs.aivax.net/docs/rag/reranking.html

# Rerankers

Rerankers reorder an existing set of candidate documents for a query. They do not search a collection or recover text that is absent from the input. Use the autonomous reranking API when your application already owns the candidates, or use [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md) to retrieve candidates from an AIVAX collection before reranking them.

[Reflex](https://docs.aivax.net/docs/rag/reflex.md) is the collection-less search experience built on this same endpoint with the default ranker — see that page when you want retrieval-style ranking without managing a collection.

## Rerank documents directly

Authenticate with an AIVAX API key and send a query with the candidate document strings. The API returns the candidates in relevance order, with the input position needed to associate each result with your application data.

Use direct reranking when candidates are dynamic, come from another search system, or do not need to be stored in an AIVAX collection. Use managed semantic search when AIVAX should retrieve candidates from a persistent corpus.

For the supported request, response, authentication, and error contract, use the API Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Rerank%20documents)

## Build candidates worth ranking

Reranking only reorders what it receives, so candidate quality decides the ceiling. Keep each candidate string focused on one idea — a paragraph or a short section rather than a whole page — so the relevance score reflects one topic instead of an average over many. When candidates come from chunking, prefer boundaries that preserve complete statements; see [Text segmentation](https://docs.aivax.net/docs/rag/text-segmentation.md).

Use the [RAG pipeline checklist](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/) to distinguish preparation, retrieval, and ordering failures.

Send enough candidates to cover plausible answers (over-retrieval first, precise ranking second) and use `top_n` to keep only the head of the ranked list. Use `min_score` to drop low-relevance tail results, but calibrate the threshold on your own queries: score scales differ between rankers and a cutoff tuned on one workload rarely transfers to another.

## Choosing a reranker

Start with the default reranker unless you have a measured reason to select another available option. The default is Reflex; alternatives include lexical matching, reciprocal-rank fusion of several signals, and third-party cross-encoders. Note that fusion (`rrf`) is retrieval-only and is rejected by this endpoint — it exists for combining signals inside collection search, not for autonomous ranking.

To compare options, fix a set of representative queries with known relevant documents from your own workload — covering your languages, document lengths, and jargon — and measure whether swapping rankers moves the right document upward. If the relevant document is missing from the candidates entirely, improve candidate retrieval, chunking, query formulation, or candidate count before comparing rerankers: no ranker recovers what was never submitted.

For current availability, supported options, and account limits, see the API Reference and [Plans and Limits](https://docs.aivax.net/docs/limits.md).

---

Source: https://docs.aivax.net/docs/rag/reflex.html

# Reflex

Reflex is AIVAX's collection-less search for RAG. Send a query together with candidate document strings and receive the most relevant items in ranked order—without indexing, storing, or maintaining a RAG collection first.

Reflex is the default ranker of the autonomous [reranking endpoint](https://docs.aivax.net/docs/rag/reranking.md): calling that endpoint without a `model` selects Reflex. This page covers when to reach for Reflex; that page covers candidate preparation and ranker comparison in depth.

Use Reflex when your application already owns the candidate documents, the candidate set changes frequently, or you want a retrieval step without collection indexing and storage. Use [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md) when AIVAX should store, index, and search a persistent knowledge base or narrow a corpus that is too large to submit as candidates on every request.

## Reflex or Semantic Search?

| Choose Reflex when... | Choose Semantic Search when... |
| --- | --- |
| Your application already has the candidate document strings. | Documents should live in managed AIVAX collections. |
| You need retrieval immediately, without an indexing step. | The knowledge base is persistent and searched repeatedly. |
| The candidate set is dynamic or request-specific. | The corpus is too large to submit as candidates on every request. |
| You want collection-less ranking. | You want collection filtering, stored metadata, document references, and managed retrieval. |

Reflex returns ranked text candidates; it does not generate an answer. Pass the selected documents to your language model or AI Gateway as RAG context.

## Use Reflex

Call the reranking API with a query and the candidate documents your application wants to compare. Results are returned in relevance order and retain the input position needed to associate them with your application data.

Use concise, focused candidate documents. Reranking can improve their order, but it cannot recover information that was not included in the candidates. If the expected document is consistently absent, improve candidate selection, chunking, or query formulation before tuning the ranker.

Reflex accepts up to 10,000 candidate documents per request and returns at most 200 ranked results. If your candidate pool exceeds 10,000 documents, narrow it first — with lexical pre-filtering, a cheap first-pass rank, or collection retrieval — and let Reflex order the shortlist.

For the supported request, response, authentication, and error contract, use the API Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Rerank%20documents)

## Use Reflex with RAG

Reflex is also the default reranker after AIVAX retrieves candidates from RAG collections. In this flow, it can improve the order of retrieved candidates but cannot recover a document that the retrieval stage did not select. If relevant documents are consistently absent, adjust retrieval, chunking, query formulation, or candidate count before tuning reranking.

Free, Pro, and Max include a daily reranking allowance for Reflex, shared between autonomous calls and RAG reranking. Cached and uncached input both consume this allowance. It is separate from the RAG embedding allowance and the processing-time cap; other rerankers are billed normally. For relative plan capacity and coverage rules, see [Plans and Limits](https://docs.aivax.net/docs/limits.md#included-daily-subscription-allowances).

---

Source: https://docs.aivax.net/docs/rag/best-practices.html

# Best Practices for RAG

Use this guide when preparing documents for AIVAX collections and semantic search. The search pipeline indexes document text as embeddings, stores the resulting vectors, and later compares user queries with those indexed vectors. Search quality depends heavily on how clear, focused, and self-contained each document is.

## Document Size

A document should represent one limited piece of knowledge.

As a practical target, keep most documents between 20 and 700 words. This range is not a hard API rule, but it usually gives the embedding model enough context without mixing unrelated topics.

The current indexer also records warnings for unusually small or large documents:

- Documents below about 10 tokens are accepted, but may be too small to retrieve reliably.
- Documents above about 1,562 tokens are accepted, but the indexer truncates the text used for embedding to about 5,000 characters and records a warning.

If a document is longer than that, split it before indexing. Use paragraphs, sections, policy clauses, FAQ entries, product descriptions, or other logical units.

## What to Avoid

- Empty, tiny, or title-only documents.
- Entire PDFs, chapters, logs, or manuals as one document.
- Multiple unrelated subjects in one document.
- Text that depends on surrounding pages to make sense.
- Mixed languages inside the same document, unless the user is expected to search that way.
- Raw JSON, code, tables, or logs without a short natural-language explanation.
- Generic pronouns and references such as "it", "this process", or "the product" when the document does not identify the subject.

## What to Do

- Give each document a clear name and focused text.
- Put the subject near the start of the document.
- Use natural language similar to the way users ask questions.
- Repeat important identifiers, product names, policy names, acronyms, and terms when they matter.
- Keep one document focused on one answerable topic.
- Use `__tags` to organize documents operationally.
- Use `__ref` to group chunks that belong to the same logical source.
- Use `__meta` for structured data your application needs to keep, such as source URL, version, author, or publication date.

Example:

Prefer:

```text
The color of the 2015 Honda Civic registered in fleet record CAR-123 is yellow.
```

Avoid:

```text
The car is yellow.
```

The first version can be retrieved and understood without external context.

## Metadata, Tags, and References

Only the document text is embedded for semantic matching. Metadata is returned with results and can be useful for applications, audits, source links, versioning, or display, but it should not replace searchable text.

Use tags for maintenance and filtering, not as the only place where important meaning appears. If a user may search for "refund policy", those words should appear in the document text, not only in a tag.

Use references when several chunks represent the same source item. When reference expansion is enabled, a matched chunk can return other documents that share the same reference.

## Chunking Larger Sources

When importing PDFs, spreadsheets, web pages, or manuals, inspect the generated chunks before relying on the collection. Remove repeated headers, footers, navigation menus, broken tables, boilerplate, and irrelevant disclaimers when possible.

See [what a vector database leaves to the application](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/) before choosing an ingestion workflow.

Good chunks usually include:

- A title or source heading.
- The immediate section context.
- The complete rule, answer, instruction, or explanation.
- Enough surrounding text to answer a question without neighboring pages.

Poor chunks often contain:

- Half of a table row.
- A sentence that depends on the previous page.
- Several policies mixed into one block.
- Repeated layout text from the original file.

## Query Quality

Semantic search works best when documents and queries use compatible language. If users ask full questions, prepare documents that contain complete explanations. If users search by product codes, policy IDs, plan names, or procedure names, include those identifiers in the text.

When search results are poor, check the basics first:

- Confirm the documents are indexed.
- Search the collection directly before testing through an AI gateway.
- Try a complete question instead of isolated keywords.
- Compare the query language with the document language.
- Review whether the relevant answer is split across too many small chunks or buried inside a very large one.

Well-prepared documents make RAG predictable: the model receives clearer source material, search returns fewer irrelevant matches, and answers become easier to audit.

---

Source: https://docs.aivax.net/docs/inference/ai-gateway.html

# AI Gateway

An AI Gateway is a persistent inference configuration. It lets you call a gateway by model name while AIVAX applies the gateway's model settings, instructions, RAG collections, tools, skills, workers, moderation, and context controls.

Use a gateway when the same behavior must be reused by multiple clients or changed without redeploying the calling application.

## How to think about a gateway

A direct call to `/v1/chat/completions` can call an integrated AIVAX model directly, for example `@openai/gpt-5-mini`. A gateway stores the decisions you do not want to repeat on every request:

- Model provider and model name.
- System instructions, remote instruction sources, user prompt template, and assistant prefill.
- RAG collections, result limits, score threshold, reranker, reference behavior, and query strategy.
- OpenAI-compatible tools, AIVAX built-in tools, MCP tools, protocol functions, skills, and the optional bash environment.
- Context window behavior, tool-message truncation, moderation, workers, model routing, and tool-call handling.

This creates a responsibility boundary. The client application sends messages and optional request overrides. The gateway administrator controls the operational policy.

In production, start with a conservative configuration: clear system instructions, a model that supports the required modalities and tools, one well-prepared RAG collection, and only the tools that are actually needed. Adding many tools, skills, or collections increases input tokens, cost, and the chance of the model choosing the wrong path.

## Models and gateway names

There are three common ways to choose what `/v1/chat/completions` uses:

- Use an integrated AIVAX model tag, usually beginning with `@`.
- Use a gateway full ID.
- Use a gateway slug in the format `name:final-id`, such as `support:50c3`.

Private API keys can resolve a gateway by full ID or by slug. Public API keys are more restricted: they can use AI gateways only by full ID, and only a limited set of chat completion request parameters is accepted.

When choosing a model, validate three points before putting it into production:

- The model supports the input modalities you intend to send, such as image, audio, video, or file.
- The model supports function calling if the gateway uses tools, RAG through `QueryFunction`, MCP, protocol functions, skills, or built-in functions.
- The model accepts the parameters you configure. Some integrated models reject assistant prefill, temperature, stop sequences, or reasoning effort.

Gateways can also use model routing. For the complexity router, AIVAX classifies the latest user request as low, medium, or high complexity, selects the configured model for that level, and emits `X-Model-Routed-Complexity` on the HTTP response when available.

## Using an AI Gateway

AIVAX provides an OpenAI-compatible chat completions endpoint:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Inference%20(chat%20completions))

Gateway values can be overridden by the request for supported parameters such as `temperature`, `top_p`, `seed`, `reasoning_effort`, `max_completion_tokens`, `stop`, `tools`, `response_schema`, `response_format`, `builtin_tools`, `multimodal_preprocess`, and `tool_invocation_explanations`. For direct inference behavior, including response rendering options, see [Inference](https://docs.aivax.net/docs/inference/inference.md).

## Using SDKs

Because the endpoint follows the OpenAI chat completions shape, you can use existing OpenAI-compatible SDKs.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.aivax.net/v1",
    api_key="<AIVAX_API_KEY>"
)

response = client.chat.completions.create(
    model="my-gateway:50c3",
    messages=[
        {"role": "user", "content": "Explain why AI gateways are useful."}
    ]
)

print(response.choices[0].message.content)
```

OpenAI-compatible inference uses `/v1/chat/completions`. The `/v1/responses` endpoint is not supported.

## Recommended production configuration

Write the system instructions so the model understands its role, audience, sources of truth, and limits. Include when to use RAG, when to use tools, and how to respond when information is unavailable. Avoid repeating operational settings that already exist in the gateway, such as truncation limits or tool lists.

For RAG, link collections with short, self-contained, well-named documents. Choose the query strategy based on the conversation type:

- `Plain`: Uses the last user message as the search term.
- `Concatenate`: Joins the last configured number of user messages line by line.
- `UserRewrite`: Rewrites recent user messages into one or more search queries using a resolver model.
- `FullRewrite`: Rewrites recent user and assistant messages using a resolver model.
- `QueryFunction`: Exposes a search function to the model instead of injecting a search result before inference.

For tools, enable only those with a clear role. Built-in tools cover common capabilities such as current date and time, web search, opening URLs, code execution, image generation, document generation, page generation, calendar actions, memory, HTTP requests, and X post lookup. External MCP is better when you already have an MCP server with business tools. Protocol functions are useful when you want to expose specific HTTP callbacks to the model without installing a full MCP server.

When enabling memory, define what the assistant may retain and how your application will review and remove stored records. The [memory-poisoning guide](https://aivax.net/blog/persistent-memory-is-a-write-path/) outlines these controls.

Use a tool handler only when the selected model needs help producing tool calls. The available handler is `react.v1.selfcall`; `native` or no value uses the model's native tool calling.

Use workers when an external system must decide something during the inference flow. A worker can block a message, rewrite context, add tools, or replace a server-side tool result. Because the worker is called in the critical path, keep it fast and deterministic.

Validate your gateway configuration with a [simulated-user and LLM-judge scenario](https://aivax.net/blog/introducing-agentic-tests/) before relying on it in production.

## Inference MCP

To expose an integrated model or AI Gateway as a tool for an external MCP client, see [Inference MCP](https://docs.aivax.net/docs/mcp-utilities/inference-mcp.md).

---

Source: https://docs.aivax.net/docs/inference/inference.html

# Inference

AIVAX exposes an OpenAI-compatible `chat/completions` API with additional AIVAX parameters. The additions are optional and are designed to support gateways, RAG, built-in tools, structured responses, multimodal pre-processing, model routing, and billing metadata.

Use this page for direct inference calls. Use [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) when the same configuration must be reused or centrally managed.

## Endpoint

<div class="request-item post">
    <span>POST</span>
    <span>
        /v1/chat/completions
    </span>
</div>

The endpoint also has the API alias `/api/v1/chat/completions`.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Inference%20(chat%20completions))

## Provider routing

Some integrated models are available through more than one provider. Provider routing lets AIVAX choose among those providers without changing the model requested by your application. This differs from model routing, which can select a different model based on request complexity.

AIVAX considers providers that are currently available and compatible with the request. If only one provider is eligible, the routing preference does not change the result. Provider routing applies only to integrated AIVAX models; a bring-your-own-key gateway uses the provider endpoint configured in that gateway.

Available routing preferences are:

| Preference | Behavior |
|---|---|
| `Balanced` | Balances price, speed, and quality. This is the default. |
| `Cheapest` | Selects the provider with the lowest applicable input and output token price. |
| `Fastest` | Prioritizes the provider with the highest available throughput. |
| `Quality` | Selects the provider that AIVAX ranks highest for quality, without optimizing for price or speed. |

### Configure routing in an AI Gateway

Use an AI Gateway when the same routing preference should apply to every request. In the gateway editor, select an integrated model, open **Routing preference**, choose the preferred strategy, and save the gateway.

The equivalent gateway configuration uses `parameters.routingOption`:

```json
{
    "name": "Cost-optimized assistant",
    "parameters": {
        "baseAddress": "@integrated",
        "modelName": "YOUR_INTEGRATED_MODEL",
        "routingOption": "Cheapest"
    }
}
```

After saving, call the gateway normally by using its ID or slug as `model`. AIVAX applies the stored routing preference while preserving the gateway's instructions, tools, RAG configuration, and other settings. See [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) for the complete gateway workflow.

### Override routing in `chat/completions`

Use `routing_preset` to choose a provider strategy for one request. The override works with a direct integrated model or an AI Gateway that uses an integrated model:

```json
{
    "model": "YOUR_INTEGRATED_MODEL_OR_GATEWAY_ID",
    "messages": [
        {
            "role": "user",
            "content": "Summarize this incident report."
        }
    ],
    "routing_preset": "Fastest"
}
```

The accepted values are `Balanced`, `Cheapest`, `Fastest`, and `Quality`. The request value overrides the gateway's saved `routingOption` for that request only; it does not update the gateway. Because `routing_preset` is an AIVAX extension, send it as an extra request-body field when using an OpenAI-compatible SDK. Request-level routing overrides require a private API key.

## Input and multimodality

AIVAX accepts OpenAI-compatible message content parts for text, images, audio, videos, and files. The selected model must support the modality unless you ask AIVAX to pre-process the media into text.

```json
{
    "model": "@google/gemini-3-flash",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Describe these inputs briefly."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "data:image/png;base64,<BASE64_PNG_CONTENT>",
                        "detail": "auto"
                    }
                },
                {
                    "type": "input_audio",
                    "input_audio": {
                        "data": "base64-encoded-audio",
                        "format": "wav"
                    }
                },
                {
                    "type": "file",
                    "file": {
                        "filename": "document.pdf",
                        "file_data": "data:application/pdf;base64,<BASE64_PDF_CONTENT>"
                    }
                }
            ]
        }
    ]
}
```

Supported content part mappings:

- `text`: Plain text.
- `image_url`: Image content. `image_url.url` can be an external URL or a base64 data URL. `image_url.detail` can be `low`, `high`, or `auto` when the model supports it.
- `video_url`: Video content. `video_url.url` can be an external URL or a base64 data URL. Prefer URLs for large videos.
- `input_audio`: Audio content. `input_audio.data` is base64 audio data, and `input_audio.format` names the format.
- `file`: File content. `file.filename` names the file, and `file.file_data` can be an external URL or a base64 data URL.

For video input, send a `video_url` content part. The following example uses a base64 Data URL; prefer a publicly reachable URL for large videos:

```json
{
    "model": "@google/gemini-3-flash",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Summarize the main actions in this video and identify any visible safety risks."
                },
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "data:video/mp4;base64,<BASE64_MP4_CONTENT>"
                    }
                }
            ]
        }
    ]
}
```

External links must be accessible to AIVAX without authentication, firewall restrictions, or JavaScript-only rendering. Failed downloads, redirects, blocked URLs, unsupported formats, or provider-specific size limits can fail the inference.

You can also send a simple text request with `prompt`:

```json
{
    "model": "@google/gemini-3-flash",
    "prompt": "Say hello"
}
```

## Request idempotency

Set `idempotency_key` when your integration needs repeat calls to update the same stored conversation record instead of creating a new conversation token. AIVAX uses this value to correlate the AI Gateway context and conversation logging.

```json
{
    "model": "your-model-or-gateway-id",
    "messages": [
        {
            "role": "user",
            "content": "Summarize order 123."
        }
    ],
    "idempotency_key": "order-123-summary"
}
```

The value must be a non-empty string with 128 characters or less. When omitted, AIVAX generates a conversation token automatically.

## Request metadata

Set `metadata` to attach string key/value information to the inference request. AIVAX stores this object with the logged conversation and exposes it to gateway events, so it is useful for operational correlation such as an order ID, tenant, workflow, or internal trace key.

```json
{
    "model": "your-model-or-gateway-id",
    "messages": [
        {
            "role": "user",
            "content": "Summarize this support ticket."
        }
    ],
    "metadata": {
        "ticket_id": "SUP-1042",
        "workflow": "support-triage"
    }
}
```

`metadata` must be a JSON object whose property names and values are strings. Do not place secrets, credentials, payment data, or large payloads in this field.

## Response and conversation records

The standard `/v1/chat/completions` response envelope includes `generation_context`. Its `generated_usage` entries contain `sku`, `amount`, `unit_price`, `quantity`, and `description`. Set `json_only: true` to return only the final JSON without this envelope.

When conversation logging is enabled, the stored record includes its ID, origin, model name, request ID, response schema, tools and tool input schemas, usage, linked resources, created and updated timestamps, token count, external user ID, error message, messages, and metadata. Gateway and API key context is available through the linked resources.

Use `idempotency_key` and `metadata` to correlate these records with your own workflow.

## Multimodal pre-processing

Use `multimodal_preprocess` when the main model should receive a textual description of media instead of the original media object. This is useful for text-first models or when you want AIVAX to normalize files before the main inference.

```json
{
    "model": "@metaai/llama-3.3-70b",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Describe this file briefly."
                },
                {
                    "type": "file",
                    "file": {
                        "filename": "document.pdf",
                        "file_data": "data:application/pdf;base64,BASE64_PDF_CONTENT"
                    }
                }
            ]
        }
    ],
    "multimodal_preprocess": "File"
}
```

Available pre-processing flags are:

- `Image`
- `Audio`
- `Video`
- `File`
- `OtherFiles`
- `All`

The resolver caches media descriptions by content hash for reuse. `Image`, `Audio`, `Video`, and PDF `File` pre-processing use auxiliary multimodal inference. Supported non-PDF files use local text extraction.

Multimodal inputs can have account requirements. Review [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md) before using them in production.

When a multimodal inference fails, narrow down the problem:

1. Test a simple text message with the same model.
2. Test one small attachment.
3. Test the same attachment with `multimodal_preprocess`.
4. Review the URL, format, size, and model modality support.

## Structured responses

AIVAX supports structured responses through `response_schema`, `response_format`, and `json_only`.

```json
{
    "model": "@google/gemini-2.5-flash",
    "prompt": "Search for recent news about electric vehicles.",
    "stream": true,
    "builtin_tools": {
        "tools": [
            "WebSearch"
        ],
        "options": {
            "web_search_mode": "full"
        }
    },
    "response_schema": {
        "type": "object",
        "properties": {
            "news": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "title": {
                            "type": "string",
                            "description": "News title"
                        },
                        "summary": {
                            "type": "string",
                            "description": "News summary"
                        }
                    },
                    "required": ["title", "summary"]
                }
            }
        },
        "required": ["news"]
    }
}
```

`response_schema` enables JSON Healing. AIVAX asks the model for JSON, extracts JSON from the generated text or markdown blocks, validates it against the schema, and retries with validation feedback until the output is valid or the attempt limit is reached.

Read more on [Structured responses](https://docs.aivax.net/docs/inference/structured-responses.md).

If your application cannot parse or validate the result, follow the [invalid JSON troubleshooting guide](https://aivax.net/blog/structured-output-healing-boundary/) before increasing the retry budget.

## On-demand functions

Use `builtin_tools` to enable AIVAX built-in tools for a direct request without creating a gateway:

```json
{
    "model": "@google/gemini-2.5-flash",
    "prompt": "Search for recent news about electric vehicles.",
    "stream": true,
    "builtin_tools": {
        "tools": [
            "WebSearch"
        ],
        "options": {
            "web_search_mode": "full",
            "web_search_max_results": 5
        }
    }
}
```

Built-in tools include `DateTime`, `WebSearch`, `AdvancedWebUsage` (disabled; returns an unavailable response; see [Changelogs](https://docs.aivax.net/docs/changelogs.md)), `OpenUrl`, `Code`, `Request`, `Calendar`, `Remember`, `GenerateWebPage`, `GenerateDocument`, `XPostsSearch`, and `ImageGeneration`.

`DateTime` exposes `get_date_time`, a no-argument tool returning the current date, time, English weekday, time zone, UTC offset, and ISO 8601 timestamp. Set `builtin_tools.options.dateTimeTimeZone` to an IANA identifier; the default is `America/Los_Angeles` (Pacific Time), with automatic daylight-saving adjustments. This setting is independent of the user's browser time zone. See [Current Date and Time](https://docs.aivax.net/docs/tools/builtin-tools.md#current-date-and-time) for configuration and output examples.

On-demand tools are suitable for occasional calls, prototypes, and integrations that do not need a persistent gateway. If the same application always uses the same tools, prefer configuring them in an AI Gateway so the policy is centralized.

## Custom provider request body

When a gateway uses a provided API key and an OpenAI-compatible provider endpoint, `extra_body` can merge custom JSON into the provider request body:

```json
{
    "model": "my-custom-model:abc4",
    "messages": [
        {
            "role": "user",
            "content": "Explain the tradeoff."
        }
    ],
    "extra_body": {
        "reasoning": {
            "enabled": true
        }
    }
}
```

`extra_body` is not allowed with integrated AIVAX models.

## Tool explanations

Set `tool_invocation_explanations: true` to ask AIVAX to include explanation fields in server-side tool arguments. When the model supplies `_tool_reason` and `_tool_goal`, `servertool.explanation` contains a client-friendly copy:

```json
{
    "model": "@x-ai/grok-4.3",
    "messages": [
        {
            "role": "user",
            "content": "What's the weather forecast for today?"
        }
    ],
    "stream": true,
    "builtin_tools": {
        "tools": ["WebSearch"]
    },
    "tool_invocation_explanations": true
}
```

Example stream event:

```json
{
    "choices": [],
    "servertool": {
        "name": "web_search",
        "id": "call-example-id-0",
        "contents": "{\"query\":\"weather forecast today\",\"_tool_reason\":\"Searching for today's weather forecast online\",\"_tool_goal\":\"I need current weather information to answer accurately.\"}",
        "state": "Created",
        "explanation": {
            "reason": "Searching for today's weather forecast online",
            "goal": "I need current weather information to answer accurately."
        }
    },
    "usage": null
}
```

## Response rendering mode

Set `rendering_mode: "textual_blocks"` when your client wants AIVAX to place reasoning and server-side tool activity in the same textual response flow that the chat UI already renders. This is useful for clients that build a single response timeline and want to turn reasoning and tool activity into visible components without keeping separate event-handling paths for every marker type.

```json
{
    "model": "@openai/gpt-5-mini",
    "messages": [
        {
            "role": "user",
            "content": "Search for recent product updates and summarize the important changes."
        }
    ],
    "stream": true,
    "builtin_tools": {
        "tools": ["WebSearch"]
    },
    "rendering_mode": "textual_blocks"
}
```

In this mode, reasoning can be emitted as `<thinking-group>` and `<think>` blocks, assistant-facing text can be emitted as `<assistant-answer>` blocks, and server-side tool markers can appear as tool result elements such as `<div class="tool-result reason" data-tool-name="...">`. Treat these blocks as presentation markers inside the response stream: parse them into chat timeline components, collapsible reasoning sections, assistant answer fragments, or tool status rows, but do not blindly concatenate every marker into the final assistant answer.

Clients that do not understand this markup should keep the default rendering mode and handle the structured stream events directly. In the default mode, reasoning arrives through `delta.reasoning`, and server-side tool activity arrives through `servertool` events. Preserve the order in which stream events arrive so reasoning, tool activity, partial content, and the final answer remain in the same response timeline.

### Raw multi-turn example

The example below shows the shape of a streamed response when server-side reasoning is visible to the client, `tool_invocation_explanations` is enabled, and `textual_blocks` is used to keep the response timeline textual. The exact tool result attributes can vary by renderer, but the important behavior is the ordering: reasoning, assistant answer fragments, tool activity, more reasoning, and the final answer can all belong to the same assistant turn.

```json
{
    "model": "my-custom-model:abc4",
    "messages": [
        {
            "role": "user",
            "content": "Which cheap and fast multimodal models should I use for security camera analysis?"
        }
    ],
    "stream": true,
    "builtin_tools": {
        "tools": ["WebSearch"]
    },
    "tool_invocation_explanations": true,
    "rendering_mode": "textual_blocks",
    "extra_body": {
        "reasoning": {
            "enabled": true
        }
    }
}
```

Raw streamed assistant timeline:

```text
<thinking-group>
<think>
The user is asking for cheap, fast multimodal models for security camera analysis.
I should list available AIVAX models and search the documentation before recommending options.
</think>
</thinking-group>

<assistant-answer>
I will check the available multimodal models and identify the best options for security camera analysis.
</assistant-answer>

<thinking-group>
<div class="tool-result reason" data-tool-name="aivax_list_models"><b>aivax_list_models</b><span>Listing the available models in AIVAX</span></div>

<div class="tool-result reason" data-tool-name="aivax_search_context"><b>aivax_search_context</b><span>Searching documentation about multimodal models and image analysis in AIVAX</span></div>

<think>
The relevant models should support VideoInput or ImageInput, have low input cost, and be fast enough for camera workflows.
I found several candidates and should rank them by cost, speed, and modality support.
</think>
</thinking-group>

<assistant-answer>
For security camera analysis, prioritize models with VideoInput, low input pricing, and high speed.

Model availability and prices change over time; the picks below are example output — see [Pricing](https://docs.aivax.net/docs/pricing.md) for current values.

Top picks:

1. @google/gemini-2.5-flash-lite: fast, inexpensive, and supports video.
2. @qwen/qwen3.5-9b: low input cost in this example output with video support.
3. @amazon/nova-lite: low input cost and a large context window.

Use VideoInput for clips when possible. If a model only supports ImageInput, extract frames from the camera stream before sending them.
</assistant-answer>
```

When the user replies, keep the conversation history focused on the user-visible assistant result. Store reasoning and tool details as timeline or audit metadata if your product needs them, but do not turn them into a new user message. The assistant message should use the content from the final `<assistant-answer>` block, not the full reasoning transcript.

```json
{
    "model": "my-custom-model:abc4",
    "messages": [
        {
            "role": "user",
            "content": "Which cheap and fast multimodal models should I use for security camera analysis?"
        },
        {
            "role": "assistant",
            "content": "For security camera analysis, prioritize models with VideoInput, low input pricing, and high speed.\n\nModel availability and prices change over time; the picks below are example output — see [Pricing](https://docs.aivax.net/docs/pricing.md) for current values.\n\nTop picks:\n\n1. @google/gemini-2.5-flash-lite: fast, inexpensive, and supports video.\n2. @qwen/qwen3.5-9b: low input cost in this example output with video support.\n3. @amazon/nova-lite: low input cost and a large context window.\n\nUse VideoInput for clips when possible. If a model only supports ImageInput, extract frames from the camera stream before sending them."
        },
        {
            "role": "user",
            "content": "Now recommend one model for real-time alerts and one for deeper review."
        }
    ],
    "stream": true,
    "builtin_tools": {
        "tools": ["WebSearch"]
    },
    "tool_invocation_explanations": true,
    "rendering_mode": "textual_blocks",
    "extra_body": {
        "reasoning": {
            "enabled": true
        }
    }
}
```

### Presentation guidance

During generation, reasoning is useful because it lets the user follow what the model is doing before the final answer exists. The assistant can "speak" while it reasons by emitting user-facing process updates or provisional answer fragments. These updates may be interleaved with reasoning blocks, tool calls, and partial answer content as the response develops.

Once the final assistant answer is generated, that answer becomes the main product of the inference. The intermediate reasoning is still useful for audit, orientation, and debugging, but it usually stops being the user's primary goal. Collapse or minimize reasoning by default after completion so the final answer receives the strongest visual emphasis, while keeping the process available for users who want to inspect it.

Use progressive disclosure around that lifecycle. Reasoning can be visible while the model is still working, then become a quieter, secondary element after the final answer appears. Tool activity should read as status, not speech: use concise labels such as "Searching", "Opening source", "Running tool", "Finished", or "Failed", and keep each tool invocation grouped as one timeline item even if its state changes over time.

A good visual hierarchy is:

- Assistant answer: highest prominence, normal reading typography, part of the main conversation.
- In-progress reasoning: visible enough to show what the model is doing while the response is being generated.
- Completed reasoning: lower prominence, subdued color or container, collapsed or minimized by default.
- Tool blocks: compact status rows with clear loading, success, and error states.
- Raw details: hidden by default unless the client is a developer, audit, or debugging surface.

Avoid exposing noisy internals directly to end users. Show tool names, states, source labels, or short summaries when they help the user understand what happened. Hide raw arguments, large payloads, and implementation details unless the user explicitly asks for details or the product surface is built for technical inspection.

For accessibility, make every collapsed block keyboard-toggleable, give each status row a readable label, avoid relying on color alone for state, and keep motion subtle. A streaming response should feel stable while it updates: new reasoning or tool rows can appear in order, but existing content should not jump around or force the user to lose their reading position.

## Direct call or gateway

Use a direct call for simple tasks, tests, internal routines, and integrations where the application controls the model, prompt, tools, and context for each request.

Use an AI Gateway when behavior needs to be stable, auditable, and reusable. Gateways are better for support assistants, chat bots, RAG agents, permanent tools, workers, skills, and configurations shared by multiple clients.

---

Source: https://docs.aivax.net/docs/inference/agentic-tests.html

# Agentic Tests

Agentic Tests evaluates how an AI Gateway behaves across a complete, goal-oriented conversation instead of grading one isolated response. AIVAX simulates the user's next message, sends each turn to the selected gateway, and uses an independent judge to determine whether the conversation reached its goal, remains recoverable, or has persistently moved away from the expected outcome.

Use Agentic Tests to create repeatable regression checks for support, sales, onboarding, tool use, RAG, and other multi-turn agent flows. Because a test runs through the configured AI Gateway, it exercises the gateway's model, instructions, tools, skills, knowledge, and inference settings together.

## Persistent tests in the dashboard

Open **Agentic Tests** in the AIVAX dashboard to create and manage reusable test cases. A test stores:

- the AI Gateway under test;
- a goal that describes the expected conversational outcome and is shared with the simulated user and judge;
- optional validation criteria used only by the judge;
- optional starting messages, external resources, and an external user identifier;
- simulated-user sampling, turn limits, exit behavior, and evaluation thresholds;
- an optional recurring schedule;
- failure and recovery notification settings.

The test definition is reusable. Each execution creates a separate run, so changing a test later does not replace the history already collected for previous runs.

### Create a useful test

Write the goal from the simulated user's perspective: describe who they are, what they want, and how they should progress through the conversation. Do not write it as instructions for the assistant. The goal is shared with both the simulated user, which pursues it, and the judge, which evaluates it. For example:

> You are choosing a plan for your team. Explain your team's size and needs when asked, ask which plan fits, and continue until you understand the recommendation and how to sign up.

Use **Validation criteria** for optional requirements that should affect only the judge's evaluation, not the simulated user's behavior. For example:

> The recommendation must name the selected plan and connect it to the stated team size. The final response must include a direct signup step.

Keeping these criteria separate prevents the simulated user from unnaturally steering the conversation toward the checks that the judge will apply.

Use **Start messages** when the scenario requires an established context, such as a customer objection, a prior assistant response, or a specific point in an existing flow. Use `external_user_id` when the gateway behavior depends on an identity from your own application. The value is forwarded to gateway inference for every run of that test.

Use `resources` to give the simulated user and judge shared context that does not belong in the initial conversation. Provide up to 16 objects with a `type` and non-empty `data` value. `Text` uses `data` as literal context; `RemoteResource` retrieves the content from the URL in `data`. For example:

```json
{
  "resources": [
    {
      "type": "Text",
      "data": "The customer has a 14-day refund window."
    },
    {
      "type": "RemoteResource",
      "data": "https://example.com/refund-policy"
    }
  ]
}
```

Resources are visible to the simulated user and judge; they are not sent to the gateway under test as conversation history. They do not add knowledge to the gateway. If the assistant must retrieve the same material, make it available through the gateway's own knowledge or tools. The simulated user may still reveal resource information naturally in its messages, so do not treat resources as hidden judge-only criteria.

Remote content can change between test runs and contributes to usage. Use only trusted, publicly reachable URLs whose content is appropriate for the test.

A focused test usually gives more actionable results than one broad scenario. Separate unrelated goals into different tests so that a failure identifies the behavior that regressed.

Follow the [multi-turn conversation testing walkthrough](https://aivax.net/blog/introducing-agentic-tests/) to define observable goals, run a simulated conversation, and inspect recovery.

### Validation hooks

Persisted Agentic Tests may call external validation hooks during a run. Configure the `hooks` array when creating or updating a test:

```json
{
  "hooks": [
    {
      "event": "before-test",
      "url": "https://validator.example/hooks/agentic-tests"
    },
    {
      "event": "after-test",
      "url": "https://validator.example/hooks/agentic-tests"
    },
    {
      "event": "before-inference",
      "url": "https://validator.example/hooks/gateway"
    },
    {
      "event": "after-inference",
      "url": "https://validator.example/hooks/gateway"
    },
    {
      "event": "context-changed",
      "url": "https://validator.example/hooks/agentic-tests"
    }
  ]
}
```

The supported events are:

| Event | When it is sent | Event data |
|---|---|---|
| `before-test` | Before the first simulated turn. | `gateway`, `goal`, and `metadata`. |
| `after-test` | After the test reaches a normal terminal outcome and before the final event is emitted. | The final outcome, reason, state, turn number, score, conversation delta, and loss streak. |
| `before-inference` | Once before the primary gateway inference of each turn. | `turn_number` and the current `messages` array. |
| `after-inference` | Once after the primary gateway inference of each turn. | `turn_number` and the current `messages` array. |
| `context-changed` | Once per turn after the gateway response has completed. | `turn_number` and the current `messages` array. |

`before-inference` and `after-inference` are available only when the test targets an AI Gateway. Hooks are supported for persisted dashboard tests and scheduled runs; the direct SSE validation endpoint does not accept `hooks`.

Each hook receives a worker-compatible JSON envelope:

```json
{
  "testId": "<AGENTIC_TEST_ID>",
  "runId": "<AGENTIC_TEST_RUN_ID>",
  "gatewayId": "<GATEWAY_ID>",
  "moment": "2026-08-16T03:00:00Z",
  "event": {
    "name": "before-inference",
    "data": {
      "turn_number": 1,
      "messages": []
    }
  }
}
```

The hook URL must be an absolute HTTP or HTTPS URL without embedded credentials and must not resolve to localhost, loopback, private, link-local, or other blocked local addresses. When the account has a hook key, AIVAX also sends `X-Request-Nonce`; validate it before trusting the payload. Hook requests use `POST` with `Content-Type: application/json`.

Hook responses follow the worker convention: any `2xx` response continues the run; a non-`2xx` response or HTTP request failure interrupts it immediately. The response body does not select another action. The interrupted run is stored as `failed`, emits a terminal result with `reason: "validation_hook_interrupted"`, and includes hook call audit entries in the run `result.hooks` array. Audit entries contain the event, URL, timestamp, status or error, whether execution continued, and up to 4,000 characters of the response body.

### Run and inspect a test

Select **Run test** to queue an execution. Runs can be `pending`, `running`, `succeeded`, `failed`, or `cancelled`. Both the rate of new runs and account-level concurrency depend on the current plan. See [Plans and limits](https://docs.aivax.net/docs/limits.md#plan-limits) for current values.

Manual runs, scheduled runs, and direct API evaluations share one new-run quota across the account's API keys. A persisted run counts when it is queued and does not count again when execution begins. Conversation turns do not consume additional run units, although applicable inference limits still apply. A manual run request above the quota returns HTTP 429 without creating a run; wait for the rate-limit window to clear before retrying.

A run processes its conversation sequentially, while eligible runs from the same account can execute concurrently. Every turn checks that the account can continue operating. A run can fail if the balance is exhausted or inference cannot continue, and a pending or running run can be cancelled from the dashboard.

The run inspector retains:

- simulated-user, assistant, and judge messages in chronological order;
- a precise timestamp for every retained message;
- prompt, cached prompt, and completion token usage per message;
- every judge opinion, including its reasoning, score, state, and trajectory values;
- the final evaluation result, failure information, and total cost charged to the run.

Use the judge opinions to identify the turn where the conversation improved, became at risk, succeeded, or entered a persistent loss. The run detail can also be exported as JSON for offline review.

### Schedule recurring tests

A test can run automatically from a standard five-field cron expression. The minimum supported interval is five minutes. For example, `*/15 * * * *` runs every 15 minutes.

If the account's new-run quota is exhausted, a due scheduled test waits for a later scheduling check without creating a run. Scheduling does not bypass the quota or reserve capacity separately from manual runs and direct evaluations.

Disable scheduling when you want to preserve the test definition without creating new scheduled runs. Manual runs remain available from the test page.

### Failure and recovery notifications

Enable failure notifications when repeated run execution failures should alert the account owner. **Notification threshold** controls how many consecutive runs in the `failed` state are required before AIVAX sends an alert. The default is `1`.

These notifications track execution errors, not behavioral test failures. A run in the `succeeded` state completed without an execution error, but its behavioral outcome can still be `loss` or `incomplete`. Those outcomes do not count toward the failure notification threshold.

When **Recovery notification** is enabled, AIVAX also notifies the account after a run completes successfully following enough consecutive execution failures to reach the configured threshold. A successfully completed run resets the consecutive-failure counter, even if its behavioral outcome is `loss` or `incomplete`. Recovery therefore means execution recovered, not that the assistant passed the behavioral checks.

### Retention

Succeeded and failed runs are retained for one month. Cancelled runs are retained for one day. Export any result that must remain available beyond those periods.

## Evaluation settings

| Setting | Default | Accepted values | Description |
| --- | ---: | --- | --- |
| `validation_criteria` | `null` | String, message part, or list of message parts | Optional requirements supplied only to the judge. They do not guide the simulated user or the gateway under test. |
| `resources` | `[]` | Up to 16 `{ "type", "data" }` objects | Additional context supplied to the simulated user and judge. Use `Text` for literal `data` or `RemoteResource` for content retrieved from the URL in `data`. |
| `hooks` | `[]` | Up to 16 `{ "event", "url" }` objects | External callbacks for persisted runs. Supported events are `before-test`, `after-test`, `before-inference`, `after-inference`, and `context-changed`; `before-inference` and `after-inference` require an AI Gateway. |
| `profile` | `medium` | `low`, `medium`, `high` | Selects the capability and price tier used by the simulated user and judge. It does not replace the model configured on the gateway under test. |
| `max_turns` | `10` | `2`–`64` | Maximum number of simulated-user turns before the run ends. |
| `minimum_turns` | `1` | `1`–`63`, less than `max_turns` | First turn when the simulated user may receive the option to end the conversation. |
| `allow_user_exit` | `true` | Boolean | When enabled, the simulated-user prompt exposes the conversation exit token from `minimum_turns` onward. When disabled, that option is omitted from every simulated-user prompt. |
| `judge_start_turn` | `1` | `1`–`63`, less than `max_turns` | First turn evaluated by the judge. The final turn is always evaluated. |
| `loss_threshold` | `0.2` | `0.01`–`0.99` | Boundary used to identify a persistently unsuccessful trajectory. |
| `base_threshold` | `0.9` | `0.01`–`0.99` | Score at or above which the goal is considered reached. It must be greater than `loss_threshold`, with more than `0.1` between them. |
| `user_sampling.top_k` | `0.4` | `0`–`2` | Controls how many sampled communication characteristics guide the simulated user. Higher values increase variation. |
| `user_sampling.max_decay` | `0.02` | `0`–`1` | Controls how sampled user characteristics may change between turns. |

Reduce `max_turns` for fast, bounded regression checks. Increase it for flows that naturally require discovery or several tool calls. `minimum_turns` and `judge_start_turn` must each be less than `max_turns`; they are otherwise independent. Delay `judge_start_turn` when early clarification is expected and intermediate scores are not useful. Disable `allow_user_exit` when only the judge or the turn budget should end the test; `minimum_turns` controls only when the simulated user sees its exit option and does not delay judge decisions. Keep a wide gap between the loss and success thresholds unless the policy has been calibrated against representative conversations.

Agentic Tests bills the selected gateway inference plus simulated-user and judge usage at the rates of the selected profile. See [Pricing](https://docs.aivax.net/docs/pricing.md#agentic-tests) for current rates.

## Direct API execution

Each direct evaluation consumes one unit from the same account quota as persisted runs. If that quota is exceeded, the request returns HTTP 429 before the SSE stream opens. Check the HTTP status before processing events, and use bounded retries with backoff. See [Plans and limits](https://docs.aivax.net/docs/limits.md#semantic-decision-and-agentic-test-rate-limits).

Use the direct generation endpoint when an application needs to run an ephemeral test and consume its events immediately. A direct execution does **not** create a persistent test case or run in the dashboard.

Authenticate with a private AIVAX API key, send `Accept: text/event-stream`, and keep the key on a trusted backend. Do not expose a private key in browser code or a distributed application bundle.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Evaluate%20Agentic%20Test)

The request accepts the same core evaluation settings as a persistent test. Use `model` for the AI Gateway slug, `goal` for the desired outcome shared with the simulated user and judge, `validation_criteria` for optional judge-only requirements, `minimum_turns` and `allow_user_exit` to control when the simulated user sees its exit option, `judge_start_turn` to schedule judge evaluation, `start` for optional initial messages, `resources` for additional `Text` or `RemoteResource` context, and `external_user_id` for an identity forwarded to gateway inference.

Every Server-Sent Events message contains this envelope:

```json
{
  "timestamp": 1786329000000,
  "event": {
    "type": "chat.start",
    "data": {
      "turn_number": 1,
      "max_remaining_turns": 10
    }
  }
}
```

Route messages by `event.type` and concatenate streamed content chunks in order. The examples below show the `event` object inside the SSE envelope. Lifecycle markers (`start_generation`, `end_generation`, `turn_analysis_start`, `turn_analysis_end`) carry an empty object (`data: {}`); reasoning chunks share the `{ "reasoning_content": "..." }` shape. Only content-bearing events are shown in full.

- **`chat.start`** — Starts a turn and reports its number and remaining turn budget.

  ```json
  {
    "type": "chat.start",
    "data": {
      "turn_number": 1,
      "max_remaining_turns": 10
    }
  }
  ```

- **`chat.user_message.start_generation`** — Marks the start of simulated-user message generation (`data: {}`).

- **`chat.user_message.reasoning`** — Streams one reasoning chunk exposed by the simulated-user model. Use this for debugging only.

- **`chat.user_message.content`** — Streams one simulated-user message content chunk. Concatenate consecutive chunks in arrival order.

  ```json
  {
    "type": "chat.user_message.content",
    "data": {
      "content": "Can you explain the refund policy?"
    }
  }
  ```

- **`chat.user_message.end_generation`** — Marks the end of simulated-user message generation (`data: {}`).

- **`chat.user_message.end_conversation`** — Reports a permitted simulated-user exit after `minimum_turns`. This event is not emitted when `allow_user_exit` is disabled.

  ```json
  {
    "type": "chat.user_message.end_conversation",
    "data": {
      "reason": "simulated_user_ended_conversation"
    }
  }
  ```

- **`chat.assistant_message.start_generation`** — Marks the start of the selected gateway response (`data: {}`).

- **`chat.assistant_message.reasoning`** — Streams one reasoning chunk exposed by the gateway model (same `reasoning_content` shape).

- **`chat.assistant_message.refusal`** — Reports a refusal returned by the gateway model.

- **`chat.assistant_message.tool_call`** — Reports an assistant tool call, including its ID, name, and arguments.

- **`chat.assistant_message.tool_result`** — Reports a tool result, including the associated call ID, name, and content.

- **`chat.assistant_message.content`** — Streams one assistant response content chunk. Concatenate consecutive chunks in arrival order.

  ```json
  {
    "type": "chat.assistant_message.content",
    "data": {
      "content": "Refunds are available within 30 days."
    }
  }
  ```

- **`chat.assistant_message.end_generation`** — Marks the end of the selected gateway response (`data: {}`).

- **`chat.judge.turn_analysis_start`** — Marks the start of an evaluation against the goal and any judge-only validation criteria (`data: {}`).

- **`chat.judge.turn_analysis_result_ready`** — Returns the judge reasoning, normalized score, current state, trajectory measurements, and continuation decision. `score` ranges from `0.001` to `0.999`; `pass` is false only after a persistent loss is established.

  ```json
  {
    "type": "chat.judge.turn_analysis_result_ready",
    "data": {
      "result": {
        "reasoning": "The response satisfied the requested outcome and validation criteria.",
        "score": 0.92,
        "pass": true,
        "should_continue": false,
        "state": "success",
        "turn_delta": 0.84,
        "conversation_delta": 1.0,
        "loss_streak": 0,
        "required_loss_streak": 2
      }
    }
  }
  ```

- **`chat.judge.turn_analysis_end`** — Marks the end of the current turn evaluation (`data: {}`).

- **`usage_updated`** — Reports prompt, cached prompt, and completion token usage. `role` is `user`, `assistant`, or `judge` depending on the inference that produced the usage.

  ```json
  {
    "type": "usage_updated",
    "data": {
      "role": "judge",
      "usage": {
        "prompt_tokens": 1240,
        "cached_prompt_tokens": 320,
        "completion_tokens": 180
      }
    }
  }
  ```

- **`unhandled_error`** — Reports an inference error, its scope, and whether the operation will be retried. `scope` is `user_inference`, `gateway_inference`, or `judge_analysis`.

  ```json
  {
    "type": "unhandled_error",
    "data": {
      "error": "The inference provider is temporarily unavailable.",
      "scope": "gateway_inference",
      "will_retry": true
    }
  }
  ```

- **`chat.validation.end`** — Reports the final outcome and closes the evaluation. The legacy event name is preserved for compatibility. `score` is included when the final outcome follows a judge evaluation, but may be absent when the turn budget is exhausted.

  ```json
  {
    "type": "chat.validation.end",
    "data": {
      "outcome": "success",
      "reason": "baseline_reached",
      "state": "success",
      "turn_number": 3,
      "score": 0.92,
      "conversation_delta": 1.0,
      "loss_streak": 0
    }
  }
  ```

The judge state can be `active`, `at_risk`, `success`, or `loss`. A weak turn does not immediately fail a recoverable conversation: the low score and cumulative trajectory must remain at or below the configured loss threshold for the required consecutive evaluations.

Final outcomes are:

| Outcome | Meaning |
| --- | --- |
| `success` | The judge reached `base_threshold`. A permitted simulated-user exit triggers judging but does not itself pass the test; without the threshold the outcome is `incomplete`. |
| `loss` | The score and cumulative trajectory remained at or below `loss_threshold` for the required consecutive judged turns. |
| `incomplete` | The conversation exhausted `max_turns` without reaching success or a persistent loss. |
| `interrupted` | A validation rule stopped the evaluation before it completed. |

A missing or invalid key returns `401 Unauthorized`; a public API key returns `403 Forbidden`; insufficient balance returns `402 Payment Required`; and malformed fields, unavailable gateway slugs, or invalid threshold combinations return `400 Bad Request`. An inference failure can instead arrive as an SSE event after streaming begins.

To investigate an inference failure, review the [AI Gateway configuration](https://docs.aivax.net/docs/inference/ai-gateway.md) used by the test.

---

Source: https://docs.aivax.net/docs/inference/voice-session.html

# Voice Session

Voice Session is AIVAX's low-latency, stateful voice API. It keeps the authenticated AIVAX WebSocket endpoint while connecting to a realtime inference service. Events follow the OpenAI-compatible GA Realtime JSON protocol after the connection is upgraded.

Use Voice Session for natural spoken conversations with streaming audio, assistant speech transcripts, server voice activity detection (VAD), interruptions, and tool calls. Use [Audio Transcriptions](https://docs.aivax.net/docs/generations/audio-transcriptions.md) for transcription-only workloads or [Speech Generation](https://docs.aivax.net/docs/generations/speech.md) when the text to synthesize is already known.

## Connect securely

Open a WebSocket upgrade request to:

```text
wss://inference.aivax.net/api/v1/voice-session
```

Authenticate the upgrade with a **private** account API key:

```http
Authorization: Bearer <AIVAX_API_KEY>
```

The `?api-key=<AIVAX_API_KEY>` query parameter is also accepted when the WebSocket client cannot set headers, but headers are preferred because URLs are commonly retained in logs and monitoring systems. Public API keys cannot open Voice Sessions.

> [!WARNING]
> Never place a private API key in browser JavaScript, a mobile application bundle, or a browser-visible WebSocket URL. Browsers also cannot add an `Authorization` header through the native `WebSocket` constructor. Terminate the end-user connection at your backend, then let that trusted backend open and relay the authenticated AIVAX WebSocket.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Open%20voice%20session)

## Migrate to GA Realtime events

After the WebSocket upgrade, exchange OpenAI-compatible GA Realtime JSON events directly. Do not send the legacy `session_start` envelope or depend on the former AIVAX-specific STT, TTS, WAV-segment, or playback-confirmation events.

The main migration changes are:

| Legacy integration | GA Realtime integration |
| --- | --- |
| Custom session-start message | `session.update` |
| Custom STT and VAD events | Native `input_audio_buffer.speech_*` events; caller transcription is currently unavailable |
| WAV response segments | Base64 PCM in `response.output_audio.delta` |
| Custom playback confirmation | Native cancel and conversation-item truncation events |
| Custom tool-call envelopes | Native function-call items and function-call outputs |
| Flat per-minute charge | Selected model's text, audio, and image token rates |

Use the `type` property to route every event. Audio remains base64-encoded inside JSON text messages; do not send binary WebSocket frames.

## Configure the session

Send `session.update` as the first client event. AIVAX supports the GA Realtime session fields and adds two optional selectors:

- `gateway`: an AI Gateway slug available to the authenticated account;
- `model`: `gpt-realtime-2.1` or `gpt-realtime-2.1-mini`.

The selector supplied in the event determines the AIVAX configuration used for the session. Configure the output voice at `session.audio.output.voice` and reasoning effort at `session.reasoning.effort`.

```json
{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "gateway": "<GATEWAY_SLUG>",
    "model": "gpt-realtime-2.1",
    "instructions": "Help the caller complete the requested task.",
    "audio": {
      "input": {
        "format": {
          "type": "audio/pcm",
          "rate": 24000
        },
        "turn_detection": {
          "type": "server_vad"
        }
      },
      "output": {
        "format": {
          "type": "audio/pcm",
          "rate": 24000
        },
        "voice": "marin"
      }
    },
    "reasoning": {
      "effort": "low"
    }
  }
}
```

Choose the model based on the experience you need:

- `gpt-realtime-2.1` for the highest-quality realtime conversations;
- `gpt-realtime-2.1-mini` for lower-cost, lighter-weight realtime workloads.

You can also provide standard GA Realtime instructions, tools, and turn-detection settings in the same update. Input-transcription sessions are not currently supported, so do not configure `audio.input.transcription` or depend on `conversation.item.input_audio_transcription.*` events. Wait for the server's session confirmation event before treating the configuration as active. A voice cannot be changed after the session has already produced audio; reconnect to select a different voice.

## Stream microphone audio

Capture raw mono signed 16-bit little-endian PCM (`s16le`) at 24,000 Hz, base64-encode each chunk in chronological byte order, and append it to the input buffer:

```json
{
  "type": "input_audio_buffer.append",
  "audio": "<BASE64_PCM_24KHZ_AUDIO>"
}
```

Send small chunks continuously for low latency. Do not add WAV, RIFF, or other container headers to each chunk. If automatic turn detection is enabled, server VAD determines when the user starts and stops speaking and creates responses according to the session configuration.

If turn detection is disabled, control the turn explicitly with the native buffer events:

1. Send one or more `input_audio_buffer.append` events.
2. Send `input_audio_buffer.commit` when the utterance is complete.
3. Send `response.create` to request the assistant response.

Use `input_audio_buffer.clear` to discard buffered audio that should not become a conversation turn.

## Receive transcripts and audio

Handle the native GA Realtime event stream rather than assuming one fixed sequence. In particular:

- `input_audio_buffer.speech_started` and `input_audio_buffer.speech_stopped` report server-VAD boundaries;
- `conversation.item.input_audio_transcription.*` events are not currently emitted because input-transcription sessions are not supported;
- `response.output_audio_transcript.delta` and `response.output_audio_transcript.done` provide the assistant's spoken transcript;
- `response.output_audio.delta` carries a base64 PCM audio chunk;
- `response.output_audio.done` marks the end of an output-audio stream;
- `response.done` reports the final response status and usage;
- `error` reports a request or session error.

Decode each `response.output_audio.delta` value as raw mono signed 16-bit little-endian PCM at 24,000 Hz and queue the samples for gapless playback:

```json
{
  "type": "response.output_audio.delta",
  "response_id": "<RESPONSE_ID>",
  "item_id": "<ITEM_ID>",
  "output_index": 0,
  "content_index": 0,
  "delta": "<BASE64_PCM_24KHZ_AUDIO>"
}
```

Event fields can evolve with the GA Realtime protocol. Route by `type`, retain the identifiers needed to correlate responses and items, and ignore unknown event types or fields that your client does not use.

## Handle interruptions

When the caller speaks over the assistant:

1. Stop or duck local playback when the confirmed `input_audio_buffer.speech_started` event arrives.
2. Send `response.cancel` if a response is still being generated.
3. Send the native conversation-item truncation event with the assistant item identifier and the amount of audio that was actually played.

Cancellation stops generation; truncation keeps the server conversation aligned with what the caller heard. Track played audio duration in the client rather than assuming every received chunk reached the speaker. Treat already-finished response races as normal and make cancel handling idempotent.

## Use tools

Declare client tools through the standard GA Realtime `tools` session field. Handle `response.function_call_arguments.done` using its `response_id`, `call_id`, `name`, and JSON-encoded `arguments`. Execute each requested function, add one native `function_call_output` conversation item per `call_id`, then send a single `response.create` after the originating `response.done` and every call from that response has an output.

Tools configured as AIVAX server `InternalFunctions` execute within AIVAX and require no client action. Client-defined tools continue to be returned normally and remain the client's responsibility.

Your tool handler should validate arguments, preserve call identifiers, enforce timeouts, and return a useful error result when execution fails. Do not wait for a client tool result that is being executed as an AIVAX server function.

## Billing and limits

For current pricing, availability, and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md).

## Disconnect gracefully

A Voice Session lasts only for the active WebSocket. It is not resumable after disconnecting.

For a normal shutdown:

1. Stop microphone capture and stop sending audio.
2. Let any response or required tool-result exchange finish, or cancel it explicitly.
3. Stop and drain local playback as appropriate for the product experience.
4. Close the WebSocket with a normal close code.
5. Release audio devices, playback queues, pending calls, and session state.

Handle peer closure, network loss, authentication failure, model errors, and server shutdown as terminal session outcomes. Reconnecting starts a new session: open a new authenticated WebSocket, send a new `session.update`, and rebuild any application context your experience requires. Use bounded retry with backoff for transient failures, but do not automatically retry authentication or configuration errors.

---

Source: https://docs.aivax.net/docs/inference/pipelines.html

# AI Pipelines

AI Gateway pipelines are the processing steps AIVAX applies before and during inference. They can add context, rewrite queries, expose tools, moderate input, route models, truncate conversations, and call external workers.

Most pipelines are configured in the gateway parameters. Request-level options can override some inference parameters on direct `chat/completions` calls.

## RAG

RAG links [collections](https://docs.aivax.net/docs/rag/collections.md) to an AI Gateway. The gateway controls:

- Collections included in retrieval.
- Maximum number of retrieved documents.
- Minimum score.
- Reranker name.
- Whether chunk references are included.
- Query strategy.

When a gateway has knowledge collections and the latest user message contains text, AIVAX can retrieve matching documents before the model call. For injection strategies, the retrieved context is inserted at the beginning of the last user message. If a linked collection has its own context text, that collection context is added to system instructions.

Query strategies:

- `Plain`: Uses the latest user message as the search query.
- `Concatenate`: Joins the latest configured number of user messages line by line and searches with the combined text.
- `UserRewrite`: Rewrites recent user messages into one or more search queries using a resolver model.
- `FullRewrite`: Rewrites recent user and assistant messages into one or more search queries using a resolver model.
- `QueryFunction`: Adds a query function to the model. The model decides when to search the linked collections, and search results are returned as tool responses.

Rewrite strategies add resolver-model cost (see [Pricing](https://docs.aivax.net/docs/pricing.md)) and latency. They are useful when users ask follow-up questions such as "what about this case?" because the resolver can turn the recent conversation into a clearer search query.

Defining many RAG results increases input token usage and can increase final inference cost (see [Pricing](https://docs.aivax.net/docs/pricing.md)). Start with a small result count and raise it only when the model lacks enough evidence.

## Instructions

Instruction settings shape the provider-facing prompt:

- **System instructions**: Added to the system instruction set.
- **Remote system instruction sources**: Fetched from configured URLs and added to the system instruction set.
- **User prompt template**: Replaces `{prompt}` with each user message's text before sending it to the model.
- **Assistant prefill**: Adds initial assistant content before generation when the model supports prefill.

Remote instruction sources are fetched as text with a 10 MB maximum response size. Their cache duration is configurable; the default is 600 seconds.

Some models do not support assistant prefill, temperature, stop sequences, or reasoning effort. Integrated model validation rejects incompatible gateway settings when those limitations are known.

## Skills

Skills are on-demand instruction packs available to the model. When a gateway enables skills, AIVAX loads the account skills configured on the gateway and can expose skill-related built-in functions.

Read more about [skills](https://docs.aivax.net/docs/features/skills.md).

## Multimodal pre-processing

Multimodal pre-processing converts selected media content into text before the main model call. The available flags are `Image`, `Audio`, `Video`, `File`, `OtherFiles`, and `All`.

Use pre-processing when the main model is text-first or when you want AIVAX to normalize media into textual context. For direct multimodal models, send the original media without pre-processing so the model can inspect it directly.

Media descriptions are cached by content hash for reuse.

## Parameterization

The parameterization pipeline configures model request options such as:

- `temperature`
- `top_p`
- `presence_penalty`
- `frequency_penalty`
- `stop`
- `max_completion_tokens`
- `reasoning_effort`
- `verbosity`
- `seed`

Request-level values can override gateway values when the endpoint supports the parameter. Some integrated models reject specific parameters, and BYOK providers may have their own restrictions.

## Context truncation

The context truncation pipeline uses an approximate token count. When `ContextMaximumSize` is set and the conversation exceeds the limit, the gateway follows `ContextOverflowAction`:

- `Throw`: Return an error instead of calling the model.
- `Truncate`: Remove older non-system messages until the conversation fits.

Truncation preserves system messages and keeps at least one user message when possible. If the remaining user message still exceeds the limit, the request fails with a message-size error.

On lower plans, the effective input context may be capped even when a larger context is configured; see [Plans and limits](https://docs.aivax.net/docs/limits.md#plan-limits).

## Tool message truncation

`ToolContextCount` controls how many recent tool response messages keep their original content. When it is set to a value greater than zero, older tool messages remain in the conversation but their content is replaced with:

```text
[tool response truncated - call this tool again]
```

This can reduce context usage in long agentic conversations. It can also hurt chains where an old tool result remains important, so use it only when the model can safely call the tool again.

## Server-side tools

Server-side tools are internal functions executed by AIVAX during inference. They can come from:

- Built-in tools.
- Protocol functions.
- Remote protocol function sources.
- MCP sources.
- QueryFunction RAG.
- Skills.
- Optional bash environment.

Server-side tool events can be streamed to clients as `servertool` updates.

## Built-in tools

Built-in tools can be configured on a gateway or supplied per request with `builtin_tools`. Available built-in tool flags include:

- `DateTime` — current date and time through `get_date_time`; configure `dateTimeTimeZone` in built-in tool options (default: `America/Los_Angeles`, Pacific Time).

- `WebSearch`
- `AdvancedWebUsage` (disabled; returns an unavailable response. See [Changelogs](https://docs.aivax.net/docs/changelogs.md).)
- `OpenUrl`
- `Code`
- `Request`
- `Calendar`
- `Remember`
- `GenerateWebPage`
- `GenerateDocument`
- `XPostsSearch`
- `ImageGeneration`

See [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md).

## MCP and protocol functions

Tools listed by an MCP source become available to the model with their declared schemas. MCP tool results can include text, images, and audio; media results are attached to the conversation as additional messages when supported.

Protocol functions expose HTTP callbacks or AIVAX callback URLs as model-callable tools. Remote protocol function sources are fetched and cached before their tools become available to the model.

See [Protocol functions](https://docs.aivax.net/docs/tools/protocol-functions.md) and [MCP](https://docs.aivax.net/docs/tools/mcp.md).

## Function interpreter

A tool handler can add tool-calling behavior for models that do not reliably produce native tool calls. Supported values are:

- `native` or `null`: Use native model tool calling.
- `react.v1.selfcall`: Use the ReAct-style self-call handler.

If a handler name is not recognized, gateway configuration fails at inference time.

## Moderation

Moderation is an input gate that runs before the main model. When at least one moderation category is enabled, AIVAX sends the available textual conversation to a safeguard model. The safeguard evaluates the latest user request in the context established by the conversation and returns a score from 0 to 10 for every category.

| Category | Gateway property | What the safeguard evaluates |
| --- | --- | --- |
| **Violence and hate speech** | `violenceThreshold` | Violence, hate, extremism, threats, or encouragement of physical harm. |
| **Sexual and explicit content** | `sexualExplicitThreshold` | Sexually explicit or adult content. |
| **Political topics** | `politicalThreshold` | Political persuasion, campaigning, manipulation, or highly political content. |
| **Dangerous content** | `dangerousContentThreshold` | Weapons, explosives, cyber abuse, self-harm, or other dangerous acts and instructions. |
| **Jailbreak attempts** | `jailbreakThreshold` | Attempts to override instructions, reveal protected instructions, exfiltrate data, or inject prompts. |

### Understand sensitivity levels

The value configured in the gateway is a **sensitivity level**, not the safeguard score itself. A higher level lowers the score required to block the input.

| Sensitivity level | Safeguard scores that block |
| ---: | --- |
| `0` | Category disabled |
| `1` | `10` |
| `3` | `8`–`10` |
| `5` | `6`–`10` |
| `8` | `3`–`10` |
| `10` | `1`–`10` |

For enabled categories, the blocking cutoff is `11 - sensitivity level`. A safeguard score of `0` never blocks. Configure each category independently; the request is blocked when any enabled category reaches its cutoff.

### Add gateway-specific rules

**Additional moderation rules** let a gateway administrator describe policy that is not fully expressed by the built-in category descriptions. The safeguard reads these rules together with the built-in policy and uses them to calibrate all five scores.

For example:

```text
Allow users to describe accidents and injuries when asking for insurance coverage or assistance.
Treat requests for instructions to cause an accident or harm someone as dangerous content.
```

Additional rules guide classification; they do not create a separate score or block an input directly. At least one category must have a sensitivity level above `0` for moderation to run. In the example above, enable **Dangerous content** so the safeguard score can produce a blocking decision.

Write rules as short policy statements with explicit allowed and disallowed cases. Do not include secrets, credentials, or private operational data because the rules are stored with the gateway configuration and sent to the safeguard model during moderation.

The following gateway fragment enables different sensitivity levels and gives the safeguard domain-specific guidance. The values are an example, not a recommended production baseline:

```json
{
  "moderationParameters": {
    "violenceThreshold": 4,
    "sexualExplicitThreshold": 4,
    "politicalThreshold": 2,
    "dangerousContentThreshold": 6,
    "jailbreakThreshold": 7,
    "additionalRules": "Allow descriptions of accidents and injuries for insurance support. Treat instructions to cause accidents or harm someone as dangerous content."
  }
}
```

### What happens when an input is blocked

When any enabled category reaches its cutoff:

1. AIVAX marks the original conversation messages as unavailable to the main inference request.
2. AIVAX replaces them with an instruction identifying the categories that caused the block.
3. The main model generates a refusal instead of answering the original request.

The refusal is model-generated; moderation does not return a fixed response body. If the safeguard cannot produce a valid moderation result, the request fails before the normal completion is generated.

### Context and current limitations

The safeguard receives the available conversation history, not only the latest message. Message roles and text are preserved as untrusted serialized conversation data so instructions inside the conversation cannot replace the safeguard policy. If the conversation exceeds the safeguard context window, older context can be truncated.

Moderation currently applies only to input text:

- Generated output is not moderated.
- Images, audio, video, and file contents are not analyzed. The safeguard receives only a marker indicating that media was present.
- Tool or worker authorization still requires application-level policy; moderation is not an authorization mechanism.
- Moderation adds a safeguard inference before the main inference, which adds latency and billable moderation usage.

Use moderation for broad safety policy. Use workers when the decision depends on external identity, account state, or business-specific policy.

## Workers

Configure worker events and endpoint details in the gateway parameters; implement the event behavior at your external endpoint. See [AI Workers](https://docs.aivax.net/docs/inference/workers.md).

---

Source: https://docs.aivax.net/docs/inference/structured-responses.html

# Structured Responses

AIVAX can produce structured JSON through two paths:

- `response_schema`: AIVAX validates the final model output against a JSON Schema and retries with validation feedback when the output is invalid. This is the JSON Healing path.
- `response_format`: AIVAX passes a native OpenAI-compatible response format to the provider, or applies healing when `healing_options` is present or automatic JSON Healing is enabled on the account.

Use structured responses when another system will consume the output and free-form text would be fragile.

## How JSON Healing works

When `response_schema` is present, AIVAX adds schema instructions to the model request. After the model generates an answer, AIVAX tries to extract JSON from:

- The complete generated text.
- Common repaired variants, such as missing opening or closing braces.
- JSON code blocks found in the generated text.

If the extracted JSON does not validate against the schema, AIVAX adds a feedback message with the validation errors and asks the model to generate the JSON again. This continues until a valid JSON value is generated or the configured attempt limit is reached.

JSON Healing improves reliability, but it is not an absolute guarantee. If the model repeatedly fails the schema, the request can fail after the attempt limit.

## Choosing a mode

Use `response_schema` when AIVAX should own validation and repair. This is the safest mode for models that do not natively support structured output, for responses that use tools before producing JSON, and for systems that cannot tolerate malformed JSON.

Use `response_format` with `type: "json_schema"` when the provider model should handle the structured output natively. AIVAX will still use the schema internally, and healing is applied when `response_format.json_schema.healing_options` is provided or the account has automatic JSON Healing enabled.

Use `json_only: true` when the HTTP response body should be only the final JSON. This removes the normal chat completion envelope, choices, usage, and generation metadata from the response body.

For a troubleshooting checklist covering refusals, incomplete streams, and native enforcement versus healing, read [How to fix invalid JSON from an LLM API with structured outputs](https://aivax.net/blog/structured-output-healing-boundary/).

## Basic example

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Inference%20(chat%20completions))

<div class="request-item post">
    <span>POST</span>
    <span>
        /v1/chat/completions
    </span>
</div>

```json
{
    "model": "@google/gemini-2.5-flash",
    "prompt": "Search for recent news about electric vehicles.",
    "stream": true,
    "builtin_tools": {
        "tools": [
            "WebSearch"
        ],
        "options": {
            "web_search_mode": "full"
        }
    },
    "response_schema": {
        "type": "object",
        "properties": {
            "news": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "title": {
                            "type": "string",
                            "description": "News title"
                        },
                        "summary": {
                            "type": "string",
                            "description": "News summary"
                        }
                    },
                    "required": ["title", "summary"]
                }
            }
        },
        "required": ["news"]
    }
}
```

`builtin_tools` is optional. If you enable tools, the selected model must support tool calling or the gateway must provide a tool handler.

## Native structured output

Use `response_format` when you want to use the provider's native JSON Schema support:

```json
{
    "model": "@openai/gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "List 3 European capitals."
        }
    ],
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "capitals": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "city": { "type": "string" },
                                "country": { "type": "string" }
                            },
                            "required": ["city", "country"]
                        }
                    }
                },
                "required": ["capitals"]
            }
        }
    }
}
```

If the selected integrated model has strict JSON support, AIVAX can forward the schema using the provider's JSON Schema response format.

## Enabling healing in `response_format`

You can explicitly enable JSON Healing inside `response_format.json_schema`:

```json
{
    "model": "@openai/gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "Return a short status object."
        }
    ],
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "status": { "type": "string" },
                    "message": { "type": "string" }
                },
                "required": ["status", "message"]
            },
            "healing_options": {
                "max_attempts": 5
            }
        }
    }
}
```

`max_attempts` must be between 1 and 10. Each retry is another model generation attempt and can increase cost and latency.

If healing frequently reaches the attempt limit, adjust the instruction and schema before raising the limit. Common causes are overly rigid schemas, missing `required` fields, vague prompts, noisy tool results, or a model that is too small for the task.

## `json_only` mode

Set `json_only: true` when the client should receive only the generated JSON:

```json
{
    "model": "@openai/gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "List 3 European capitals."
        }
    ],
    "response_schema": {
        "type": "object",
        "properties": {
            "capitals": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "city": { "type": "string" },
                        "country": { "type": "string" }
                    },
                    "required": ["city", "country"]
                }
            }
        },
        "required": ["capitals"]
    },
    "json_only": true
}
```

With `stream: false`, the HTTP response body is the final JSON with `Content-Type: application/json`. With `stream: true`, AIVAX sends the complete JSON as one SSE data event and then sends `[DONE]`.

## Supported schema features

AIVAX validates the generated JSON with JSON Schema. The documented supported features are:

- `string`: `minLength`, `maxLength`, `pattern`, `format`, and `enum`.
- `number` and `integer`: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, and `multipleOf`.
- `array`: `items`, `uniqueItems`, `minItems`, and `maxItems`.
- `object`: `properties` and `required`.
- `boolean` and `bool`.
- `null`.
- Multiple types, for example `"type": ["string", "number"]`.

Use `required` for fields your application must receive. Use explicit `items` for arrays. Use `enum`, `format`, and length or pattern restrictions when the accepted values are known.

## Practical patterns

For extraction, tell the model which fields must be inferred, which fields should be `null` when missing, and which fields must not be invented.

For classification, use `enum` for the final label and add a short `reason` field when humans need to audit the decision.

For tool-backed JSON, allow the model to use tools before producing the JSON and include source fields when the application needs provenance.

For database writes or external API calls, validate the JSON again in your application. AIVAX validates the JSON shape, but your application remains responsible for business rules such as permissions, valid IDs, date ranges, and account-specific constraints.

---

Source: https://docs.aivax.net/docs/inference/workers.html

# AI Workers

AI Gateway workers are HTTP hooks that let an external service control gateway execution at runtime. A worker can allow an event, stop it, rewrite context, add instructions or tools, or replace a server-side tool result.

Use workers when a rule must be decided outside the prompt. Common cases include subscription checks, CRM enrichment, tenant-specific policy, audit logging, dynamic tool blocking, and replacing a model-visible tool result with data from an internal system.

Workers run in the inference critical path. Every worker event adds an HTTP request before the gateway can continue, so the endpoint must respond quickly and predictably.

## Request format

When a configured worker event fires, AIVAX sends a `POST` request to the gateway's worker URL.

```json
{
    "gatewayId": "your-gateway-id",
    "moment": "2025-12-29T17:04:39",
    "event": {
        "name": "message.received",
        "data": {
            "messages": [
                {
                    "role": "system",
                    "content": "User local date is Monday, December 29, 2025 (timezone is America/Sao_Paulo)"
                },
                {
                    "role": "user",
                    "content": "Good morning"
                }
            ],
            "origin": "ChatCompletionsApi",
            "externalUserId": "customer-123",
            "metadata": {}
        }
    }
}
```

The exact `event.data` shape depends on the event. Always validate `gatewayId` when one endpoint serves more than one gateway.

## Authentication

When the account has a hook key, AIVAX sends `X-Request-Nonce`. The nonce is a BCrypt hash derived from the account hook key. Validate this header before trusting the body, especially when the worker releases private data, changes context, or authorizes tool usage.

Treat `externalUserId`, `metadata`, messages, and tool arguments as untrusted input.

## Response behavior

After sending the request, AIVAX handles the worker response as follows:

| Response | Behavior |
|---|---|
| `Content-Type: application/json+worker-action` | Execute the action described in the JSON body. |
| `2xx` without `application/json+worker-action` | Continue normally. |
| Non-OK response without `application/json+worker-action` | Stop the event. |

If the worker request fails with an HTTP request exception, AIVAX logs the failure and stops the event.

Choose fail-open or fail-closed behavior intentionally. Return `2xx` when enrichment is optional. Return a non-OK response when authorization, compliance, or business policy cannot fail open.

## `message.received`

The `message.received` event fires after the gateway prepares the incoming message context and before the model call.

```json
{
    "name": "message.received",
    "data": {
        "messages": [],
        "origin": "ChatCompletionsApi",
        "externalUserId": "customer-123",
        "metadata": {}
    }
}
```

To modify the context, return `Content-Type: application/json+worker-action` with `type: "message.received.response"`:

```json
{
    "type": "message.received.response",
    "data": {
        "rewrites": [
            {
                "type": "add-system",
                "message": "Answer in formal English."
            }
        ]
    }
}
```

Available rewrite actions:

| Action | Description | Parameters |
|---|---|---|
| `clear` | Removes context elements. | `argument`: `messages`, `meta`, `system`, `tools`, `skills`, `all`, or omitted. |
| `add-message` | Adds a message to the conversation. | `message`: OpenAI-compatible message object. |
| `remove-message` | Removes a message by index. | `index`: zero-based message index. |
| `add-system` | Adds a system instruction. | `message`: instruction text. |
| `add-tool` | Adds an OpenAI-compatible tool definition. | `tool`: tool JSON object. |
| `add-protocol-tool` | Adds a [protocol function](https://docs.aivax.net/docs/tools/protocol-functions.md). | `tool`: protocol function definition. |
| `add-mcp-source` | Adds the tools discovered from an [MCP](https://docs.aivax.net/docs/tools/mcp.md) source to the context. | `source`: MCP source object with `url`, `headers`, `name`, and/or `cacheDuration`. |

### Replace the user context

```json
{
    "type": "message.received.response",
    "data": {
        "rewrites": [
            {
                "type": "clear"
            },
            {
                "type": "add-message",
                "message": {
                    "role": "user",
                    "content": "The original message was removed by an external policy check. Tell the user they need an active subscription to continue."
                }
            }
        ]
    }
}
```

### Remove a message

```json
{
    "type": "message.received.response",
    "data": {
        "rewrites": [
            {
                "type": "remove-message",
                "index": 0
            }
        ]
    }
}
```

### Add a temporary MCP source

```json
{
    "type": "message.received.response",
    "data": {
        "rewrites": [
            {
                "type": "add-mcp-source",
                "source": {
                    "name": "Internal CRM",
                    "url": "https://crm.example.com/mcp",
                    "headers": {
                        "Authorization": "Bearer <INTERNAL_CRM_TOKEN>"
                    },
                    "cacheDuration": 600
                }
            }
        ]
    }
}
```

Use `add-mcp-source` when the tool list needs to depend on the message, user, channel, or an external policy. AIVAX lists the tools from the MCP server, converts each schema into a model-callable function, and makes those tools available only for that inference. For permanent sources, configure MCP directly in the AI Gateway.

## `tool.called`

The `tool.called` event fires before AIVAX executes an internal server-side tool.

```json
{
    "name": "tool.called",
    "data": {
        "toolName": "check_order",
        "toolArguments": {
            "order_id": "A123"
        },
        "origin": "ChatCompletionsApi",
        "externalUserId": "customer-123",
        "metadata": {}
    }
}
```

Return a non-OK response to block the tool call. Return `2xx` to let AIVAX execute the tool normally.

To replace the tool result, return `Content-Type: application/json+worker-action` with `type: "tool.called.response"`:

```json
{
    "type": "tool.called.response",
    "data": {
        "result": "Order A123 is paid and scheduled for delivery tomorrow.",
        "messages": []
    }
}
```

Fields of `data`:

| Field | Description |
|---|---|
| `result` | Textual content injected as the tool result. |
| `messages` | Optional additional OpenAI-format messages attached to the conversation context. |

When `tool.called.response` is returned, AIVAX uses the worker-provided result instead of executing the default tool handler.

## Example: blocking unauthorized users

The example below shows a Cloudflare Worker that blocks a gateway call when the external user is not allowed.

```js
export default {
  async fetch(request, env) {
    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 });
    }

    const body = await request.json();

    if (body.gatewayId !== env.CHECKING_GATEWAY_ID) {
      return new Response();
    }

    if (body.event?.name !== "message.received") {
      return new Response();
    }

    const externalUserId = body.event.data.externalUserId;
    const allowedUsers = new Set((env.ALLOWED_USERS || "").split(","));

    if (!allowedUsers.has(externalUserId)) {
      return new Response("User is not authorized", { status: 403 });
    }

    return new Response();
  }
};
```

## Example: replacing a tool with an internal system

Use `tool.called` when the model should see a tool result, but the actual data should come from your system.

```js
export default {
  async fetch(request, env) {
    const body = await request.json();

    if (body.event?.name !== "tool.called") {
      return new Response();
    }

    const { toolName, toolArguments, externalUserId } = body.event.data;

    if (toolName !== "check_order") {
      return new Response();
    }

    const orderId = toolArguments?.order_id;
    const orderResponse = await fetch(`${env.INTERNAL_API}/orders/${orderId}`, {
      headers: {
        "Authorization": `Bearer ${env.INTERNAL_API_TOKEN}`
      }
    });

    if (!orderResponse.ok) {
      return new Response(JSON.stringify({
        type: "tool.called.response",
        data: {
          result: `The order ${orderId} could not be retrieved for user ${externalUserId}. Ask the user to confirm the order number.`
        }
      }), {
        headers: {
          "Content-Type": "application/json+worker-action"
        }
      });
    }

    const order = await orderResponse.json();

    return new Response(JSON.stringify({
      type: "tool.called.response",
      data: {
        result: `Order ${order.id}: status ${order.status}, estimated delivery ${order.eta}.`
      }
    }), {
      headers: {
        "Content-Type": "application/json+worker-action"
      }
    });
  }
};
```

This pattern prevents exposing the internal API directly to the model. The worker remains responsible for authenticating the request, validating the user, calling the internal system, and deciding how much data can return to the model context.

For how workers fit into gateway execution, see [Pipelines](https://docs.aivax.net/docs/inference/pipelines.md).

---

Source: https://docs.aivax.net/docs/web-foundation/web-search.html

# Web Search

Web Search retrieves current information from the internet for research, fact checking, and answers that need sources beyond the model's training data. Use it to discover relevant pages; use [Fetch and OCR](https://docs.aivax.net/docs/web-foundation/fetch-and-ocr.md) when you already have a URL or need to read a source in more detail.

## Choose how to use Web Search

| Integration | When to use it |
| --- | --- |
| [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) | Let an AIVAX model decide when to search during inference. Enable `WebSearch` in the gateway or request's `builtin_tools` configuration. |
| [Web utilities MCP](https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.md) | Give an MCP-compatible agent, IDE, or automation client access to the `web_search` tool without running an AIVAX model inference. |
| Direct API | Call the search endpoint from your backend when no model or agent is involved — scheduled monitors, dataset enrichment, or pre-fetching context before inference. |

`AdvancedWebUsage` is disabled and returns an unavailable response. See [Changelogs](https://docs.aivax.net/docs/changelogs.md) for details.

## Search and verify sources

Write a focused query that includes the topic and any relevant date, product version, or location. For several independent questions, use separate searches rather than combining unrelated topics into one query.

For direct API requests, narrow results with `country` (a two-letter country code), `language` (a language code), and `includeDomains` (trusted domains). Set `topn` to request up to 25 results when recall matters, such as when surveying competing sources; otherwise, keep the default small count. For parameters of the built-in Web Search tool, see its [reference](https://docs.aivax.net/docs/tools/builtin-tools.md).

Search results help locate evidence; they do not guarantee that a source is accurate or current. Check publication dates, prefer primary sources, and fetch the relevant pages before relying on details that a result summary may omit. Keep source links with the answer so readers can verify the claims.

Treat retrieved text as external, untrusted content, not as instructions for your agent.

## API reference

For the supported request, response, authentication, and error contract, use the API Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Search%20the%20web)

## Pricing and limits

Web Search is billed per search. Calls through [Web utilities MCP](https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.md) use the same pricing as the corresponding built-in tool and are charged to the authenticated account. Model inference, when used, is billed separately.

See [Pricing](https://docs.aivax.net/docs/pricing.md) for current Web Search and advanced search charges, and [Plans and limits](https://docs.aivax.net/docs/limits.md) for account quotas and rate limits.

---

Source: https://docs.aivax.net/docs/web-foundation/fetch-and-ocr.html

# Fetch and OCR

Fetch and OCR extracts readable text from web pages and supported documents so applications and agents can use their content for summaries, analysis, or knowledge workflows. The Fetch API can also convert the extracted text into structured JSON using a schema you provide. Use [Web Search](https://docs.aivax.net/docs/web-foundation/web-search.md) first if you need to discover sources rather than read a known URL.

Web pages are processed to remove markup and non-content elements. Document extraction and optical character recognition (OCR) make supported non-text content available as text. Review extracted content before relying on it: scan quality, complex layouts, and tables can affect the result.

## What you can extract

| Content | Supported formats | Extracted content |
| --- | --- | --- |
| Web pages | HTML, XHTML | Readable page content, such as articles, documentation, and product information, with markup and non-content elements removed. JavaScript and CSS is rendered before extraction. |
| Plain text and Markdown | TXT, Markdown | The document's text, including existing Markdown formatting. |
| PDFs | PDF | Text from digital documents and OCR text from scanned pages. Mixed PDFs can combine direct text extraction with OCR where needed. |
| Images containing text | PNG, JPEG, WebP, TIFF, BMP | Recognized text from screenshots, scanned documents, receipts, and other images with legible writing. |
| Word-processing documents | DOC, DOCX, ODT, RTF | Document text converted into a readable textual representation. |
| Presentations | PPT, PPTX, ODP | Textual slide content. |
| Spreadsheets and tabular files | XLSX, ODS, CSV | Cell and row content as text, for downstream reading or analysis. |
| E-books | EPUB | The publication's text. |

For example, you can fetch an online article, read a PDF manual, extract text from a receipt image, or turn a spreadsheet into text for an agent to analyze. Provide a URL that returns the actual page or file, not a sharing page that requires a login. When submitting a data URI through the API, declare the content's correct MIME type.

The result is extracted text, not a pixel-perfect copy of the original document. Do not assume that layout, table structure, charts, or embedded images will be reproduced exactly. Spreadsheet extraction does not execute formulas or macros.

## Fetch API vs. Media Descriptions

Use the **Fetch API to extract existing text**. Use [Media Descriptions](https://docs.aivax.net/docs/generations/media-descriptions.md) to **interpret media with AI**, optionally guided by what your application needs to learn from it.

| | Fetch API | Media Descriptions |
| --- | --- | --- |
| Main purpose | Retrieve readable text from pages and documents, using OCR for supported images and scanned PDFs. | Generate descriptions or extract information from images, PDFs, audio, and video using AI. |
| Output | Extracted text, optional schema-guided JSON generated from that text, processing-unit usage, and per-item errors. | Model-generated content focused by your guidance, which can describe visual or audiovisual information beyond the text present in the source. |
| Images and PDFs | Read text, such as the words on a receipt or the paragraphs in a manual. | Describe visual content or interpret a document, such as explaining a diagram or identifying information relevant to a question. |
| Audio and video | Not an audio/video understanding or transcription API. | Analyze audio and video content. For a dedicated speech-to-text workflow, use [Audio Transcriptions](https://docs.aivax.net/docs/generations/audio-transcriptions.md). |
| Billing | Extraction processing units (PUs), with plan-dependent allowances and rates. Optional JSON conversion is billed separately and is not covered by the extraction allowance. | AI usage charges under Media Descriptions pricing; Fetch PU allowances do not replace these charges. |

For a PDF report, choose Fetch when you need its text for indexing or later analysis. Choose Media Descriptions when you need an explanation of its charts or a guided interpretation of the content. For a receipt image, Fetch reads the printed text; Media Descriptions can interpret the receipt according to your extraction guidance.

Neither guarantees perfect results. Fetch can lose text or structure because of OCR and layout limitations. Media Descriptions can omit details or introduce incorrect interpretations because its output is model-generated. Verify consequential details against the original source and compare the billing paths in the pricing section below.

## Choose an integration

| Integration | When to use it |
| --- | --- |
| Fetch API | Your application controls which URLs or inline base64 data URIs to process and needs structured results, processing-unit usage, and per-item errors. |
| [Web utilities MCP](https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.md) | An MCP-compatible client needs to read public URLs through `fetch_url`. This tool accepts one to five URLs per call. |
| [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) | An AIVAX model needs to read a URL during inference. Enable `OpenUrl` and follow the URL Context configuration guide. |

The integrations have different input contracts. In particular, the MCP tool accepts public URLs; use the Fetch API for inline base64 data URIs.

## Fetch content with the API

Authenticate with an AIVAX API key; see [Authentication](https://docs.aivax.net/docs/authentication.md). Supply a non-empty `contents` array containing URLs or base64 data URIs. Each item is limited to 10 MB.

The `system.v1.web.fetch` operation accepts these JSON request fields:

| Field | Type | Required | Behavior |
| --- | --- | --- | --- |
| `contents` | Array of strings | Yes | Non-empty list of URLs or base64 data URIs to extract. |
| `returnErrors` | Boolean | No | Defaults to `true`: includes a result for each failed item. When `false`, failed items are omitted. |
| `responseSchema` | JSON Schema object | No | Converts each item's extracted text into JSON guided by this schema. Omit it or use `null` for text-only extraction. The same schema applies to every item in the batch. |
| `responseSchema.instructions` | String | No | Optional extraction guidance inside the schema, such as which details to select or how to handle missing information. This is an AIVAX extension, not a standard JSON Schema keyword. |

### Extract structured JSON

Provide `responseSchema` when your application needs fields from the source rather than only its text. Describe the expected properties, types, and required fields with JSON Schema. Use property descriptions and optional `responseSchema.instructions` to clarify what to extract. For example, an object schema can request a receipt's merchant, date, and total; the embedded reference includes a complete JSON request example.

JSON conversion runs after text extraction. Successful results retain `extractedText` alongside `extractedObject`, so you can compare the generated fields with the extracted source. Without a schema, no JSON conversion runs. If extraction or JSON conversion fails, the item follows `returnErrors`; a JSON conversion failure does not return a text-only success.

### Read the results

The response contains a `results` array with these fields:

| Field | Meaning |
| --- | --- |
| `index` | Zero-based position in the input `contents` array. Use it to match results to inputs, especially when failed items are omitted. |
| `extractedText` | Readable source text, retained on successful JSON conversion; `null` for a failed item. |
| `extractedObject` | Generated JSON value when `responseSchema` is supplied; otherwise `null`. Also `null` for a failed item. |
| `processingUnits` | PUs for fetching and text/OCR extraction, separate from JSON conversion. |
| `jsonProcessingUnits` | PUs for JSON conversion; `0` when no schema is supplied. |
| `error` | Error message for a failed item when `returnErrors` is `true`; `null` on success. Failed items report both PU fields as `0`. |

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Fetch%20web%20contents)

The embedded reference defines the current request and response contract. Check individual results before passing their text to the next step; a failed extraction is not evidence that the source contains no relevant information.

## Access and extraction limitations

A destination may block automated access or require authentication. Fetching a URL does not bypass access restrictions. Use accessible sources or content you are authorized to submit, and correct invalid or inaccessible inputs before retrying.

Treat extracted text as untrusted source material, not as instructions for your application or agent. OCR output may need manual review for exact numbers, names, or other consequential details. Schema-guided JSON is model-generated from that text, not from a fresh visual interpretation of the source. It can inherit extraction errors or contain incorrect values; validate its structure and verify consequential fields against the original.

## Pricing and limits

Fetch and OCR extraction is metered in `processingUnits`. Daily included allowances and rates for uncovered extraction depend on the account plan. Each extraction is either fully covered or billed in full; coverage is not split within one extraction. Optional JSON conversion is metered separately in `jsonProcessingUnits`, with a variable inference-based PU price and no coverage from the daily extraction allowance. Do not add both counts and apply the OCR rate to the total. See [Pricing](https://docs.aivax.net/docs/pricing.md#web-search-ocr-and-fetch) for allowances and charges rather than estimating cost from extracted text length.

[Web utilities MCP](https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.md) uses the same pricing as the corresponding built-in tool. Model inference, when used to analyze the extracted content, is billed separately. See [Plans and limits](https://docs.aivax.net/docs/limits.md) for account quotas and rate limits. The Fetch API requires a positive account balance.

---

Source: https://docs.aivax.net/docs/generations/decisions.html

# Semantic decisions

Semantic decisions evaluate named questions against a shared state and return structured answers rather than a generated explanation. Use them to route support requests, select a category, check a condition, or assign an ordered score.

A request supplies the model, the evidence in `state`, and a `questions` object. Each question has an ID you choose; the response uses the same ID in `answers`. You can ask different question types in one request without making separate API calls.

Use [structured responses](https://docs.aivax.net/docs/inference/structured-responses.md) instead when you need a larger generated object or a written explanation. For embedding-based similarity between documents and labels, see [Text classification](https://docs.aivax.net/docs/rag/classification.md).

## Choose a model

All models below support `choice`, `noul`, and `score`. See [Pricing](https://docs.aivax.net/docs/pricing.md#semantic-decisions) for model rates and [Plans and limits](https://docs.aivax.net/docs/limits.md#semantic-decision-model-limits) for model-specific limits.

| Model |
| --- |
| `@supersonic-labs/julia-1` |
| `@typesafe/jev-1.13` |
| `@respan/span-01` |
| `@respan/span-01-lite` |
| `@jaredpalmer/kev-4b` |

`@typesafe/jev` is also accepted and currently resolves to `@typesafe/jev-1.13`. An unspecified context limit does not mean unlimited input. Model-specific limits and interpretation of scores can differ; validate a model on representative examples before switching production traffic.

An authenticated API key and a positive account balance are required, including when selecting a model with a zero base token price. See [Authentication](https://docs.aivax.net/docs/authentication.md), [Pricing](https://docs.aivax.net/docs/pricing.md), and [Plans and limits](https://docs.aivax.net/docs/limits.md).

### Discover models programmatically

`GET /api/v1/information/decisions-models.json` lists the current decision-model catalog without authentication. The response's `data` array contains:

- `name`: the canonical identifier to use in a decision request.
- `aliases`: other accepted identifiers for that model.
- `contextLength`: the advertised context in tokens, or `null` when unspecified.
- `releaseDate`: the catalog release date in `yyyy-MM-dd` format.
- `capabilities`: supported question types (`noul`, `choice`, and/or `score`).
- `inputPricePerMillionTokens` and `outputPricePerMillionTokens`: base USD prices, before account and plan adjustments.

Use this listing to populate model selectors rather than maintaining a separate hardcoded catalog. It describes configured models, not a real-time provider health check. Model-specific option and question limits are not included in this listing.

## Write the questions

| Type | `criteria` format | Use it for |
| --- | --- | --- |
| `choice` | Object mapping choice IDs to descriptions | Select one destination or category |
| `noul` | Object with nonempty `false` and `true` descriptions | Evaluate a Boolean condition |
| `score` | Ordered array of level descriptions | Evaluate a graded condition |

Every question requires nonempty `instructions`. Use descriptions that distinguish the options, not just opaque IDs. For example, `"billing": "Charges, payments, and refunds"` provides more evidence than `"billing": "B"`.

For `noul`, both criteria are required. Describe what counts as false as carefully as what counts as true, especially when the state might omit the relevant information. For `score`, keep the level order consistent across requests.

## Evaluate a support request

Send the following JSON body to `POST /api/v1/generations/decisions`. The example uses Julia-1; its state may be text, an object, or an array.

```json
{
  "model": "@supersonic-labs/julia-1",
  "state": {
    "message": "I was charged twice. Please return the extra payment."
  },
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Charges, payments, and refunds",
        "technical": "Software errors and service outages",
        "sales": "Plans, pricing, and purchases"
      }
    },
    "refund_requested": {
      "type": "noul",
      "instructions": "Does the customer explicitly ask for money back?",
      "criteria": {
        "false": "The customer does not ask for money to be returned",
        "true": "The customer asks for a refund or return of a payment"
      }
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is the request based on the stated deadline?",
      "criteria": [
        "No deadline stated",
        "A deadline is stated, but it is not today",
        "The customer explicitly needs resolution today"
      ]
    }
  }
}
```

Keep only relevant evidence in the state. Instructions should explain the decision, not ask for a chain of reasoning or additional text. Avoid overlapping choice descriptions unless that ambiguity is intentional.

### Read the response

The success response is a JSON object directly, **without a `data` envelope**. It contains:

- `id`: the decision identifier.
- `model`: the canonical model identifier used for the request.
- `provider`: the provider reported for the result.
- `answers`: an object keyed by your question IDs.
- `usage`: `input_tokens`, `output_tokens`, and the billed `cost`.

For Julia-1, an illustrative `answers` object for the example above is shown below. These numbers explain the format; they are not a recorded response or a quality guarantee.

```json
{
  "department": {
    "type": "choice",
    "choice": "billing",
    "probabilities": {
      "billing": 0.90,
      "technical": 0.06,
      "sales": 0.04
    }
  },
  "refund_requested": {
    "type": "noul",
    "noul": 0.95,
    "probabilities": {
      "false": 0.05,
      "true": 0.95
    }
  },
  "urgency": {
    "type": "score",
    "score": 0.3,
    "legend": {
      "0": "No deadline stated",
      "1": "A deadline is stated, but it is not today",
      "2": "The customer explicitly needs resolution today"
    },
    "probabilities": {
      "0": 0.8,
      "1": 0.1,
      "2": 0.1
    }
  }
}
```

With Julia-1:

- `choice` is the selected caller-defined ID, not the option description.
- `noul` is the probability assigned to the true criterion, not a JSON Boolean. Your application chooses the threshold and how to handle uncertain cases.
- `score` is the expected **zero-based level index**. In the example, `0 × 0.8 + 1 × 0.1 + 2 × 0.1 = 0.3`. It is not necessarily an integer, and it is not a normalized 0–1 score when there are more than two levels.
- `probabilities` are keyed by choice IDs, `false`/`true`, or score-level indices. `legend` describes the score levels.

Other models may omit optional fields such as `probabilities`, `legend`, or `confidence`. Do not assume that every provider uses the same score scale or confidence definition. A high probability is not proof that the decision is correct; validate thresholds and escalation rules with labeled examples from your own domain.

## Account rate limits

Semantic decision requests share an account-level rate limit across models and API keys. Each request counts once, even when it contains multiple questions. This quota is separate from the daily subscription allowance and applies to both included and paid usage. See [Plans and Limits](https://docs.aivax.net/docs/limits.md#plan-limits) for the Free, Pro, and Max thresholds.

Requests above the limit return `429 Too Many Requests` before evaluation. Pace calls across the account and retry with backoff after the rate-limit window clears; changing API keys within the same account does not provide a separate quota.

## Model-specific limits

See [Semantic decision model limits](https://docs.aivax.net/docs/limits.md#semantic-decision-model-limits) for the current context, question, option, and payload limits. Limits interact: shorten descriptions or reduce the option count instead of assuming every maximum can be used at once.

Inputs exceeding the context or question/options budget are rejected, not silently truncated. The literal `<mask>` is reserved and cannot appear in Julia-1 state, instructions, or option descriptions.

## Usage and cost

For Julia-1, input usage sums the encoded sequence for each question, excluding padding. The shared state is therefore counted again for each question. Multiple questions over one state do not have the same input usage as one question over that state. Julia-1 does not generate text, so its `output_tokens` value is zero.

Julia-1 is currently eligible for the daily semantic decision allowance on Free, Pro, and Max. Other decision models are billed normally. The allowance is shared across eligible decision calls, not reserved for each question or API key. See [Plans and Limits](https://docs.aivax.net/docs/limits.md#included-daily-subscription-allowances) for relative plan capacity and coverage rules.

When not covered, Julia-1 input is billed at the [listed rate](https://docs.aivax.net/docs/pricing.md#semantic-decisions), subject to account and plan adjustments. Use the returned `usage.cost` for the actual billed amount; it is zero when the input is fully covered by the allowance.

## Errors and reliable use

- **Invalid model or question:** check the exact model identifier, question type, instructions, and criteria shape. Question IDs and choice IDs must be nonempty.
- **Context or option limit exceeded:** shorten the state or descriptions, reduce the option count, or select a model with suitable limits. Retrying the same invalid input will not resolve it.
- **Authentication or balance error:** verify the API key and account balance before retrying. A zero-priced model still requires a positive balance.
- **Rate limit (429):** reduce the account's request rate and retry with backoff. Multiple questions in one request still count as one request, but model-specific payload limits and per-question usage remain applicable.
- **Temporary capacity or provider unavailability:** avoid an immediate parallel retry storm. Reduce concurrency and use bounded retries with backoff for transient failures.

Evaluate `choice`, `noul`, and `score` separately when validating a model: success at routing does not establish reliable scoring or Boolean behavior. Include ambiguous and incomplete states in your test set, and use human review where a wrong decision has material consequences. A retry is a new request; do not assume automatic deduplication or identical model outputs.

## API reference

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Evaluate%20semantic%20decisions)

---

Source: https://docs.aivax.net/docs/generations/speech.html

# Speech Generation

Use Speech Generation when your application already has final text and needs playable audio without running a chat completion. Typical uses include narrating an article or notification, voicing an IVR prompt, producing a draft voice-over for review, or generating audio files for offline playback.

Authenticate requests with an AIVAX API key. See [Authentication](https://docs.aivax.net/docs/authentication.md) for authorization guidance.

## Choose the delivery form

The endpoint can return audio in two forms, and the right choice depends on the consumer:

- **Binary audio (`raw: true`)** — the response body is the audio file itself, served inline with its MIME type. Use this when a player, browser `<audio>` element, or download flow consumes the response directly.
- **Base64 in the JSON envelope (`raw: false`)** — the response contains the format, MIME type, and base64-encoded audio inside the standard envelope. Use this when the result travels through JSON pipelines, gets stored in a database, or needs to be inspected alongside billing metadata.

Supported formats are `mp3`, `wav`, and `ogg`. When the consumer accepts any of them, prefer `mp3` for smaller payloads and `wav` for maximum compatibility with audio tooling.

## Generate speech

Write text that is ready to be spoken aloud, including punctuation and formatting that communicate pauses or emphasis. Expand abbreviations and numerals the way they should be heard ("twenty twenty-six", "doctor" instead of "Dr.") rather than trusting the voice to guess.

Test the selected voice with representative content before using it in production, especially for names, abbreviations, or specialized terms. Voices differ per speech model, so confirm the voice against the model you will use rather than assuming one model's voices exist in another.

The embedded reference is the source of truth for available models and voices, request options, output formats, and response fields.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Generate%20speech)

To let an MCP-compatible agent or IDE synthesize speech without calling this endpoint directly, use the [Media generation MCP](https://docs.aivax.net/docs/mcp-utilities/media-generation-mcp.md). It returns a public URL to MP3 audio instead of inline audio.

## Speech or Voice Sessions

Use Speech Generation for one-shot synthesis of known text. Use [Voice Sessions](https://docs.aivax.net/docs/inference/voice-session.md) when the experience is an interactive spoken conversation with interruptions, turn-taking, and tool calls — chaining transcription, inference, and synthesis manually adds latency that the realtime session avoids.

## Pricing, limits, and errors

For current pricing, availability, and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md). Review reported validation errors before retrying a failed request.

---

Source: https://docs.aivax.net/docs/generations/audio-transcriptions.html

# Audio Transcriptions

Use Audio Transcriptions to convert recorded speech into text for search, review, captions, or downstream automation. Typical uses include transcribing meetings and interviews, captioning recorded videos, making voice notes searchable, and feeding spoken input into classification or RAG pipelines.

Authenticate requests with an AIVAX API key. See [Authentication](https://docs.aivax.net/docs/authentication.md) for authorization guidance.

## Choose a transcription model

Query the available transcription models for the authenticated account before selecting one. Availability can vary by account configuration, so do not hard-code a catalog in your application.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Get%20Audio%20Transcription%20Models)

## Transcribe audio

Send the audio as base64 with its format (`wav`, `mp3`, `m4a`, `flac`, `ogg`, `webm`, or `aac`). Decoded audio must not exceed 75 MB, and billing follows the measured media duration (see [Pricing](https://docs.aivax.net/docs/pricing.md)) — files with no measurable duration are rejected.

Provide the optional `language` hint when the spoken language is known. It steers recognition toward that language and helps with accented speech and domain vocabulary; omit it when the language genuinely varies within one file and let detection handle it.

Review the returned text before using it in user-visible or irreversible actions: noisy recordings, multiple speakers, specialized vocabulary, and poor microphone quality can affect transcription quality. For files where speaker attribution matters, plan a diarization or review step downstream — the endpoint returns the transcript text, not speaker labels.

The embedded reference is the source of truth for accepted request forms, options, and response fields.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Transcribe%20audio)

## After the transcript

A transcript is usually the start of a pipeline, not the end. Index it into a [RAG collection](https://docs.aivax.net/docs/rag/collections.md) to make recordings searchable, run it through [Batch](https://docs.aivax.net/docs/features/batch.md) when there are many files, or feed it to [Text classification](https://docs.aivax.net/docs/rag/classification.md) to label conversations at scale. For live conversation instead of recorded audio, use [Voice Sessions](https://docs.aivax.net/docs/inference/voice-session.md).

## Pricing, limits, and errors

For current availability, pricing, and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md). Correct invalid or inaccessible audio before retrying a failed request.

---

Source: https://docs.aivax.net/docs/generations/media-descriptions.html

# Media Descriptions

Use Media Descriptions when an application needs structured information from audio, images, video, or PDF content. Typical uses include preparing media for search, moderation review, accessibility workflows, and downstream automation.

Choose the more specialized API when the task is limited to a single medium, such as [Audio Transcriptions](https://docs.aivax.net/docs/generations/audio-transcriptions.md) for speech-to-text.

## Describe media or reason over it directly

There are two ways to extract information from media, and they serve different needs:

- **Describe media, then decide** — use this endpoint when you need a reusable text artifact: a description stored for search, a transcript-like record for audit, or input text for a later workflow stage that runs independently.
- **Multimodal inference in one request** — send the media straight to a chat model that supports the input type when the model should reason over it immediately and no intermediate artifact is needed. See [Inference](https://docs.aivax.net/docs/inference/inference.md).

Prefer Describe when the extraction guidance is stable and the result feeds several consumers; prefer direct multimodal inference when the question about the media changes on every request.

## Describe media

Provide media that AIVAX can access, and use guidance that focuses the extraction on the information your workflow needs. The optional `extractionGuidance` instruction narrows the output — "identify the main subject and visible text" for an image, "summarize the sequence of events" for a video — so write it as the question your workflow needs answered.

Each submitted item is handled independently and results come back in input order, so keep the positions aligned when correlating results with the original media. Remote file URLs are downloaded before processing under the `auto` preset (up to 5 MB); switch to `high` when the service should receive the remote URL directly.

For remote media, make sure the resource remains accessible for the duration of processing and does not require interactive sign-in. Avoid sending credentials, personal data, or other content that must not appear in a generated description.

The embedded reference is the source of truth for accepted media forms, request options, and response fields.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Describe%20media)

## Pricing, limits, and errors

For current pricing, media availability, and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md). Correct inaccessible media or invalid content before retrying.

---

Source: https://docs.aivax.net/docs/generations/teach-skill.html

# Teach Skill

Use Teach Skill to turn recorded demonstrations into reusable, step-by-step skill instructions. Typical uses include capturing an expert's screen workflow so support agents repeat it consistently, converting onboarding walkthroughs into assistant behavior, and bootstrapping a skill draft that a human then tightens.

Submit tutorial videos as `video_url` content parts — hosted URLs or base64 data URIs. Since analysis takes a while on longer recordings, the API recommends `direct.inference.aivax.net`.

## Record a demonstration that teaches well

The quality of the draft follows the quality of the recording. Before submitting:

- Show the workflow in the order it should be understood, one step at a time, without jumping between screens.
- Narrate or caption the intent behind each action ("I open this panel because..."), not just the click itself — silent recordings leave the required context implicit and force the model to guess.
- Keep credentials, personal data, and customer information out of frame; anything visible can end up in the resulting instructions.
- Prefer a few short, focused recordings over one long session when the procedure has natural phases.

## Create a skill draft

Organize recordings in the order in which the procedure should be understood. The response uses the standard JSON envelope. `data.resultText` contains a structured Markdown draft that can include front matter, steps, notes, and assumptions when the recording leaves required context implicit. `data.usage.processedUnits` reports the processed usage units for the request.

A generated draft is not automatically published as an account skill. Validate every step against the real workflow, remove recording-specific details (window sizes, test names, one-off values), confirm prerequisites, and rewrite vague steps as imperative instructions before saving it. See [Skills](https://docs.aivax.net/docs/features/skills.md) for skill structure and activation guidance.

The embedded reference is the source of truth for accepted video input and response behavior.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Teach%20skill)

## Pricing, limits, and errors

For current availability and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md). Correct invalid or inaccessible video content before retrying a failed request.

---

Source: https://docs.aivax.net/docs/generations/images.html

# Image Generation

Use Image Generation to create images from a text prompt in an application workflow. Typical uses include draft illustrations for editorial review, product mockups, marketing variations for A/B testing, and placeholder art that a designer refines later.

Authenticate requests with an AIVAX API key. See [Authentication](https://docs.aivax.net/docs/authentication.md) for authorization guidance.

## Choose a model

Select an image-generation model available to the authenticated account. Model availability and capabilities can change, so obtain current options from the platform instead of depending on a fixed list in this guide.

When several models are available, decide by capability: whether you need reference-image support (only some models accept `referenceImages`), how many variations per prompt you need (`count` accepts 1 to 4), and how long your application can wait — image requests on some models take a while, in which case the API recommends `direct.inference.aivax.net`.

## Generate an image

Use prompts that state the outcome you need rather than relying on vague visual labels. Include the important details, such as the subject, environment, framing, and any text that must be present. One request can carry a single prompt or an array of prompts, and each prompt generates `count` images whose URLs come back grouped by input prompt.

Supply up to four HTTP(S) reference images when the selected model supports them and the output must follow an existing subject, style, or composition. References guide the result; they do not guarantee identity preservation, so inspect the output before publishing.

Treat generated assets as drafts and review them for accuracy, brand suitability, and unintended content before release. When the first result is close but not right, iterate by tightening one element at a time — subject detail, composition, style constraint — rather than rewriting the whole prompt at once.

The embedded reference is the source of truth for the request shape, supported options, and response fields.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Generate%20images)

To let an MCP-compatible agent or IDE generate images without calling this endpoint directly, use the [Media generation MCP](https://docs.aivax.net/docs/mcp-utilities/media-generation-mcp.md).

## Pricing, limits, and errors

The Models catalog lists a fixed price per output image and, where applicable, a price per reference image. The charge is the number of delivered outputs multiplied by the output price plus the reference price for every reference sent with each output. For example, two outputs using three references cost `2 × (output price + 3 × reference price)`. A zero reference price means references have no separate charge.

These tariffs use rounded-up estimates where the provider bills tokens or megapixels; they are not the provider's exact charge for each request. Prompt processing is included in the output tariff, with no separate token charge. No AIVAX image-generation markup or account and plan pricing multiplier is added. Only delivered images count toward the image charge.

For current pricing, availability, and account limits, see [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md). If a request fails, revise the reported validation issue before retrying.

---

Source: https://docs.aivax.net/docs/features/skills.html

# Skills

Skills (also known as abilities) can be used to improve how your agent performs on specific tasks. Skills are special instructions that are retrieved on demand, and your agent loads these skills as needed.

Ideally, your agent should only use a skill when it is relevant to the task it is performing. It is embedded in the conversation through a special tool, and skill instructions are added to the context, indicating that the agent has context provided in the context that can be used in the conversation.

## How do skills work?

Skills are primarily provided through a slug and a brief description of what that skill is and when it should be used. You can have multiple skills in your account, but only use a subset of them in your [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md). The skills enabled in the AI Gateway are listed in the model’s system instructions and the `read_skill` function is added to the context.

When the model calls `read_skill`, it passes an array of skill slugs. The instructions for the matching skills are inserted into the system instructions of the next context turn inside `<activated_skill>` blocks. More than one skill can be activated when a task spans multiple domains, and the model can call `read_skill` again with a different list when the active task changes.

For this to work, the chosen base model must support **function calls** and **system instructions**.

- If your model does not support function calls, consider using a [tool handler](https://docs.aivax.net/docs/inference/pipelines.md) to handle function calls.
- If your model does not support system instructions, consider using the `No system instructions` flag, which provides system instructions as a user message.

Larger models tend to follow instructions and function calls very strictly. Run tests to see if your model is changing its skills as required.

## Structure and operation

A skill has `slug`, `description`, `instructions`, and `options`. The `slug` is the technical identifier used by the account and must be 64 characters or fewer, using letters, numbers, underscores, periods, or hyphens. The `description` is the text that helps the model decide when that skill should be loaded; it needs to explain the usage scenario instead of repeating the name. The `instructions` are the full content that will be inserted when the skill is activated. In the current runtime path, `instructionSources` is stored and exported with the skill configuration, but only the inline `instructions` field is injected when the model activates a skill.

Write the description as a routing rule. A description like “legal skill” is weak because it doesn’t say when to use it. A better description would be “use when the user asks for analysis, review, or explanation of contractual clauses in simple language.” The instructions, on the other hand, should be operational: explain how to respond, what questions to ask when data is missing, which tools may be useful, what limits must be respected, and the expected final format. The model reads the description to choose the skill and reads the instructions to perform the task.

`allowedToolsNames` is part of the skill configuration and is intended to associate runtime tool names with a skill, such as `web_search`, `open_url`, `generate_image`, or `generate_web_page`. In the current runtime path, the general always-visible tool list is the reliable way to keep shared tools visible when the gateway hides tools without a skill.

Skills can also be imported and exported in JSONL. This format is useful for versioning skills, migrating settings between accounts, keeping a set of skills in a repository, or reviewing changes before publishing. Each line must represent a skill with at least `slug` and `instructions`; `description` and `options` can complement the configuration. When importing a skill with an existing `slug`, AIVAX updates the corresponding skill instead of creating a duplicate.

## How to write skills?

Writing effective skills requires clarity and specificity in the instructions. Here are the main guidelines:

### Skill structure

A well‐written skill should contain:

1. **Clear, descriptive slug**: Use a short slug that immediately identifies the purpose of the skill, such as `python_code_analysis`, `customer_support`, or `technical_translation`.

2. **Concise description**: Provide a brief description (1–2 sentences) that explains when the skill should be activated. This description is crucial because the model uses it to decide whether to load the skill.

3. **Detailed instructions**: The actual instructions should include:
   - Clear objectives of the task
   - Expected response format
   - Specific rules to follow
   - Examples where appropriate
   - Important constraints or limitations

### Best practices

- **Be specific**: Avoid vague instructions. Instead of “be helpful,” say “provide step‐by‐step explanations with code examples.”
- **Use imperative language**: Start sentences with action verbs (analyze, explain, compare, list).
- **Keep the scope limited**: Each skill should focus on a specific knowledge area or type of task.
- **Iteratively test**: Refine your skills based on how the model responds in practice.
- **Avoid redundancy**: Do not repeat information already present in the base system instructions.

### Skill example
- Name: Code Performance Analysis
- Description: Use when the user asks for performance analysis, optimization, or bottleneck identification in code.

```markdown
- Analyze the provided code identifying possible performance bottlenecks
- Consider temporal complexity (Big O) and memory usage
- Suggest specific optimizations with code examples
- Explain the impact of each proposed optimization
- Prioritize readability and maintainability alongside performance
```

## When to use skills?

Skills are most useful in specific scenarios where you need specialized behavior:

### Ideal scenarios

**1. Domain‐specific expertise**
- Technical terminology specific to an industry
- Compliance with regulations or standards
- Specific methodologies (Scrum, ITIL, SOC 2, etc.)

**2. Tone or style shift**
- Formal vs. casual service
- Technical vs. simplified communication
- Different personas or roles

**3. Complex, structured processes**
- Multi‐step workflows
- Analyses that follow specific frameworks
- Document generation with strict formats

**4. Tasks that require extensive context**
- When instructions are too long to include every time
- Knowledge that is only occasionally relevant
- Multiple mutually exclusive rule sets

### When NOT to use skills

- **Permanent instructions**: If instructions should always be active, put them in the base system instructions.
- **Simple tasks**: For direct answers that don’t require special context.
- **General knowledge**: Information the model already knows well does not need to be reinforced via skills.
- **Too few or too many skills**: Aim for 3–10 skills. Less than that, consider using base instructions. More than that can confuse the model.

### Comparison: Skills vs. System Instructions

| Aspect | Skills | System Instructions |
|--------|--------|----------------------|
| When to apply | On demand, when relevant | Always active |
| Ideal size | Can be extensive | Should be concise |
| Context switching | Yes, can alternate | No, fixed |
| Token usage | Economical (only when needed) | Constant |
| Best for | Specialized knowledge | Base agent behavior |

### Implementation tips

- **Start small**: Implement 2–3 skills initially and expand as needed.
- **Monitor usage**: Check that the model is activating skills correctly.
- **Avoid overlap**: Skills with similar descriptions can confuse the model.
- **Test transitions**: Ensure the model switches skills when appropriate.

## When to choose RAG or system instructions instead

Use skills together with [collections](https://docs.aivax.net/docs/rag/collections.md) when the task needs searchable knowledge, and with [pipelines](https://docs.aivax.net/docs/inference/pipelines.md) when it needs event-time policy or message rewriting. For broader background, see the [Prompt Engineering Guide](https://www.promptingguide.ai/), [Building Production‐Ready RAG Applications](https://www.anthropic.com/index/building-effective-agents), and the [AI Gateway Documentation](https://docs.aivax.net/docs/inference/ai-gateway.md).

---

Source: https://docs.aivax.net/docs/features/chat-clients.html

# Chat Clients

A chat client provides a user interface via an [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) that allows the user to converse with their assistant. A chat client is integrated with the AI gateway inference and supports deep thinking, search, text conversation, and image sending. Audio features depend on the integration and client configuration.

You can customize the chat client interface with CSS, custom JavaScript, colors, labels, suggestion buttons, frame origins, input modes, and the language used by the chat resources.

## How the chat client works

A chat client is a session layer on top of an AI Gateway. The gateway defines the assistant’s behavior; the chat client defines how an end user converses with it, how the session is identified, how long it lasts, what limits are applied, which visual resources appear, and how messages enter and exit through external channels. This separation is important: you can use the same gateway in an internal API, a web widget, Telegram, and WhatsApp, but each channel will have its own rules for identity, attachments, formatting, commands, and message delivery.

Each session maintains a message history, additional context, metadata, conversation token, and an optional external identifier. When you create a session with a `tag`, AIVAX tries to reuse the active session for that tag instead of creating a new conversation. This allows a user to return to the widget or send another message through the same channel without immediately losing context. When the session has no `tag`, it functions as an independent conversation controlled by the access token generated at creation.

The `tag` also serves as the connection point between the chat client, memory, calendar, workers, and integrations. Tools like memory need a stable identifier to know who a preference or persistent information belongs to. Workers receive `externalUserId` to apply rules per user, per channel, or per external account. WhatsApp and Telegram integrations use the conversation ID, phone number, or user to retrieve the correct session. Therefore, choose a stable, non‐sensitive, unique `tag` per user or conversation.

## Creating a chat session

A chat session is where you create a conversation between your chat client and the user. You can call this endpoint providing additional context for the conversation, such as the user’s name, location, etc.

A session can be identified with a stable, non-sensitive tag so the client can continue the appropriate conversation. See the embedded API Reference for supported session behavior and configuration.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20Web%20Chat%20Session)

When creating a session, use `extraContext` for information that helps the assistant in that conversation but should not become permanent memory: displayed name, client plan, preferred language, referrer page, product the user is viewing, order number, or current flow state. Do not use this field for secrets, internal tokens, or data the model should not see. The additional context goes into inference and can influence responses, tools, and workers.

You can also provide `contextLocation`, a URL that AIVAX loads during response generation and appends to the session context. Use it for server-controlled context that may change over time, and make sure the URL is reachable by AIVAX.

The web chat accepts text messages and attachments. Images, files, videos, and audio are materialized before inference; supported image, file, video, and audio types can be forwarded as multimodal content when the selected model and gateway configuration support them. Audio can also be synthesized as a response when the chat client’s audio synthesis setting is active. When a channel cannot embed an attachment, AIVAX turns unsupported content into a textual attachment notice so that the assistant can reply clearly.

## Sending prompts from your application

Use **Send Prompt** when your application needs a synchronous response using an existing chat-client session. The session access key authorizes the request; keep it private. The request uses the session history and the associated AI Gateway configuration.

For long-running inference, send `POST /api/v1/public/chat-clients/<access-key>/prompt` to `https://direct.inference.aivax.net` to bypass the Cloudflare Tunnel path. Configure your HTTP client's timeout for the expected generation duration. The direct domain exposes selected routes, not the entire chat-client API.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Send%20Prompt)

### Choose the input format

The required `prompt` accepts plain text, one OpenAI-compatible message object, or an ordered array of message objects. Plain text becomes a user message. Use message objects for multimodal content or tool results, and an array when several messages must be supplied together. At least one message must contain content or tool calls.

The response contains `completionText`, `reasoning` when available, `toolCalls` for your application to execute, `usage`, and `createdMessages`. The last field contains only the messages generated during this request, in generation order—not the submitted messages or the previous session history. Its messages use the OpenAI-compatible format and may include tool calls, tool results, reasoning and per-message metadata.

`completionText` is the text of the last assistant message in `createdMessages`, not a concatenation of the turn: after a tool call, it holds only the final answer. It falls back to the accumulated inference text when the last assistant message is missing or empty, so a tool-call-only turn may return an empty `completionText` — check `toolCalls` to decide whether to continue.

### Decide whether to save the turn

`commit` defaults to `true`: submitted messages and generated messages are saved to the session. Use `commit: false` for a one-off inference against the current history without saving that turn. You still receive the completion and `createdMessages`.

This is not a dry run: inference is still billed and tools can still act. Uncommitted messages stay out of later session history — to continue that branch, resubmit them in an ordered `prompt` array.

### Add context for one inference

Use `instructions` for context that should apply only to the current request, such as a temporary response format or the item currently selected in your application. It accepts a string or an array of strings; array entries are joined with blank lines. This context is appended after the session's `extraContext` and any context loaded from `contextLocation`.

Unlike session `extraContext`, `instructions` is not saved as session context, even when `commit` is `true`. It is still sent to the model and can influence responses and tools; do not include secrets or data the model should not see.

### Complete client-side tool calls

1. Configure the desired client-side tool in the AI Gateway and send a prompt.
2. When `toolCalls` is nonempty, execute the requested function in your application. Each entry exposes `id`, `functionName`, `contents` (JSON-encoded arguments), and `isProtocolFunction`. Validate the arguments and enforce your application's permissions before execution.
3. Send a new prompt with a `role: "tool"` message. Set `tool_call_id` to the returned call's `id`, `name` to its `functionName`, and `content` to the tool result as text. For multiple calls, submit an array of matching result messages.
4. Read the next completion, or repeat if it requests more tools.

With the default `commit: true`, the assistant's tool-call message is already in the session: submit only the tool results, without duplicating that assistant message. If the preceding request used `commit: false`, include the unsaved conversation messages—including the assistant message containing `tool_calls`—before the results. The top-level `toolCalls` entries are not message objects; use the OpenAI-compatible messages in `createdMessages` when reconstructing that exchange.

For server-side tools, AIVAX executes the tools and continues generation within the same request. `createdMessages` can therefore contain an assistant tool call, its tool result, and the final assistant reply, while top-level `toolCalls` is empty. Do not execute those server-side calls again. The API reference above includes examples for simple completions, client-side calls, submitted client-side results, and multiple messages from server-side calling.

## Integration sessions

AIVAX provides integrations for chat clients via Telegram and WhatsApp, including [Z-Api](https://www.z-api.io/), Evolution API, and Kapso. Each conversation in an app is an individual session, identified by the conversation ID, chat ID, or the user’s phone number, depending on the provider. Integration sessions default to a three-hour duration unless the integration parameters specify another value.

These sessions obey the original chat client rules. Additionally, chat sessions in these integrations have two special commands:

- `/reset`: clears the current session context.
- `/usage`: when `debug` is active in the chat client, displays the current chat usage in tokens.

Integrations treat the channel as the source of messages, but inference continues to be performed by the AI Gateway associated with the chat client. In Telegram, the conversation receives additional instructions about formatting and expected channel behavior. In WhatsApp, each provider has its own webhook details, media download, and response sending; Z-Api, Evolution API, and Kapso are different paths to the same operational goal. In all cases, user messages enter the session, are materialized as inference‐compatible messages, and the assistant’s response is sent back via the integration’s messenger.

Use Telegram when you need a simple bot with users identifiable by chat and easy‐to‐test commands. Use WhatsApp when the user’s primary support channel is already the phone and the conversation needs to happen in a daily app. Use the web widget when you want to embed the assistant in a website, product, support center, or dashboard. The channel choice should not change the gateway’s essential content, but may require adjustments to tone, response size, formatting, and attachment tolerance.

Before opening a channel to the public, review the chat client configuration and explain memory behavior to users when applicable. When an integration does not respond as expected, first verify the associated gateway and integration configuration, then retry with a simple message before investigating optional capabilities.

Treat persistent memory separately from session history. See [the memory-poisoning guide](https://aivax.net/blog/persistent-memory-is-a-write-path/) for reviewing stored context and testing its influence across conversations.

For the next steps, review [AI Gateway configuration](https://docs.aivax.net/docs/inference/ai-gateway.md) and [Authentication](https://docs.aivax.net/docs/authentication.md), especially the boundary between public chat access and API credentials.

---

Source: https://docs.aivax.net/docs/features/batch.html

# Batch

Batch is AIVAX's feature for running the same AI workflow over many independent items. It turns a list of inputs into a background-processed queue with fixed instructions, structured output, optional validation, progress tracking, retries, and result export.

Use Batch when you have dozens, hundreds, or thousands of records that need to undergo the same reasoning: classifying leads, extracting fields from text, enriching records, summarizing short documents, evaluating responses, moderating content, generating structured data, or invoking built-in tools for each line of a list.

## What Batch solves

Processing many items with AI usually requires a queue, error handling, retries, JSON validation, and result export. Batch consolidates these parts in AIVAX.

In practice, it mainly solves:

- **Repeatable processing:** the same instruction, model, and schema are applied to all items.
- **Asynchronous execution:** the work continues in the background, without keeping a request open.
- **Structured output:** each item can be required to return an object compatible with a JSON Schema.
- **Correction and validation:** AIVAX tries to reprocess invalid responses and can run a second validation step.
- **Operation at scale:** jobs can be started, paused, resumed, monitored, filtered, cleaned, resent for retry, and exported.
- **Operational control:** the UI shows progress, failures, confidence, and job events.

## When to use

Use Batch when items can be processed independently and do not need to share memory. Good examples are one line per client, URL, product, ticket, message, short document, contract snippet, or raw record.

Batch is a good choice when:

- the same prompt applies to all items;
- you need tabular or JSON results for later consumption;
- response time can be asynchronous;
- you want to track errors and retry only the problematic items;
- you want to use built-in tools, such as web search, for each item;
- you need to measure confidence and success rate per run.

Do **not** use Batch for real‐time conversations, flows where one item depends on the previous item's response, document indexing for RAG, or purely deterministic tasks that do not require an AI model. To index searchable knowledge, use [RAG collections](https://docs.aivax.net/docs/rag/collections.md). For a single immediate response to a user, use [inference](https://docs.aivax.net/docs/inference/inference.md).

## Concepts

### Workflow

The workflow is the processing recipe. It defines how future items will be handled:

- title;
- processing instructions;
- model;
- expected result schema;
- enabled built-in tools;
- validation instructions; and
- retry and error-handling behavior.

Changing a workflow affects subsequent jobs and items processed with that configuration. Use separate workflows when the instruction, schema, model, or validation rules change in a significant way.

### Job

A job is a concrete execution created from a workflow. It groups the items of a workload, maintains state, events, and metrics.

A job represents the workload while it is prepared, processed, paused, or completed. See the embedded API Reference for supported job states.

### Item

An item is a row from the imported list. Each row becomes an independent input sent to the model with the workflow's instructions.

Each item records its processing outcome, output, confidence, and validation details. See the embedded API Reference for supported item states.

## How to use in the console

In the AIVAX console, go to **Batch**.

### Create a workflow

In **Workflows**, create a workflow and configure:

1. **Basic:** set a title, the processing instruction, and the JSON Schema of the result.
2. **Behavior:** choose the supported assistant capabilities for the workflow.
3. **Validation:** enable validation when the response needs to be checked against business rules.
4. **Handling:** configure the workflow's error-handling behavior.

Write the instruction as a general rule, not as a single question. The imported item will be the variable input.

Instruction example:

```text
Classify the company provided in the input. Return the likely sector, a short justification, and signals found in the text. If the input does not contain enough information, use sector "Undefined".
```

Schema example:

```json
{
  "type": "object",
  "properties": {
    "sector": { "type": "string" },
    "reason": { "type": "string" },
    "signals": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["sector", "reason", "signals"],
  "additionalProperties": false
}
```

### Create and run a job

After creating the workflow, create a job for the load you want to process. New jobs are created in `Paused` state so you can import and inspect the workload before starting processing.

You can import items in four modes:

- `lines`: reads one uploaded text file and imports each non-empty line as one item.
- `files`: imports each uploaded plain-text file as one item.
- `zip`: imports each plain-text entry in an uploaded ZIP file as one item.
- `text`: imports the submitted text field as a single item.

Lines can be plain text, delimited CSV, URLs, IDs, compact JSON, or any format the instruction knows how to interpret. For structured line-based inputs, prefer JSONL: one JSON object per line.

Example:

```jsonl
{"name":"Company A","description":"B2B auto-parts marketplace"}
{"name":"Company B","description":"Office specializing in employment contracts"}
{"name":"Company C","description":"Regional pharmacy chain"}
```

With the items imported, start the job. The job screen lets you monitor:

- overall progress;
- pending, completed, and failed items;
- average confidence;
- job events;
- most recent processed items;
- full item list with filters by state and confidence.

### Operate on failed items

Use the list filters to find items with execution error, validation error, refusal, or low confidence. Then you can:

- retry all errors;
- retry only execution errors;
- retry only validation errors;
- retry completed items with low confidence;
- remove pending, completed, error, or all non‐running items;
- open an individual item to review input, output, state, and confidence.

### Export results

When the job finishes, export the results in JSONL. Each exported line contains metadata, the original input, and the output. Use this export to import into a spreadsheet, database, data pipeline, or manual review step.

## How to use via the API

Use the API when you want to integrate Batch into your internal system, data pipeline, or automation. Authentication follows the same pattern as the AIVAX API.

The API flow is the same as the console flow, only expressed as separate operations. First create the workflow, which is the reusable recipe. Then create a job, import the items, and start the job when the workload is ready. After processing begins, use the listing, retry, cleanup, and export endpoints to operate the job without losing track of individual records.

If you are still deciding whether Batch is the right feature, compare it with [RAG collections](https://docs.aivax.net/docs/rag/collections.md) and [direct inference](https://docs.aivax.net/docs/inference/inference.md). Batch is for repeated reasoning over independent items. RAG collections are for searchable knowledge that should be retrieved later. Direct inference is for one immediate answer.

### Create workflow

Create a workflow when you want to save the processing rule that future jobs will reuse. This is where you define the instruction, model, output schema, validation behavior, retries, and enabled tools. A good workflow reads like a policy for every item, not like a one-time prompt for a single record.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20Batch%20Workflow)

### Create job

Create a job when you have a concrete workload to run through an existing workflow. Jobs are created paused so you can import and inspect items before processing.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20Batch%20Job)

Jobs are created paused. Import the items before starting.

### Import items

Import items after the job exists. Each imported item becomes one independent unit of work, so choose the mode that best matches your source data: one line per record, one file per record, one ZIP entry per record, or one submitted text as a single record.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Import%20Batch%20Job%20Items)

Choose the import format that matches the source data. See the embedded API Reference for supported import modes, fields, and current constraints.

### Start, pause, or finish

Start the job only after the item list looks correct. Pause it to investigate errors or adjust the workflow, and finish it when the job should be terminated rather than resumed.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Edit%20Batch%20Job)

See the embedded API Reference for the supported job states.

### Monitor

Monitoring is how you decide whether the workflow is healthy. The job view gives the overall state; the item list tells you where the work is getting stuck, which items failed validation, and which low-confidence results deserve human review.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=View%20Batch%20Job)

To list items:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=List%20Batch%20Job%20Items)

Use the list endpoint to filter items by state, confidence, or input text. See the embedded API Reference for supported filters.

### Retry and clean

Retries are best used after you understand the failure shape. Retry execution errors when the provider or request failed, validation errors when the response can likely be regenerated into the expected shape, and low-confidence results when the item succeeded but deserves another model attempt. Cleanup endpoints are for removing non-running items from a job when they are no longer useful.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Retry%20Batch%20Job%20Items)

Use the retry endpoint after reviewing the failure pattern. See the embedded API Reference for supported retry options and resulting job behavior.

To remove non‐running items:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Remove%20Batch%20Job%20Items)

Use the removal endpoint only for items that are no longer useful. See the embedded API Reference for supported removal options.

### Export

Export is the handoff point from AIVAX back into your own workflow. Use it after the job finishes, or export only a subset when a review process needs finished items first and errors later. The JSONL format is convenient for spreadsheets, databases, queues, and manual audit tools because each line remains tied to the original input and its generated output.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Export%20Batch%20Job)

Use the export endpoint to select the completed results that your review or downstream process needs. See the embedded API Reference for supported export filters.

## Availability

Review [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and Limits](https://docs.aivax.net/docs/limits.md) before processing a large workload.

## Best practices

- Test the workflow with a few items before importing a large list.
- Use restrictive schemas with `required` and `additionalProperties: false` when the output will be consumed by a system.
- Include input and output examples in the instruction when the format is ambiguous.
- Prefer one line per item; if you need to send complex objects, use JSONL.
- Keep validation enabled for sensitive tasks such as legal, financial extraction, or data that feeds automations.
- Use `maxRetries` to fix occasional failures, but investigate repeated errors in the prompt or schema.
- Set a low `errorStopThreshold` in new workflows to avoid spending on a batch with a wrong configuration.
- Retry low‐confidence items separately; low confidence does not mean error, but indicates the response deserves review.
- Export results by state when manual review is needed, e.g., first `finished`, then `errors`.

---

Source: https://docs.aivax.net/docs/tools/builtin-tools.html

# Built-in Tools

AIVAX provides a list of built-in tools for you to enable in your model. These tools can be used together with the [server-side functions](https://docs.aivax.net/docs/tools/protocol-functions.md).

Some functions have usage charges. See [Pricing](https://docs.aivax.net/docs/pricing.md) before enabling them in a production workflow.

Note that each model decides which function to call and its parameters. Not all models can obey the call rules.

## How to Choose and Combine Tools

Built-in tools should be enabled as work capabilities, not as agent decoration. Each tool adds a decision to the model: it needs to perceive that the tool exists, understand when to use it, assemble valid arguments, wait for the result, and continue the response. The more similar tools are available at the same time, the higher the chance of redundant use or poor choice. Start with the smallest set that solves the use case and write clear instructions on when to use each.

Use `WebSearch` when the answer depends on public, recent, or variable information. Use `OpenUrl` when the user has already provided a URL and wants the assistant to analyze that specific content. `AdvancedWebUsage` is disabled and returns an unavailable response; see [Changelogs](https://docs.aivax.net/docs/changelogs.md). Use `Code` for calculation, data transformation, and small algorithmic reasoning. Use `Request` when the model needs to call an HTTP API with method, headers, or custom body. Use `Remember` and `Calendar` only in chat clients or calls with an identifiable user, because these tools depend on persistent per-user context.

Generation tools, such as image, document, and web page, should be treated as output actions. They do more than improve an answer; they create artifacts hosted or attached to the conversation. Therefore, instruct the model on when to generate an artifact and when to reply in text. In support, for example, generating a document can be useful for a quote, proposal, or formal summary; generating a web page can be useful for a visual report; generating an image can be useful for creative ideation. If the user only asked for an explanation, plain text is usually sufficient.

When tools are available via `builtin_tools` in a direct call, the application making the request decides the list for each inference. When configured in the AI Gateway, the list is centralized and can be combined with skills, workers, MCP, protocol functions, and shell. In production, prefer the gateway for permanent policies, because it prevents different clients from enabling different tools without control. Use direct calls for testing, internal routines, and flows where the application truly needs to choose tools dynamically.

The values in `builtin_tools.tools` are configuration flags such as `WebSearch`, `Code`, and `OpenUrl`. The model sees runtime function names such as `web_search`, `evaluate_code`, and `open_url`. Use runtime function names when configuring skill tool allowlists or shell tool allowlists.

## Current Date and Time

Enable `DateTime` to expose `get_date_time`. This tool takes no arguments and reads the current time when it is called. It returns the date, time, day of the week, time zone, UTC offset, and an ISO 8601 timestamp.

In the dashboard, select **Current date and time** in the gateway's built-in tools, then edit its **Time zone** under **Current date and time configuration**. The Functions playground and Batch workflow tool options also expose this setting.

Configure `dateTimeTimeZone` with an IANA time zone identifier. The default is `America/Los_Angeles` (Pacific Time), which automatically follows PST/PDT daylight-saving changes rather than using a fixed UTC offset. For example, use `America/Sao_Paulo` for São Paulo or `UTC` for UTC. Invalid identifiers are rejected. The tool uses this configured zone, not the browser or user-context time zone.

Activation via `builtin_tools`:

```json
{
    "tools": ["DateTime"],
    "options": {
        "dateTimeTimeZone": "America/Los_Angeles"
    }
}
```

For a saved gateway, include `DateTime` in `parameters.sentinelOptions.enabledFunctions` and set `parameters.builtinFunctionsOptions.dateTimeTimeZone`. For a Batch workflow, use `enabledTools.enabledFunctions` and `enabledTools.options.dateTimeTimeZone`.

Example tool result (illustrative, not a live reading):

```json
{
    "date": "2026-07-15",
    "time": "09:30:00",
    "day_of_week": "Wednesday",
    "time_zone": "America/Los_Angeles",
    "utc_offset": "-07:00",
    "date_time": "2026-07-15T09:30:00-07:00"
}
```

The date uses `yyyy-MM-dd`, time uses 24-hour `HH:mm:ss`, and weekday names are returned in English. All fields describe the same instant.

## Internet Search

This function enables internet search in your model. With this, the model can query specific or real‐time information such as weather data, news, game results, etc.

Internet search is performed by multiple providers, chosen based on network availability and latency. AIVAX uses a mix of providers to perform internet searches.

AIVAX provides two types of searches configurable via its dashboard:

- **Full**: the performed search is complete, inserting the entire content of each result into the conversation context.
- **Summarized**: the performed search is summarized, inserting into the conversation context a summary generated by AI by the search provider itself.

The `Full` mode may consume more input tokens from the conversation, but can provide more precise results. See [Pricing](https://docs.aivax.net/docs/pricing.md) and [Plans and limits](https://docs.aivax.net/docs/limits.md) before enabling internet search in production.

> [!NOTE] 
>
> **Important:** the `Full` search is not always available.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "WebSearch"
    ],
    "options": {
        "web_search_max_results": 10,
        "web_search_mode": "full"
    }
}
```

## Advanced Internet Search

`AdvancedWebUsage` is disabled and returns an unavailable response. See [Changelogs](https://docs.aivax.net/docs/changelogs.md) for details.

## Code Execution

This function allows the model to execute JavaScript code and inspect the execution result. With this, the model can evaluate algorithmic results of mathematical expressions and other situations that are better represented through code.

The code runs in a protected JavaScript environment. It is intended for calculations and small transformations, not for file I/O, network access, or importing external scripts.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "Code"
    ],
    "options": {
    }
}
```

## URL Context

This function allows the model to access external content at URLs and links provided by the user. With this function, the model can access links and evaluate their content.

Note that some destinations may identify the access as a bot and block it, as this function is not crawling but a simple GET to the destination.

Upon obtaining link content, the system checks the return content and handles it according to each type:

- HTML content is rendered: HTML tags, scripts, CSS, and “noise” are removed from the access result, keeping only the plain text of the link.
- Other textual content: the content is read directly and no transformation is performed.
- Non‐textual content: when the link responds with non‐textual content and the response indicates a filename (either by path or by the `Content‐Disposition` header), the system attempts to convert the downloaded file to a textual version.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "OpenUrl"
    ],
    "options": {
    }
}
```

## Memory

This function allows the model to store relevant content to be used across multiple conversations.

> Currently, this function is only available when used in [chat clients](https://docs.aivax.net/docs/features/chat-clients.md) and when the session is identified by a `tag`.

Through the session `tag`, the model stores a relevant piece of conversation data, such as name preferences or persistent context the assistant should remember.

The memory tool requires an identifiable session. Without a user reference ID, memory operations return an error instead of storing or searching information. The memory instruction tells the model not to save sensitive or personal data, however, it is not guaranteed that the model will always follow this rule.

Each saved memory can include a retention period. Memory items can be searched, updated, removed individually, or cleared for the user.

> Note: in chat/completions requests, the `tag` is specified in the `$.user` parameter.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "Remember"
    ],
    "options": {
        "include_all_memory_context": true
    }
}
```

For application-level controls around memory writes, retention, and review, see [How to protect LLM agent memory from poisoning](https://aivax.net/blog/persistent-memory-is-a-write-path/).

## Image Generation

This function allows the model to create AI images.

AI‐generated images are attached to the conversation context, but are not directly visible to the assistant.

Image generation can incur usage charges. See [Pricing](https://docs.aivax.net/docs/pricing.md) before enabling it in production.

You can also enable the generation of explicit and adult images in image generation. When this feature is enabled, the model will be allowed to generate adult material. For this to happen, the model must also “agree” to generate such content. Some models have a lower security filter than others. For example, Gemini models have the lowest security filter, making them a viable option for role‐play and generating such material.

You are always responsible for the [material you generate](https://docs.aivax.net/docs/legal/terms-of-service.md) and the generated material must be compatible with our terms of service.

The available image generation models are listed in the AIVAX console.

Generated images are stored on AIVAX servers for a few months before being permanently removed.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "ImageGeneration"
    ],
    "options": {
        "image_generation_model_name": "grok-imagine",
        "image_generation_allow_reference_usage": true,
        "image_generation_quality": "high",
        "image_generation_max_results": 2,
        "image_generation_allow_mature_content": false
    }
}
```

## X Posts Search

This function allows the model to search for posts on X (formerly Twitter), and to read a specific post when the model has a post ID.

It is a direct alternative to `web_search`, as it can be used to look for up‐to‐date information in real time, such as news, information, game results, etc. This tool provides much more recent results than the conventional internet search tool.

It is not recommended to use both functions together because they have the same purpose.

This function can incur usage charges. See [Pricing](https://docs.aivax.net/docs/pricing.md) before enabling it in production.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "XPostsSearch"
    ],
    "options": {
    }
}
```

## Document Generation

This function allows the model to create PDFs from HTML text.

The created files are hosted on AIVAX servers and made available by the assistant.

The content is hosted for a few months before being permanently deleted.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "GenerateDocument"
    ],
    "options": {
    }
}
```

## Web Page Generation

This function allows the model to host HTML pages on AIVAX servers.

This allows the model to host reports, landing pages, and other HTML infographics.

The content is hosted for a few months before being permanently deleted.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "GenerateWebPage"
    ],
    "options": {
    }
}
```

## Advanced Request

This function provides the model with an advanced HTTP request tool. With this function, the model can set headers, forms, contents, and methods to perform advanced HTTP requests.

Text responses are read up to the platform content limit. Binary responses are not expanded into the context; the tool returns a short binary-content marker with the content type and size when available.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "Request"
    ],
    "options": {
    }
}
```

## Calendar

Calendar is backed by the same persistent information store as memory, but stores date-based reminder objects instead of loose memory text. It can create, search, find, update, and delete appointments for an identified user.

It is not recommended to activate this function together with the memory function or message scheduling functions of the chat client.

Activation via `builtin_tools`:

```json
{
    "tools": [
        "Calendar"
    ],
    "options": {
    }
}
```

## Tool Diagnosis

When a tool is not called, first confirm that it is enabled in the gateway or in the `builtin_tools` field of the request. Then, check whether the selected model supports function calls or if a tool handler is configured for models without native support. Next, review the instruction: if it does not specify when to search, open a URL, generate an image, or query memory, the model may respond only with its own knowledge. Finally, test a direct question that clearly requires the tool, such as requesting a recent news article for `WebSearch` or asking to open a specific URL for `OpenUrl`.

When a tool is called too often, reduce ambiguity. Tools like `WebSearch`, `AdvancedWebUsage`, and `XPostsSearch` compete for recent information; `OpenUrl` and `Request` can seem similar when the user sends a link; `Remember` and `Calendar` can overlap when the user talks about preferences and dates. Remove unnecessary tools, make gateway instruction descriptions more restrictive, and, when possible, use workers to block or replace calls in specific scenarios.

When a tool fails, treat it as a normal part of the experience. Searches may return little content, URLs may block bots, APIs may deny authorization, image generation may refuse content, and code execution may receive ambiguous input. Instruct the model to explain the limitation objectively and offer the next step, such as requesting another link, trying a more specific query, asking for authorization, or responding based only on the available context. Do not rely on an external tool as the sole way to conclude a critical conversation without an experience fallback.

---

Source: https://docs.aivax.net/docs/tools/mcp.html

# Support for Model Context Protocol (MCP)

You can bind external MCP protocol tools to your [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md). The protocol defines tools that run on the server side and enable the assistant to interact with real-time services.

AIVAX acts as an MCP client for gateway inference: it connects to the configured MCP source, lists tools, converts each tool schema into a model-callable function, and calls the remote MCP server when the model selects that tool.

<img src="/assets/diagrams/mcp-1.drawio.svg">

## When to use MCP

Use MCP when you already have external tools that need to be discovered and called by models in a standardized way. An MCP server is suitable for tool catalogs, integrations with internal systems, stateful operations, tools shared among multiple agents, and environments where you want to keep logic outside of AIVAX. AIVAX acts as an MCP client: it connects the AI Gateway to the remote server, reads the available tools, and allows the model to call those tools during inference.

Do not use MCP only to replace a single simple HTTP call. When you need to expose an isolated function with a specific callback and nonce authentication, [protocol functions](https://docs.aivax.net/docs/tools/protocol-functions.md) are usually simpler. When the capability already exists in AIVAX, such as web search, URL opening, code execution, or image generation, [built‐in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) are usually the most direct path. MCP is better when there is a set of tools with their own schema, when another system already speaks MCP, or when you want the same server to be used by different clients.

In production, treat the MCP server as an API exposed to an agent. Tool descriptions should be clear, schemas should be restrictive, and authentication should be configured in the server headers. The model should not receive overly generic tools such as `execute`, `request`, or `search` without strong descriptions and controlled parameters. Ambiguous tools increase wrong calls; specific tools like `lookup_customer_by_email` or `create_support_ticket` help the model decide better.

### Choosing the function name

The function name should be simple and deterministic about what the function does. Avoid names that are hard to guess or that do not hint at the function’s role, as the assistant may get confused and not call the function when appropriate.

As an example, let’s think of a function that queries a user in an external database. The following names are good examples to consider for the call:

- `search_user`
- `query_user`

Bad names include:

- `search` (implicit, possibly ambiguous)
- `search user` (name with improper characters)

Having the function name, we can think about the function description.

### Choosing the function description

The function description should conceptually explain two situations: what it does and when the assistant should call it. This description should include the scenarios the assistant should consider calling it and when it should not be called, providing a few one‐shot call examples and/or making the function rules explicit.

### Defining MCP servers

You can define your MCP servers in the gateway through a JSON array:

```json
[
    {
        "name": "My MCP server",
        "url": "https://example-server.io/mcp",
        "headers": {
            "Authorization": "<AIVAX_API_KEY>"
        }
    }
]
```

Your MCP server must support **Streamable HTTP** to work with AIVAX as a gateway tool source. You can define custom headers in your MCP server configuration to set up authentication or other needs. Tools, server instructions, and remote skill discovery are cached according to `cacheDuration`; the default is 600 seconds.

## Server instructions and remote skills

Each MCP source accepts `allowClientInstructions` and `allowRemoteSkills`, both enabled by default. Set either option to `false` to exclude that content. Only connect servers whose content you trust: server instructions are added to the gateway context, and remote skills become available through `read_skill` alongside your account skills.

AIVAX requests tools only when the server advertises tools. Skill discovery additionally requires the `resources` capability and the `io.modelcontextprotocol/skills` extension in the server capabilities returned during initialization. AIVAX follows `skills/list` pagination and reads each listed `SKILL.md` from the same server. Remote skills receive origin-qualified identifiers so they do not replace account skills or same-named skills from another source.

This is a limited compatibility integration, not full support for the extension's `2026-07-28` base protocol. The server must accept the existing Streamable HTTP initialization flow and advertise the extension there; servers requiring `server/discover` are not supported by this integration.

Only the root `SKILL.md` is imported. Its byte length, SHA-256 digest, and YAML frontmatter must match the advertised entry; a mismatch fails discovery instead of loading unverified content. Manifests are limited to 512 resources and 16 MiB total declared content per skill. Skills advertising `resources: "dynamic"` are skipped. Supporting files, directory browsing, direct `skills/get` lookup, and scripts are not imported. Frontmatter does not grant tool permissions or populate the gateway's allowed-tool lists; existing tool policies still apply. This integration does not add a per-skill approval system or a new execution sandbox. Disable remote skills when your policy requires those controls.

The same options apply to MCP sources added by gateway workers. Existing account skills remain available. New connections fetch skill content during discovery, while `read_skill` controls when that content enters the model's active skill context.

## Metadata sent with tool calls

For each `tools/call` request, AIVAX adds execution context in `params._meta`, alongside `params.arguments`. Arguments contain the tool input; metadata identifies the calling context and carries application-supplied values. These fields describe tool execution requests, not tool discovery (`tools/list`).

The following example illustrates a call with an identified user, a conversation token, and custom metadata:

```json
{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/call",
    "params": {
        "name": "get_weather",
        "arguments": {
            "location": "New York"
        },
        "_meta": {
            "_aiv_nonce": "<BCRYPT_HASH>",
            "_aiv_external_user_id": "<EXTERNAL_USER_ID>",
            "_aiv_call_source": "WebChatClient",
            "_aiv_conversation_token": "<CONVERSATION_TOKEN>",
            "_aiv_moment": "2025-09-09T16:58:05.0000000+00:00",
            "tenant_id": "<TENANT_ID>",
            "request_id": "<APPLICATION_REQUEST_ID>"
        }
    }
}
```

### AIVAX fields

All paths below are relative to `params._meta`. Names beginning with `_aiv` are reserved; do not use them for custom metadata.

| Field | JSON type | Meaning and availability |
| --- | --- | --- |
| `_aiv_nonce` | `string` or `null` | BCrypt hash derived from the calling account's hook key. Without a configured hook key, its value is `null`. Verify the configured plain-text hook key against this hash as described in [hook authentication](https://docs.aivax.net/docs/authentication.md#hook-authentication); do not compare hash strings or expect the hook key itself. |
| `_aiv_external_user_id` | `string` or `null` | External user identifier carried by the inference context. For chat clients it comes from the session; for chat completions it comes from the request's `user` field. It can be `null` when no user was identified. Use it to look up the user in your application, not as an AIVAX account ID or proof of authorization. |
| `_aiv_call_source` | `string` | Origin of the inference, not the outbound tool transport. An MCP tool called during web chat inference still receives `WebChatClient`, not `McpClient`. See the values below. |
| `_aiv_conversation_token` | `string` or `null` | Conversation correlation token carried by the session or inference request. For chat completions, it comes from `idempotency_key` when supplied. It can be `null`; it is neither an authentication credential nor a unique tool-call ID. Multiple calls in the same conversation may share it. |
| `_aiv_moment` | `string` | Timestamp created when AIVAX prepares this tool call, in ISO 8601 round-trip format with fractional seconds and a UTC offset. It uses the AIVAX server's local clock, not the user's timezone or the conversation start time. Parse the offset and convert to your application's timezone when needed. |

### Call source values

The same string values are used by protocol functions in `context.callSource`:

| Value | Inference origin |
| --- | --- |
| `WebChatClient` | AIVAX web chat client. |
| `ChatCompletionsApi` | OpenAI-compatible chat completions API; the default origin for that API. |
| `FunctionsApi` | Functions API. |
| `IntegrationBot` | Messaging integration bot. |
| `OpenWebUiClient` | Open WebUI client. |
| `McpClient` | Inference initiated through an MCP client. |
| `ValidationApi` | Agentic test validation. |

Treat the source as routing and diagnostic context, not as an authorization role. Consumers should tolerate future source values.

### Custom metadata and security

Custom inference metadata is a map of string keys to string values. It comes from request `metadata` for chat completions or session metadata for chat clients. AIVAX copies these entries directly into `params._meta`: in the example, `tenant_id` and `request_id` are application-defined, not built-in AIVAX fields. There is no nested `metadata` object in the MCP envelope. With no custom metadata, only the AIVAX fields remain.

Do not send secrets in custom metadata: these values are forwarded to the remote tool server. Validate tenant and user access against your own trusted records before using metadata to select data or perform writes. Neither an external user ID, a conversation token, nor a call-source label grants permission by itself.

The nonce authenticates the configured account hook key; it is not a signature of the arguments, a unique request ID, or a replay-prevention mechanism. Keep HTTPS and the MCP server's configured authentication headers, and apply your own authorization and duplicate-operation controls. If your server requires nonce authentication, reject a missing or invalid nonce.

For the equivalent HTTP callback envelope, see [protocol function context](https://docs.aivax.net/docs/tools/protocol-functions.md#context-fields).

## Tool results

Tool results can include text, image, and audio content blocks. Text is added directly to the tool result. Image and audio blocks are attached back into the conversation as multimodal content with generated IDs. Unsupported content block types are reported as unsupported text.

When an MCP tool does not appear for the model, verify that the remote server is reachable, that it supports Streamable HTTP, that the authentication headers are correct, and that the gateway is actually configured with the MCP source. When the tool appears but is not called, review the name, description, and schema. When it is called with bad arguments, restrict the JSON Schema and include property descriptions. When the call fails, make the MCP server return readable errors, because the model needs to understand whether to try another argument, ask the user for information, or terminate the action.

---

Source: https://docs.aivax.net/docs/tools/protocol-functions.html

# Protocol Functions

Protocol functions are AIVAX server-side tools. They let a model request a named action while AIVAX executes the action from the server by calling your HTTP callback or an internal AIVAX callback URL.

Use them when an assistant needs a controlled way to interact with your application, such as checking an order, opening a support ticket, validating a coupon, registering a lead, or querying a private service. The model sees the tool name, description, and JSON argument schema. It does not see the callback URL or headers.

<img src="/assets/diagrams/protocol-functions-1.drawio.svg">

A protocol function has two contracts:

- **The model contract:** the public tool name, description, and JSON Schema that guide the model's tool call.
- **The callback contract:** the hidden URL and optional headers that AIVAX uses to execute the tool after the model calls it.

This separation keeps implementation details out of the prompt while still giving the model a precise capability to use.

## When to use protocol functions

Use protocol functions when you want to expose a specific HTTP action to an AI Gateway without creating a full MCP server. They are good for point integrations, such as checking an order, opening a ticket, fetching a user, validating a coupon, registering a lead, or invoking an internal automation. AIVAX keeps the URL invisible to the model, sends the request from the server side, and adds the textual result to the conversation context.

If you have many tools, dynamic tools, or a system that already implements Model Context Protocol, prefer [MCP](https://docs.aivax.net/docs/tools/mcp.md). If you want to use capabilities already maintained by AIVAX, prefer [built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md). If you need to make decisions before or after inference events, prefer [workers](https://docs.aivax.net/docs/inference/workers.md). Protocol functions sit in the middle: they are simpler than MCP and more specific than workers, but still give the model a controlled tool to execute an external action.

A protocol function must have a tool name, a description, a callback URL, and an argument schema. The name helps the model recognize the action; the description explains when to call; the schema limits the argument format. Function names must be valid JavaScript identifiers with at least three characters. The callback URL and authentication details are not visible to the model. This allows creating specialized tools without exposing internal endpoints, provided your service validates the `X-Request-Nonce`, validates received arguments, and applies its own authorization when the call depends on the user.

Use the following rule of thumb:

| Need | Prefer |
|---|---|
| One or a few stable HTTP callbacks owned by your app | Protocol functions |
| A larger tool catalog, standardized discovery, or an existing MCP server | [MCP](https://docs.aivax.net/docs/tools/mcp.md) |
| Web search, URL reading, code execution, image generation, memory, calendar, or HTTP request tools maintained by AIVAX | [Built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) |
| Event-time policy, message rewriting, dynamic tool injection, or blocking a tool call before execution | [Workers](https://docs.aivax.net/docs/inference/workers.md) |
| Provider-native tool definitions that your own client will execute | Raw `tools` in an OpenAI-compatible request |

### Choosing the function name

The function name should be simple and specific about what the function does. Avoid names that are hard to guess or that do not reflect the function's role, because the assistant may not call the function when it should.

For example, consider a function that queries a user in an external database. Good names include:

- `search_user`
- `query_user`

Weak names include:

- `search` (too broad)
- `query_user_in_database_data` (long and noisy)
- `pesquisa-usuario` (name not in English)
- `search user` (name with improper characters)

Having the function name, we can think about the function description.

### Choosing the function description

The function description should explain what the function does, when the assistant should call it, and when it should not call it. Good descriptions include the expected input, the kind of result returned, and any decision rule the assistant should follow before calling the tool.

For example, a useful description is:

> Use this tool to fetch a customer profile by customer ID when the user asks about account status, recent orders, or support eligibility. Do not call it when the user only provides a name or email; ask for the customer ID first.

That gives the model a clear action, trigger, and limitation. A vague description such as "Search customers" gives the model much less guidance.

## Defining protocol functions

Protocol functions are defined in the [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md). You can declare them directly in the gateway when the list is stable, or provide a remote source that returns the function list when the catalog is managed by another service.

The direct form is the simplest option. Use it when the function list changes rarely and belongs to the gateway configuration itself.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20AI%20Gateway)

```json
{
    "name": "my-ai-gateway",
    "parameters": {
        "baseAddress": "@integrated",
        "modelName": "@openai/gpt-5-mini",
        "protocolFunctions": [
            {
                "name": "list_clients",
                "description": "Use this tool to list and search for the user's clients.",
                "callbackUrl": "https://my-external-api.com/api/scp/users",
                "contentFormat": {
                    "type": "object",
                    "properties": {}
                }
            },
            {
                "name": "view_client",
                "description": "Use this tool to obtain details and orders of a client via its ID.",
                "callbackUrl": "https://my-external-api.com/api/scp/users",
                "contentFormat": {
                    "type": "object",
                    "properties": {
                        "user_id": {
                            "type": "string",
                            "format": "uuid"
                        }
                    },
                    "required": ["user_id"]
                }
            }
        ]
    }
}
```

In the snippet above, the gateway exposes `list_clients` and `view_client` to the model. The model decides whether to call one of them during generation. If it calls `view_client`, AIVAX validates the arguments against `contentFormat`, then sends the callback request to your service.

## Protocol function sources

Protocol function sources let a gateway load its protocol functions from one or more remote endpoints instead of storing every function inline. In the gateway configuration, `protocolFunctionSources` is an array of URL strings, not an array of source objects. Use sources when the function catalog belongs to another service, changes independently from the gateway, or needs to be shared by multiple gateways.

Each source URL is an HTTP endpoint that returns a JSON object with a `functions` array. AIVAX fetches the source before preparing the inference options, converts each returned definition into a server-side tool, and caches the result for 10 minutes per account and source URL. During the cache window, AIVAX reuses the function definitions instead of making another listing request.

Use inline `protocolFunctions` for stable, gateway-owned tools. Use `protocolFunctionSources` for remotely managed catalogs. You can use both in the same gateway, but function names must remain unique after inline functions, sourced functions, MCP tools, built-in tools, and raw tools are combined.

<img src="/assets/diagrams/protocol-functions-2.drawio.svg">

Define the function listing endpoints in your AI Gateway:

```json
{
    "name": "my-ai-gateway",
    "parameters": {
        "baseAddress": "@integrated",
        "modelName": "@openai/gpt-5-mini",
        "protocolFunctionSources": [
            "https://my-external-api.com/api/scp/listings"
        ]
    }
}
```

The function source endpoint receives a `GET` request. It must return a successful HTTP status and a JSON object in this format:

<div class="request-item get">
    <span>GET</span>
    <span>
        https://my-external-api.com/api/scp/listings
    </span>
</div>

```json
{
    "functions": [
        {
            "name": "list_clients",
            "description": "Use this tool to list and search for the user's clients.",
            "callbackUrl": "https://my-external-api.com/api/scp/users",
            "contentFormat": {
                "type": "object",
                "properties": {}
            }
        },
        {
            "name": "view_client",
            "description": "Use this tool to obtain details and orders of a client via its ID.",
            "callbackUrl": "https://my-external-api.com/api/scp/users",
            "contentFormat": {
                "type": "object",
                "properties": {
                    "user_id": {
                        "type": "string",
                        "format": "uuid"
                    }
                },
                "required": ["user_id"]
            }
        }
    ]
}
```

The returned function objects use the same shape as inline `protocolFunctions`: `name`, `description`, `callbackUrl`, optional `headers`, and `contentFormat`.

If the source endpoint returns a non-success status, AIVAX logs the error and the inference request fails while preparing the tools. Make the source endpoint fast, stable, and small enough to return only the functions the gateway should expose. If the catalog is user- or tenant-specific, use the request nonce and your own authorization rules to decide which functions to return.

### Handling function calls

Functions are invoked at the provided endpoint in `callbackUrl` via an HTTP POST request. AIVAX sends `Content-Type: application/json` and a body equivalent to:

```json
{
    "type": "tool_call",
    "function": {
        "name": "view_client",
        "content": {
            "user_id": "example-user-id"
        }
    },
    "context": {
        "externalUserId": "<EXTERNAL_USER_ID>",
        "metadata": {
            "tenant_id": "<TENANT_ID>",
            "request_id": "<APPLICATION_REQUEST_ID>"
        },
        "callSource": "WebChatClient",
        "conversationToken": "<CONVERSATION_TOKEN>",
        "moment": "2025-05-18T03:36:27+00:00"
    }
}
```

The response to this action must return a successful HTTP status (2xx), even for errors the assistant may have made. A non-success response is logged and the model receives a generic tool-call error instead of your response body.

#### Context fields

AIVAX builds `context` separately from the model-generated arguments in `function.content`. Your callback should validate the arguments against the function schema and use context to correlate the request with your application. Context is not an additional set of model-selected function arguments.

| Field | JSON type | Meaning and availability |
| --- | --- | --- |
| `context.externalUserId` | `string` or `null` | External user identifier from the inference context. For a [chat session](https://docs.aivax.net/docs/features/chat-clients.md), this is the session's external user ID; for chat completions, it comes from the request's `user` field. It can be `null` when no user was identified. It is not an AIVAX account ID or an authentication credential. |
| `context.metadata` | `object` | Application-supplied string key/value pairs from the inference request or chat session. For chat completions, these are the request's `metadata` entries. The object is empty (`{}`) when none are supplied. The example's `tenant_id` and `request_id` are custom fields, not names generated by AIVAX. |
| `context.callSource` | `string` | Origin of the inference, such as `WebChatClient`, `ChatCompletionsApi`, or `IntegrationBot`. It does not describe the callback transport: calling a protocol function does not automatically make the value `FunctionsApi`. See the shared [call source values](https://docs.aivax.net/docs/tools/mcp.md#call-source-values) for all current values. |
| `context.conversationToken` | `string` or `null` | Conversation correlation token carried by the session or inference request. For chat completions, it comes from `idempotency_key` when supplied. It can be `null`, and multiple function calls may share it. Do not treat it as a unique callback ID, a credential, or a guarantee that an operation has not already run. |
| `context.moment` | `string` | JSON date-time timestamp created when AIVAX prepares the callback, using the server's local clock. It is not the user's local time or the conversation start time. Parse it as a date-time with its UTC offset rather than relying on a fixed fractional-second precision; convert to your application's timezone when needed. |

Custom metadata stays nested under `context.metadata`; it is not merged into `context`. This differs from [MCP metadata](https://docs.aivax.net/docs/tools/mcp.md#metadata-sent-with-tool-calls), where the same custom entries sit alongside reserved `_aiv_*` fields in `params._meta`.

The authentication nonce is **not** a `context` or `metadata` field. Protocol functions receive it in the `X-Request-Nonce` HTTP header when the account has a hook key. MCP tool calls carry the equivalent value in `params._meta._aiv_nonce` instead. These context fields describe execution POST requests; do not assume that function source-listing requests have this JSON body.

#### Response format

Successful responses should be textual and will be attached as the function response in whatever way the endpoint returns. There is no JSON format or structure for this response, but it is advisable to give a simple, human-readable answer so that the assistant can read the action result.

Errors can be common, such as not finding a client by ID or a field not being in the desired format. In these cases, respond with an OK status and include a human description of the error in the response body and how the assistant can work around it.

Arguments are validated against the JSON Schema before execution. Your callback should still validate received arguments, because schemas can limit structure but cannot replace business authorization or consistency checks. Functions that do not expect arguments should use an empty object schema.

The function response should be written for the model, not for the end user. It may contain data, warnings, and short instructions on how to use the result. For example, if an order search finds the status, respond with the status, expected date, and relevant restrictions. If not found, respond that the order was not located and indicate which data the assistant should ask the user for. Avoid returning huge objects, HTML, raw logs, or internal messages, because this content enters the context and can confuse the next response.

> [!IMPORTANT]
>
> The more functions you define, the more input tokens you will consume in the reasoning process. The function definition, as well as its format, consumes tokens from the reasoning process.

#### Authentication

Request authentication is done via the `X-Request-Nonce` header sent in protocol function calls and source-listing requests.

See the [authentication](https://docs.aivax.net/docs/authentication.md) manual to understand how to authenticate reverse requests from AIVAX.

#### User authentication

Use `context.externalUserId` to look up the caller in your application, then check whether that user may access the requested resource or perform the requested action. If a function requires an identified user, reject calls without a usable identifier rather than silently granting anonymous access.

Validate the nonce and enforce your own user and tenant authorization rules. Request and session metadata can contain caller-supplied values: a `tenant_id`, `callSource`, or `conversationToken` is not proof of permission. Avoid putting secrets in metadata because it is forwarded to your callback. The nonce verifies the account hook key, not the individual user's permissions, and does not sign the body or prevent replay; use your own duplicate-operation controls for writes.

#### Security considerations

For the AI model, only the name, description, and format of the function are visible. It cannot see the endpoint the function points to. It also does not receive the callback headers configured for the function. Treat all callback requests as privileged server-to-server calls: validate the nonce, validate arguments, enforce authorization, and avoid exposing broad write actions unless your application has its own approval rules.

## Specialist functions

In addition to [built-in functions](https://docs.aivax.net/docs/tools/builtin-tools.md), you can define specialist functions that perform specific tasks in your AIVAX account.

You define specialist functions using the `aivax://` URL scheme, following the example below:

```json
{
    "name": "my-ai-gateway",
    "parameters": {
        "baseAddress": "@integrated",
        "modelName": "@openai/gpt-5-mini",
        "protocolFunctions": [
            {
                "name": "search_disease",
                "description": "Use this tool to search for diseases, treatments, and symptoms.",
                "callbackUrl": "aivax://query-collection?collection-id=your-collection-id&top=5&min=0.4",
                "contentFormat": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "type": "string",
                            "description": "Name of the disease, treatment, or symptoms."
                        }
                    },
                    "required": [
                        "query"
                    ]
                }
            }
        ]
    }
}
```

The above function creates a tool for the AI to query a specific [document collection](https://docs.aivax.net/docs/rag/collections.md), guiding the assistant on what to search in that collection and what to expect from a response. This way, you can link multiple RAG collections for an assistant to retrieve specialist content.

You can customize the description of the JSON Schema properties for specialist functions, but their structure is fixed. Specialist function parameters are supplied in the URL via query parameters.

Currently, only one specialist function exists:

- `query-collection`: performs an RAG search on a specified collection.

Query parameters:

- `collection-id`: the UUID of the collection to be searched.
- `top`: a number indicating how many documents should be returned in the search.
- `min`: a decimal indicating the minimum similarity score of the search.
- `refs`: when present, includes referenced parent documents in the RAG result.

Function JSON format:

```json
{
    "type": "object",
    "properties": {
        "query": {
            "type": "string",
            "description": "Search content."
        }
    },
    "required": [
        "query"
    ]
}
```

---

Source: https://docs.aivax.net/docs/tools/shell.html

# Shell

AIVAX offers a virtual shell environment that can be used by agent assistants to execute terminal commands during inference. This feature is especially useful for tasks such as data manipulation, API calls, script execution, and workflows that are easier to express as command-line operations.

The shell environment allows moving selected model tools to the shell side, turning them into CLI commands. This is useful when you have many tools and do not want to expose all of them directly to the model, or when a tool is easier to use through command-line arguments and pipes.

When enabled in an AI Gateway, the model sees a `shell` tool with one argument: `command`. Commands run in a sandboxed shell with network modules, filesystem defaults, and a workspace mounted at `/home/workspace`. Each command is limited to 60 seconds and returns up to 4,096 characters of output to the model.

## Design for the limits

The 60-second timeout and 4,096-character output cap shape how shell tools should behave. Keep commands fast and their output narrow: filter server-side with `grep`, `awk`, or query flags before printing, and prefer tools that return CSV or line-delimited rows the model can slice with pipes. When a result legitimately exceeds the cap, split the work — one command to list or count, follow-ups to fetch slices — or write the full output to a workspace file and read the relevant part back through the Shell file API below.

Long-running operations do not belong in an inference command. Move exports, bulk transforms, and polling loops to [Batch](https://docs.aivax.net/docs/features/batch.md) or an external job, and let the shell handle the interactive slices.

## Adapting tools for shell

In the virtual shell interface, standard command-line utilities and registered shell modules are available. This way, you can adapt your tools to return raw or long outputs, and the model can use the shell's text manipulation tools to extract the relevant information, for example:

```bash
get-users --filter active --format csv | grep "John Doe" | awk -F, '{print $1, $2}'
```

In the line above, `get-users` is a custom tool that returns a list of users in CSV format. The `grep` command filters the results to find "John Doe", and `awk` extracts and formats the desired columns. This tool may have been defined by [MCP](https://docs.aivax.net/docs/tools/mcp.md), [built-in tools](https://docs.aivax.net/docs/tools/builtin-tools.md) or be a [protocol tool](https://docs.aivax.net/docs/tools/protocol-functions.md).

Tools moved into the shell are no longer exposed as direct model functions, except reserved tools such as `shell` and `read_skill`. Configure the shell tool list as either:

- `WhiteList`: only listed tools are exposed as shell commands.
- `BlackList`: listed tools remain direct model tools, and the other non-reserved tools are exposed as shell commands.

Use the runtime function name when listing tools, such as `web_search`, `open_url`, `request`, or a protocol/MCP function name. Each shell command generated from a tool supports `--help` and maps JSON Schema properties to command-line options.

Prefer the whitelist when the model needs a small, predictable command set — every new tool otherwise leaks into the shell automatically. Prefer the blacklist when most tools are shell-friendly and only a few must stay as direct functions for latency or reliability reasons.

## Data persistence

It is possible to define data persistence for the shell environment. When `allowDataPersistence` is enabled and the inference context has a user external ID, AIVAX mounts a persistent workspace scoped to the account and user. This allows the agent to keep files across conversations and sessions for that identified user.

If persistence is disabled, or the inference context has no user external ID, the shell uses an in-memory filesystem and the workspace is discarded after the inference iteration.

Enable persistence only for data the user expects to survive — working documents, generated reports, configuration they manage. Keep secrets, credentials, and other users' data out of the persistent workspace: anything written there endures beyond the session that created it.

## Shell file API

AIVAX also exposes Shell I/O endpoints under `/api/v1/shell/io` for authenticated accounts. These endpoints use the required `X-Shell-User-Id` header to scope the filesystem sandbox and support listing directories, downloading files, inspecting file metadata, creating temporary public file addresses, uploading files, creating directories, and deleting files or directories. Uploads are documented with a 100 MB maximum request body.

Reference:

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=List%20Directory)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Download%20File)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Get%20File%20Details)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Get%20File%20Public%20Address)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Upload%20File)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20Directory)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Delete%20File)
[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Delete%20Directory)

---

Source: https://docs.aivax.net/docs/mcp-utilities/collections-mcp.html

# Collections MCP

The Collections MCP exposes one or more AIVAX RAG collections as tools for compatible MCP clients. Use it when an external model, agent, IDE, or desktop assistant should decide when to search an AIVAX knowledge base.

For information about creating collections, preparing documents, and improving retrieval quality, see [Collections and Documents](https://docs.aivax.net/docs/rag/collections.md) and [Semantic Search](https://docs.aivax.net/docs/rag/semantic-search.md).

## Endpoint

```text
https://inference.aivax.net/v1/mcp/collections
```

## Headers

| Header | Description | Default |
| --- | --- | --- |
| `Authorization` | Bearer token of your API key. | Required |
| `X-Mcp-Collection-Id` | One or more collection IDs. Use commas for multiple collections. | Required |
| `X-Mcp-Collection-Name` | Collection name used to generate tool names. | `collection` |
| `X-Mcp-Reranker` | Selects the ranker used to order search results. Use a canonical `@provider/name`, `lexical`, `rrf`, `smart`, or `none`. | `@aivax/reflex-v1` |
| `X-Mcp-Top-K` | Maximum number of results to return. | `5` |
| `X-Mcp-Min-Score` | Minimum relevance score greater than 0 and up to 1.0. | `0.4` |
| `X-Mcp-Use-References` | Set to `none` to enable references in search results; omit the header to disable them. | disabled |
| `X-Mcp-Allow-Write` | Use `yes` to expose document write and delete tools. | disabled |
| `X-Mcp-Naming-Convention` | Controls how generated tools are named. Use `default` or `agent`. | `default` |

## Configuration example

Visual Studio Code:

```json
{
  "servers": {
    "my-rag-collection-mcp": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/collections",
      "headers": {
        "Authorization": "Bearer <AIVAX_API_KEY>",
        "X-Mcp-Collection-Id": "<COLLECTION_ID>",
        "X-Mcp-Collection-Name": "my_collection",
        "X-Mcp-Top-K": "5",
        "X-Mcp-Min-Score": "0.4",
        // Enables references in search results.
        "X-Mcp-Use-References": "none"
      }
    }
  }
}
```

## Generated tools

With the default naming convention, the read tool is named:

```text
{collection_name}_search
```

It accepts:

- `search_terms` (`string[]`): one or more search terms.

The MCP read tool enforces two request-shaping limits:

- At most 10 search terms per call.
- At most 500 total characters across all search terms.

When `X-Mcp-Allow-Write` is disabled, only the search tool is exposed. This is the recommended mode for assistants that only need to read a knowledge base.

When `X-Mcp-Allow-Write: yes` is sent, the server also exposes document creation/update and delete tools. Enable this only for trusted clients, because a model with write access can change collection contents.

Use Collections MCP when an external model or MCP client should decide when to search. For a typical AIVAX chat client, it is often simpler to attach the collection directly to an [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md) and let the gateway RAG pipeline retrieve documents automatically.

---

Source: https://docs.aivax.net/docs/mcp-utilities/documentation-mcp.html

# Documentation MCP

The AIVAX documentation MCP exposes AIVAX documentation, API reference content, and model metadata to MCP-compatible clients. It is designed for assistants, IDEs, internal agents, and implementation workflows that need current AIVAX context before they answer, write code, configure a gateway, or troubleshoot an integration.

This MCP is read-oriented. It does not expose a generic account API invocation tool. Use it when an agent needs to understand AIVAX features, find the right API route, compare model capabilities, or ground its answer in the product manual. Use the [account management MCP](https://docs.aivax.net/docs/mcp-utilities/account-management-mcp.md) only when the client also needs to inspect or change authenticated account resources through API calls.

> [!NOTE]
> Do not configure the documentation MCP together with the [account management MCP](https://docs.aivax.net/docs/mcp-utilities/account-management-mcp.md) in the same client unless you have a specific reason to duplicate tools. The account management MCP already includes documentation search functions, so adding both servers usually creates redundant documentation tools and can make tool selection less predictable.

## Endpoint

```text
https://inference.aivax.net/v1/mcp/documentation
```

Authenticate with an AIVAX account API key:

```text
Authorization: Bearer <AIVAX_PRIVATE_API_KEY>
```

For key types and authentication options, see [Authentication](https://docs.aivax.net/docs/authentication.md).

## Configuration example

The exact configuration depends on the MCP client. For clients that accept a Streamable HTTP server entry, configure the AIVAX documentation endpoint and pass the API key as a header.

```json
{
  "servers": {
    "aivax-docs": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/documentation",
      "headers": {
        "Authorization": "Bearer <AIVAX_PRIVATE_API_KEY>"
      }
    }
  }
}
```

After the client connects, it can discover the tools exposed by the server. Tool names are prefixed with `aivax_` so they remain clear when the client also has project, database, browser, or code tools available.

## Available tools

### `aivax_search_documentation`

Searches AIVAX documentation and API reference content. Use this tool when the assistant needs product context before answering or acting.

The tool accepts:

| Argument | Type | Required | Description |
| --- | --- | --- | --- |
| `search_terms` | `string[]` | Yes | Search terms to query against AIVAX documentation and API reference. |
| `search_type` | `string` | No | Search scope. Use `documentation-manual`, `api-function-reference`, or `all`. |

The search can query the documentation manual, the API function reference, or both. It returns relevant documentation snippets in text form so the client can use them directly as context. Search is limited to 10 terms per call and 500 total characters across all terms.

Example arguments:

```json
{
  "search_terms": [
    "AI Gateway MCP headers",
    "tool metadata"
  ],
  "search_type": "all"
}
```

Use more complete phrases when the question has a clear intent, such as `semantic search reranker settings` or `public key chat completion restrictions`. Use multiple terms when you want to cover neighboring concepts, alternate names, or likely API-reference terms.

Search calls use the per-account quotas documented in [Plans and limits](https://docs.aivax.net/docs/limits.md#plan-limits).

### `aivax_list_models`

Lists integrated AIVAX chat models and returns a model-readable summary for each match. Use it when the assistant needs to choose a model, explain whether a model is available to the current plan, compare capabilities, or understand pricing (see [Pricing](https://docs.aivax.net/docs/pricing.md)).

The tool accepts:

| Argument | Type | Required | Description |
| --- | --- | --- | --- |
| `name_filter` | `string` | No | Optional fuzzy filter for model names, such as `gpt 5`, `sonnet`, `qwen coder`, or `@openai/gpt-5-mini`. |

The response includes model description, stability, type, capabilities, flags, rate-limit group, routing model, subscription multiplier, technical metadata, token pricing, providers, and availability for the authenticated account plan. For current pricing details, see [Pricing](https://docs.aivax.net/docs/pricing.md).

Example arguments:

```json
{
  "name_filter": "gemini flash"
}
```

Model listing calls use the per-account quotas documented in [Plans and limits](https://docs.aivax.net/docs/limits.md#plan-limits).

## When to use it

Use the documentation MCP when you want an assistant to answer AIVAX questions from source-backed context instead of memory. This is useful in IDEs, support tools, onboarding agents, internal implementation copilots, and evaluation workflows where the assistant should search the manual before recommending a route, parameter, feature, model, or debugging step.

It is also helpful for agent-building workflows. Before creating or editing an [AI Gateway](https://docs.aivax.net/docs/inference/ai-gateway.md), an assistant can search for the relevant feature, check model capabilities, then explain what configuration should be used and why. For example, it can compare built-in tools, MCP functions, server-side functions, workers, RAG collections, structured responses, and multimodal pre-processing before suggesting a design.

For troubleshooting, the documentation MCP helps the assistant move from an error message to the likely product boundary. It can search for authentication rules, plan limits, multimodal balance requirements, RAG search parameters, gateway behavior, or public-key restrictions, then produce a focused checklist that reflects AIVAX behavior.

For model selection, combine `aivax_search_documentation` with `aivax_list_models`. Search the manual for the feature requirement, such as tool calling, video input, structured output, or long context, then list matching models and choose one that is available to the account plan.

## Choosing search terms

Good search terms should describe the user's goal, not only one keyword. Prefer:

```text
public key chat completion restrictions
semantic search include references
AI Gateway MCP source headers
multimodal preprocess video
```

Over:

```text
key
search
headers
video
```

When the assistant is unsure which page contains the answer, use `search_type: "all"`. When it needs route names, request bodies, or endpoint behavior, use `api-function-reference`. When it needs conceptual guidance, tradeoffs, or workflow explanations, use `documentation-manual`.

## Security guidance

The documentation MCP is safer than a management tool because it is read-oriented, but it still authenticates as an AIVAX account and may expose account-plan-sensitive model availability. Connect it only to clients that should know the account's available models and documentation context.

Use a dedicated API key for each MCP client. Store it in the client's secret mechanism or local configuration store, not in source control. If a client only needs public documentation and does not need account-specific model availability, prefer linking directly to the public documentation site instead of connecting an authenticated MCP server.

---

Source: https://docs.aivax.net/docs/mcp-utilities/account-management-mcp.html

# Account management MCP

The account management MCP exposes selected AIVAX account operations to an MCP-compatible client, such as an IDE, desktop assistant, internal agent, or automation environment. It is designed for trusted operators and backend workflows that need to inspect account capabilities, discover documentation, or call authenticated AIVAX API routes without leaving the MCP client.

This endpoint is different from configuring an external MCP source inside an [AI Gateway](https://docs.aivax.net/docs/tools/mcp.md). In that flow, AIVAX is the MCP client and your gateway calls another server during inference. With the account management MCP, AIVAX is the MCP server. Your MCP client connects to AIVAX and receives tools for operating the authenticated account.

Use this MCP when an agent needs account-aware context before it acts: which models are available to the current plan, how a feature is documented, what an API route returns, or whether an account resource can be created, updated, or inspected through the existing AIVAX API. It is especially useful for internal support assistants, development environments, account administration copilots, and implementation agents that need to combine documentation lookup with real API calls.

Because the MCP can invoke authenticated AIVAX functions, connect it only from trusted clients and use a private API key. Do not expose this server to end users or browser-side applications.

> [!NOTE]
> Do not configure the account management MCP together with the [documentation MCP](https://docs.aivax.net/docs/mcp-utilities/documentation-mcp.md) in the same client unless you have a specific reason to duplicate tools. The account management MCP already includes documentation search functions, so adding both servers usually creates redundant documentation tools and can make tool selection less predictable.

## Endpoint

```text
https://inference.aivax.net/v1/mcp/account-management
```

The request must authenticate with an account API key. Use a private key with the `Authorization` header:

```text
Authorization: Bearer <AIVAX_PRIVATE_API_KEY>
```

For key types and authentication options, see [Authentication](https://docs.aivax.net/docs/authentication.md).

## Configuration example

The exact MCP configuration shape depends on the client. For clients that accept a Streamable HTTP server entry, configure the AIVAX endpoint and send the private key as a header.

```json
{
  "servers": {
    "aivax-account": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/account-management",
      "headers": {
        "Authorization": "Bearer <AIVAX_PRIVATE_API_KEY>"
      }
    }
  }
}
```

After the client connects, it can list the tools exposed by the account management server. The tool names are stable and intentionally prefixed with `aivax_` so they remain clear when mixed with tools from other MCP servers.

## What you can use it for

The account management MCP is useful when the assistant needs to reason about the account itself, not only answer an end-user prompt. It gives an MCP client a controlled way to combine AIVAX documentation, model metadata, and authenticated account API calls. This makes it a good fit for operational agents, implementation copilots, and internal assistants that need to inspect how an AIVAX workspace is configured before recommending or changing something.

### Create and maintain agents

An internal development agent can use the MCP to help create, review, and adjust [AI Gateways](https://docs.aivax.net/docs/inference/ai-gateway.md). Before changing a gateway, the agent can search the AIVAX manual for the relevant feature, list available models for the current account, compare model capabilities and plan availability, then invoke the appropriate account API route.

This is useful when teams frequently create assistants for different departments, tenants, or products. The MCP lets the operator ask for an agent in product terms, such as "create a support assistant for refund policies with the CRM MCP enabled", while the implementation assistant checks which models, tools, RAG collections, and gateway options are available in the account.

For production workflows, keep a human approval step before writes. The MCP can help prepare the configuration, explain the tradeoffs, and show the API action it intends to perform before it modifies account state.

### Monitor costs and model choices

The MCP can help an operations assistant investigate cost patterns and model selection. By listing models and invoking account API routes for usage, billing, gateway, or key information, the assistant can explain which models are expensive, which routes are using a subscription multiplier, and whether a cheaper model could handle part of the workload.

This is especially useful when a team has many gateways or batch jobs and wants to understand why spend changed. Instead of looking only at totals, an assistant can connect usage to model metadata, plan availability, gateway configuration, and feature choices such as RAG, tools, batch, or multimodal input.

Use this for recurring checks such as "which gateways are likely to drive cost this week?", "which model families are being used most?", or "can we move this low-risk workflow to a smaller model without losing required capabilities?"

### Understand errors and improve observability

When an integration fails, the account management MCP can help an assistant move from a generic error to a useful diagnosis. The assistant can search documentation for the failing route or feature, inspect account resources through API calls, and compare the observed response with the expected behavior.

For example, a support assistant can investigate whether a failure is caused by an expired API key, insufficient balance, plan restrictions, a missing collection, a disabled provider route, an invalid gateway configuration, or a tool schema problem. The result is a clearer explanation: what failed, where it likely failed, what evidence supports that conclusion, and what the operator should check next.

This kind of observability is most valuable when the assistant is allowed to read relevant account state but not automatically change it. Give write access only to trusted maintenance workflows.

### Meta-prompting and conversation review

The MCP can support meta-prompting workflows where an assistant reviews prior conversations, gateway behavior, and documentation to suggest improvements. The goal is not to answer the original user again; it is to inspect how the agent behaved and identify what could be improved in prompts, instructions, tools, model choice, or RAG setup.

A review agent can look for patterns such as overlong answers, missing questions, wrong tool selection, repeated clarification loops, unsafe assumptions, or responses that ignore retrieved context. It can then propose concrete changes: a better system instruction, a narrower tool description, a stronger structured-output schema, a different model, or an added retrieval source.

This is useful for teams that treat assistants as products. Instead of tuning prompts only by intuition, they can review real interactions and turn the findings into smaller gateway or knowledge-base changes.

### Identify why models fail in specific situations

Some model failures are not caused by the model alone. A wrong answer can come from missing context, a weak prompt, a bad retrieval result, an unavailable tool, a mismatched model capability, or a schema that lets the model produce ambiguous output.

With account context available through MCP, an assistant can investigate these layers together. It can check which model was selected, whether that model supports the needed capability, what gateway configuration was active, whether the relevant RAG collection exists, and which documentation explains the expected behavior.

This helps answer questions like "why does the assistant fail when users ask about refunds?", "why does this model ignore a tool?", or "why do answers degrade when the request includes a file?" The result should be an evidence-based explanation and a focused fix, such as changing the model, improving the tool description, adding retrieval material, or rewriting the gateway instruction.

### Improve RAG quality

The account management MCP is also useful for maintaining RAG systems. An assistant can inspect collection-related API behavior, search the AIVAX documentation for retrieval guidance, and help compare gateway configuration with the intended retrieval flow.

Use it to investigate weak retrieval, missing citations, irrelevant chunks, overly broad searches, low-quality documents, or cases where a gateway should use a collection but does not. A RAG maintenance assistant can suggest better query phrasing, chunking changes, collection organization, reranker settings, `top` and `minScore` adjustments, or when to expose a collection through [collection MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md).

The best workflow is iterative: inspect a failing answer, identify which context should have been retrieved, test or review the retrieval path, update documents or gateway settings, then re-check the same conversation pattern.

## Security guidance

Treat this MCP server as an administrative integration. A private key connected to it may be able to read or mutate account resources depending on the routes the agent invokes.

Use a dedicated API key for each MCP client or automation. Label it clearly, set an expiration when possible, and rotate it if the client is shared, compromised, or no longer needed. Store the key in the MCP client's secret mechanism or local configuration store, not in source control.

Do not connect the account management MCP to untrusted agents, public chat clients, or user-controlled browser sessions. If a workflow only needs retrieval from a RAG collection, use the [collection MCP](https://docs.aivax.net/docs/mcp-utilities/collections-mcp.md) with read-only configuration. If a gateway needs to call your external tools during inference, configure [MCP functions](https://docs.aivax.net/docs/tools/mcp.md) or [server-side functions](https://docs.aivax.net/docs/tools/protocol-functions.md) instead.

---

Source: https://docs.aivax.net/docs/mcp-utilities/web-utilities-mcp.html

# Web utilities MCP

The web utilities MCP exposes AIVAX web retrieval tools to any MCP-compatible client. Use it when an agent, IDE, desktop assistant, or automation environment needs AIVAX web search and URL fetching without running those tools through an AIVAX model inference.

AIVAX hosts this MCP server and performs the web operations for the authenticated account. The tools use the same responses, billing, and applicable limits as their built-in counterparts.

## Endpoint

```text
https://inference.aivax.net/v1/mcp/web-utilities
```

The server uses Streamable HTTP. Authenticate requests with an account API key:

```text
Authorization: Bearer <AIVAX_API_KEY>
```

For key types and authentication options, see [Authentication](https://docs.aivax.net/docs/authentication.md).

## Configuration example

The exact configuration shape depends on the MCP client. The following example enables both tools:

```json
{
  "servers": {
    "aivax-web": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/web-utilities",
      "headers": {
        "Authorization": "Bearer <AIVAX_API_KEY>",
        "X-Mcp-Enabled-Tools": "fetch_url, web_search"
      }
    }
  }
}
```

After the client connects, it can discover and call the enabled tools through the standard MCP `tools/list` and `tools/call` methods.

## Select which tools are exposed

Use the optional `X-Mcp-Enabled-Tools` request header to control which tools the server exposes to that client. Provide a comma-separated allowlist containing `fetch_url`, `web_search`, or both:

```text
X-Mcp-Enabled-Tools: fetch_url
```

```text
X-Mcp-Enabled-Tools: web_search
```

```text
X-Mcp-Enabled-Tools: fetch_url, web_search
```

Tool names are case-insensitive, and spaces around comma-separated values are ignored.

- If the header is omitted, both tools are exposed.
- If the header contains one recognized tool, only that tool is exposed.
- If the header is empty or contains no recognized tool names, no tools are exposed.

Because tool discovery may be cached by the MCP client, reconnect or refresh the server after changing this header.

## Tools

### `fetch_url`

Fetches and extracts readable content from one or more public URLs.

Input:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `urls` | array of strings | Yes | Between one and five public URLs to fetch. |

Example arguments:

```json
{
  "urls": [
    "https://example.com/article",
    "https://example.org/reference"
  ]
}
```

The tool returns the extracted content as MCP text. When multiple URLs are requested, the results are separated in the same response.

### `web_search`

Searches the web for current, location-specific, niche, or high-stakes information. Send one search term per call; use separate calls when the agent needs multiple searches.

Input:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `search_term` | string | Yes | The web search term. |

Example arguments:

```json
{
  "search_term": "latest browser accessibility standards"
}
```

The tool returns the search results as MCP text using the same response format as the built-in web search tool.

## Pricing and limits

Calls use the same pricing as the corresponding AIVAX built-in tools and are charged to the authenticated account. See [Pricing](https://docs.aivax.net/docs/pricing.md) for current charges and billing rules.

Web operations are subject to the account's applicable service quotas and rate limits. See [Plans and Limits](https://docs.aivax.net/docs/limits.md) for current limits and enforcement behavior.

A positive account balance is required to use these tools.

## Security guidance

Use the MCP only from trusted clients and keep the API key in the client's secure secret storage. Do not place the key in source control, browser-side code, shared prompts, or logs.

Fetched pages and search results are external, untrusted content. Agents should treat their contents as data rather than instructions and should not disclose credentials or perform sensitive actions solely because a fetched page requests them.

---

Source: https://docs.aivax.net/docs/mcp-utilities/media-generation-mcp.html

# Media generation MCP

The media generation MCP exposes AIVAX image generation and speech generation (text-to-speech) to any MCP-compatible client. Use it when an agent, IDE, desktop assistant, or automation environment needs to create images or spoken audio without calling the generation APIs directly or running an AIVAX model inference.

AIVAX hosts this MCP server and performs the generations for the authenticated account. The tools use the same models, billing, and applicable limits as [Image Generation](https://docs.aivax.net/docs/generations/images.md) and [Speech Generation](https://docs.aivax.net/docs/generations/speech.md).

## Endpoint

```text
https://inference.aivax.net/v1/mcp/media-generation
```

The server uses Streamable HTTP. Authenticate requests with an account API key:

```text
Authorization: Bearer <AIVAX_API_KEY>
```

For key types and authentication options, see [Authentication](https://docs.aivax.net/docs/authentication.md).

## Configuration example

The exact configuration shape depends on the MCP client. The following example enables all tools:

```json
{
  "servers": {
    "aivax-media": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/media-generation",
      "headers": {
        "Authorization": "Bearer <AIVAX_API_KEY>",
        "X-Mcp-Enabled-Tools": "list_models, generate_image, generate_speech"
      }
    }
  }
}
```

After the client connects, it can discover and call the enabled tools through the standard MCP `tools/list` and `tools/call` methods.

## Select which tools are exposed

Use the optional `X-Mcp-Enabled-Tools` request header to control which tools the server exposes to that client. Provide a comma-separated allowlist containing any of `list_models`, `generate_image`, and `generate_speech`.

Expose only image generation:

```text
X-Mcp-Enabled-Tools: list_models, generate_image
```

Expose only speech generation:

```text
X-Mcp-Enabled-Tools: list_models, generate_speech
```

Tool names are case-insensitive, and spaces around comma-separated values are ignored.

- If the header is omitted, all tools are exposed.
- If the header lists recognized tools, only those tools are exposed.
- If the header is empty or contains no recognized tool names, no tools are exposed.

Keep `list_models` enabled alongside a generation tool so the agent can discover valid model names instead of guessing them. Because tool discovery may be cached by the MCP client, reconnect or refresh the server after changing this header.

## Tools

### `list_models`

Lists the models available for image generation or speech generation, including each model's description and pricing. Image models also indicate whether they accept reference images. Deprecated image models are not listed.

Input:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `type` | string | Yes | `image` to list image generation models, or `audio` to list speech generation models. |

Example arguments:

```json
{
  "type": "audio"
}
```

The tool returns the model list as MCP text. Use the returned model name as the `model` argument of the corresponding generation tool. Calling this tool has no charge.

### `generate_image`

Generates one to four images from a prompt and returns their URLs.

Input:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `prompt` | string | Yes | A non-empty description of the image to generate. |
| `model` | string | Yes | An image model name returned by `list_models` with `type` set to `image`. Case-insensitive. |
| `count` | integer | No | How many images to generate, from 1 to 4. Defaults to 1. |
| `reference_images` | array of strings | No | Up to four public HTTP(S) image URLs that guide the result. Only accepted by models that support reference images. |

Example arguments:

```json
{
  "prompt": "A flat illustration of a lighthouse at dusk, warm palette, no text",
  "model": "<IMAGE_MODEL_NAME>",
  "count": 2
}
```

The tool returns the generated image URLs as MCP text. The URLs are publicly accessible, so anyone with a URL can open the image. Download and store the images if your workflow needs to keep them under its own access control.

For prompt guidance and reference-image behavior, see [Image Generation](https://docs.aivax.net/docs/generations/images.md).

### `generate_speech`

Synthesizes speech from text and returns the URL of the generated MP3 audio.

Input:

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `input` | string | Yes | The non-empty text to synthesize. |
| `model` | string | Yes | A speech model name returned by `list_models` with `type` set to `audio`. Case-insensitive. |
| `voice` | string | No | A voice supported by the selected model. The model's default voice is used when omitted. |
| `instructions` | string | No | Guidance about tone, emotion, or pacing for the generated speech. |

Example arguments:

```json
{
  "input": "Your order has shipped and should arrive on Thursday.",
  "model": "<SPEECH_MODEL_NAME>",
  "instructions": "Friendly and calm, moderate pace."
}
```

The tool returns the URL of the MP3 audio as MCP text. The URL is publicly accessible, so anyone with it can play the audio. Unlike the Speech Generation API, this tool does not return the audio inline and does not offer other output formats; convert the downloaded file if your workflow needs WAV or OGG.

Voices differ per speech model. For voice selection and text preparation guidance, see [Speech Generation](https://docs.aivax.net/docs/generations/speech.md).

## Pricing and limits

Generation calls use the same pricing as [Image Generation](https://docs.aivax.net/docs/generations/images.md) and [Speech Generation](https://docs.aivax.net/docs/generations/speech.md) and are charged to the authenticated account. See [Pricing](https://docs.aivax.net/docs/pricing.md) for current charges and billing rules.

Generations are subject to the account's applicable service quotas and rate limits. See [Plans and Limits](https://docs.aivax.net/docs/limits.md) for current limits and enforcement behavior.

A positive account balance is required to use these tools.

## Security guidance

Use the MCP only from trusted clients and keep the API key in the client's secure secret storage. Do not place the key in source control, browser-side code, shared prompts, or logs.

Generated media URLs are public. Do not generate images or audio containing personal, confidential, or otherwise sensitive information unless the workflow accounts for that exposure. Review generated assets for accuracy and suitability before publishing them.

---

Source: https://docs.aivax.net/docs/mcp-utilities/inference-mcp.html

# Inference MCP

The Inference MCP exposes an AIVAX integrated model or AI Gateway as a tool for compatible MCP clients. Use it when another model, agent, IDE, or desktop assistant should call the configured AIVAX model or gateway as a sub-agent.

For information about configuring models, instructions, RAG, tools, and workers on the underlying gateway, see [AI Gateways](https://docs.aivax.net/docs/inference/ai-gateway.md).

## Endpoint

```text
https://inference.aivax.net/v1/mcp/inference
```

## Headers

| Header | Description | Required |
| --- | --- | --- |
| `Authorization` | Bearer token for your AIVAX API key. | Yes |
| `X-Mcp-Model-Name` | Integrated model tag, gateway full ID, or gateway slug. | Yes |
| `X-Mcp-Tool-Name` | Base tool name. AIVAX converts it to identifier format and exposes `invoke_{tool_name}`. | No, defaults to `ai_model` |
| `X-Mcp-Tool-Description` | Description shown to the MCP client. | No |
| `X-Mcp-Tool-Title` | Friendly title shown to the MCP client. | No |
| `X-Mcp-User` | External user ID stored in the inference context. | No |

## Configuration example

```json
{
  "servers": {
    "my-ai-gateway-mcp": {
      "type": "http",
      "url": "https://inference.aivax.net/v1/mcp/inference",
      "headers": {
        "Authorization": "Bearer <AIVAX_API_KEY>",
        "X-Mcp-Model-Name": "<MODEL_TAG_OR_GATEWAY_ID>",
        "X-Mcp-Tool-Name": "data_assistant",
        "X-Mcp-Tool-Description": "Use this tool to invoke the specialized assistant for data analysis.",
        "X-Mcp-Tool-Title": "Data Analysis Assistant"
      }
    }
  }
}
```

The generated MCP tool accepts one argument:

| Parameter | Type | Description |
| --- | --- | --- |
| `prompt` | string | Prompt sent to the configured model or gateway. |

The MCP tool returns the gateway response as text and shares the same inference billing and rate-limit path as the underlying chat completion.

---

Source: https://docs.aivax.net/docs/legal/privacy-policy.html

# Privacy Policy

Revision: 1.4
Effective date: August 1, 2026
Previous update: July 20, 2026
Last updated: September 26, 2026

---

Welcome to AIVAX. This Privacy Policy describes how AIVAX collects, uses, stores, shares, and protects information in its AI inference services. It is written for transparency and technical clarity, and it does not constitute legal advice. Account Managers should review it with their own legal counsel when needed.

By using AIVAX services, the Account Manager acknowledges and agrees to the terms of this policy.

### 1. Definitions

- **AIVAX:** Company providing the platform and AI model orchestration services.
- **AIVAX Account Manager ("Account Manager"):** Natural or legal person who creates and administers the account and integrates the API.
- **End User:** Individual who interacts with the Account Manager's application that consumes the AIVAX API.
- **Account Data:** Registration and administrative data, such as name, email, company, role, internal identifiers, preferences, and API key settings.
- **Billing Data:** Data required for invoices, receipts, account balance, credits, payment events, and payment processing through third-party processors.
- **Inference Data:** Inputs sent to models and resulting outputs. In the Terms of Use, this maps to Input Content and Generated Content.
- **Semantic Training Data:** Anonymized eligible RAG and reranking content collected after the Account Manager enables semantic data collection, as described in [Data Collecting](https://docs.aivax.net/docs/data-collecting.md). Document indexing and storage are excluded.
- **Conversations:** Stored sequences of inference interactions, including messages, metadata, model name, usage object, tools, resources, and error information when conversation logging is enabled.
- **Technical Metadata:** Inference and access logs, IP addresses or forwarding information, user-agent strings, timestamps, latency, session identifiers, token usage, response codes, request identifiers, and security or abuse signals.
- **Processor:** AIVAX when it processes data according to the Account Manager's instructions.
- **Controller:** AIVAX when it defines purposes for account, billing, security, and compliance data.
- **Subprocessor:** Third party contracted or configured by AIVAX to support processing, such as infrastructure, email, billing, model providers, search providers, or object storage.

---

### 2. Processing Roles

| Data Type | AIVAX Role | Account Manager Role |
| --- | --- | --- |
| Account Data | Controller | Data subject or controller of its own internal relationship |
| Billing Data | Controller for legal, contractual, and operational billing purposes | Provides and verifies |
| Inference Data / Conversations | Processor for customer-directed processing; Controller for security, abuse prevention, and operational logs where applicable | Controller of content and purpose |
| Semantic Training Data | Controller for model and semantic-system development based on the Account Manager's consent | Controller of the originating content and responsible for the legal basis and required notices to end users |
| Technical Metadata | Controller for security, reliability, abuse prevention, and platform operations; Processor when generated as part of service execution | Controller of the originating application context |

When acting as Processor, AIVAX follows the Account Manager's instructions as expressed through API calls, dashboard settings, model selection, and configured integrations.

For contractual consistency, Inference Data in this policy corresponds to Input Content and Generated Content in the Terms of Use.

---

### 3. Categories of Collected Data

1. **Provided directly by the Account Manager:** Account Data, preferences, account settings, organization information, generated API keys, and credentials stored as hashes or tokens where applicable.
2. **Generated by use:** Technical metadata, usage records, token counts, latency, model usage, request resources, error information, and billing events.
3. **Inference Data and Conversations:** Text and other content sent to models and model outputs are processed to execute requests. Conversation content and message history are collected and logged only when the Account Manager enables the observability feature ("Conversations") for the request or globally for the account, as described in Section 5.
4. **Semantic Training Data:** Anonymized RAG query terms, contents and relevance scores of documents returned by a search, selected reranker, reranking queries, submitted documents, and ranking results collected while semantic data collection is enabled. Document indexing and storage are not collected. These anonymized records exclude account, API key, request, collection, and document identifiers, document names, billing data, and collection timestamps.
5. **Support and Communication:** Ticket messages, emails sent to support or contact channels, and operational communications.
6. **Billing:** Fiscal, payment, invoice, credit, account balance, payment intent, and payment webhook data.
7. **Aggregated or Anonymized Data:** Derived metrics that do not identify the Account Manager or end users.

AIVAX does not require special categories of sensitive personal data. If the Account Manager sends sensitive personal data in Inference Data, the Account Manager is responsible for having the appropriate legal basis and notices.

---

### 4. Purposes, Legal Bases, and Retention

| Category | Primary Purpose | Legal Basis (LGPD) | Technical Retention or Limit |
| --- | --- | --- | --- |
| Account Data | Account creation, authentication, account management, operational communications | Contract execution / legitimate interest | While the account is active; disabled or inactive accounts may be deleted by scheduled cleanup according to platform rules |
| Billing Data | Invoices, credits, payment confirmation, fraud prevention, tax and accounting records | Legal obligations / contract execution | According to legal, accounting, and contractual requirements |
| Technical Metadata | Security, abuse prevention, debugging, reliability, rate limiting, cost accounting | Legitimate interest / contract execution | Inference and access logs and metadata, including IP addresses and user-agent strings, are retained for up to 1 year |
| Inference Data | Execution of requested inference and configured integrations | Contract execution | Processed for the request and may be stored inside Conversations when conversation logging is enabled |
| Semantic Training Data | Develop, train, fine-tune, evaluate, test, and improve models and semantic retrieval or ranking systems | Consent | Retained in anonymized form while reasonably necessary for these purposes, legal obligations, security, and audits; then deleted |
| Conversations | Monitoring, support, export, debugging, usage review, and user-facing history | Legitimate interest / contract execution | Visible/exportable according to plan retention: Free up to 2 hours, Pro up to 2 days, Max up to 30 days |
| Support | Resolve questions, incidents, and compliance requests | Contract execution / legitimate interest | Retained as needed to resolve the request and maintain business records |
| Aggregated or Anonymized Data | Capacity planning, reliability, abuse prevention, service improvement | Outside LGPD scope when irreversibly anonymized | Indeterminate while anonymized |

Conversation export periods are 2 hours, 1 day, 7 days, and 30 days, capped by the account plan's retention period.

---

### 5. Conversation Logging and Deletion

AIVAX collects and logs conversations only when the Account Manager enables the observability feature ("Conversations"), either for the individual request or globally for the account. If it is not enabled for the request or through the account-wide setting, AIVAX does not collect, log, or store that request's conversation content. Processing inputs and outputs to execute inference does not create a stored conversation record.

AIVAX cannot retrieve or provide conversations that were not collected because this feature was not enabled, including in response to a court order. Technical logs and metadata described in Section 4 are separate from conversation content. Disclosure of stored conversations to authorities is subject to the court-order requirements in Section 7.1.

The authenticated Account Manager can list, view, export, and delete stored conversations through the conversations API, subject to authorization and retention windows. Deleting a conversation removes the matching conversation record for that account from production storage.

---

### 6. RAG, Memories, and Storage

AIVAX can store RAG collections, documents, embeddings, document metadata, user memories, media descriptions, web chat session data, and shell workspace files when those features are used.

Current storage accounting includes:

- RAG document text and embedding bytes.
- User persistent information.
- Media descriptions.
- Web chat session messages, extra context, and metadata.
- Account shell files.

Storage quotas are plan-based. Current included storage is 30 MB for Free, 2 GB for Pro, and 20 GB for Max.

---

### 7. Optional Semantic Data Collection and Model Training

By default, AIVAX does not use Account Manager Inference Data or Conversations to train proprietary AIVAX models. When an authorized Account Manager enables semantic data collection, AIVAX may use eligible anonymized RAG and reranking records generated while the setting is enabled for the purposes described in [Data Collecting](https://docs.aivax.net/docs/data-collecting.md). RAG document indexing and storage are not included and receive no program discount.

Before storage, AIVAX anonymizes these records by removing the account relationship and excluding operational identifiers, document names, billing data, and collection timestamps. The account is consulted only to verify consent and apply the eligible discount. AIVAX does not retain an account-to-record mapping.

The setting is disabled by default. Disabling it stops new collection but does not automatically delete records collected while consent was active or reverse training already completed. Because account-to-record mappings are not stored, AIVAX cannot locate a training record from an account ID alone. The Account Manager remains responsible for the legal basis, notices, and permissions required for personal data submitted by its end users. Requests concerning personal data present inside semantic content may be sent to **privacy@aivax.net** or **wm@aivax.net** with enough information to locate the content where applicable.

Third-party model providers and aggregators may have their own processing terms, retention terms, and model-improvement policies independently of this optional program. The Account Manager should review the selected provider's policy before sending personal or sensitive data.

---

### 7.1. Court Orders and Requests from Foreign Authorities

For judicial disclosure of account data, AIVAX executes only orders issued by competent Brazilian courts. Orders issued in other countries must be submitted through the Brazilian courts and result in a Brazilian court order before AIVAX discloses data. A foreign order alone does not authorize disclosure.

Foreign judicial authorities may request preservation of existing logs for up to 1 year while pursuing the applicable proceedings through Brazilian courts. Preservation is not disclosure and does not authorize collection or reconstruction of conversation content that was not logged.

To comply with a Brazilian court order, AIVAX may provide the following data, limited to the scope of the order and the records actually available:

- **Logs and technical metadata:** Up to 1 year of inference and access logs and metadata, including IP addresses and user-agent strings. These records do not imply that conversation content was collected.
- **Stored conversations:** Only conversations collected while the Account Manager had enabled "Conversations" for the request or globally for the account, and which remain available. AIVAX provides these conversations to authorities only under a Brazilian court order; it cannot provide conversations that were never collected.
- **Other account information and stored resources:** Account information, user memories, AI gateways and their configurations, RAG collections and documents, and other stored account resources. Available records may include backups dating back up to 3 months, as described in Section 21.

These periods describe the limits of available retention, not a guarantee that every record exists throughout the entire period. A preservation request does not restore records that were never collected or are no longer available.

---

### 8. Data Subject Rights (Art. 18, LGPD)

When AIVAX acts as Controller, data subjects may request confirmation of processing, access, correction, anonymization, blocking or deletion, portability, information about sharing, revocation of consent where applicable, opposition to processing based on legitimate interest, and review of automated decisions where applicable.

Channel: **privacy@aivax.net** or **wm@aivax.net** (Data Protection Officer). We may request identity verification. For data where AIVAX acts as Processor, AIVAX may direct the data subject to the Controller Account Manager.

---

### 9. Data Protection Officer (DPO)

Data Protection Officer (Art. 41): **(Anonymized identity)**
Contact: **wm@aivax.net**

Functions include communication with data subjects and the ANPD, internal compliance guidance, and support for privacy impact assessments.

---

### 10. Subprocessors and Third-Party Providers

AIVAX uses third-party services for infrastructure, object storage, transactional email, billing, web search, image generation, reranking, and AI model inference. The current technical list is maintained in [Data Processors](https://docs.aivax.net/docs/legal/third-party-processors.md).

When selecting a model or enabling a tool, the Account Manager may cause content to be sent to the selected model provider, aggregator, or tool provider. AIVAX does not control third-party policies and recommends prior review.

---

### 11. International Data Transfers

Data may be processed or stored outside Brazil depending on the selected infrastructure, model provider, search provider, object storage provider, or payment provider. AIVAX applies technical and contractual safeguards appropriate to the service, including access controls, encryption in transit, minimization, and logical segregation where applicable.

---

### 12. Information Security

Key measures include:

- Encryption in transit with HTTPS/TLS.
- API key authentication and account-scoped authorization.
- Role-based administrative access.
- Rate limiting for inference, RAG search, document insertion, tools, and payment operations.
- Balance, minimum-balance, and storage-quota checks before cost-incurring operations.
- Operational logs and error reporting for troubleshooting and abuse prevention.
- Separation between account-owned resources, such as collections, documents, conversations, memories, and shell files.
- Secure credential configuration through application initialization parameters rather than hardcoded secrets.

No security measure is absolute; AIVAX maintains a continuous improvement process.

---

### 13. Incident Management

Relevant security incidents are assessed based on impact, data nature, and risk to data subjects. When required, AIVAX will notify affected Account Managers and competent authorities with available information about the event, affected data categories, mitigation steps, and recommended actions.

---

### 14. Automated Decisions

AIVAX uses automation for rate limiting, balance checks, storage quota checks, abuse prevention, fraud prevention, indexing, routing, and operational monitoring. These automations may temporarily restrict requests or keys. The Account Manager may request review through support channels.

---

### 15. Cookies and Tracking Technologies

Dashboard interfaces may use strictly necessary cookies or equivalent browser storage for sessions, authentication, and preferences. AIVAX does not use behavioral advertising cookies in the documented platform flow.

---

### 16. Children, Adolescents, and Emancipated Minors

The services are not intended for persons under 18, except legally emancipated minors aged 16 or older under Brazilian law. The Account Manager is responsible for implementing appropriate checks when their use case may involve minors.

---

### 17. Sensitive Data

AIVAX does not require sensitive data to use the platform. The Account Manager should avoid sending health, biometric, genetic, religious belief, political opinion, or other sensitive data unless they have an appropriate legal basis and clear notices for data subjects.

---

### 18. Use Limitations and Prohibited Content

It is prohibited to use the platform to store or process illegal content, rights-infringing material, malware, defamatory material, or content that infringes third-party rights. AIVAX may suspend, restrict, or block keys upon reasonable suspicion of violation, preserving logs necessary for investigation.

---

### 19. Aggregated Data

AIVAX may generate aggregate statistics such as token volume, error rate, model distribution, storage usage, and latency. Aggregated statistics are used for capacity planning, reliability, billing, and abuse prevention.

---

### 20. Export and Portability

AIVAX provides APIs to export conversation history in JSON or JSONL within the configured retention window. Collection documents can be exported in JSONL format. Export availability is subject to authentication, authorization, retention, and account state.

---

### 21. Backups and Disaster Recovery

AIVAX maintains backups for operational continuity and disaster recovery, with historical copies dating back up to 3 months. These backups may contain account information and stored resources, including user memories, AI gateways, and RAG collections and documents. Deleted production data may remain in backups within this period until backup rotation completes. Available backup data may be provided to comply with a Brazilian court order under Section 7.1. Backups do not contain conversations that were never collected because "Conversations" was not enabled.

---

### 22. Changes to This Policy

Material changes may be notified by email, dashboard notice, or publication of an updated policy. Continued use after the effective date constitutes acceptance where permitted by law and contract.

---

### 23. Contact Channel and Complaints

Questions, rights requests, or complaints: **privacy@aivax.net** / **wm@aivax.net**.
If unsatisfied, the data subject may appeal to the **ANPD** (National Data Protection Authority).

---

### 24. Revision History

| Version | Effective Date | Major Changes |
| --- | --- | --- |
| 1.0 | 07/30/2025 | Initial version published |
| 1.1 | 10/05/2025 | Added legal bases, rights, detailed retention, subprocessors, transfers, expanded security, incidents, automated decisions, cookies, sensitive data, versioning |

---

### 25. General Contact

Legal / Privacy: **legal@aivax.net**
Data Protection Officer: **wm@aivax.net**

Always use official channels to avoid social engineering.

---

### 26. Final Provisions

If any clause of this policy is deemed invalid, the remaining provisions remain in full force. In case of conflict between this policy and specific product terms, the more protective provision for data subjects will prevail unless a different legal obligation applies.

> Note: This policy may be complemented by a specific Data Processing Agreement (DPA) between AIVAX and the Account Manager, when applicable.

---

Source: https://docs.aivax.net/docs/legal/terms-of-service.html

# Terms of Use

Revision: 1.4
Effective date: August 1, 2026
Previous update: July 20, 2026
Last updated: September 26, 2026

---

Welcome to AIVAX. These Terms of Use ("Terms") govern your access to and use of our AI inference services, APIs, website, and any associated software (collectively, the "Services"). These Terms are not legal advice.

By creating an account, accessing, or using our Services, you ("AIVAX Account Manager") agree to be bound by these Terms and our [Privacy Policy](https://docs.aivax.net/docs/legal/privacy-policy.md). If you do not agree to these Terms, do not use our Services.

### 1. Definitions

- **AIVAX:** The company providing the Services.
- **AIVAX Account Manager:** The natural or legal person who creates and manages an account on the AIVAX platform and agrees to these Terms.
- **Input Content:** Data, text, prompts, files, messages, metadata, or any other information that the AIVAX Account Manager sends to the Services for processing.
- **Generated Content:** Responses, text, images, files, or any other data generated by AI models or platform tools as a result of processing Input Content.
- **Conversations:** Stored sequences of Input Content, Generated Content, metadata, model information, usage data, tools, and related technical information when conversation logging is enabled.
- **RAG Data:** Collections, documents, document metadata, references, tags, and embedding vectors stored for retrieval-augmented generation.
- **Semantic Training Data:** Anonymized eligible RAG and reranking content collected after the AIVAX Account Manager enables semantic data collection, as described in [Data Collecting](https://docs.aivax.net/docs/data-collecting.md). Document indexing and storage are excluded.

In the Privacy Policy, "Inference Data" jointly covers Input Content and Generated Content.

---

### 2. Use of Services and Responsibilities

#### 2.1. Responsible Use and Conduct

You agree to use AIVAX Services ethically and responsibly. It is prohibited to:

- Abuse, interfere with, disrupt, overload, or harm the Services, servers, networks, or third-party integrations.
- Attempt to bypass authentication, authorization, rate limits, billing controls, storage quotas, or security controls.
- Probe, exploit, or disclose vulnerabilities without authorization.
- Use the Services to harass, threaten, defame, deceive, or violate the rights and dignity of third parties.
- Store, generate, or distribute illegal content, malware, or rights-infringing material.

#### 2.2. Legal Compliance

You are solely responsible for ensuring that your use of the Services complies with all applicable laws and regulations, including Brazilian law, international obligations that apply to your use case, intellectual property rules, privacy rules, and sector-specific rules.

Your use must respect the principles established by the Marco Civil da Internet (Law No. 12.965/2014) and the General Data Protection Law (LGPD - Law No. 13.709/2018), where applicable.

#### 2.3. Personal Data

The AIVAX Account Manager is the controller of data entered into the Services. If Input Content includes personal data of third parties, you represent that:

- You have the appropriate legal basis to collect, process, and send that data to AIVAX for the configured purpose.
- You are responsible for notices, consents where required, data-subject requests, retention choices, and downstream use of Generated Content.
- AIVAX acts as processor for customer-directed inference processing, subject to the Privacy Policy and applicable agreements.

#### 2.4. Eligibility

To use AIVAX services, you must be legally capable, at least 18 years old, or 16 years old if legally emancipated under Brazilian law. By using our services, you represent that:

- you have not previously been suspended, removed, or banned from the Services;
- your account is linked to a person or organization in compliance with applicable laws and account obligations;
- you meet the minimum age and capacity requirements.

If you use the Services on behalf of another person, organization, or company, you represent that you have authority to bind that entity to these Terms.

#### 2.5. Balance, Credits, Storage, and Refunds

Most AIVAX services require prepaid balance ("credits"). Cost-incurring operations may be blocked when the account balance is zero or negative, when the account lacks the minimum required balance for a feature, or when storage usage exceeds the plan quota.

The current platform applies plan-based storage quotas to stored resources such as RAG documents and embeddings, user memories, media descriptions, web chat session data, and shell files. Current included storage is 30 MB for Free, 2 GB for Pro, and 20 GB for Max.

Payment providers may collect and process billing and payment information required to add credits. AIVAX payment flows currently include InfinitePay for payment invoice creation, and the platform also contains Stripe webhook support for payment event handling.

Before paying for credits, you can view applicable fees and refuse the payment before balance addition. Credits added expire one year after addition. Expired credit no longer counts toward your account balance.

**Refund Policy:** In accordance with Art. 49 of the Consumer Defense Code (Law No. 8.078/1990), you may exercise the right of withdrawal within 7 (seven) calendar days from the date of balance addition by requesting a refund via legal@aivax.net. From the refundable amount, the following may be deducted:

- costs corresponding to computing services, inference, processing, RAG data, vectors, memories, storage, or other resources actually consumed up to the request moment;
- non-refundable payment provider service fees.

The refund will be processed using the same payment method within up to 15 (fifteen) business days after approval. The remaining account balance may be deducted or zeroed.

**Service interruption for insufficient balance or storage:** The platform may deny new cost-incurring requests when the account has insufficient balance or exceeds storage quota. You are responsible for exporting or deleting stored data as needed and for maintaining enough balance and quota for continued service operation.

---

### 3. Generated Content and Intellectual Property

#### 3.1. Ownership and Responsibility for Generated Content

Subject to these Terms, AIVAX grants you all rights, title, and interest it may have in Generated Content. In other words: what you create is yours, subject to applicable law and third-party rights.

You are solely responsible for Generated Content and its subsequent use, including legality, accuracy, suitability, and possible third-party rights infringements. AIVAX is not responsible for how you use Generated Content.

**Non-exclusivity and similarity:** Due to the statistical nature of AI models, similar content may be generated for different users without cross-access to original inputs. AIVAX does not guarantee absolute exclusivity of expressions or ideas generated by models. You do not acquire rights to model weights, internal prompts, techniques, platform code, or AIVAX trade secrets.

**Limited license granted to AIVAX:** By submitting Input Content, you grant AIVAX a worldwide, non-exclusive, royalty-free license limited to what is necessary to process inferences, provide configured tools and integrations, maintain technical logs and security, detect abuse, calculate billing, and comply with legal obligations. This license is subject to the conversation collection, retention, and judicial disclosure limits in Section 3.4 and the Privacy Policy; it does not authorize conversation logging when "Conversations" is not enabled.

By default, AIVAX does not use Input Content or Generated Content to train proprietary AIVAX models. The optional semantic data collection program described below is the exception to this default rule.

**High-risk sectors:** Generated Content must not be used as the sole basis for medical, legal, financial, critical safety engineering, or other high-risk decisions without qualified human validation.

#### 3.2. Optional Semantic Data Collection

When an authorized Account Manager enables semantic data collection, the Account Manager:

- authorizes AIVAX to collect eligible anonymized RAG and reranking records generated while the setting is enabled;
- grants AIVAX a worldwide, non-exclusive, royalty-free license to store, reproduce, transform, annotate, combine, analyze, and use Semantic Training Data to develop, train, fine-tune, evaluate, test, and improve models and systems related to embeddings, retrieval, ranking, reranking, and other semantic processing;
- represents that it has the authority, legal basis, notices, and permissions required for this use, including for personal data submitted by end users; and
- acknowledges that eligible RAG and reranking operations receive the documented 10% discount, while document indexing, storage, and unrelated services remain at their regular prices.

AIVAX anonymizes these records before storage by removing the account relationship and excluding operational identifiers, document names, billing data, and collection timestamps.

The setting is disabled by default. Disabling it applies prospectively: it stops new collection and ends the discount for future eligible operations, but does not automatically delete records already collected or require AIVAX to reverse completed training. Because AIVAX does not retain an account-to-record mapping, records cannot be located or selectively deleted from an account ID alone. Deletion requests remain subject to the Privacy Policy and applicable law. The complete operational terms are documented in [Data Collecting](https://docs.aivax.net/docs/data-collecting.md).

#### 3.3. Adult, Explicit, and Sensitive Content

AIVAX is a tool that can be used to generate many kinds of content. Adult or explicit content is permitted only when all of the following conditions are met:

1. You assume full responsibility for creating, storing, and distributing the material.
2. The material does not violate any applicable law, with zero tolerance for content involving child exploitation, non-consensual sexual content, non-consensual violence, or illegal abuse.
3. If the material involves representations of real people, you have explicit and verifiable consent from those individuals.
4. You implement your own access-control and age-verification mechanisms if you make this content available to third parties.

AIVAX does not endorse this type of content and may investigate or suspend accounts that violate these conditions.

#### 3.4. Conversation Logging, Retention, and Judicial Disclosure

AIVAX collects and logs conversation content only when the Account Manager enables the observability feature ("Conversations") for the individual request or globally for the account. If neither enables logging for the request, its conversation content is not collected, logged, or stored. Processing Input Content and Generated Content to execute a request does not create a stored conversation record. AIVAX cannot retrieve or provide conversations that were never collected, even under a court order.

Technical logs and metadata are separate from conversation content. AIVAX retains inference and access logs and metadata, including IP addresses and user-agent strings, for up to 1 year. Account information and stored resources, including user memories, AI gateways and their configurations, and RAG collections and documents, may also remain in backups dating back up to 3 months. These periods do not guarantee that every record remains available; conversation retention and other applicable limits are described in the [Privacy Policy](https://docs.aivax.net/docs/legal/privacy-policy.md).

For judicial disclosure of account data, AIVAX executes only orders issued by competent Brazilian courts. Orders from other countries must be submitted through Brazilian courts and result in a Brazilian court order before data is disclosed. Foreign judicial authorities may request preservation of existing logs for up to 1 year while pursuing those proceedings. Preservation alone does not authorize disclosure, restore unavailable records, or authorize collection of conversation content that was not logged.

Under a Brazilian court order, AIVAX may provide available logs and metadata, stored conversations, and other account information and resources, including available backup data, limited to the scope of the order. Stored conversations are provided to authorities only under such an order. See Section 7.1 of the Privacy Policy for the disclosure rules and Section 21 for backups.

---

### 4. Third-Party Model Providers, Tools, and BYOK

#### 4.1. Bring Your Own Key (BYOK)

AIVAX may offer BYOK (Bring Your Own Key) functionality, allowing you to use your own API keys from external providers for inference or related services. By using BYOK, you agree that:

- You are responsible for obtaining, storing, rotating, and using your own provider keys.
- AIVAX has no control over the original provider's policies, usage limits, bans, suspensions, prices, or terms.
- If your external key is blocked, revoked, rate-limited, or exhausted, Services that depend on it may fail.
- AIVAX peripheral services, including storage, RAG processing, vector management, user memories, agent orchestration, and auxiliary tools, may continue to be charged against your AIVAX balance according to current pricing.
- BYOK does not exempt you from these Terms, legal compliance, abuse rules, or acceptable-use policies.

#### 4.2. Third-Party Models and Tools Provided by AIVAX

AIVAX Services may route Input Content to AI models, rerankers, search providers, image generation providers, payment providers, object storage providers, messaging providers, or other processors. The specific provider depends on the selected model, configured provider, tool, gateway, or integration.

When using a specific model, provider, or tool, you may also be subject to that provider's terms. You are responsible for reviewing provider suitability before sending personal, confidential, regulated, or sensitive data.

---

### 5. Suspension and Termination

AIVAX may suspend, restrict, terminate, or ban access to the Services for violations of these Terms, security risks, legal requirements, repeated non-payment, abuse, fraud, or risk to the platform or third parties. Suspensions may be temporary or permanent depending on the severity of the issue.

---

### 6. Limitation of Liability and Disclaimer of Warranties

THE SERVICES ARE PROVIDED "AS IS" AND "AS AVAILABLE", WITHOUT WARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED. AIVAX DOES NOT WARRANT THAT THE SERVICES WILL BE UNINTERRUPTED, SECURE, ERROR-FREE, OR SUITABLE FOR A PARTICULAR PURPOSE.

TO THE MAXIMUM EXTENT PERMITTED BY LAW, AIVAX WILL NOT BE LIABLE FOR INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES ARISING FROM YOUR ACCESS TO OR USE OF THE SERVICES.

GENERATED CONTENT MAY CONTAIN INACCURACIES, OMISSIONS, OR HALLUCINATIONS AND IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. YOU ARE RESPONSIBLE FOR HUMAN REVIEW BEFORE CRITICAL USE.

---

### 7. Indemnification

You agree to indemnify, defend, and hold harmless AIVAX, its affiliates, administrators, collaborators, and partners from claims, losses, damages, liabilities, costs, and expenses resulting from: (i) Input Content; (ii) misuse of the Services; (iii) breach of these Terms or applicable law; (iv) infringement of third-party intellectual property, privacy, or personality rights; (v) improper use or exposure of API keys or credentials associated with your account.

### 8. Termination and Post-Termination

After termination or suspension: (a) access and keys may be deactivated; (b) export of available data may be limited by retention windows, account state, and security requirements; (c) data may be deleted or anonymized according to the Privacy Policy and operational procedures; (d) outstanding amounts remain payable.

### 9. Force Majeure

No party will be liable for failures or delays caused by events beyond reasonable control, including natural disasters, governmental actions, widespread infrastructure failures, cyber-attacks, pandemics, wars, or broad power outages. Payment obligations for consumed services are not discharged by force majeure.

### 10. Export Controls and Sanctions

You represent that you are not located in, nor acting on behalf of, any entity or person subject to applicable sanctions or trade restrictions. You will not use the Services for purposes prohibited by export, anti-corruption, or anti-terrorism laws. Breach of this clause may result in immediate suspension or termination.

### 11. Confidentiality and Credential Security

You must keep API keys, tokens, and credentials confidential and implement appropriate access controls. Activity performed with your credentials may be presumed authorized until a compromise is reported. You must promptly notify AIVAX of suspected unauthorized use.

### 12. Feedback and Improvements

Any comments, suggestions, ideas, or feedback you provide may be used by AIVAX to improve or develop products and services, without compensation, credit, or additional confidentiality obligations.

### 13. Beta Features and Model Discontinuation

Features identified as "Beta", "Experimental", or equivalent may be unstable, change behavior, or be removed. AIVAX may discontinue models, providers, or technical limits for performance, cost, compliance, availability, or security reasons.

### 14. Takedown Procedure

If you believe output or use of the Services infringes copyright, trademark, privacy, personality, or other rights, send a notice to legal@aivax.net containing: (i) precise identification of the material; (ii) basis of the claim; (iii) your contact information; (iv) a statement of good faith and truthfulness. AIVAX may remove or limit access preemptively and terminate repeat-offender accounts.

### 15. Assignment

You may not assign or transfer these Terms without prior written consent from AIVAX. AIVAX may assign these Terms, in whole or in part, including in corporate transactions, mergers, acquisitions, or reorganizations, upon notice where required.

### 16. Survival

The following provisions survive termination: Intellectual Property, Limitation of Liability, Indemnification, Confidentiality, Termination, Assignment, Governing Law and Jurisdiction, and any others that by their nature must persist.

### 17. Entire Agreement

These Terms, together with the Privacy Policy and any expressly referenced additional documents, constitute the entire agreement between you and AIVAX regarding the Services.

### 18. Notices

Formal notices may be delivered by registered email, dashboard notice, or publication on the official Terms page. A notice sent by email is considered received after 24 hours unless a proven technical error occurs.

### 19. Price and Limit Updates

AIVAX may adjust prices, billing models, usage limits, storage quotas, or rate-limit policies. Material changes affecting future costs may be communicated with reasonable advance notice unless required by law, security, or emergency conditions. Continued use after the changes take effect implies acceptance where permitted.

### 20. Language

The Portuguese-language version of these Terms prevails over translations provided solely for convenience.

### 21. Modifications to the Terms

AIVAX may modify these Terms. Material changes may be notified by publication, email, or dashboard notice and may require additional acceptance. Continued use after the effective date constitutes acceptance where permitted.

### 22. General Provisions

These Terms are governed by the laws of the Federative Republic of Brazil. The court of the County of Sao Paulo, State of Sao Paulo, Brazil, is elected to resolve disputes, with waiver of any other jurisdiction, however privileged. The invalidity of any clause does not affect the validity of the remaining provisions.

For questions about these Terms of Use, contact: **legal@aivax.net**.

---

Source: https://docs.aivax.net/docs/legal/third-party-processors.html

# Data Processors

AIVAX uses third-party services for specific operations such as infrastructure, object storage, email delivery, payment processing, web search, AI inference, reranking, and image generation. The providers involved in a request depend on the selected model, gateway, tool, and integration.

This page is a disclosed technical inventory, not legal advice. It does not list every provider that may be available through model catalogs or aggregators. Provider policies may change, and Account Managers should review the provider terms that apply to their selected models and tools before sending personal, confidential, regulated, or sensitive data.

The data protection law column summarizes the principal frameworks identified in each provider's current privacy policy or data processing addendum. Applicability may vary by contracting entity, data subject location, and processing region.

## Operations, Services, and Infrastructure

| Provider | Used for | Data protection law |
| --- | --- | --- |
| [Cloudflare](https://www.cloudflare.com/) | Reverse proxy/security where deployed and embedding services | [EU/UK GDPR, Swiss FADP, and CCPA/CPRA](https://www.cloudflare.com/cloudflare-customer-dpa/) |
| [Hetzner](https://www.hetzner.com/) | Compute infrastructure and hosting | [GDPR and BDSG](https://docs.hetzner.com/general/company-and-policy/data-protection-at-hetzner/) |
| [netcup](https://www.netcup.com/) | Compute infrastructure and hosting | [GDPR and BDSG](https://www.netcup.com/en/contact/data-privacy) |
| [Backblaze](https://www.backblaze.com/) | Object and file storage for generated media, uploaded media, generated documents, exposed files, and error artifacts | [EU/UK GDPR and CCPA/CPRA](https://www.backblaze.com/company/privacy) |
| [Brevo](https://www.brevo.com/) | Transactional email and notifications | [GDPR, French Data Protection Act, CCPA/CPRA, PIPEDA, and LGPD](https://www.brevo.com/legal/privacypolicy/) |
| [InfinitePay](https://infinitepay.io/) | Payment invoice creation and payment confirmation | [LGPD (Brazilian Law No. 13,709/2018)](https://www.infinitepay.io/legal/aviso-de-privacidade) |
| [Stripe](https://stripe.com/) | Payment event processing where Stripe checkout is configured | [EU/UK GDPR, Irish Data Protection Act 2018, and CCPA/CPRA](https://stripe.com/privacy) |
| [Twitter/X](https://developer.x.com/) | Built-in X/Twitter search and post reading tools | [EU/UK GDPR, Swiss FADP, CCPA, and LGPD](https://x.com/en/privacy) |
| [Linkup](https://www.linkup.so/) | Web search for context enrichment | [GDPR, French Data Protection Act, ePrivacy Directive, and CCPA/CPRA](https://www.linkup.so/privacy-policy) |
| [Tavily](https://www.tavily.com/) | Web search for context enrichment | [GDPR, UK Data Protection Act 2018, and applicable U.S. state privacy laws](https://www.tavily.com/privacy) |
| [Sinkin AI](https://sinkin.ai/) | Image generation for selected image models | [EU/UK GDPR, CCPA, and Virginia CDPA](https://sinkin.ai/privacy) |
| [Pollinations](https://pollinations.ai/) | Image generation for selected image models | [GDPR](https://pollinations.ai/privacy) |

## Direct Inference, Embedding, and Reranking Providers

| Provider | Used for | Data protection law |
| --- | --- | --- |
| [Groq](https://groq.com/) | LLM inference through an OpenAI-compatible endpoint | [EU/UK GDPR, Swiss FADP, CCPA/CPRA, and Saudi PDPL](https://console.groq.com/docs/legal/customer-data-processing-addendum) |
| [Jina AI](https://jina.ai/) | Embeddings, reranking, and web research | [GDPR and BDSG](https://jina.ai/legal/) |
| [OpenRouter](https://openrouter.ai/) | Model routing, fallback, and speech synthesis | [GDPR, CCPA/CPRA, and applicable U.S. state privacy laws](https://openrouter.ai/privacy/) |
| [DeepInfra](https://deepinfra.com/) | AI inference | [GDPR and CCPA/CPRA](https://deepinfra.com/privacy) |
| [Xiaomi MiMo](https://platform.xiaomimimo.com/) | Xiaomi model inference | [PIPL, EU/UK GDPR, and Swiss FADP](https://privacy.mi.com/all/en_US) |
| [Inception Labs](https://www.inceptionlabs.ai/) | Mercury model inference | [California Civil Code §§ 1798.83–1798.84 and Nevada Revised Statutes Chapter 603A](https://www.inceptionlabs.ai/docs/privacy-policy) |
| [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) | Embedding services | [EU/UK GDPR, Swiss FADP, and CCPA/CPRA](https://www.cloudflare.com/cloudflare-customer-dpa/) |

## Model Families and Underlying Providers

The model catalog also identifies model families or underlying providers that may be selected directly or reached through aggregators such as OpenRouter. Not every request is sent to every provider.

| Provider | Used for | Data protection law |
| --- | --- | --- |
| [OpenAI](https://openai.com/) | OpenAI model families exposed in the catalog or compatible integrations | [EU/UK GDPR, CCPA/CPRA, and applicable U.S. state privacy laws](https://openai.com/policies/privacy-policy/) |
| [Google Vertex AI](https://cloud.google.com/vertex-ai) | Gemini and other Google model families | [EU/UK GDPR, Swiss FADP, and CCPA/CPRA](https://cloud.google.com/terms/data-processing-addendum/) |
| [Anthropic](https://www.anthropic.com/) | Claude model families | [EU/UK GDPR, Swiss FADP, LGPD, and applicable U.S. state privacy laws](https://www.anthropic.com/legal/privacy) |
| [AWS](https://aws.amazon.com/) | AWS-hosted model families such as Amazon Nova or Bedrock-backed providers | [EU/UK GDPR, Swiss FADP, and CCPA/CPRA](https://aws.amazon.com/privacy/) |
| [Cohere](https://cohere.com/) | Cohere model families | [PIPEDA, GDPR, and CCPA/CPRA](https://cohere.com/privacy) |
| [xAI](https://x.ai/) | Grok model families and Grok Voice through configured routes | [EU/UK GDPR, Swiss FADP, and CCPA/CPRA](https://x.ai/legal/data-processing-addendum) |
| [Mistral AI](https://mistral.ai/) | Mistral model families | [GDPR, French Data Protection Act, and CCPA/CPRA](https://legal.mistral.ai/terms/privacy-policy) |
| [DeepSeek](https://www.deepseek.com/) | DeepSeek model families | [PIPL, Data Security Law, and Cybersecurity Law of the People's Republic of China](https://cdn.deepseek.com/policies/en-US/deepseek-privacy-policy.html) |
| [Z.ai](https://z.ai/) | GLM/Z.ai model families | [Singapore PDPA and GDPR/UK GDPR where applicable](https://docs.z.ai/legal-agreement/privacy-policy) |
| [Alibaba Cloud](https://www.alibabacloud.com/product/modelstudio) | Qwen/Alibaba model families | [GDPR/UK GDPR and applicable regional laws, including PIPL](https://www.alibabacloud.com/help/en/legal/latest/alibaba-cloud-international-website-privacy-policy-history) |
| [Cerebras](https://www.cerebras.ai/) | Cerebras model families | [GDPR and CCPA/CPRA](https://trust.cerebras.ai/) |
| [Nebius](https://nebius.com/) | Nebius-backed model families | [EU/UK GDPR, Dutch GDPR Implementation Act, and Swiss FADP](https://docs.nebius.com/legal/dpa) |
| [Fireworks AI](https://fireworks.ai/) | Fireworks-backed model families | [EU/UK GDPR, CCPA/CPRA, and applicable U.S. state privacy laws](https://fireworks.ai/privacy-policy) |
| [Novita](https://novita.ai/) | Novita-backed model families | [CCPA/CPRA](https://novita.ai/legal) |
| [Azure](https://azure.microsoft.com/) | Azure-backed model families | [EU/UK GDPR, CCPA/CPRA, and PIPEDA](https://www.microsoft.com/licensing/docs/view/Microsoft-Products-and-Services-Data-Protection-Addendum-DPA) |
| [LongCat](https://longcat.chat/) | LongCat/Meituan model family metadata | [PIPL, Data Security Law, and Cybersecurity Law of the People's Republic of China; GDPR and CCPA where applicable](https://longcat.chat/platform/private/) |

## Data Handling Notes

By default, AIVAX does not use Account Manager Input Content, Generated Content, or Conversations to train proprietary AIVAX models. Eligible anonymized RAG and reranking records are used for model development only when an authorized Account Manager enables the optional program described in [Data Collecting](https://docs.aivax.net/docs/data-collecting.md). Document indexing and storage are excluded.

Third-party providers and aggregators may have their own processing, retention, abuse-monitoring, and model-improvement rules. The selected model, provider, tool, or integration determines which third party receives data for a specific request.

Avoid sending sensitive, confidential, regulated, or personal information to a provider unless you have reviewed that provider's current terms and have an appropriate legal basis for the processing.