JSON Healing

JSON Healing is how AIVAX makes any model return JSON that matches a schema, including models without native structured output. Use it when the application must parse a predictable JSON value instead of free text.

Turn it on in one of these ways:

• Send response_schema.
• Send healing_options in response_format.
• Turn on automatic JSON Healing in the account settings.

Use response_format when you want the provider’s native JSON Schema support. Add healing_options inside response_format.json_schema when that request should also use JSON Healing.

How JSON Healing works #

When response_schema is present, AIVAX follows this loop.

1. AIVAX adds schema instructions to the model request.
2. After the model generates an answer, AIVAX tries to extract JSON from:
   • The complete generated text.
   • Common repaired variants, such as missing opening or closing braces.
   • JSON code blocks found in the generated text.
3. If the extracted JSON does not validate against the schema, AIVAX adds a feedback message with the validation errors and asks the model to generate the JSON again.
4. This continues until a valid JSON value is generated or the configured attempt limit is reached.

JSON Healing improves reliability, but it is not an absolute guarantee. If the model repeatedly fails the schema, the request can fail after the attempt limit; see When healing fails.

Basic example #

Use response_schema when you want JSON Healing on a chat completion, including after tools run. The request below lets the model search the web and then return the results as JSON validated against response_schema.

Reference:

{
    "model": "@google/gemini-3.8-flash",
    "prompt": "Search for recent news about electric vehicles.",
    "stream": true,
    "builtin_tools": {
        "tools": [
            "WebSearch"
        ],
        "options": {
            "web_search_mode": "full"
        }
    },
    "response_schema": {
        "type": "object",
        "properties": {
            "news": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "title": {
                            "type": "string",
                            "description": "News title"
                        },
                        "summary": {
                            "type": "string",
                            "description": "News summary"
                        }
                    },
                    "required": ["title", "summary"]
                }
            }
        },
        "required": ["news"]
    }
}

builtin_tools is optional. If you enable tools, the selected model must support tool calling or the gateway must provide a tool handler.

Native structured output #

Use response_format when you want the provider’s native JSON Schema support. If the selected integrated model has strict JSON support, AIVAX can forward the schema using the provider’s JSON Schema response format.

{
    "model": "@openai/gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "List 3 European capitals."
        }
    ],
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "capitals": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "city": { "type": "string" },
                                "country": { "type": "string" }
                            },
                            "required": ["city", "country"]
                        }
                    }
                },
                "required": ["capitals"]
            }
        }
    }
}

Enabling healing in response_format #

Use healing_options when the request already uses response_format and you also want JSON Healing. Place the options inside response_format.json_schema:

{
    "model": "@openai/gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "Return a short status object."
        }
    ],
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "status": { "type": "string" },
                    "message": { "type": "string" }
                },
                "required": ["status", "message"]
            },
            "healing_options": {
                "max_attempts": 5
            }
        }
    }
}

max_attempts must be between 1 and 10. Each retry is another model generation attempt and can increase cost and latency.

When healing fails #

Healing fails when every attempt produces invalid JSON. The request then fails with the message Failed to generate a valid JSON response. Please, try again later.

What the client receives:

• Without streaming, the response is HTTP 503 with the error code service_unavailable and the header Retry-After: 5.
• With streaming, the stream ends with a chunk whose finish_reason is error and whose error object carries the code and message, followed by [DONE]. See Streaming.

If healing frequently reaches the attempt limit, adjust the instruction and schema before raising the limit. Common causes:

• Overly rigid schemas.
• Missing required fields.
• Vague prompts.
• Noisy tool results.
• A model that is too small for the task.

Supported schema features #

AIVAX validates the generated JSON with JSON Schema. Use these features when you need to constrain a value. The documented supported features are:

• string: minLength, maxLength, pattern, format, and enum.
• number and integer: minimum, maximum, exclusiveMinimum, exclusiveMaximum, and multipleOf.
• array: items, uniqueItems, minItems, and maxItems.
• object: properties and required.
• boolean and bool.
• null.
• Multiple types, for example "type": ["string", "number"].

Apply restrictions this way:

• Use required for fields your application must receive.
• Use explicit items for arrays.
• Use enum, format, and length or pattern restrictions when the accepted values are known.

Practical patterns #

Use these patterns when you design the schema, not only when you turn healing on.

Extraction #

Tell the model which fields must be inferred, which fields should be null when missing, and which fields must not be invented.

Classification #

Use enum for the final label. Add a short reason field when humans need to audit the decision.

Tool-backed JSON #

Allow the model to use tools before producing the JSON. Include source fields when the application needs provenance.

Writes to other systems #

For database writes or external API calls, validate the JSON again in your application. AIVAX validates the JSON shape, but your application remains responsible for business rules such as permissions, valid IDs, date ranges, and account-specific constraints.