Source: http://localhost:1313/docs/rag/collections.html

# Collections and Documents

AIVAX provides a RAG (Retrieval-Augmented Generation) service for storing documents and retrieving them later through semantic search. A collection is an account-owned group of documents. Each document stores text, optional tags, an optional reference, optional metadata, and the vectors generated by the indexing job.

Collections can be searched directly through the RAG API or attached to an AI Gateway so retrieved documents can be injected into the model context.

Compare vector storage with the surrounding ingestion and retrieval pipeline in [RAG vs vector database](https://aivax.net/blog/a-vector-database-is-not-a-rag-system/).

## Collections

Use collections to group documents that belong to the same knowledge base, product, tenant, language, or operational purpose.

Create a collection before adding searchable knowledge. For example, a support collection can hold help-center answers, a legal collection can hold contract clauses, and a product collection can hold descriptions, policies, and troubleshooting notes.

After adding and indexing documents, search the collection directly with the [Semantic Search](http://localhost:1313/docs/rag/semantic-search.md) API, expose it through [Collections MCP](http://localhost:1313/docs/mcp-utilities/collections-mcp.md), or attach it to an [AI Gateway](http://localhost:1313/docs/inference/ai-gateway.md) so retrieved documents are placed into the model context automatically.

Each collection has:

- A unique collection ID.
- A name.
- Optional context and contextual tags.
- A set of documents.
- Usage statistics based on RAG transactions.

Collection availability and account limits depend on the current account configuration. See [Plans and limits](http://localhost:1313/docs/limits.md) before creating collections for production use.

## Documents

A document is the unit that gets indexed and retrieved. It should be small enough to match a specific question and complete enough to be useful on its own.

This is the part that most affects RAG quality. A document should not be "everything you know" about a source; it should be one piece of knowledge that can stand alone when the model reads it later. If a user asks about cancellation fees, the retrieved document should already contain the relevant rule, product, condition, and exception. If the answer only makes sense when the model also sees the previous page, the document is probably too dependent on surrounding context.

A good document usually has:

- A stable name.
- Focused text.
- Optional tags for filtering or maintenance.
- Optional metadata for application-specific data.
- An optional reference ID when the document is one chunk of a larger logical item.

For example, a car manual should not be indexed as one document. Index separate documents for topics such as starting the vehicle, checking tire pressure, pairing Bluetooth, and replacing a headlight. Each document should include enough context to be read independently. For broader chunking guidance, see [Best Practices for RAG](http://localhost:1313/docs/rag/best-practices.md); for query behavior after indexing, see [Semantic Search](http://localhost:1313/docs/rag/semantic-search.md).

## Document Fields

When you import documents in JSONL, each line represents one document that can be created or updated. Use a stable `docid` so AIVAX can recognize the same document on future imports. Sending the same `docid` with different text updates and reindexes the existing document.

For extra application data that does not belong in the searchable text, use `__meta`.

The JSONL import endpoint accepts one JSON object per line:

| Property | Type | Required | Description |
| --- | --- | --- | --- |
| `docid` | `string` | Yes | Stable document name. Existing documents are matched by this value. |
| `text` | `string` | Yes | Text content to index semantically. |
| `__ref` | `string` | No | Reference ID used to group related chunks. Maximum stored length is 64 characters. |
| `__tags` | `string[]` | No | Tags for filtering, browsing, and maintenance. |
| `__meta` | `object` | No | Metadata returned with document details and search results. Metadata is not the semantic text used for embeddings. |

The document name must be non-empty and is limited by the API to 256 characters. The stored document content is required and cannot be empty.

## Upserts and Reindexing

Documents are matched by name (`docid` in JSONL, `Name` in the single-document API).

When a document is created, it is queued for indexing. When existing document text changes, the document is queued again and its vectors are regenerated by the background indexer. When only `__meta` changes, metadata is updated without reindexing the document text.

Reference and tag values are stored with the document. In the single-document API, changed reference, tags, or metadata can update an existing document without reindexing when the text is unchanged. In the JSONL import endpoint, changed text queues reindexing, metadata-only changes update metadata without reindexing, and changed text can also update reference, tags, and metadata. A JSONL entry that changes only the reference or tags is skipped.

## References

Use `__ref` when multiple documents represent parts of the same logical source, such as:

- Sections of the same contract.
- Clauses from the same policy.
- Chunks from the same PDF.
- Product fragments that should be shown together.

When search reference expansion is enabled, if one chunk matches, other documents in the same collection with the same reference can be included in the response.

## Media File Import

The AIVAX dashboard can upload a source file and process it into RAG documents with [Media Injector](http://localhost:1313/docs/rag/media-injector.md). Use it when you have a source file but do not already have focused, self-contained document text prepared for direct or JSONL import.

The original file name is normalized to Unicode NFC and preserved during upload, including accented letters, non-Latin scripts, typographic punctuation, and other Unicode characters. You do not need to rename a file to an ASCII-only name before importing it.

A Media Injector job is created only after every file chunk has uploaded and the dashboard successfully completes the upload. You can then follow it under **Batch > Media Processing**. If no job appears, the upload did not reach its completion step; retry the upload and check the error shown by the dashboard.

## Batch Import Limits

Batch import is sent as a JSONL file in the `documents` multipart field.

Use batch import when you already have many documents prepared outside AIVAX, such as chunks generated from PDFs, product catalogs, policies, or help-center articles. If you are creating or updating one document from an application flow, the single-document endpoint below is usually easier. If you are preparing a large knowledge base, import in batches, wait for indexing, and then test retrieval through [Semantic Search](http://localhost:1313/docs/rag/semantic-search.md) before attaching the collection to a production gateway.

Per-request JSONL line limits and daily RAG insertion limits vary by plan; see [Plans and limits](http://localhost:1313/docs/limits.md#plan-limits). If your import exceeds the request limit, split it into multiple files. If your account reaches the daily insertion limit, wait for the rate window to reset or upgrade the plan.

> [!WARNING]
> Indexing incurs cost based on document text tokens when documents are created or when their text changes.

The embedded API reference is the source of truth for the import request and response contract.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Index%20Documents%20(JSONL))

## Document Management

### Create or update document

This endpoint is useful when your application manages documents one at a time. For example, an admin screen can save one FAQ entry, one policy clause, or one product note directly into a collection. AIVAX matches the document by name: changed text queues reindexing, while metadata-only changes update metadata without reindexing the content.

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Create%20or%20Update%20Document)

### List documents

The browse endpoint helps you inspect what is already inside a collection. Use it when you need to verify an import, find a document by name, review queued versus indexed documents, or filter content before deciding whether to update, delete, or reimport part of the knowledge base.

Supported filters:

- `-t "tag"`: documents containing the tag.
- `-r "reference"`: documents with the exact reference ID.
- `-c "content"`: documents whose content contains the text snippet.
- `-n "name"`: documents whose name contains the text snippet.
- `-i "id"`: documents whose ID contains the supplied text.

Supported states:

- `queued`: documents waiting for indexing.
- `indexed`: documents already indexed.

Supported sort values:

- `created_at_asce`
- `created_at_desc`
- `updated_at_asce`
- `updated_at_desc`
- `indexed_at_asce`
- `indexed_at_desc`

[API endpoint reference](https://inference.aivax.net/apidocs?embed=iframe&embed-endpoint=Browse%20Documents)