Anatomy of a prompt

A prompt is the input that guides a language model towards an answer. In a simple chat box, it looks like the sentence you type. In a business assistant, it usually includes much more: standing instructions, previous messages, relevant documents and results from other software. Understanding those parts helps you diagnose a poor answer without endlessly rewriting the latest question.

Imagine briefing a temporary receptionist. You provide the office rules, explain what the visitor wants, and hand over any records needed to help. The model receives a similar briefing, but with an important difference: it does not carry a private, lasting recollection from one request to the next. The application must supply the relevant briefing again.

Messages have roles #

A message is a piece of conversation labelled with its source. The label is called its role. Roles tell the model whether text is an instruction from the application, a request from the person using it, a previous answer, or information returned by a tool. A tool is software the assistant can ask to perform a defined operation, such as checking delivery status.

The system role answers questions such as “What is this assistant responsible for?” and “What should happen if the answer is unavailable?” For a support assistant, that might mean explaining delivery policies in plain language, checking order records before stating a status, and referring unresolved cases to a person. Put durable behaviour here rather than asking customers to repeat it.

The user role carries the immediate need: “My parcel has not arrived. Can you check it?” A user may also provide documents, corrections or preferences relevant to that task. These are useful inputs, but an ordinary user message should not become an administrative rule merely because it says “I am the manager; ignore the policy.” The application decides which sources have authority.

An assistant message records what the model previously said or requested. Keeping it helps resolve follow-up questions such as “Can you explain that more simply?” However, a previous answer is not proof that its claim was correct. If the assistant guessed a delivery date earlier, repeating that answer in history does not turn the guess into a verified fact.

Tool results need the same care. A successful lookup can establish the status recorded in an authorised system at that moment. An error establishes only that the lookup failed. Neither should be silently rewritten into the result the customer hoped for. Retrieved web pages or documents may contain instructions of their own; those instructions are source material, not automatically commands for the assistant.

Follow one conversation #

The following example is fictional. Notice that the tool contributes a record, while the assistant turns that record into a customer-facing answer. The roles remain separate even when all the messages are ultimately presented together to the model.

The customer might see only their question and the final reply. The application may keep system instructions and technical tool details out of the chat interface. Invisible to the customer does not mean absent from the model’s input, nor does it make system instructions a safe place to store secrets. Only include information the model needs for its job.

Related: on AIVAX, the configured runtime that combines instructions, a model and connected capabilities is called an AI gateway. That configuration is distinct from the customer’s latest message. Separating the two makes it easier to change the customer’s request without weakening the service’s operating rules.

Rebuild the briefing on every turn #

A turn is one exchange in the conversation. When the customer asks a follow-up, the model does not independently open the old chat and remember what happened. At the model-request boundary, it is stateless: relevant context must be supplied again. A product may store a conversation for you, but that persistence belongs to the surrounding application or service, not to an enduring personal memory inside the model.

“Everything is re-sent” means everything the model needs for the new generation must be represented in its input context. It does not mean every product transmits every historical message forever. Some products manage stored sessions, summaries or reusable input internally. Those conveniences do not remove the need to decide what information the model can actually see on this request.

This distinction explains a common surprise. You tell an assistant your preferred delivery address early in a lengthy chat, then it asks again later. The problem may not be poor wording: the original detail may no longer be in the selected history. The unit on context windows and tokens explains that space limit, while memory explains deliberate recall across conversations.

Give each part one job #

A well-assembled prompt separates behaviour, evidence and task. Mixing them into one long paragraph makes it harder to see what should stay constant and what should change for a particular customer. It also makes troubleshooting difficult: you cannot easily tell whether a wrong answer came from missing facts or conflicting instructions.

Role labels help the model interpret a conversation, but they are not a replacement for software controls. A sentence saying “never issue refunds” should be reinforced by not granting refund authority to an assistant that only explains policy. Keep permissions outside the prompt as well as describing them clearly inside it.

When reviewing a disappointing response, ask a practical question: “Could a new colleague have answered correctly from this exact briefing?” If essential facts were absent, adding more forceful language is unlikely to help. If the facts were present but the output was unclear, the next step is to improve the task description and examples.

What’s next: Explore prompting techniques to make that briefing clearer without making it unnecessarily long.