Prompt

What a user types is rarely what the model receives. Chat APIs accept a structured list of messages with roles - system (or developer), user, assistant and tool - plus tool definitions and parameters. The serving stack renders this list through the model's chat template into one flat token sequence, inserting special delimiter tokens around each turn (ChatML-style markers such as im_start and im_end, or model-specific equivalents), and appends the opening of an assistant turn so that next-token prediction continues as the assistant. Tool schemas, retrieved documents, images, the current date and safety instructions added by the platform all end up in that same sequence. Using the wrong template with an open-weight model is a common cause of degraded or bizarre output.

Role markers are learned conventions, not access controls. Instruction and preference training teach the model to treat system text as higher-priority and tool output as data, and OpenAI's instruction-hierarchy work (Wallace et al., 2024) trains this explicitly, but at inference everything is tokens attending to tokens. That is the structural reason prompt injection exists: an instruction embedded in a pasted document, an email or a web page is processed by the same mechanism as a legitimate instruction. Greshake et al. (2023) named the case where the attacker never talks to the model directly indirect prompt injection; OWASP lists prompt injection as LLM01 in its 2025 Top 10 for LLM applications.

Prompt structure affects cost and latency. Input tokens are billed per request, so a long static prefix (system prompt, tool definitions, reference documents) is paid again on every call unless the provider's prompt caching reuses the precomputed KV state for an identical prefix; caching works only if the stable content comes first and the variable content last. Many assistants also support prefilling the start of the assistant turn to force a format, and stop sequences to end generation. The prompt plus the maximum requested output must fit in the context window.

From a data-protection perspective the prompt is the data flow. Everything in it is transmitted to the model operator, may be logged for abuse monitoring or debugging for a period set in the provider's terms, and may appear in the provider's telemetry. Where the prompt contains personal data the provider is typically a data processor under GDPR Art. 28, requiring a data processing agreement, and retention, training-use and region settings must be verified rather than assumed. Logging prompts on the application side is useful for audit and incident response but creates a new store of sensitive text that needs its own access control.