Context Engineering for Sovereign Multi-Agent AI: Shared Memory, Tool Scoping, and Prompt Payload Discipline

Most multi-agent systems on private infrastructure don’t struggle because the model is weak. They struggle because every agent is handed too much text, too many tools, and too little structure. Prompts swell, agents repeat each other’s work, the wrong tool gets called, and one bad assumption quietly travels through the whole workflow.

Context engineering is the fix. Think of each agent’s context as a supply chain you manage on purpose. Instead of forwarding transcripts, raw logs, and a global tool catalog, you deliver a small, typed package of state, evidence, and capabilities that fits the agent’s role.

This article covers the context, memory, and tool layer only. Orchestration graphs, GPU tuning, tracing, and compliance each get their own piece in this series.

The hidden performance tax: context bloat

Context bloat is everything an agent has to read that it doesn’t need to do its current job. Duplicated instructions, full conversation histories, raw tool output, irrelevant documents, and tool definitions for tools it will never touch.

In a multi-agent workflow, it compounds. A research agent pulls a pile of pages and passes its whole browsing history to an analyst. The analyst adds its own working notes and passes the bigger bundle to a writer. By the third hop, nobody is sure which statements were verified and which were guesses.

Illustrative line chart comparing prompt size across three agent handoffs. Forwarding full transcripts makes the prompt grow steeply, from about 2K to roughly 38K tokens, like a snowball gathering mass. Passing a structured brief keeps it nearly flat at 2K to 3K tokens. The gap between the lines is labeled "wasted context."

The cost isn’t only tokens. Bigger prompts mean more prefill work, and they make it less likely that a reusable prompt prefix stays intact between calls. They also make tool selection harder, because the agent is sifting through noise to find the signal.

The principle to hold onto: an agent doesn’t need the whole story. It needs the smallest sufficient, current, and structured slice of it. In practice that means each agent gets its role and rules, the current task and constraints, relevant facts and artifact references, a bounded view of prior decisions, only the tools it’s allowed to use, and a clear description of what it must return.

Four levers get you there, and the rest of this article walks through them: structured handoffs, cache-aware prompt layout, tool scoping, and typed tool contracts.

Context as a contract, not a transcript

The most common failure pattern is context dumping. Agent A finishes, then sends its entire chat history, raw tool results, and internal reasoning to Agent B.

Here is what goes wrong when you do that:

  • The receiving agent has to re-derive what actually matters.
  • Tentative reasoning gets mistaken for established fact.
  • Raw tool output eats context without helping any decision.
  • Contradictions and unverified claims spread downstream.
  • Prompt size grows with every turn, which hurts cache reuse.

The alternative is a structured handoff: a compact, versioned state object holding only what the next agent needs.

A shared-memory state model

The idea is to separate an agent’s narrative from the system’s durable state. Five components cover most workflows.

Memory component What it contains Example
Task object Objective, acceptance criteria, owner, deadline “Prepare a compliance-ready vendor-risk summary by Friday.”
Working memory Current hypothesis, active plan, open questions “Three vendors remain unverified; next step is policy review.”
Evidence store Source-attributed facts, documents, and artifacts “Policy v3.2 states X; source: policy PDF, page 12.”
Decision log What was decided, by whom, and why “Analyst selected vendor B because risk score exceeded threshold.”
Handoff envelope Recipient role, required inputs, expected output “Send findings brief to report writer; output must include citations.”

This makes context auditable. Rather than asking “what did the research agent say?”, the next agent receives claims with confidence levels, source references, and a list of what’s still unknown. Storing artifacts and claims instead of transcripts is also what stops a wrong assumption from spreading. A claim carries its source, so it can be checked. A line buried in a transcript can’t.

Take a concrete case. A research agent shouldn’t hand 40,000 tokens of browsing history to a report writer. It should hand over a findings brief: the research question, the verified findings, supporting citations, a confidence level, known gaps, and the required output format. The writer can produce a good report without ever seeing how the research was done.

The takeaway: shared memory should be an auditable, role-aware workspace, not a growing transcript.

Designing schema-enforced handoffs

Two side-by-side JSON handoff cards between AI agents. The left, marked valid, lists five annotated fields: task, context_refs, constraints, required_output, and validation_rules. The right is stamped "Rejected: missing context_refs" because that field is absent. A validator gate lets the valid handoff pass to the next agent and blocks the malformed one before any downstream work starts.

Structured handoffs work best when they’re machine-checkable. If a handoff is just a paragraph of prose, you’re relying on the next agent to interpret it correctly every time. If it follows a schema, a malformed handoff can be caught before any downstream work starts.

A handoff contract should specify:

  • The sender role and the recipient role
  • The task ID and workflow stage
  • The input fields the recipient requires
  • The output schema and its validation rules
  • Confidence, evidence references, and failure reasons

You can express this with JSON Schema or Pydantic-style typed models. The format matters less than the discipline. Schemas stop required fields from going missing. They stop a free-text “status” field from becoming a dumping ground for everything the sender didn’t know where to put. And they make an invalid handoff visible immediately instead of three agents later.

Versioning matters too. If one agent’s output shape changes, a versioned schema means the change is explicit instead of silently breaking whoever consumes it.

A handoff envelope, described in plain terms, has five parts: the task, the context_refs pointing to evidence and artifacts, the constraints, the required_output, and the validation_rules. That’s enough to tell any recipient what it’s doing, what it can rely on, and what a good answer looks like.

One note on scope: this section is about what a handoff contains. How the pipeline routes it, rejects it, or retries it is an orchestration question, and that belongs with the topology patterns, not here.

Memory tiers: what to keep, summarize, and expire

You don’t need a vector database tutorial to get memory right. You need a retention policy. A simple three-tier model works well.

Working context is the current task, the immediate constraints, and recent decisions. It’s small and changes constantly.

Task memory holds the durable facts, artifacts, and decisions that matter for the whole workflow. It survives across steps but not forever.

Episodic or archive memory keeps historical runs, prior outputs, and reusable patterns. It’s searched when relevant, not loaded by default.

A few retention rules keep this healthy. Keep claims that are attributed to a source. Summarize phases once they’re complete. Expire stale state after a milestone. And keep mutable state, like the current plan, separate from immutable evidence, like a retrieved document, so you never wonder whether a fact changed or was simply overwritten.

Routing matters as much as storage. Each agent should receive the slice of memory that fits its function and the current stage of the workflow. A writer drafting section three doesn’t need the analyst’s discarded hypotheses.

Be wary of memory sprawl. If shared state can grow without limit, you’ve rebuilt the transcript dump one level up. Memory isn’t “more context.” It’s the right context, kept for the right length of time, scoped to the right role.

Cache-aware prompt architecture

Two horizontal strips of prompt blocks compared. In the top strip, a changed timestamp and request ID sit at the start of the prompt, so every block after it (system prompt, tool definitions, examples, current request) is marked red as recomputed. In the bottom strip, the system prompt, tool definitions, examples, and history are green and reused from cache, and only the final timestamp and request block, placed at the end, is recomputed.

Local inference stacks can reuse work when the beginning of a prompt is identical across requests. This is usually called prefix caching or KV-cache reuse. You don’t need to know how any inference server implements it to benefit. You just need to lay out prompts so the stable parts come first and the volatile parts come last.

The recommended order:

  1. Stable system prompt and role definition
  2. Stable tool definitions
  3. Reusable templates and few-shot examples
  4. Task-specific history
  5. The current request and dynamic data

The reason is mechanical. Reuse works from the start of the prompt forward, and it breaks at the first token that differs. A timestamp in the system prompt, a request ID, a changing counter, tools listed in a different order, or dynamic metadata near the top can invalidate everything after it. Even a one-token difference early on forces the rest to be recomputed.

Two habits help a lot. First, serialize deterministically: keep field ordering consistent between calls. Second, update by appending where you can, rather than rewriting earlier content, so the reusable prefix stays intact.

Here’s how that maps out in practice:

Prompt element Cache-friendly placement Why
Agent role and instructions Beginning Stable across runs
Tool definitions Early, stable section Changes invalidate later context
Reusable examples After stable instructions Reused across similar tasks
Conversation history Later Grows during the run
Current request and metadata End Most volatile content

The cache-hostile version of this is the mirror image: a clock reading and request ID at the top, a shuffled tool list, and instructions that get edited on every call. Same information, far less reuse.

This is about prompt layout only. How the inference server manages memory, batching, or quantization is a separate topic.

Tool scoping: fewer choices, better decisions

Give an agent thirty tools and you get predictable trouble. It hesitates between near-duplicates, invents arguments, makes calls it didn’t need, and carries all thirty definitions in every prompt.

A useful rule of thumb is a tool budget: no more than about four targeted tools per specialist agent. Role-based bundles make that easy to apply:

  • Researcher: search, fetch, extract, cite
  • Analyst: calculate, query, transform, summarize
  • Writer: retrieve approved source, draft, revise
  • Executor: validate request, perform action, confirm result

Separate reading from writing. Broad read access is often acceptable. Actions that change something in the world should belong to a narrowly authorized executor, so a confused researcher can’t do real damage.

You can go a step further with dynamic tool routing, exposing only the tools relevant to the current stage of the task. Just remember the connection to caching: changing the tool set changes the prompt prefix. Routing should be deliberate and predictable, not improvised call by call.

Tool scope is a reliability control, not just a way to shrink the prompt.

Typed tool contracts and lightweight arguments

Scoping decides which tools an agent can see. Typed contracts decide how it uses them.

A well-defined tool has a declared purpose, strongly typed inputs, a predictable output structure, and errors that say what went wrong in a form the agent can act on.

Compare two ways of asking for the same thing. The unstructured version is “look this up and tell me what you find.” The typed version is a narrow operation with named parameters, a defined set of output fields, and explicit failure modes. The second one leaves far less room for guesswork.

Typed contracts reduce hallucinated arguments, redundant calls, oversized results, and parsing failures downstream. They also push you toward a good habit: return compact, structured results. A tool that hands back raw HTML, a full log file, or an entire database row is quietly refilling the context you just worked to shrink. Return the fields that drive the decision and a reference to the rest.

To keep the boundary clear: the schema here governs the tool interface. Validation of what passes between agents is a separate concern.

Context budgets and payload hygiene

"Two stacked bars showing an example context budget for an AI agent: role instructions, tool definitions, task state, evidence, recent history, and output spec. In the second bar, evidence grows from 35% to 45% while recent history shrinks from 15% to 5%, illustrating that when one part of the context grows, another must shrink."

Give every agent a context budget, the same way you’d give a service a memory limit. Divide it across role instructions, tool definitions, task state, relevant evidence, recent history, and output instructions. When something has to grow, something else has to shrink.

A few working rules:

  • Remove duplicated instructions.
  • Replace narrative logs with structured facts.
  • Summarize completed phases.
  • Pass references to artifacts instead of embedding them.
  • Cap tool-result size and keep only decision-relevant fields.
  • Use a consistent vocabulary for task types, priorities, confidence levels, and evidence requirements.

It’s tempting to respond to any failure by adding more context. It often backfires. Irrelevant material competes for the model’s attention and raises the odds of a wrong tool choice or a muddled answer.

A before-and-after shows the scale of the difference.

Before: a 12,000-token transcript, 18 tools, and a free-text handoff.

After: a 2,000-token task brief, 3 tools, a schema-enforced handoff, and references to artifacts instead of the artifacts themselves.

The second agent is cheaper to run and easier to get right.

Measuring context quality

You can’t improve what you don’t measure, but you don’t need a full observability stack to start. Six metrics capture context quality:

  • Context precision: the share of provided context that’s relevant to the task
  • Context recall: the share of necessary facts the agent actually had
  • Handoff validity rate: the percentage of handoffs that pass schema checks
  • Tool-selection accuracy: the right tool with the right arguments for the intended action
  • Payload compression ratio: tokens removed without losing task-relevant information
  • Cache-prefix stability: the share of requests that reuse the intended stable prefix

Treat the last two together. Pruning and summarization save tokens, but if they keep changing the early part of your prompt, you lose cache reuse. Judge every compression strategy on both token reduction and prefix stability.

A light cadence is enough. Check schema validity and payload size at each handoff. Check argument validity and result relevance at each tool call. Check completion quality and unnecessary tool calls at the task level.

Instrumentation, trace analysis, and compliance controls are covered in the observability article.

A five-step context engineering checklist

  1. Define the agent’s job. Work out the minimum information and capabilities that role actually requires.
  2. Create a handoff contract. Specify required fields, output shape, evidence references, and validation rules.
  3. Build tiered memory. Keep working context, durable task memory, and archival history separate.
  4. Make prompts cache-stable. Put stable instructions and tool definitions first and volatile content last.
  5. Scope and type every tool. Limit each agent to a small tool set, with typed arguments and compact outputs.

Common anti-patterns

  • Passing complete agent transcripts between every step
  • Giving every agent the same global tool catalog
  • Embedding raw web pages, logs, or database records directly in prompts
  • Putting timestamps, request IDs, or changing status fields in the system prompt
  • Using free-text status fields instead of typed enums and structured fields
  • Letting shared memory grow indefinitely without milestone-based summarization
  • Treating context compression as a substitute for clear task boundaries

What this unlocks

Done well, context engineering gives you smaller prompts and less repeated reasoning. Tool selection gets more reliable because each agent has a small, purpose-built toolset. Handoffs become consistent because state is schema-enforced. Cache reuse goes up because prefixes stay stable. And because responsibilities are cleanly separated, the rest of the sovereign stack becomes easier to tune.

The shift in mindset is the real payoff. The goal was never to use less context for its own sake. It’s to design a context supply chain, where every agent receives a compact, typed, role-appropriate package of state, evidence, and tools, and nothing it doesn’t need.

Where to go next: For orchestration patterns that enforce handoff validation and prevent loops, read the topology article. For how prefix caching interacts with local inference throughput and model tiering, read the compute and inference article. For tracing handoffs and measuring tool accuracy at the span level, read the observability and governance article.

Leave a Comment