Agentic Context Engineering (ACE): How AI Agents Learn From Experience

Most AI agents begin every task with the same system prompt, a few tools, and perhaps a transcript of recent messages. When they make a mistake, the lesson usually disappears with the session. The next run can fail in exactly the same way.

Teams often respond by making the prompt longer. They add another rule, another example, another warning. Eventually the prompt becomes a document that nobody fully understands and nobody wants to edit.

Agentic Context Engineering, or ACE, proposes a different approach: treat the agent’s context as a playbook that can learn from execution. The model’s weights stay unchanged. What evolves is the structured knowledge supplied to the model when it works.

The idea comes from the ICLR 2026 paper Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. Its central claim is practical: an agent can improve by collecting useful strategies, failure patterns, and domain rules in its context instead of repeatedly rewriting one large prompt.[1]

What ACE changes

A normal agent receives a static context:

system instructions + retrieved documents + conversation history + current task

An ACE-style agent also receives an evolving playbook:

system instructions + relevant playbook entries + current task

After the task, execution feedback is converted into small updates to that playbook. A successful API sequence might become a reusable procedure. A failed attempt might become a warning. A correction from a validator might refine an existing rule.

This is more specific than the broad term “context engineering.” Context engineering covers the selection, construction, and delivery of everything placed in a model’s context window. ACE is a method for improving part of that context over time.

The distinction matters. A prompt tells the agent what to do now. An ACE playbook records what previous runs discovered about doing it well.

Why static context breaks down

The ACE paper describes two recurring problems in context adaptation.

The first is brevity bias. Prompt optimizers often converge on short, general instructions because concise prompts are easy to score and rewrite. Those summaries can drop the exact details an operational agent needs, such as tool constraints, domain terminology, API quirks, and known failure modes.[1]

The second is context collapse. If an LLM rewrites the entire accumulated context after every task, useful details can disappear in one pass. In the paper’s AppWorld case study, a context containing 18,282 tokens at one step was compressed to 122 tokens at the next. Accuracy fell from 66.7 to 57.1, below the 63.7 baseline without adaptation.[1]

The failure is familiar if you have used long-running agents. A summary looks clean, but it silently replaces specific knowledge with phrases such as “validate inputs” or “use the appropriate API.” Those rules sound reasonable and help almost nobody.

ACE avoids full rewrites. It stores knowledge as small entries and applies localized changes.

The generator, reflector, and curator

ACE separates context improvement into three roles.

1. The generator does the work

The Generator is the task-solving agent. It receives the current query and the relevant playbook entries, then produces a trajectory: reasoning, tool calls, observations, and a final result.

It should also identify which playbook entries helped and which were misleading. This gives the learning process a traceable connection between a rule and an outcome.

2. The reflector extracts the lesson

The Reflector reviews the trajectory and its feedback. It asks concrete questions:

• What worked?
• What failed?
• Was the failure caused by a bad strategy, missing knowledge, or incorrect execution?
• Which lesson could help on a future task?
• Does an existing playbook entry need correction?

Separating reflection from task execution reduces the pressure on one model call. The Generator focuses on solving. The Reflector focuses on learning from the result.

3. The curator updates the playbook

The Curator turns the Reflector’s lessons into delta updates. A delta can add a new entry, revise an existing one, increment usefulness counters, or mark an entry for pruning.

The paper represents the playbook as structured bullets with unique identifiers, content, and counters showing how often each bullet was helpful or harmful. It merges updates using deterministic code rather than asking an LLM to rewrite the full context.[1]

A simplified entry could look like this:

{
  "id": "payments-014",
  "scope": ["refunds", "card-payments"],
  "content": "Before refunding a captured payment, retrieve the payment and verify that its status is refundable.",
  "helpful": 12,
  "harmful": 1,
  "source": "execution-feedback",
  "updatedAt": "2026-08-05T00:00:00Z"
}

This structure is easier to inspect, retrieve, update, and delete than a single 20,000-token prompt.

A practical example

Consider an agent that resolves payment support tickets.

A customer asks for a refund. The agent immediately calls the refund endpoint, but the provider rejects the request because the payment is still pending. The task fails, and the environment returns a structured error.

The Reflector should avoid recording a vague lesson such as “check payment status first.” It can extract a more useful rule:

For refund requests, retrieve the payment before calling the refund endpoint.
Proceed only when the provider status is captured and refundable.
If the payment is pending, explain the delay and stop instead of retrying the refund.

The Curator then checks the existing playbook:

• If no refund rule exists, add this entry.
• If a partial rule exists, update it in place.
• If several entries express the same procedure, merge them.
• If later executions show that the rule is wrong for one provider, narrow its scope instead of deleting the entire lesson.

On the next similar ticket, retrieval supplies this entry to the Generator. The agent starts with knowledge obtained from the earlier failure.

That is the useful form of “self-improvement” in ACE. The model has not become more intelligent in a general sense. The system has preserved a verified lesson and made it available at the right time.

Grow and refine instead of rewrite and hope

An evolving playbook cannot grow forever without maintenance. ACE uses a grow-and-refine strategy.

New knowledge is appended as new entries. Existing knowledge is edited locally. A deduplication stage compares semantically similar entries and removes redundancy. Refinement can happen after each update or only when the playbook approaches its token budget.[1]

This gives the system two competing responsibilities:

1. Preserve useful detail.
2. Keep the context relevant enough for the model to use.

Longer is not automatically better. The goal is a comprehensive playbook with strong retrieval and clear scopes, not a dump of every observation the agent has ever seen.

A production curator therefore needs rules that an LLM cannot override casually:

- Never replace the full playbook in one generation.
- Every mutation must target an entry ID.
- Preserve source and evaluation history.
- Require repeated evidence before deleting a useful rule.
- Quarantine contradictory entries for review.
- Enforce a token budget per domain or tool.

Offline and online adaptation

ACE supports two modes.

Offline adaptation builds the playbook from a training set before deployment. The agent runs tasks, observes results, reflects on them, and produces a playbook that is later evaluated on held-out tasks. This works well when you have representative scenarios and a reliable test environment.

Online adaptation updates the playbook during use. The agent first attempts a task with its current context, then learns from the outcome before handling later tasks. This is useful when production behavior changes or when the full problem distribution is unknown.[1]

The safest production design usually combines both:

1. Build an initial playbook offline from tests and historical cases.
2. Deploy it as a versioned artifact.
3. Collect candidate lessons online.
4. Validate those lessons before promotion.
5. Roll out a new playbook version gradually.

Updating a live playbook after every unverified user interaction is risky. A malicious instruction, a transient provider error, or a poor reflection could become durable context. Online learning needs the same controls as code changes: review, tests, versioning, and rollback.

ACE compared with nearby techniques

Technique	What changes	Best use
Prompt engineering	Instructions for one model call or workflow	Roles, constraints, formats, and immediate behavior
Retrieval-augmented generation	External evidence selected for the current query	Facts from documents, databases, or search
Agent memory	Information retained across tasks	User preferences, prior events, and working state
Fine-tuning	Model weights	Broad behavior or capabilities that should apply across contexts
ACE	A structured playbook built from execution feedback	Reusable strategies, failure patterns, and domain procedures

These techniques can work together. An ACE system may use RAG to retrieve source documents, memory to retain user-specific facts, and a playbook to supply learned operating procedures. Fine-tuning remains useful when the desired behavior is too broad or too frequent to carry efficiently in context.

The boundary worth protecting is provenance. Retrieved facts should not silently turn into permanent procedural rules. User-specific memory should not leak into a shared playbook. A failed execution should not become a lesson until the system understands why it failed.

What the reported results show

The paper evaluates ACE on AppWorld and domain-specific reasoning tasks. AppWorld is a benchmark with simulated applications, hundreds of APIs, and tasks that require agents to write and execute code across multiple apps.[3]

The authors report average gains of 10.6% on agent tasks and 8.6% on financial benchmarks compared with the selected baselines. They also report an average 86.9% reduction in adaptation latency. In two detailed comparisons, ACE reduced offline AppWorld adaptation latency by 82.3% and rollouts by 75.1% versus GEPA; on online FiNER adaptation, it reduced latency by 91.5% and token cost by 83.6% versus Dynamic Cheatsheet.[1]

The official implementation exposes offline, online, and evaluation-only modes. It also includes configurable playbook token budgets, curator frequency, reflection rounds, deduplication, and separate models for each of the three roles.[2]

These are promising benchmark results, not a guarantee that any agent will improve after adding a reflection loop. The quality of the feedback signal matters more than the label “self-improving.”

Where ACE can fail

The paper is direct about its main limitation: if the Reflector cannot extract a sound lesson, the playbook becomes noisy or harmful. The finance experiments also show degradation when neither ground-truth labels nor reliable execution signals are available.[1]

Several practical failure modes follow from that:

Bad feedback creates bad memory

A timeout does not prove that a procedure is wrong. A flaky test does not justify a new rule. The system must separate task quality from infrastructure noise.

Local success can produce a global mistake

A workaround that succeeds for one provider, tenant, or software version may fail elsewhere. Every entry needs scope and provenance.

Retrieval can hide good knowledge

A correct lesson is useless if retrieval does not select it. Evaluate retrieval recall separately from Generator performance.

Context can still become expensive

Incremental updates prevent destructive rewrites, but the playbook can still grow. Track tokens, latency, cache hit rate, and the percentage of retrieved entries that the agent actually uses.

The playbook becomes a security boundary

If external content can influence reflection, attackers may try to plant durable instructions. Candidate updates should pass policy checks, source validation, and approval rules before entering shared context.

ACE is also unnecessary for simple tasks with stable strategies. The paper notes that some question-answering tasks and fixed-strategy games benefit more from concise instructions than from a rich playbook.[1]

A production blueprint

A minimal ACE-inspired system needs six components:

1. A task runner that records trajectories and tool results.
2. Evaluators that produce reliable, structured feedback.
3. A Reflector that proposes lessons with evidence.
4. A Curator that emits typed delta operations.
5. A versioned playbook store with deterministic merging.
6. Retrieval that selects entries by task, tool, domain, and similarity.

The update path can be expressed as a small loop:

playbook = loadVersion("production")

for task in tasks:
    entries = retrieve(playbook, task)
    trajectory = generator.run(task, entries)
    feedback = evaluate(task, trajectory)

    candidateLessons = reflector.review(
        task,
        trajectory,
        feedback,
        entries
    )

    deltas = curator.propose(candidateLessons, playbook)
    validatedDeltas = validate(deltas, feedback, policies)
    playbook = deterministicMerge(playbook, validatedDeltas)

publish(playbook, after = [regressionTests, humanReview, canary])

The evaluator is the part teams most often underestimate. Before automating reflection, define what counts as success, what evidence is trustworthy, and which failures should produce no learning at all.

What to measure

Do not judge an ACE system only by the final task score. Track the learning system itself:

• Task success before and after each playbook version.
• Improvement on held-out tasks, not only tasks used to generate lessons.
• Number of entries added, revised, merged, quarantined, and deleted.
• Retrieval recall and entry usage.
• Contradiction and duplicate rates.
• Token cost and end-to-end latency.
• Regressions caused by a specific delta.
• Time required to remove a bad or sensitive lesson.

A playbook version should be reproducible. Given the same base version and the same approved deltas, the merge should produce the same result.

The larger lesson

ACE moves agent development away from endlessly editing one system prompt. It treats operational knowledge as data with structure, history, evaluation, and lifecycle rules.

That is the part worth adopting even if you never use the official framework. Record lessons as small, scoped entries. Tie them to evidence. Retrieve them deliberately. Update them locally. Test the resulting context as seriously as you test code.

An agent does not learn merely because it has memory. It learns when experience becomes a reliable instruction for the next run.

Sources

[1] https://arxiv.org/abs/2510.04618 — Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models [2] https://github.com/ace-agent/ace — ACE official implementation [3] https://arxiv.org/abs/2407.18901 — AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents