How Much Memory Does an AI Agent Need?

Published August 18, 2026

Agentic memory can improve an AI system's performance, but adding more guidance does not always produce better results. In evaluations across eight models, the most effective amount of memory depended on each model's capabilities, remaining performance headroom, and context requirements.

The central finding was that agentic memory must be calibrated to the model rather than enabled as a uniform feature.

Key findings

• ALTK-Evolve enables an agent to learn from its own previous trajectories by distilling reusable guidelines and adding them during inference. It does not update model weights or require human annotation.
• Strong models with available performance headroom often benefit from the complete guideline set.
• Weaker models generally perform better with a compact, high-confidence core supplemented by task-specific retrieval.
• Models that are already near their observed ceiling may show no measurable improvement.
• Curated retrieval can improve accuracy while adding little token overhead. On gpt-oss-120b, it increased task completion by 16.1 percentage points with only a 5% increase in tokens.

Memory dosage depends on model capability

The evaluation identified three recurring patterns across eight models, ranging from a 30B dense model to proprietary frontier systems.

Strong models with headroom

Models that have substantial room for improvement can often use the complete set of self-distilled guidelines, including lessons about uncommon edge cases. DeepSeek-V3.2, a 671B MoE model, improved task completion by 9.5 percentage points when given its full guideline set.

Smaller or weaker models

A large collection of instructions can overwhelm less capable models. These models performed best with a compact set of high-confidence guidelines and a small number of task-relevant guidelines retrieved for each task.

gpt-oss-120b, a 117B MoE model, improved by 16.1 percentage points with curated retrieval. The full guideline set produced a smaller gain while using approximately 50% more tokens.

Saturated models

Some models showed no measurable improvement. This pattern was observed with GLM-5, a 745B MoE model. The result does not establish why the model failed to improve. It may already have been close to its ceiling on the evaluated tasks, the guidelines may not have addressed its remaining errors, or the model may not have applied the guidance effectively.

Parameter count alone does not determine which pattern a model follows. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution may all contribute. The practical conclusion remains that memory dosage can be calibrated to the model.

Learning around the model

In this system, memory is not a replay of previous conversations. It is a set of distilled guidelines covering effective strategies, mistakes to avoid, and relevant edge cases from the agent's earlier trajectories.

The process is:

1. The agent attempts tasks and generates trajectories.
2. ALTK-Evolve extracts behavioral guidelines from successful and unsuccessful runs.
3. The guidelines are consolidated into a reusable set.
4. During inference, the agent receives either the complete set or a task-relevant selection.

The model's weights are not changed. Instead, the system modifies the guidance available during inference. This makes the process portable across the eight evaluated models.

Evaluation on AppWorld

The models were evaluated on AppWorld, which contains 585 multi-step tasks: 168 test_normal tasks and 417 test_challenge tasks across nine simulated applications, including calendars, messaging, and payments.

Two metrics were used:

• TGC, or Task Goal Completion: the percentage of individual tasks completed fully and correctly.
• SGC, or Scenario Goal Completion: a stricter measure that counts a scenario as successful only when every variant of that scenario passes.

Compared configurations

Both memory configurations used the same guideline set, mined once from AppWorld's training split. The test split was not used to build the guidelines. The difference was how the guidelines were supplied to the agent.

Because the number of guidelines mined varies by model, the results are described by strategy rather than by raw guideline counts.

Representative results

The following results report TGC and SGC on the evaluated configurations:

SGC often increased more than TGC. DeepSeek-V3.2, for example, gained 16.1 percentage points on SGC compared with 9.5 percentage points on TGC. This indicates that effective guidelines can help agents succeed across every variant of a scenario, rather than only improving average task performance.

The effect was also observed in models near the top of the TGC range. GPT-5.5 and Claude Opus 4.6 improved SGC by 7.2 and 7.1 percentage points, respectively.

Token cost of memory strategies

Adding a full guideline set to every ReAct step increases input-token usage. The evaluation measured the following average token totals per task:

Curated retrieval kept gpt-oss-120b close to its baseline token usage while delivering the strongest improvement: +16.1 percentage points in TGC at +5% tokens.

Memory also did not substantially lengthen the reasoning loop. DeepSeek-V3.2 used approximately 18 to 19 ReAct steps on average both with and without memory. The additional cost therefore came primarily from larger inputs, not from longer trajectories.

Prompt caching can reduce the production cost of a static guideline set. Since the shared portion remains identical across steps, keeping the guideline prefix stable can make it cacheable.

Context-window size may also influence which strategy is most effective. Models with larger windows may handle complete guideline sets more effectively, while models with smaller contexts may benefit from retrieval that limits the injected content. Controlled experiments isolating this factor have not yet been conducted.

Calibrating memory instead of accumulating it

The results suggest that an agent should not automatically receive everything it has learned. The memory supplied should match the amount of experience the model can use effectively.

• Weaker models: use a compact core with a few task-specific lessons. This approach can also minimize token cost.
• Strong models with headroom: retain the full guideline set, with prompt caching helping control production costs.
• Saturated models: avoid adding context until their remaining failure modes are better understood.

The improvements were obtained automatically from the agent's own trajectories, without human annotation, but they depended on matching the memory strategy to the model.

Future work

Several areas remain open:

• Learned selection: Current retrieval ranks guidelines by cosine similarity, but similarity does not perfectly predict which guidelines will help a task. A selector trained from outcome signals could improve selection.
• Very weak models: Self-distillation may lack sufficient signal below a minimum capability level. Teacher-distilled memory is being considered as a separate approach.
• Additional benchmarks: The findings have been validated on AppWorld, a rigorous but single benchmark. Evaluation on broader agent benchmarks and real-world deployments remains in progress.
• Context-window effects: Controlled studies are needed to separate context-window size from general model capability.

Appendix: Understanding TGC and SGC

Both AppWorld metrics are percentages, with higher values indicating better performance.

TGC, or Task Goal Completion, measures the share of individual tasks that the agent completes fully and correctly. It answers whether the agent completed the requested task.

SGC, or Scenario Goal Completion, is an all-or-nothing reliability measure. Each scenario contains several variants of a task, such as requests with different data, wording, or edge conditions. A scenario passes only when the agent succeeds on every variant. An agent that succeeds on most variants but fails one can receive credit under TGC but not under SGC.