Skip to content

Research / Library

Library

An annotated account of the work this lab builds on: what each effort measures, and where it stops. Naming what others do well is the point; a benchmark is only credible if its related work is honest. Corrections are welcome at contact.

00 / METHOD

Every entry is read before it is listed, and carries the same two annotations: covers, what the work establishes, and stops at, the boundary past which it does not speak. The second is not a criticism. Every honest piece of work has one, including this lab's.

Nothing is listed here to make this lab look better by comparison, and the lab's own entry sits at the bottom under the same rules. Where a result is quoted, it belongs to the cited authors.

Section I

Agent and MCP attack surface

How agent systems are actually attacked, and what containment exists. This is the reading behind ECHOS and AEGIS, and it comes first because the lab's work does.

01 / INJECTION

Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Greshake, Abdelnabi, Mishra, Endres, Holz, and Fritz; February 2023. The paper that named the primitive everything else in this section elaborates: text a model retrieves is text a model obeys, so any untrusted document reaching the context window is an instruction channel rather than data.

Covers. The attack class itself, with a taxonomy of delivery paths demonstrated against applications people were already running.

Stops at. It predates MCP and the tool-calling agent stack. The mechanism transfers intact; the specific integrations it breaks no longer exist in that form.

arxiv.org/abs/2302.12173
arXiv:2302.12173
02 / THREAT MODEL

The lethal trifecta for AI agents

Simon Willison; June 2025. The clearest short statement of when an agent is exploitable at all: access to private data, exposure to untrusted content, and the ability to communicate externally. Any two of the three are survivable. All three is an exfiltration path, and no amount of prompt-level instruction closes it.

Covers. A threat model a practitioner can apply in about a minute. This lab uses it as the first filter when scoping an ECHOS target.

Stops at. It is a framing, not a measurement, and says so. Nothing here is quantified, and nothing here tells you how often the three overlap in deployed systems.

simonwillison.net
Essay, June 2025
03 / HARNESS

AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

Debenedetti, Zhang, Balunović, Beurer-Kellner, Fischer, and Tramèr; June 2024, NeurIPS 2024. Ninety-seven realistic tasks and 629 security test cases in an environment built to be extended rather than frozen. Two results have aged well: more capable models were often easier to attack, and plain tool isolation outperformed the more elaborate prompting defenses.

Covers. Attacks and defenses measured against each other in one place, on an evaluation designed to keep moving as both sides do.

Stops at. The environments are synthetic by construction: an email client, an e-banking site, a travel booker. Nothing here is a third-party server as it actually ships.

arxiv.org/abs/2406.13352
arXiv:2406.13352
04 / MCP

MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits

Radosevich and Halloran; April 2025. Drives industry-leading models, using tools from default MCP servers, into malicious code execution, remote access control, and credential theft. Ships MCPSafetyScanner, an agent-driven auditor, alongside the findings.

Covers. That the exposure is real and reachable from stock configurations, not only from contrived ones. The closest published work to the ECHOS scope, and worth reading before anything this lab publishes on MCP.

Stops at. A demonstration across a handful of servers rather than systematic coverage of the ecosystem, and a snapshot of a protocol that has changed since.

arxiv.org/abs/2504.03767
arXiv:2504.03767
05 / TOOL POISONING

Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoning

Huang, Huang, Tran, and Milani Fard; March 2026. STRIDE and DREAD applied across five components of an MCP deployment: host and client, the model, the server, external data stores, and the authorization server. Tool poisoning, malicious instructions embedded in tool metadata, comes out as the most prevalent and impactful client-side vulnerability.

Covers. A structured threat model rather than a bug list, which makes it usable for deciding where to look next instead of only what has already been found.

Stops at. The severity ranking is analytic, derived from the model rather than from disclosed incidents in the wild. Whether the ordering survives contact with real deployments is open.

arxiv.org/abs/2603.22489
arXiv:2603.22489
06 / CONTAINMENT

Defeating Prompt Injections by Design

Debenedetti, Shumailov, Fan, Hayes, Carlini, Fabian, Kern, Shi, and Tramèr; March 2025. CaMeL extracts control and data flow from the trusted query so untrusted data can never alter program flow, and gates exfiltration behind explicit capabilities. It clears a large share of AgentDojo's security cases with provable guarantees rather than empirical ones.

Covers. The design-level argument this lab's AEGIS work shares: containment belongs in the runtime around the tool, not in instructions inside the prompt.

Stops at. It constrains the agent's own program. It does not tell you what a third-party server does with the authority it was granted once the call goes through.

arxiv.org/abs/2503.18813
arXiv:2503.18813

Section II

Memory, staleness, and forgetting

Stored beliefs going stale, and how well the field measures it. State-drift stays on this site because it is an attack surface: a belief held past its expiry is a reliability bug when the world moves on its own and a vulnerability when someone moves it on purpose.

07 / STALENESS

STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?

Chao, Bai, Sheng, Li, and Sun; May 2026. Four hundred expert-validated conflict scenarios, 1,200 evaluation queries, contexts up to 150K tokens. Probes three abilities: recognizing that an old belief is outdated, rejecting a query that falsely presupposes the stale state, and proactively applying the updated state. The strongest frontier model reached 55.2% overall; most specialized memory frameworks fell below 10%.

Covers. Whether agents notice staleness at all, measured with unusual care, and the implicit-conflict case where nothing is explicitly negated.

Stops at. The conflicts are constructed and expert-validated rather than arising from an operational workload.

arxiv.org/abs/2605.06527
arXiv:2605.06527
08 / CONTROL PLANE

Control-Plane Placement Shapes Forgetting

An architectural study of agent memory across thirteen system configurations; June 2026. Introduces ForgetEval, a 1,000-case templated suite with a 385-case adversarial layer, released under MIT. Documents that production memory failures are predominantly forgetting failures, and that the control plane, the operations that supersede, release, and purge stored facts, is far less tested than recall.

Covers. Its sharpest result: systems scoring 63 to 68 percent on their own test cases dropped to 28 to 33 percent on an externally authored subset. That is the strongest third-party evidence for why neutral, externally runnable harnesses matter.

Stops at. Thirteen configurations is a study, not a census, and the adversarial layer is partly LLM-drafted. The direction of the finding is more durable than its exact numbers.

arxiv.org/abs/2606.15903
arXiv:2606.15903
09 / RECALL

LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

Wu, Wang, Yu, Zhang, Chang, and Yu; October 2024, ICLR 2025. Five hundred curated questions over freely scalable chat histories, decomposed into five abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Commercial assistants and long-context models lost around 30 percent accuracy across sustained interaction.

Covers. The reference point for recall, and the earliest of these to treat knowledge updates as an ability distinct from retrieval.

Stops at. Chat history between one user and one assistant. No adversary, no operational workload, and updating is one dimension of five rather than the object of study.

arxiv.org/abs/2410.10813
arXiv:2410.10813
10 / FORGETTING

Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions

Hu, Wang, and McAuley; July 2025. MemoryAgentBench evaluates four competencies: accurate retrieval, test-time learning, long-range understanding, and selective forgetting. Its FactConsolidation dataset applies counterfactual edits and tests whether an agent correctly prefers later information over the earlier fact it replaced.

Covers. Selective forgetting as a first-class competency rather than a footnote to recall. This is the FactConsolidation reference an earlier version of this page promised.

Stops at. The counterfactual edits are constructed, so the conflict signal is cleaner than anything a production system sees.

arxiv.org/abs/2507.05257
arXiv:2507.05257
11 / METRIC

From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents

Uddin, Shubham, Blanco, Baral, and Wang; April 2026. Introduces the Memora benchmark, spanning weeks-to-months conversations across remembering, reasoning, and recommending, and with it Forgetting-Aware Memory Accuracy: a metric that penalizes reliance on obsolete or invalidated memory instead of only rewarding correct recall. Across four models and six memory agents, reuse of invalid memories was frequent.

Covers. The scoring rule, which is the contribution that matters here. A metric that makes staleness cost something is the design closest to this lab's own.

Stops at. Still a conversational benchmark, and the naming is a trap worth flagging: this Memora is a benchmark, while the Memora in the next entry is a memory system from a different group.

arxiv.org/abs/2604.20006
arXiv:2604.20006
12 / SYSTEM

Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity

Xia, Zhang, Dixit, Harimurugan, Wang, Ruhle, Sim, Bansal, and Rajmohan (Microsoft Research); February 2026. Organizes memory through primary abstractions that index concrete values and consolidate related updates, with cue anchors widening retrieval. Reports state-of-the-art results on LoCoMo and LongMemEval while storing roughly half the entries Mem0 does.

Covers. A system, not a measurement. It is listed here because it is a candidate subject for the State-Drift Benchmark, not a peer of it.

Stops at. Its evaluation is recall-oriented. Consolidating related updates into unified entries is exactly the operation that decides whether a superseded fact survives, and the behavior under adversarial staleness is untested.

arxiv.org/abs/2602.03315
arXiv:2602.03315
13 / BELIEF UPDATING

When Does Belief-Based Agent Memory Help?

Singh; June 2026. Stores knowledge as a probability distribution per entity-attribute pair, updated by Bayesian inference and capped by provenance as a poisoning defense. The controlled ablation returns a negative result: Bayesian updating gains little over naive last-write-wins, because the conversational benchmarks it was tested on rarely contain contradictory or differently reliable evidence.

Covers. A negative result worth more than most positive ones. It locates the limiting factor in the benchmarks rather than in the method, which is the same conclusion this lab reached from the other direction.

Stops at. Preliminary single-author work on one benchmark. The finding deserves replication before it carries weight.

arxiv.org/abs/2606.22030
arXiv:2606.22030

Section III

Evaluation methodology

Who writes the test cases, and what that does to the scores. Section II's control-plane study belongs to this theme too: its externally authored subset cut reported scores roughly in half.

14 / ADAPTIVE ATTACKS

The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections

Nasr, Carlini, Sitawarin, Schulhoff, Hayes, Ilie, Pluto, Song, Chaudhari, Shumailov, Thakurta, Xiao, Terzis, and Tramèr; October 2025. Twelve prominent defenses, spanning prompting, adversarial training, filtering, and secret-knowledge mechanisms, all fall to adaptive attackers, most at success rates above 90 percent. The argument is about evaluation practice: static attack sets and weak optimization make a defense look stronger than it is.

Covers. Why this lab publishes the harness with the numbers. A defense evaluated against fixed attacks reports its own best case, and a benchmark scored by its own authors has the same problem in a different coat.

Stops at. The subject is jailbreaks and prompt injection, not memory. The methodological point transfers unchanged; the attack techniques do not.

arxiv.org/abs/2510.09023
arXiv:2510.09023

Section IV

This lab

Listed under the same rules as everything above, including the part about where it stops.

15 / THIS LAB

The State-Drift Benchmark

Production memory systems, run as deployed, under changing facts, in an operational harness. The intended complement to the work above: constructed-scenario studies show the judgment gap exists; this benchmark was built to measure how shipped systems behave inside it, with the dataset, oracle, and fairness scripts to publish alongside the results.

Covers. Shipped systems rather than research prototypes, scored by someone who did not write them.

Stops at. Archived in August 2026 without publishing. It produced no results, so it supports no claim, and every result cited elsewhere on this page belongs to someone else.

Read the primer
Archived 2026-08