Mem0 vs Zep vs Letta vs a local ledger
Choosing agent memory in 2026, honestly — including the cases where the answer is not us.
The question that actually separates them
Every option below can store and retrieve memories. The differences that matter in practice are four: where your memory lives (their database, their platform, or your repo), what it costs to have installed (tokens of tool definitions before your first prompt), what happens to a number you corrected (does the agent keep quoting the old one from history?), and what happens when the agent writes the old number back anyway (is the correction merely shown, or enforced?).
| Where memory lives | Shape | Honest fit | |
|---|---|---|---|
| Mem0 | Hosted memory API + vector store | Managed service; the most adopted in its tier. Free, Starter $19, Pro $249, Enterprise custom (pricing page, 31 Aug 2026) | You want a hosted memory database with a team behind it, and vendor storage passes your security review. Corrections are a free-tier feature there, not what the product is built around. |
| Zep | Hosted temporal knowledge graph | Graph-native, temporal edges; a superseded fact is invalidated, not deleted. Free tier, then Flex from $125/mo; the open-source Community Edition was discontinued in 2025 (Graphiti stays Apache-2.0) | Closest to us in spirit. You want relational/temporal structure and are fine with hosted. |
| Letta | Inside their agent runtime | Agent OS with memory built in | You are building agents on their runtime anyway. As a bolt-on to an existing MCP client, it is a big adoption. |
| Cognee / LangMem / Graphiti | Wherever you assemble it | Libraries | You want parts and are building your own memory system. |
| Cloudflare Agent Memory | Your CF account (KV/DO) | Platform-native default | Your whole agent stack is Cloudflare-native and platform defaults are fine. |
| Ambient platform memory | Inside one vendor's product | Default-on in GitHub Copilot, Claude Code auto-memory and Cursor; extracted from conversations; Copilot's expires after 28 days | You want zero setup and are fine with memory you did not choose to record, that can expire, and that your other tools cannot read. Nothing there is enforced at edit time. |
| Hand-rolled CLAUDE.md | Your repo | A file you curate | Honestly fine at small scale — this is our real competitor. No causality, no query, corrections rely on your editing discipline. |
| memory-pulse | Your repo (local JSONL ledger) | 4 MCP tools over a causal ledger | You want your agent to stop re-quoting withdrawn numbers — and to be blocked from writing them back — from a file you own, for ~670 tokens of overhead. |
What we lead with
Corrections surface first — and are enforced. Record a correction once and it outranks the history that contained the old number, at every brief size, in every session. Surfacing alone is not enough: agents re-violate corrections they were just shown (TRACE, arXiv 2606.13174, found agents re-violated more than half of the corrections they had been shown, with Mem0 installed). So the guard hook checks every edit before it lands and gives one of three verdicts: verified (the edit agrees with the ledger), blocked (it writes a withdrawn value back — the verdict names the ledger line and date that retired it), or no evidence (the ledger has nothing to say — reported as exactly that, never as a pass). A comparison that names the replacement beside the old value goes through. This is the behavior you can test in five minutes — or right now, in the browser.
Tamper-evident receipts, keyless verification. Hash-chained audit logs are no longer rare. What we have not found elsewhere is verification that needs no key at all: hand an auditor the receipt and the public verify endpoint and they check it without your key or our permission. A substituted, deleted, reordered, or truncated event fails by name.
The context tax is inverted. All four tools cost ~670 tokens of definitions, enforced by a test that fails the build past ~1k. Independent measurements put typical multi-server MCP setups at 50–67k tokens before the first prompt.
Structural privacy. The hosted engine keeps no database of your memory — state rides in the request; keys and counters are all it stores. The free tier has no account at all.
When not to pick us: you need cross-repo organizational memory shared by many teams (hosted graphs do this better today); you need multi-seat admin and SSO (we do not have it); or you want memories extracted automatically from every conversation (we record what you tell us to remember — deliberate, not ambient).
Industry benchmarks, with the method attached
LongMemEval and LoCoMo are the two benchmarks the field quotes. A score on either is only comparable when the reader model, the judge model, the retrieval unit and the question set are named — the official LongMemEval judge is gpt-4o-2024-08-06 with per-type prompts, LoCoMo's official metric is token F1 with no judge, and vendors disagree even on LoCoMo's category labels. So every row below carries its method; rows without one are not here.
| Benchmark | System | Score | Reader / judge | Source |
|---|---|---|---|---|
| LongMemEval-S | memory-pulse — retrieval only | R@5 0.970 · R@10 0.985 session recall_any@k | none / none | cert_9efd39bb0776 n=470 (30 abstention excluded); BM25 program run in our execute sandbox; 10 s for all 500 |
| LongMemEval-S | memory-pulse — QA, local 8B reader | 64.2% QA accuracy, all 500 questions (task-averaged 68.5%) | qwen3:8b, local / qwen3:8b, official per-type prompts | cert_7f78bc15fd76 n=500, no sample; the reader gets ~2k tokens (top-20 hit turns plus the following turn, grouped by session date). By type: single-session user .914 · single-session assistant .964 · knowledge-update .833 · temporal reasoning .511 · multi-session .421 · preference .467; abstention .733 (n=30). The judge is a local model, not gpt-4o-2024-08-06: exact against our own arms, approximate against vendor rows |
| LongMemEval-S | memory-pulse — fixed-size signature index (retrieval only) | R@5 0.955 · R@10 0.985 session recall_any@k | none / none | cert_2eb0328a32dd n=470; one fixed 8 KB signature per session whatever its length, term weights read from the signatures themselves, no inverted index, no embeddings |
| LoCoMo | memory-pulse — QA, local 8B reader | J 70.1 · F1 0.400 LLM-judge (J) and official token F1, cats 1–4 | qwen3:8b, local / qwen3:8b, Mem0 'be generous' prompt | cert_600c77c084fd n=1,540, every question, top-10 turns to the reader. By official id: 4 single-hop J .790 / F1 .512 · 2 temporal .670 / .363 · 1 multi-hop .553 / .191 · 3 open-domain .458 / .153. 99 of the 1,540 answer keys are known-bad (issue #27), so 93.6% is the ceiling for anyone. The judge is a local model, not gpt-4o-mini |
| LoCoMo | memory-pulse — retrieval only | turn R@5 0.574 · session R@5 0.894 recall@k over evidence dialog ids | none / none | cert_ba8ac3206539 n=1,536, categories 1–4 by official id; BM25 over every turn |
| LongMemEval-S | Paper full-context baseline | 60.6% QA accuracy | gpt-4o / gpt-4o-2024-08-06 | Oct 2024 |
| LongMemEval-S | Zep | 71.2% QA accuracy | gpt-4o-2024-11-20 / LLM judge (model not named) | Jan 2025 |
| LongMemEval-S | Supermemory (third-party rerun) | 81.6% QA accuracy | gpt-4o / gpt-4o, paper prompts | 2026 |
| LongMemEval-S | Mem0 | 94.4% QA accuracy | gpt-4o / gpt-4o | Apr 2026 |
| LongMemEval-S | agentmemory (rohitg00) | R@5 0.952 session recall_any@5 (retrieval only) | none / none | 2026 |
| LoCoMo | Full-context baseline (Mem0 paper) | J 72.9 LLM-judge (J), cats 1–4 | gpt-4o-mini / gpt-4o-mini | Apr 2025 |
| LoCoMo | Mem0 | J 66.9 LLM-judge (J), cats 1–4 | gpt-4o-mini / gpt-4o-mini | Apr 2025 |
| LoCoMo | Zep | J 75.1 ± 0.2 LLM-judge (J), cats 1–4 | gpt-4o-mini / gpt-4o-mini | May 2025 |
| LoCoMo | Letta (filesystem) | 74.0% accuracy, cats 1–4 | gpt-4o-mini agents / gpt-4.1 | Aug 2025 |
| LoCoMo | Mem0 platform v3 | 92.5% accuracy, cats 1–4 | gpt-5 / gpt-5 | 2026 |
Our retrieval rows have no model in the loop: the memory system's job is to hand the reader the right evidence, and that is measured directly against each question's labelled evidence. Our QA rows are the complete runs, every question, on a local 8B reader and judge (qwen3:8b on an M1 Pro, $0) with the official prompts; because the judge is not the official gpt-4o, read them exactly against our own arms and only approximately against vendor rows. On both benchmarks the shape is the same: single-hop questions are strong and multi-hop and temporal questions are the weakest, which is where the work goes next. Vendor rows are as each vendor published them, read 1 September 2026; the readers and judges differ from row to row, which is the point.
Try the five-minute test
Install (claude mcp add memory-pulse -- npx -y memory-pulse), record a correction with
remember (kind=correction), start a new session, and ask the agent about the old number.
Then do the same with your current setup.