Memory for AI Agents: Vector Stores and Memory Layers Compared

Memory for agents is two distinct purchases, not one: a vector store that retrieves relevant information — the shelf — and, increasingly, a memory layer that decides what an agent should remember, update, or forget across sessions — the librarian. As of this writing, the pragmatic defaults are pgvector if you already run Postgres, Qdrant or Weaviate for a dedicated open-source engine, Pinecone when you want zero operations, and Chroma for prototypes that need to grow up gradually. On the librarian side, mem0 is the most adopted open-source layer, Zep leads on temporal knowledge graphs, and Letta packages agent-plus-memory as one opinionated server. Both directories below, with verdicts and skip-if notes.

The shelf and the librarian

Retrieval-augmented generation solved "find me relevant chunks." It did not solve "remember that this user switched companies in March and stop citing the old one." That second problem — extraction, consolidation, contradiction handling, forgetting — is what the new memory layers sell. They sit above a vector store (and often a graph store) and expose a simple API: add what happened, get what matters. If your agent only answers questions over documents, you need the shelf. If it serves the same user across many sessions, you'll eventually want the librarian too — or you'll hand-roll a worse one.

Vector stores: the directory

Vector store verdicts

pgvector

pgvector turns the Postgres you already run into a competent vector store, with joins, transactions, and one backup story. Hosted Postgres providers ship it by default, and for the majority of agent products — thousands to low millions of vectors — it is simply enough.

Verdict: boring wins; the correct default for most teams in 2026.

Skip if: you are heading past roughly 50-100 million vectors or need heavy hybrid search — dedicated engines pull ahead there.

Pinecone

Pinecone made serverless vector search mainstream: no clusters, no capacity planning, pay for what you store and query.

Verdict: still the zero-ops king, and the bill is predictable — until your workload is not.

Skip if: data residency or self-hosting is a requirement; there is no self-hosted Pinecone.

Qdrant

Qdrant is a Rust engine with the best filtered-search story in open source, a generous free cloud tier, and consistently strong benchmark showings.

Verdict: the performance pick for self-hosters and the easiest OSS-to-cloud graduation path.

Skip if: you want an ecosystem beyond vectors — it is deliberately a vector database, not a platform.

Weaviate

Weaviate bundles hybrid (keyword plus vector) search, reranking modules, and multi-tenancy into one platform-flavored engine.

Verdict: the most batteries-included open-source option; complexity arrives with the batteries.

Skip if: your use case is plain similarity search — you would be operating machinery you do not need.

Chroma

Chroma is what most tutorials use for a reason: pip install, four lines, working retrieval. Chroma Cloud now gives that prototype a usage-based production path.

Verdict: the best developer experience in the category, now credible beyond prototypes.

Skip if: you are planning for billions of vectors or strict enterprise multi-tenancy from day one.

Milvus

Milvus is built for the far end of scale — billions of vectors, GPU indexes, serious distributed operations — with Zilliz Cloud as the managed route.

Verdict: the scale monster; nothing here handles more, and nothing here costs more attention.

Skip if: you are small — operating Milvus for a million vectors is a tax without a benefit (Milvus Lite notwithstanding).

MongoDB Atlas Vector Search

MongoDB Atlas puts vector indexes next to the documents you already store, which collapses a whole sync pipeline for existing Mongo shops.

Verdict: obvious if you are on Atlas already; nobody should adopt Mongo just for vectors.

Skip if: you are not a MongoDB shop — that is the entire value proposition.

Redis

Redis offers the lowest query latencies in the group and doubles as your semantic cache — a combination agent workloads increasingly exploit. Licensing drama largely ended with Redis 8 returning to an open-source license, as of this writing.

Verdict: the speed play, and the semantic-caching angle is genuinely underrated.

Skip if: you want your durable system of record for embeddings — that's not the job Redis is best at.

Memory layers: the directory

Memory layer verdicts

mem0

mem0 extracts, consolidates, and retrieves user memories behind a simple add/search API, works with any framework, and has become the most-adopted open-source memory layer by a wide margin.

Verdict: the fastest route to "my agent remembers users"; keep an eye on what it decides to store.

Skip if: you need strict auditability of every stored fact — automatic extraction requires review discipline.

Zep

Zep models memory as a temporal knowledge graph via its open-source Graphiti engine: facts carry validity intervals, so the agent knows what was true and when — the failure mode mem0-style flat memories handle worst.

Verdict: the most sophisticated answer to changing facts; more machinery than simple personalization needs.

Skip if: your memory needs are "remember preferences" — a knowledge graph is overkill for that.

Letta

Letta, from the MemGPT researchers, inverts the pattern: instead of bolting memory onto your agent, the agent runs inside Letta's server and edits its own memory blocks as a first-class behavior.

Verdict: the most principled take on agent memory; adopting it means adopting its runtime.

Skip if: you just want a memory bolt-on for an existing LangGraph or SDK stack.

LangMem

LangMem is LangChain's memory SDK — semantic, episodic, and procedural memory patterns that plug naturally into LangGraph checkpointed state.

Verdict: the low-friction choice inside the LangGraph ecosystem; little reason to use it outside.

Skip if: you are not on LangGraph — mem0 or Zep serve framework-agnostic stacks better.

Honest advice before you buy

Now the part vendors skip. Retrieval quality is decided mostly above the database — chunking, hybrid search, and a reranker move accuracy far more than switching engines, so fix those before migrating anything. Giant context windows didn't kill retrieval, either: filling a million-token window on every call is slow, expensive, and empirically less precise than good retrieval over large corpora. What long context did kill is aggressive chunking of medium-sized documents. And memory layers are young. Log every write, sample them weekly, and build a delete path early — wrong memories compound in ways wrong retrievals don't, and as of this writing no layer fully solves contradiction handling for you.

Related reading

Your framework choice shapes your memory integration — see the agent frameworks directory — and memory failures are exactly what evals and observability catch. For the whole picture, start at how to choose your agent stack.

This directory is re-checked monthly. Get the next update by email — pricing moves, new engines, one email.

Keep reading