Systems on Paper Architecture Notebook · Entry 330

AI Infrastructure

One memory layer, every AI tool: how a shared second brain works

Projects like second-brain-cloudflare let you store a fact once and recall it from Claude, ChatGPT, Cursor — any MCP client — using a self-hosted store on a free tier. Here's the architecture that makes that possible, in 3D.

The Hub-and-Spoke Memory Model

Drag to rotate · buttons fire real events
Ready. The indigo core is your memory store; satellites are MCP clients.

The problem it solves

Every AI tool keeps its own memory silo. Tell Claude your stack is "Next.js + Postgres," and ChatGPT still doesn't know. Cursor forgets your naming conventions the moment a session ends. You end up re-explaining yourself to each tool — the context tax.

The fix: externalize memory

Move memory out of the tools and into one store they all share. Each assistant becomes a stateless client; the state lives in a database you control.

Why MCP is the glue

The Model Context Protocol gives every compatible client the same way to call external tools. Your memory server exposes two capabilities — roughly store(text) and search(query) — and any MCP client (Claude, Cursor, and a growing list) can call them without custom integrations per app.

How recall actually works

Embeddings, not keywords

When you store "I prefer Tailwind over plain CSS," the server converts the sentence into an embedding — a vector of numbers capturing meaning. Later, the query "what styling does the user like?" is embedded too, and the store returns memories whose vectors sit closest, even though no words match exactly.

Why Cloudflare's free tier fits

A personal memory layer needs three cheap parts: an HTTP endpoint (Workers), a place for vectors (Vectorize), and a place for raw text (D1 or KV). Traffic is tiny — a few requests per conversation — so a personal deployment typically costs nothing.

Self-hosted = yours

Because you deploy it, your preferences, notes, and project facts never live inside any one vendor's product. Switch assistants next year; the brain comes with you.

Worked example — one fact, three tools

  1. Store once (from Claude): you say "Remember: our API rate limit is 120 req/min per key." Claude calls the MCP tool store. The server embeds the sentence and writes vector + text.
  2. Recall in Cursor: while coding a retry loop you ask "what's our rate limit?" Cursor calls search("rate limit"). Cosine similarity ranks the stored memory ~0.89 against the query — top hit, returned in milliseconds.
  3. Recall in ChatGPT: drafting docs, you ask the same thing. Same server, same answer. Zero re-explaining.
// The whole contract, conceptually
POST /mcp store { text: "API rate limit is 120 req/min" }
POST /mcp search { query: "rate limit", top_k: 3 }
// → [{ text: "...120 req/min...", score: 0.89 }]

The similarity score is the dot product of two unit vectors: 1.0 means identical meaning, ~0 means unrelated. Most memory layers return anything above roughly 0.7 and let the model judge relevance from there.

Enjoy this tool? Build your own with Super