you start a new chat session with your AI coding assistant (whether that’s Cursor, Claude Code, Windsurf, or Cortex Code), you’re essentially starting from zero.
The AI coding assistant doesn’t know that your team uses Streamlit for building web apps. It also doesn’t know that you prefer Material icons over emojis. And it also doesn’t know about that port conflict that made you switch from 8501 to 8505 three months ago.
So you repeat yourself. Session after session.
The tools are powerful, but they are also forgetful. And until you address this memory gap, you’re the human-in-the-loop who is manually managing state that could otherwise be automated.
The Stateless Reality of Large language models (LLMs)
LLMs don’t remember you. Each conversation is a blank slate, by architecture and not by accident.
Your conversation lives in a context window with a hard token limit. Once you close the chat, all traces of the conversation is gone. That’s by design for privacy reasons, but it’s a friction for anyone who needs continuity.
Let’s now take a look at the technical differences between short-term and long-term memory:
- Short-term memory: What the AI remembers within a single session. This lives in the context window and includes your current conversation, any open files, and recent actions. When you close the chat, it’s all gone.
- Long-term memory: What persists across sessions. This is what rules files, memory services, and external integrations provide. It’s knowledge that survives beyond a single conversation.
Without long-term memory, you become the memory layer, copy-paste context, assemble the context, re-explain conventions, answer the same clarifying questions that you answered yesterday and the day before that.
This obviously doesn’t scale.
The Compounding Cost of Repetition
Let’s consider the compounding cost of a lack of persistent memory. But before doing so, we’re going to take a look at what this looks like in practice:
Without persistent context:
You: Build me a dashboard for this dataAI: Here’s a React dashboard with Chart.js…
You: No, I use StreamlitAI: Here’s a Streamlit app with Plotly…
You: I prefer Altair for chartsAI: Here’s the Altair version…
You: Can you use wide layout?AI: [finally produces something usable after 4 corrections]
With persistent context (rules file):
You: Build me a dashboard for this data
AI: [reads your rules file, knows your tech stack and preferences]
Here’s a Streamlit dashboard with wide layout and Altair charts…
As you can see from both examples, same requests but dramatically different experiences. The AI with context produces usable code on the first try because it already knows your preferences.
The quality of AI-generated code is directly proportional to the quality of context that it receives. Without memory, every session starts cold. With memory, your assistant builds on top of what it already knows. The difference compounds over time.
Context Engineering as a Missing Layer
This brings us to what practitioners are calling context engineering, which is the systematic assembly of information that an AI needs to accomplish tasks reliably.
Think of it like onboarding a new team member. You don’t just assign a task and hope for the best. In strike contrast, you would provide your colleague with all of the necessary background on the project, relevant history, access to necessary tools, and clear guidelines. Memory systems do the same for AI coding assistants.
While prompt engineering focuses on asking better questions, context engineering ensures that AI has everything that it needs to give the right answer.
The truth is, there’s no single solution here. But there is a spectrum of possible for tackling this, which can be categorized into four levels: from simple to sophisticated, from manual to automatic.
Level 1: Project Rules Files
The simplest and most reliable approach: a markdown file at the root of your projects that the AI coding assistant can read automatically.
| Tool | Configuration |
| Cursor | .cursor/rules/ or AGENTS.md |
| Claude Code | CLAUDE.md |
| Windsurf | .windsurf/rules/ |
| Cortex Code | AGENTS.md |
This is explicit memory. You write down what matters in Markdown text:
– Python 3.12+ with Streamlit
– Snowflake for data warehouse
– Pandas for data wrangling
– Built-in Streamlit charts or Altair for visualization
# Conventions
– Use Material icons (`:material/icon_name:`) instead of emojis
– Wide layout by default with sidebar for controls
– @st.cache_data for data, @st.cache_resource for connections
– st.spinner() for long operations, st.error() for user-facing errors
# Commands
– Run: streamlit run app.py –server.port 8505
– Test: pytest tests/ -v
– Lint: ruff check .
Your AI coding assistant reads this at the start of every session. No repetition required.
The advantage here is version control. These files travel with your codebase. When a new team member clones the repo, the AI coding assistant immediately knows how things are to be done.
Level 2: Global Rules
Project rules solve for project-specific conventions. But what about your conventions (the ones that follow you across every project)?
Most AI coding tools support global configuration:
– Cursor: Settings → Cursor Settings → Rules → New → User Rule
– Claude Code: ~/.claude/CLAUDE.md and ~/.claude/rules/*.md for modular global rules
– Windsurf: global_rules.md via Settings
– Cortex Code: Currently supports only project-level AGENTS.md files, not global rules
Global rules should be conceptual, not technical. They encode how you think and communicate, not which framework you prefer. Here’s an example:
– Brief responses with one-liner explanations
– Casual, friendly tone
– Present 2-3 options when requirements are unclear
# Code Output
– Complete, runnable code with all imports
– Always include file paths
– No inline comments unless essential
# Coding Philosophy
– Readability over brevity
– Simple first, optimize later
– Convention over innovation
Notice what’s not here: no mention of Streamlit, Python, or any specific technology. These preferences apply whether you’re writing a data pipeline, a web app, or a CLI tool. Tech-specific conventions belong in project rules while communication style and coding preferences belong in global rules.
A Note on Emerging Standards
You may encounter skills packaged as SKILL.md files. The Agent Skills format is an emerging open standard with growing tool support. Unlike rules, skills are portable across projects and agents. They tell the AI how to do specific tasks rather than what conventions to follow.
The distinction matters because rules files (AGENTS.md, CLAUDE.md, etc.) configure behavior, while skills (SKILL.md) encode procedures.
Level 3: Implicit Memory Systems
What if you didn’t have to write anything down? What if the system just watched?
This is the promise of tools like Pieces. It runs at the OS level, capturing what you work on: code snippets, browser tabs, file activity, and screen context. It links everything together with temporal context. Nine months later, you can ask “what was that st.navigation() setup I used for the multi-page dashboard?” and it finds it.
Some tools blur the line between explicit and implicit. Claude Code’s auto memory (~/.claude/projects/<project-folder>/memory/) automatically saves project patterns, debugging insights, and preferences as you work. You don’t write these notes; Claude does.
This represents a philosophical shift. Rules files are prescriptive, meaning you decide upfront what’s worth remembering. Implicit memory systems are descriptive, capturing everything and letting you query later.
| Tool | Type | Description |
| Claude Code auto memory | Auto-generated | Automatic notes per project |
| Pieces | OS-level, local-first | Captures workflow across IDE, browser, terminal |
| ChatGPT Memory | Cloud | Built-in, chat-centric |
Model Context Protocol (MCP)
Some implicit memory tools like Pieces expose their data via MCP (Model Context Protocol), an open standard that lets AI coding assistants connect to external data sources and tools.
Instead of each AI tool building custom integrations, MCP provides a common interface. When a memory tool exposes context via MCP, any MCP-compatible assistant (Claude Code, Cursor, and others) can access it. Your Cursor session can pull context from your browser activity last week. The boundaries between tools start to dissolve.
Level 4: Custom Memory Infrastructure
For teams with specific needs, you can build your own memory layer. But this is where we need to be realistic about complexity versus benefit.
Services like Mem0 provide memory APIs that are purpose-built for LLM applications. They handle the hard parts: extracting memories from conversations, deduplication, contradiction resolution, and temporal context.
For more control, vector databases like Pinecone or Weaviate store embeddings (i.e. as numerical representations of text that capture semantic meaning) of your codebase, documentation, and past conversations. But these are low-level infrastructure. You build the retrieval pipeline yourself: chunking text, generating embeddings, running similarity searches, and injecting relevant context into prompts. This pattern is called Retrieval-Augmented Generation (RAG).
| Tool | Type | MCP Support | Description |
| Mem0 | Memory as a Service | Yes | Memory layer for custom apps |
| Supermemory | Memory as a Service | Yes | Universal memory API |
| Zep | Memory as a Service | Yes | Temporal knowledge graphs |
| Pinecone | Vector database | Yes | Managed cloud vector search |
| Weaviate | Vector database | Yes | Open-source vector search |
Most developers won’t need this, but teams building internal tooling will. Persisting institutional knowledge in a format AI can query is a real competitive advantage.
Building Your Memory Layer
If you’re not sure where to begin, start here:
1. Create a rules file (CLAUDE.md, AGENTS.md, or .cursor/rules/ depending on your tool) in your project’s root folder
2. Add your stack, conventions, and common commands
3. Start a new session and observe the difference
That’s it. The goal isn’t perfect memory. It’s reducing friction enough that AI assistance actually accelerates your workflow.
A few principles to keep in mind:
- Start with Level 1. A single project rules file delivers immediate value. Don’t over-engineer until friction justifies complexity.
- Add Level 2 when you see patterns. Once you notice preferences repeating across projects, move them to global rules.
- Keep global rules conceptual. Communication style and code quality preferences belong in global rules. Tech-specific conventions belong in project rules.
- Version control your rules files. They travel with your codebase. When someone clones the repo, the AI coding assistant immediately knows how things work.
- Review and prune regularly. Outdated rules cause more confusion more than they help. Update them regularly like you update code.
- Let the AI suggest updates. After a productive session, ask your AI coding assistant to summarize what it had learned.
As for higher levels: implicit memory (Level 3) is powerful but tool-specific and still maturing. Custom infrastructure (Level 4) offers maximum control but requires significant engineering investment. Most teams don’t need it.
Where This Is Going
Memory is becoming a first-class feature of AI development tools, not an afterthought.
MCP is gaining adoption. Implicit memory tools are maturing. Every major AI coding assistant is adding persistent context. The LLMs themselves will likely remain stateless. That’s a feature, not a bug. But the tools wrapping them don’t have to be. The stateless chat window is a temporary artifact of early tooling, not a permanent constraint.
OpenClaw takes this to its logical endpoint. Its agents maintain writable memory files (SOUL.md, MEMORY.md, USER.md) that define personality, long-term knowledge, and user preferences. The agent reads these at startup and can modify them as it learns. It’s context engineering taken to the extreme: memory that evolves autonomously. Whether that’s exciting or terrifying depends on your appetite for autonomy.
The challenge for practitioners isn’t choosing the perfect memory system. It’s recognizing that context is a resource. And like any resource, it can be managed intentionally.
Every time you repeat yourself to an AI coding assistant, you’re paying a tax. Every time you document a convention once and never explain it again, you’re investing in compounding returns. These gains compound over time, but only if the infrastructure exists to support it.
Memory persistency are coming to AI. As I am writing this article, Anthropic had literally rolled out support for memory feature in Claude.
Disclosure: I work at Snowflake Inc., the company behind Cortex Code. All other tools and services mentioned in this article are independent, and I have no affiliation with or sponsorship from them. The opinions expressed here are my own and do not represent Snowflake’s official position.