Want a product team that really knows AI?
Yesterday, you and Claude Code built half a marketing site for your real estate business. By evening, it knew your taste in UI and your sense of humor in copy. Working with AI was a piece of cake.
This morning, you open a new session and describe the scroll animations and writing for the next few pages. “No problem,” you think. “Claude Code is officially my best design partner.” Then this best partner hands you a page straight out of the AI slop playbook, with copy that sounds like every other real estate site online. Yesterday’s rapport is gone. Today, you have to start from scratch.
This kind of letdown happens to anyone working with a coding agent, because models, on their own, don’t remember anything between sessions. You could just stay in one endless conversation to avoid that, but then you run into a different set of problems. The catch is that every time you send a message, the model rereads the entire conversation from the top. So the longer it runs, the slower and pricier each reply gets.
Fortunately, there’s a reliable workaround, one that developers have been using for decades. It’s called documentation. In this article, I’ll break down a lightweight but essential documentation structure that gives your coding agent enough working memory, so you don’t have to start from scratch every time, and the rapport you build carries over from one session to the next.
Documentation: Old Practice, New Purpose
In tech, documentation is the written story of a software project: what it is, how it’s built, and why it ended up that way. For most of software history, its readers were humans. When an engineer tried to figure out what they’d just walked into, the first thing they’d do was read the docs. This works for AI too. Every new AI session you open is like a new teammate assigned to your project with zero background knowledge. Keeping a separate doc structure is a reliable way to onboard them quickly, so your collaboration runs more smoothly.
Before we dive into the doc structure, let’s look at what makes docs for AI special.
Doc format. Stick to plain text, ideally Markdown (.md files). A diagram is the fastest way for a human to take something in, but that’s not the case for an agent. Even though AI can read images, it has to interpret them first, and that usually costs more tokens than reading the same information as text. That's why plain text is the cheapest for an agent to read and the easiest for it to edit. If you need to include a diagram, don’t rely on the image alone. Describe it in detail in writing, so both you and the agent can work with it.
Doc content. You’ll have to include collaboration guidelines in your docs. This is the most distinctive part of docs written for AI. So what are collaboration guidelines? They’re the rules you set for how your agent works with you. For example, you might write: “Never write passwords, API keys, or other sensitive information anywhere in the repository.” This helps even when you don’t realize something is sensitive. You might not know that a certain key needs to be kept private, but with this rule in place, the agent will typically flag it and ask you to store it securely yourself.
Doc Upkeep. Because an agent tends to trust whatever is written in the docs, keeping them current is essential. A human reading an outdated doc will often notice when something feels off and ask around. An agent usually takes the doc at its word and builds on it. That’s what makes the wrap-up habit more important than ever. So whenever you hit a milestone, or simply wrap up for the day, update the docs with where things stand. That way, the next session starts from the right place.
Effort on the doc. Here’s the good news: writing and updating docs is now much cheaper, because the agent can do most of the work. With a simple routine in place, keeping thorough guidelines and up-to-date docs is no longer the chore it used to be.
Next, I’ll share the doc structure I usually use, and the routine I follow to keep it up to date. It works especially well for projects that are just getting started.
A Small Front Door to A Big House
Figure 1. The basic doc structure
A small front door to a big house: AGENTS.md stays short and points to the docs that hold the details.
Because the agent reads this file at the start of every session, keep it short and include only the bare minimum. Everything in it takes up space in the agent’s context, every single time. Collaboration guidelines, a project overview, a doc map, and a wrap-up routine are usually enough. The details belong in the docs the doc map points to. Keep the front door clean and compact to save tokens, and let the agent open the other rooms only when it needs them.
AGENTS.md is the file your coding agent reads automatically at the start of every session. The name matters: it’s all caps, and “agents” is plural. Call it START-HERE.md or README.md, and the agent won’t pick it up on its own (though you can still ask it to read the file at the start of a session).
The doc map is the heart of the system. For a project that’s just getting started, it points to four files, each answering a different question about your project. As the project grows, the map can grow with it. Here’s what goes in each one.
spec.md: what are we building? “Spec” is short for specification. This file describes the product itself: its features, how the paywall works, which metrics you’re tracking, the UI style you’ve chosen, your brand voice, and the roadmap ahead. It’s a lot like the PRD (product requirements document) a product manager would typically write. If someone asked “what is this product supposed to be?”, the answer should be here.
architecture.md: how is it built? This file describes the technical side of your product: the tech stack you’ve chosen (frameworks, UI library, database, hosting), how the main pieces fit together, how the project’s folders are organized, and which outside services you rely on, such as payments, sign-in, or analytics. It can also cover how to run and deploy the project. If spec.md is the blueprint of what the product should be, architecture.md is the map of how it’s actually put together.
decisions.md: why is it built this way? This file is a running log of the important choices you’ve made, and the reasoning behind them. Each entry records what you decided, why, and what alternatives you considered and turned down. For example, one of my projects has an entry like this in its decisions.md: “Search uses embeddings to shortlist results first, then passes only that shortlist to the AI, instead of sending everything to the AI directly. This keeps token costs down.” It’s a typical decision entry: what we chose, what we ruled out, and why.
progress.md: where do things stand? This file is a snapshot of the project right now: what’s done, what’s in progress, what’s next, and any known bugs or loose ends. Unlike decisions.md, it gets updated constantly, ideally at the end of every session, so it always reflects the current state rather than the full history. When a new session starts, this is the file that tells the agent where to pick up.
The routine. At the start of a project, ask your agent to set up AGENTS.md and the four docs, then read through what it writes. Before you close your session each time, ask: “What should we update in the docs before we stop?” Do this a few times, and it becomes a habit that saves you from starting over every time.
A Few Rules of Thumb
Start small. A new project doesn’t need an elaborate doc system. Begin with the basics, and add more structure only as the project grows.
Write it down in one place. If the same information lives in several files, sooner or later one of them falls out of date, and your agent won’t know which to trust. Keep each piece of information in a single place, and link to it from anywhere else that needs it.
Don’t chase perfect docs. As a project grows and more people join, the docs will get a little messy. That’s fine. Let the agent do the tidying as part of your wrap-up, and if the docs and the code ever disagree, treat the code as the truth and fix the docs.
AI has changed how fast we can build, but not what it takes to build something that lasts. A demo can live inside one good conversation. But a product needs to answer what it is, how it’s built, why it’s built that way, and where it stands. Giving your agent a memory through a simple doc structure is the first step in turning your demo into a real product.
Happy Building!
