September 24th, 2026
0 reactions

Documentation Is the New Source Code

Liza Shakury

Why writing things down is the highest-leverage thing you can do when building with AI and HVE tools

For most of my career, “good documentation” was a virtue — somewhere between writing tests and updating your status report. Nice to have. Often skipped. Quietly resented in code review.

That calculation has flipped. When your day-to-day involves Copilot, Claude, Cursor, agentic CLIs, MCP servers, and a dozen “high-velocity engineering” (HVE) tools that read your repo before they write a single line, the documentation you write is the system prompt for every future change.

Skip it and the AI guesses. Write it and the AI executes.

This post explains why we invested early in documentation — scoped instruction files, skills, and technical guides — and the concrete payoff we’ve seen.

1. The shift: docs are no longer just for humans

Last month I asked an agent to add a new endpoint to our service. With no conventions written down, it invented its own error-handling pattern, its own logger, and a folder structure we’d abandoned some time ago. Same prompt, same agent, a week later — after we’d written the conventions into copilot-instructions.md: one PR, green CI, merged the same afternoon. Nothing about the model changed. Only the input did.

That gap is what this section is about. Four things shifted at roughly the same time to make this flip possible:

  1. Coding assistants got context-aware. Tools like GitHub Copilot, Claude Code (Anthropic) and Cursor (Cursor AI) now automatically read README.md, AGENTS.md, .github/copilot-instructions.md, *.instructions.md, SKILL.md, MCP server descriptions, ADRs — anything they can find. The model’s behavior is a pure function of what lands in its context window, and docs are now the highest-signal thing you can put there. Think of the context window as the new compiler input: garbage in, plausible-looking garbage out.
  2. Agents started running multi-step tasks. A single prompt now triggers dozens — sometimes a few hundred — file reads, edits and tool calls in one turn. At that scale the model has to infer conventions at every step, and every wrong inference compounds into the next.
  3. Teams adopted “high-velocity” workflows. Short-lived branches, daily PRs, agent-authored changes, async review. Hallway knowledge no longer has time to spread by osmosis — by the time you’d have explained the convention to a teammate, three agent-authored PRs have already shipped without it.
  4. Models became interchangeable. You’re going to swap your assistant every six months — GPT (OpenAI) to Claude (Anthropic) to Gemini (Google) to whatever ships next. Prompt-engineering tricks tuned to one model don’t transfer. A good copilot-instructions.md does. Docs are the only artifact in your repo that survives the model swap.

The combined effect: whatever isn’t written down gets re-invented, badly, on every task. The model picks the most popular pattern from its training data, not the one your team actually uses.

Documentation used to compete with code for engineering time. Now it compiles into it — every doc you write becomes input to every change the AI makes.

2. What “documentation as system prompt” looks like in practice

Here’s the layered setup we use in our repo. Every layer is read by humans and by AI tools — often automatically.

The guiding principle is simple: things the model needs on every task go in the system prompt; things it occasionally needs live one grep away. Cheap-to-load context is always-on; expensive-to-load context is on-demand.

# Layer File(s) When the model reads it
1 Repo-wide instructions .github/copilot-instructions.md Every task
2 Scoped instructions .github/instructions/*.instructions.md Edits matching the applyTo glob
3 Skills (workflow recipes) .github/skills/<name>/SKILL.md When the workflow is invoked
4 ADRs ADRs/NNN-*.md When linked or searched
5 System, feature & how-to docs docs/** (architecture, features, guides, runbooks) Deep lookups, on demand
6 Code-adjacent docs docstrings, schema comments Every time the file is opened

Layer 1 — Repo-wide AI instructions

.github/copilot-instructions.md is the first file most AI tools open. Ours is short, opinionated, and information-dense:

  • Architecture overview (request flow in one line, key directories with one sentence each)
  • Configuration system (where YAML lives, how it’s loaded)
  • Environment variables (table with required / default / description)
  • Developer workflows (exact commands to run backend, frontend, tests)
  • CI gates (so the model doesn’t propose changes that will fail lint)
  • Git conventions (branch naming, commit prefixes, PR template)
  • Cross-cutting conventions (error handling, parallelism, logging, telemetry)
  • Pointers to ADRs and docs/

Notice what isn’t in it: tutorials, motivation, history. Those belong in deeper docs. The top-level file is for rules the model must follow on every task.

A redacted excerpt to make it concrete — the top of ours opens with the request flow and an env-var table, in that order:

## Request flow
Controller → RequestContext → SingleProcessPipeline.run /
Orchestrator.run() → result

## Environment variables
| Name           | Required | Default | Description              |
|----------------|----------|---------|--------------------------|
| OPENAI_API_KEY | yes      | —       | Auth for the model calls |
| DB_ENDPOINT    | yes      | —       | Cache + state storage    |
| LOG_LEVEL      | no       | INFO    | Standard Python levels   |

That’s the first thing the agent sees on every task. Everything else in the file is rules and pointers.

Layer 2 — Scoped instruction files

For larger codebases, a single file isn’t enough. We use VS Code’s applyTo glob pattern to scope instructions:

.github/instructions/backend.instructions.md   →  applyTo: backend/**
.github/instructions/frontend.instructions.md  →  applyTo: frontend/**

The agent only sees the backend rules when editing Python, only sees the frontend rules when editing TypeScript. This keeps each context window lean and prevents cross-contamination (no React advice leaking into a FastAPI controller).

The trade-off is real: scoped files only fire when the agent edits a matching path. Anything truly cross-cutting — logging, error handling, telemetry, secret handling — still belongs in the repo-wide file. Over-scope and you’ll watch the model cheerfully violate your logging conventions on every cross-cutting change.

Layer 3 — Skills (workflow recipes)

.github/skills/<name>/SKILL.md files are the equivalent of “runbooks for the model.” Each one says: here is a task you might be asked to do; here’s the exact procedure. We have an internal code-review-excellence skill, and we lean on a handful of off-the-shelf ones (create-pull-request, address-pr-comments, summarize-github-issue-pr-notification).

At their simplest they look like this:

---
name: create-pull-request
description: Open a PR using our template, labels, and reviewer rules.
---

1. Ensure the branch name matches `feat/`, `fix/`, or `chore/` prefix.
2. Fill the PR body using `.github/PULL_REQUEST_TEMPLATE.md`.
3. Add the area label (`area:backend`, `area:frontend`, ...).
4. Request review from code owners for the changed paths.

The win: instead of the model improvising “how to open a PR in this repo,” it follows the SKILL.md verbatim — including our PR template, label rules, and reviewer assignment.

Layer 4 — ADRs

ADRs/ folder contains our numbered decision records. Each one captures why a choice was made — how we retrieve context for the model, how we orchestrate agents, how we version regenerated outputs, and so on.

ADRs do something other docs can’t: they prevent the AI (and the next engineer) from undoing a deliberate decision. They work because they are durable, dated, and authoritative — the model treats a numbered decision record as a stronger signal than a comment, a Slack quote, or a half-remembered convention.

Layer 5 — System, feature, and how-to documentation (docs/)

docs/ is the home for everything too big for the system prompt but big enough that the model (or a human) will search for it. We split it into three top-level areas:

docs/
├── system/        ← how the whole thing fits together
│   ├── architecture.md
│   ├── configuration.md
│   ├── cosmos-cache-queries.md
│   ├── observability.md
│   └── onboarding.md
├── features/      ← one folder per shipped capability
│   ├── report-generation/
│   │   ├── overview.md
│   │   ├── pipeline.md
│   │   └── prompts.md
│   ├── table-extraction/
│   │   ├── overview.md
│   │   └── strategies.md
│   └── regenerate-section/
│       ├── overview.md
│       └── versioning.md
└── guides/        ← how to use or implement something
    ├── add-a-new-section-type.md
    ├── write-a-new-extraction-strategy.md
    ├── plug-in-a-new-llm-provider.md
    └── run-the-eval-harness-locally.md

docs/system/ answers questions like “how does configuration load?” or “what does observability look like end-to-end?” — cross-cutting context the model needs when it’s reasoning about anything non-trivial.

docs/features/ answers questions like “how does regenerate-section actually work?” — one folder per shipped feature, each with an overview.md (what it does, why it exists, key invariants) and as many companion files as the feature deserves. When a ticket lands that touches a feature, the agent grep-finds the right folder and pulls in the relevant context instead of guessing from code.

docs/guides/ answers questions like “how do I add a new section type?” or “how do I plug in a new LLM provider?” — task-oriented walkthroughs that take an engineer (or an agent) from intent to working code without re-deriving the design. These pay off twice: humans use them to onboard onto unfamiliar parts of the system, and agents follow them step-by-step when asked to implement the same kind of change.

These docs are too long for the always-on system prompt, but the model can — and does — open them on demand. They also serve their original purpose: onboarding humans, and giving new agent sessions a fast way to get oriented.

Layer 6 — Code-adjacent docs

Docstrings, type hints, config-schema YAML comments. Too small to be a document, too important to lose. They’re cheap, local, always co-located with the thing they describe, and the model reads them every time it opens the file.

Each layer is cheap on its own. Stacked, they give an agent a complete picture of how this codebase wants to be changed — without any single file getting bloated, and without forcing the model to guess.

3. Where the leverage shows up

With those layers in place, the payoff shows up in six places — most of them places we didn’t expect.

a. AI is a force multiplier — of whatever you’ve written down

A well-documented repo means an agent can implement a feature in one shot. A poorly-documented repo means the same agent produces plausible-looking code that violates the conventions, fails CI, and needs several rounds of human correction. The model isn’t worse — the input is.

The shape of the payoff is concrete: convention-related review comments (“we don’t structure errors that way,” “use the shared logger,” “wrong folder”) drop sharply once the conventions live in copilot-instructions.md. The hour you spent writing them down pays back on every PR after.

b. Onboarding collapses — at least the mechanical half

A new engineer (or a new agent session) used to need a buddy, a wiki page, and three meetings. Now they read onboarding.md, point their assistant at the repo, and ship a real PR the same afternoon. The agent acts as the pair programmer who already knows the codebase.

The honest caveat: docs collapse the mechanical part of onboarding — running the stack, understanding the layout, making a first safe change. The squishy part — who owns what, why a decision felt the way it did, where the bodies are buried — still needs a human. Don’t promise more than the docs can deliver.

c. Decisions stop eroding — even across model swaps

Without ADRs, every six months someone re-litigates a settled choice. With ADRs, the conversation is two minutes long: “Did you read ADR 007?”

The AI-specific version is sharper: the model has no memory between sessions. Without an ADR, it will confidently re-propose the rejected pattern — cheerfully asking “have you considered…?” — every single time, and a different model next quarter will do it again, in a different voice. The ADR is the model’s memory. Once it’s there, the agent cites it in its own suggestions and the loop closes.

d. Async velocity becomes safe

We increasingly hand whole tickets to agents on a Friday and merge the PR on a Monday, with no human in the loop on the convention-following parts. That only works when the conventions are written down — when the agent can self-serve branch naming, commit style, label rules and PR layout without asking.

Without the docs, async agent-authored work is fast and wrong. With them, it’s fast and right.

e. Tools amplify good docs across the stack

MCP servers, search subagents, code-review skills — they all read your docs. The clearest example we’ve seen: hooking up a docs-search MCP server meant the agent stopped guessing at architecture questions and started quoting docs/system/architecture.md back at us in its plans. The marginal return on a good doc keeps going up as more tools learn to read it.

f. Docs become the contract between humans and agents

When a convention lives in copilot-instructions.md, it’s no longer hallway knowledge — it’s a published interface. Reviewers check PRs against the doc instead of against memory. Agents follow the same doc across sessions and across models. Disagreements happen at the doc, not at the code, which is a much cheaper place to have them.

This is the deepest payoff and the one that took us longest to notice. Docs aren’t just instructions for the AI; they’re the shared spec that lets humans and agents collaborate without re-negotiating the rules on every change.

None of these payoffs require new tooling. They all require a few hours of writing — done once, paid back forever.

4. A style guide for AI-readable docs

Most style-guide advice was written for humans skimming. These are the patterns that change the model’s behavior — grouped by what they’re trying to do.

Structure: make the shape obvious

  • Lead with the dataflow. A one-line flow at the top — HTTP request chain, build pipeline, event sequence, whatever your equivalent is — orients the model before any rules. Put yours at the top of copilot-instructions.md and never bury it.
  • Tables beat prose. Env vars, config keys, file responsibilities — use tables. Prose hides structure the model has to re-derive; a table lets it answer “what’s the default for LOG_LEVEL?” in one glance instead of parsing three sentences and guessing.
  • Show the exact command. Not “run the backend” — the literal cd backend && uvicorn src.api.main:app --reload --port 8000. The model will paste what you wrote, so make sure what you wrote can be pasted. (This cuts both ways: if the command is stale, the model will confidently run the stale one.)
  • Date your long-lived docs. A small Last reviewed: 2026-05 at the top of system and feature docs gives both humans and the model a signal for likely-stale content. Cheap to add, huge payoff for trust over time.

Discipline: keep the prompt lean and honest

  • Link, don’t duplicate. Top-level instructions point to ADRs and guides. The model follows links when it needs depth, but it doesn’t drown in the system prompt.
  • Be the source of truth, not a pointer outward. “See the team wiki for details” is a dead end — the agent can’t follow links to private wikis, internal SharePoint sites, or Slack threads. If the model needs the information, the information has to live in the repo.
  • Encode the negatives. “Don’t add docstrings to code you didn’t change.” “Don’t create markdown files unless requested.” Explicit prohibitions matter more for AI than for humans because the model’s default behavior is helpful expansion.
  • Keep it current or delete it. Stale docs are worse than none — they actively mislead the model. We treat doc drift as a CI bug: link-check on every PR, a grep gate that fails if known-renamed symbols still appear in docs/**, and a warning when an env-var name in copilot-instructions.md no longer exists in the codebase.

Diagnose: write for the failure modes you’ve actually seen

  • Write for the model’s mistakes. If you’ve seen the AI hallucinate a path, mention the real one. If it keeps proposing a pattern you rejected, write the ADR. If it keeps inventing a fourth section type, list the three valid ones explicitly. Every recurring AI mistake is a missing line of documentation.

None of these rules are clever. They just respect the fact that the model is a literal, fast, forgetful reader — and so we write docs accordingly.

5. Unit tests are documentation too — and they matter more than ever

Everything above applies doubly to tests. A test suite is the only form of documentation an AI agent can execute — and in an AI-assisted workflow, that promotes tests from quality net to control surface.

Comments lie, READMEs go stale, ADRs describe intent — but a green test is empirical proof that the code behaves a certain way right now.

a. Coverage shapes what the agent dares to change

This is the pattern that surprised us most. In well-tested modules, the agent makes bold, correct changes — it knows it’ll find out immediately if it broke something. In poorly-tested modules, it tiptoes: adds redundant guards and duplicates logic instead of refactoring.

Coverage isn’t just about catching bugs. It changes the kind of work the agent is willing to do. A well-tested module gets braver edits; an untested one gets timid ones. If you want better agent-authored code, the cheapest lever is more tests on the modules you want the agent to touch.

b. Tests are the feedback loop that makes async velocity safe

When you hand a task to an agent — “implement this, run the tests, fix what breaks” — the test suite is the spec. The model iterates against it the same way you would: change code, run tests, read failures, adjust.

Without tests, the regression cost lands on you, after the agent is done. With tests, it lands on the agent, mid-task, where it’s cheap to fix.

That same loop is what makes async velocity safe. Human review can’t keep up when agents merge multiple PRs per day; CI is the gate. The path-filtered workflows in .github/workflows/ — lint, type-check, unit tests with coverage — are what stop “ship fast with AI” from becoming “ship bugs fast with AI.”

c. Tests pin behavior the model would otherwise “improve”

LLMs love to refactor. Left to their own devices, they rename a helper, “simplify” an edge-case branch, or reorder arguments to match the most popular pattern from their training data.

A unit test that locks the contract — input X yields output Y — turns those silent regressions into red CI checks. Either the model preserves the behavior, or it explicitly asks to change the test.

That second one is the conversation you wanted to have — instead of the silent regression you wouldn’t have caught.

d. Tests document the unhappy paths

Docs and ADRs naturally capture happy-path intent. Tests capture what happens when the input is empty, malformed or huge; when the upstream service times out; when two parallel callers race; when the config flag is missing.

These are exactly the scenarios where agents hallucinate plausible-looking behavior. Concrete example from our repo: we added one test for extract_table(html='') returning [] after watching three different agents “fix” the empty-input case by raising an exception. The test cost five minutes; it’s stopped that mistake at least a dozen times since.

A failing test on a malformed-input case is worth a thousand words of “be careful with edge cases” in copilot-instructions.md.

e. Tests as executable examples

A test like test_ensure_agent_updates_existing_when_name_matches teaches the model the contract of ensure_agent better than any docstring. When the agent needs to call that function in new code, it can grep the tests for usage, copy the pattern, and know it works.

We’ve leaned into this: the test file is often the fastest way for a new agent session (or a new engineer) to learn what a module promises.

Practical rules we follow

  • Test the public surface, not the implementation. Agents refactor internals constantly; tests on internals create churn.
  • One assertion concept per test, with a descriptive name. The name is what the model reads first when scanning failures.
  • Prefer fast unit tests over slow integration tests. Agents will run the suite many times per task; latency compounds.
  • Test the error path explicitly. A raises(...) assertion on bad input is non-negotiable for any function the model might call.

And one rule important enough to stand on its own:

Treat a failing test as a sacred signal. Don’t let agents “fix” a test by weakening it. Encode that in instructions verbatim: “If a test fails, fix the code, not the assertion, unless the requirements changed.”

This is the most AI-specific rule in the whole post. The default behavior of a helpful, unconstrained agent is to make the red thing green by any means available — including deleting the assertion that caught the bug.

If documentation is the system prompt, tests are the runtime verification that the system prompt was followed. You need both.

6. Things we got wrong (and learned from)

  • We over-wrote at first. Early copilot-instructions.md was 600 lines. The model started losing focus. We trimmed it ruthlessly and pushed depth into linked guides.
  • We under-scoped. A single repo-wide instructions file made the model apply Python conventions in TypeScript files. Splitting via applyTo fixed it.
  • We forgot to mark prohibitions. Until we wrote “do not create markdown summaries of changes,” the agent kept generating CHANGES.md files we had to delete.
  • We let ADRs go stale. A few ADRs described a “proposed” state that had since shipped. The model believed the proposed pattern was still the future. Lesson: review ADR statuses quarterly.

7. A practical starting checklist

If you’re working in an HVE / AI-assisted setup and you want to capture some of this leverage, in order:

  1. Write a one-page .github/copilot-instructions.md (or AGENTS.md) today. See §2 and §4 for the shape — the short version is: architecture in one line, key directories one line each, env vars in a table, exact dev commands, cross-cutting conventions, and explicit prohibitions.
  2. Add a PR template that the model will fill in by default.
  3. Start an ADR folder. Write ADR 001 about something already decided — a settled choice is the cheapest first ADR because there’s nothing to debate, only to record.
  4. When you find yourself correcting the model the same way twice, write it down. Either in instructions (rule) or in an ADR (decision).
  5. Scope instructions with applyTo the first time you catch the model applying Python conventions in TypeScript files (or vice versa).
  6. Create docs/ with system/, features/, and guides/ subfolders. Seed each with one real page — even a stub beats an empty folder, because it tells the model where new content belongs.
  7. Add a SKILL for any multi-step workflow you’d otherwise re-explain (creating PRs, addressing review comments, releasing).
  8. Treat your test suite as documentation. Add a unit test for every bug you fix and every contract you want the agent to preserve. Wire it into CI so agent-authored PRs cannot merge red.
  9. Treat docs the same as code: PRs, review, owners.

You don’t need to do all nine this week. Item 1 alone will change how every future PR feels.

TL;DR

Documentation is no longer a chore that competes with shipping — it’s the substrate on which shipping now happens. Tests are the executable half: the only documentation an agent can run, and the only safety net fast enough for AI-velocity merges. The repos that ship cleanly with AI are the ones whose conventions, decisions, workflows and contracts are written down in ways that both humans and models can read — and verify.

Write the README. Write the ADR. Write the SKILL.md. Write the test. Each one is a contract — with your team, with your future self, and with whichever agent reads the repo six months from now.

Attribution

The image used in this post was created using ChatGPT GPT-5 (an OpenAI model and Microsoft partner).

Category

Topics

Author

Read next