{"id":471,"date":"2026-08-21T13:31:34","date_gmt":"2026-08-21T13:31:34","guid":{"rendered":"https:\/\/marcinzygmunt.pl\/blog\/?p=471"},"modified":"2026-08-21T13:31:34","modified_gmt":"2026-08-21T13:31:34","slug":"context-is-money-how-to-work-with-a-large-knowledge-base-without-burning-tokens","status":"publish","type":"post","link":"https:\/\/marcinzygmunt.pl\/blog\/context-is-money-how-to-work-with-a-large-knowledge-base-without-burning-tokens\/","title":{"rendered":"Context is Money. How to Work with a Large Knowledge Base Without Burning Tokens"},"content":{"rendered":"\n

A model is only as good as the context you give it. Working with AI agents (Claude Code, Copilot, Cursor), you discover this quickly and painfully: context is a real cost. Literally, in tokens, and indirectly, in the quality of answers. An overstuffed context means hallucinations, worse responses, higher bills, and the frustration of constantly correcting your agent.<\/p>\n\n\n\n

The fix turns out to be a few simple habits and some structural discipline that will spare you the mistakes I made when I started working with AI. I write about code repositories here because that’s my main workshop, but the same rules apply to any body of knowledge an agent works with: product docs, a notes vault, team procedures, content.<\/p>\n\n\n\n

1. First things first: context on demand, not “just in case”<\/h2>\n\n\n\n

The biggest mistake is dumping everything into context “because it might be useful.” The model gets lost in the noise (the so-called lost in the middle<\/em> problem), and you pay for tokens that add nothing.<\/p>\n\n\n\n

Instead: structure that lets the model find what it needs on its own<\/strong>. An AI with a good entry point and a clear map of the repo will read the rest when it has to.<\/p>\n\n\n\n

2. How to AI: CLAUDE.md \/ AGENTS.md \/ a README for the agent<\/h2>\n\n\n\n

Every repo should have a short AI file. It’s not documentation for people, it’s a condensed briefing<\/strong>:<\/p>\n\n\n\n

    \n
  • what the project is (2\u20133 sentences),<\/li>\n\n\n
  • stack and versions (Java 21, Spring Boot 3.x, Gradle),<\/li>\n\n\n
  • how to build, test, and run it (exact commands),<\/li>\n\n\n
  • conventions: code style, package structure, the patterns you use,<\/li>\n\n\n
  • what NOT to do (e.g., “don’t touch the legacy-billing module without a ticket”).<\/li>\n<\/ul>\n\n\n\n

    The key part: this file must be short<\/strong>, under ~150 lines. It’s loaded into every session, so every line is a fixed token cost. Move details into separate files and link to them; the model will read them when the topic actually comes up.<\/p>\n\n\n\n

    Each tool has its own standard, and the conventions are worth knowing:<\/p>\n\n\n\n

      \n
    • CLAUDE.md<\/code><\/strong>: Claude Code (supports a hierarchy too: a global ~\/.claude\/CLAUDE.md<\/code> + per-repo + per-directory),<\/li>\n\n\n
    • AGENTS.md<\/code><\/strong>: an open standard read by a growing list of tools (Codex, Cursor, Zed, and others),<\/li>\n\n\n
    • .cursor\/rules\/<\/code><\/strong>: Cursor (rules with per-path activation conditions),<\/li>\n\n\n
    • .github\/copilot-instructions.md<\/code><\/strong>: GitHub Copilot,<\/li>\n\n\n
    • llms.txt<\/code><\/strong>: a standard for websites and docs, a model-friendly content map.<\/li>\n<\/ul>\n\n\n\n

      The pragmatic approach: keep one source of truth in AGENTS.md<\/code>, with CLAUDE.md<\/code> as a symlink or a one-line pointer. Zero duplication.<\/p>\n\n\n\n

      3. Layers: think CPU cache<\/h2>\n\n\n\n

      Think of knowledge as a memory hierarchy.<\/p>\n\n\n\n

      L1, always in context:<\/strong> the AI file. Minimal.<\/p>\n\n\n\n

      L2, read on demand:<\/strong> docs\/<\/code> with short, single-topic files: docs\/architecture.md<\/code>, docs\/auth-flow.md<\/code>, docs\/deployment.md<\/code>. One file = one topic = one read. A 2,000-line file “about everything” is an anti-pattern: the model has to ingest all of it to find one paragraph.<\/p>\n\n\n\n

      L3, discovered through search:<\/strong> code, tests, ADRs. Here naming and structure do the work (more on that below).<\/p>\n\n\n\n

      This hierarchy isn’t specific to code. A notes vault or process documentation lays out the same way: a short entry point, an index, topical files read on demand.<\/p>\n\n\n\n

      INDEX.md: a map instead of exploration<\/h3>\n\n\n\n

      Between L1 and L2, an index file<\/strong> works great: docs\/INDEX.md<\/code> (or a section in the AI file). It’s a table of contents for the knowledge in the repo, one line per topic:<\/p>\n\n\n\n

      - docs\/architecture.md - modules, boundaries, data flow\n- docs\/auth-flow.md - OAuth2 + JWT, refresh tokens\n- docs\/adr\/ - architecture decisions (ADR-0001..0014)\n- payments\/README.md - Stripe integration, webhooks<\/code><\/pre>\n\n\n\n
      The effect: instead of groping through the repo blindly (a dozen-plus reads, thousands of tokens), the model does one lookup in the index and one precise read<\/strong>. The same works for large monorepos: an INDEX.md per module describing what lives where. One condition: the index has to stay current. Put it on the PR checklist.<\/p>\n\n\n\n
      4. Code structure that explains itself<\/h2>\n\n\n\n
      AI navigates a repo the way a new team member does: by names. Everything that helps a human onboard helps the model.<\/p>\n\n\n\n
        \n
      • Package-by-feature over package-by-layer.<\/strong> com.company.orders<\/code> says more than order logic scattered across controllers\/<\/code>, services\/<\/code>, repositories\/<\/code>.<\/li>\n\n\n
      • File names match contents.<\/strong> OrderRefundPolicy.java<\/code> turns up with a single grep. Utils2.java<\/code> never will.<\/li>\n\n\n
      • Small files.<\/strong> A 3,000-line class is a context bomb. The model has to load all of it to change one method.<\/li>\n\n\n
      • Tests as documentation.<\/strong> A good test names a business scenario. AI reads tests to understand intent before it touches the code.<\/li>\n<\/ul>\n\n\n\n
        (TIP: DDD projects are naturally easier for AI to read and describe. Bounded contexts and ubiquitous language force a structure that explains itself.)<\/em><\/p>\n\n\n\n
        5. ADRs: decision memory for pennies<\/h2>\n\n\n\n
        Architecture Decision Records are the most underrated form of knowledge for AI. A short file: context \u2192 decision \u2192 consequences. When the model suggests “how about we rewrite this on Kafka?”, ADR-0012 explains why you chose RabbitMQ, and the conversation gets back on track. One page of text saves hours of wandering.<\/p>\n\n\n\n
        6. Trimming context in practice<\/h2>\n\n\n\n
          \n
        • .gitignore<\/code> and ignore lists for AI:<\/strong> generated files, build\/<\/code>, target\/<\/code>: all of it out of indexing range.<\/li>\n\n\n
        • One session = one task.<\/strong> Long, multi-threaded conversations degrade quality. Finished the feature? New session, fresh context.<\/li>\n\n\n
        • Summarize instead of scrolling.<\/strong> When a session balloons, ask for a concise summary of the work so far and start a new session from it.<\/li>\n\n\n
        • Point, don’t paste.<\/strong> Instead of pasting 500 lines of code, give the file path and let the tool load exactly as much as it needs.<\/li>\n\n\n
        • Delegate documentation updates to the agent.<\/strong> Keeping docs current is the chore nobody ever assigns, so assign it explicitly, as a rule in the AI file: “if you change something described in docs\/<\/code>, update that file and INDEX.md<\/code> in the same PR.” The agent is already making the change and has full context, so it doesn’t get cheaper than that; what’s left for you is reviewing the docs diff along with the code.<\/li>\n<\/ul>\n\n\n\n
          7. Tools that do it for you (almost)<\/h2>\n\n\n\n
          You don’t have to start from scratch: there are ready-made tools.<\/p>\n\n\n\n
          Generating the AI file:<\/strong><\/p>\n\n\n\n
            \n
          • Claude Code \/init<\/code><\/strong>: scans the repo and generates a starter CLAUDE.md,<\/li>\n\n\n
          • Cursor<\/strong>: can generate rules based on an analysis of the project.<\/li>\n<\/ul>\n\n\n\n
            A generated file is a draft, not a finished product: before it lands in the repo, review it, trim it, and add what the tool couldn’t know.<\/p>\n\n\n\n
            Packing a repo into LLM context:<\/strong><\/p>\n\n\n\n
              \n
            • Repomix<\/strong>: packs a codebase into a single file (Markdown\/XML), shows token counts per file, respects .gitignore<\/code>, and scans for secrets,<\/li>\n\n\n
            • Gitingest<\/strong>: zero setup, swap hub<\/code> for ingest<\/code> in a GitHub URL and you get a text digest of the repo,<\/li>\n\n\n
            • repo2txt<\/strong>: the browser version, you pick the files yourself.<\/li>\n<\/ul>\n\n\n\n
              Auto-documentation:<\/strong><\/p>\n\n\n\n
                \n
              • DeepWiki<\/strong> (Cognition\/Devin): generates an interactive wiki with architecture diagrams for any GitHub repo; there’s also an open-source equivalent that runs locally.<\/li>\n<\/ul>\n\n\n\n
                8. What to avoid<\/h2>\n\n\n\n
                  \n
                • A monolithic 3,000-line README as the “source of truth.”<\/li>\n\n\n
                • Documentation that has drifted from the code. Stale knowledge is worse than no knowledge, because the model trusts it.<\/li>\n\n\n
                • The same information duplicated in several places (you pay for it repeatedly and risk contradictions).<\/li>\n\n\n
                • “Preventive” context: adding files nobody asked about.<\/li>\n<\/ul>\n\n\n\n
                  Summary<\/h2>\n\n\n\n
                  A good repo for AI is a good repo for people, just held to a stricter standard: a short entry point, layered knowledge, names that speak, small files, documented decisions. And it’s not just about code: notes, documentation, and procedures benefit from the same structure.<\/p>\n\n\n\n
                  This year’s research gives the practice a mixed review. According to one study (Lulla et al., ICSE 2026), an agent in a repo with an AGENTS.md<\/code> finishes tasks faster (median runtime down ~29%) and uses fewer output tokens (~17% less). According to another (Gloaguen et al., ETH Zurich), a context file doesn’t raise the share of completed tasks, while inference cost grows by over 20% on average, whether the file was written by a human or an LLM. The difference comes down to what was measured: time and tokens on real pull requests versus success rates on a benchmark. The most practical takeaway is ETH’s finding about content: agents follow instructions diligently, but a “repository overview” helps with nothing<\/strong>.<\/p>\n\n\n\n
                  My own experience says the practice holds up regardless of the benchmarks. Since the knowledge my agents get has structure (an entry point, an index, small topical files), sessions are shorter, less time goes into correcting the agent, and I no longer start work by explaining everything from zero. Decisions come out more accurate, I can run different models at lower reasoning effort, and costs drop with them.<\/p>\n\n\n\n
                  A benchmark measures the average across other people’s repositories; the effect in yours is something you can check yourself.<\/p>\n\n\n\n
                  Treat tokens as a budget and the structure of your knowledge as the interface to the model.<\/p>\n","protected":false},"excerpt":{"rendered":"
                  Practical rules for structuring a repo and knowledge base for AI agents: short entry points, layered docs, INDEX.md, ADRs, and what research says about it.<\/p>\n","protected":false},"author":1,"featured_media":470,"comment_status":"open","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-471","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/marcinzygmunt.pl\/blog\/wp-content\/uploads\/2026\/08\/cover-v1.png","_links":{"self":[{"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/posts\/471","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/comments?post=471"}],"version-history":[{"count":1,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/posts\/471\/revisions"}],"predecessor-version":[{"id":472,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/posts\/471\/revisions\/472"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/media\/470"}],"wp:attachment":[{"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/media?parent=471"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/categories?post=471"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marcinzygmunt.pl\/blog\/wp-json\/wp\/v2\/tags?post=471"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}