LLM Wiki — Karpathy Knowledge Base Pattern

Experimental skill — iterating. Authored by Lewis Liu (lylewis@outlook.com) · Inspired by Karpathy's llm-wiki Gist

Core idea

Instead of RAG (re-retrieving raw docs on every query), the LLM compiles raw sources into a persistent, cross-linked wiki. Every ingest, query, lint, and audit pass makes the wiki richer. Knowledge compounds — and the human stays in the loop via a structured feedback channel instead of ad-hoc corrections that get lost.

• You own: sourcing raw material, asking good questions, steering direction, filing feedback on anything the AI got wrong.
• LLM owns: all writing, cross-referencing, filing, bookkeeping, and acting on your feedback.

The wiki is a living artifact with five operations —

compile

ingest

query

lint

audit

CLAUDE.md

wiki/index.md

Directory layout

<wiki-root>/
├── CLAUDE.md          ← Schema: scope, conventions, current articles, gaps
├── log/               ← Per-day operation log (one file per day)
│   ├── 20260409.md
│   └── 20260410.md
├── audit/             ← Human feedback inbox (one file per comment)
│   ├── 20260409-143022-claude-code-size.md
│   └── resolved/      ← Processed feedback, archived with resolution notes
├── raw/               ← Immutable source documents (LLM reads, never writes)
│   ├── articles/
│   ├── papers/
│   ├── notes/
│   └── refs/          ← Pointer files for large binaries kept outside raw/
├── wiki/              ← LLM-generated knowledge (LLM writes, you read)
│   ├── index.md       ← Master catalog — every page, structured by category
│   ├── concepts/      ← Concept/topic pages (split into subfolders when >1200 words)
│   ├── entities/      ← People, tools, papers, organizations
│   └── summaries/     ← Per-source summary pages
└── outputs/
    └── queries/       ← Query answers (promote durable ones to wiki/)

CLAUDE.md

references/schema-guide.md

Core principles

Four rules govern everything below. If a future instruction contradicts one, flag it to the user before acting.

1. Divide and conquer

A single concept page should never try to cover a complex topic end-to-end. Target: 400–1200 words per page. When a topic would blow past that:

• Create a subfolder:

  wiki/concepts/<topic>/

• Put a short index page at

  wiki/concepts/<topic>/index.md

  — definition, list of sub-pages, one-line summaries
• Put each aspect in its own file:

  wiki/concepts/<topic>/<aspect>.md

• In

  wiki/index.md

  , show the hierarchy via indented bullets

Example layout (from a real wiki):

wiki/tech/claude-code/
├── index.md                         (overview + links to sub-pages)
├── Claude_Code_Architecture.md
├── Claude_Code_Agent_Framework.md
├── Claude_Code_Bridge_System.md
├── Claude_Code_Query_Engine.md
├── Claude_Code_Skills_Plugins.md
├── Claude_Code_State_Management.md
└── Claude_Code_Tool_System.md

One fat file covering all seven aspects would be unreadable and unlinkable. Seven focused files + an index page give you navigation, selective reading, clean backlinks, and small audit targets.

2. Mermaid for diagrams, KaTeX for formulas

• Any flow, sequence, hierarchy, or state diagram must be written in mermaid — never ASCII art. ASCII boxes rot fast and are impossible to annotate.

  ```mermaid
  flowchart LR
      A[raw/article.md] --> B[summary]
      B --> C[concept page]
      C --> D[index.md]
  ```
  

• Any formula must be written in KaTeX: inline

  $f(x) = \sum_i w_i x_i$

  or block

  $$...$$

  .

Both render in the web viewer (server-side KaTeX, client-side mermaid) and in Obsidian with default settings.

3. Raw file policy

Small text-based sources (md, txt, small pdfs, small images) → copy into

raw/<subfolder>/

Large binaries (videos, model weights, installers, datasets, large PDFs >10 MB) → do not copy. Instead:

• Create a pointer file at

  raw/refs/<slug>.md

  with:

  ---
  kind: ref
  external_path: /Volumes/external/models/llama-3-70b/
  size: ~140 GB
  ---
  

  followed by a short description of what it is and why it matters to this wiki.
• Wiki pages cite

  [[raw/refs/<slug>]]

  exactly like any other source.

This keeps the wiki repo git-friendly and portable.

4. Audit is the human feedback surface

The wiki is AI-written; it will be wrong sometimes. The raw sources are human-written; they will contradict each other. The

audit/

• Humans file feedback via the Obsidian plugin or the web viewer. Each feedback is one file in

  audit/

  with YAML frontmatter (anchor, target, severity) and a markdown body.
• The AI must periodically run the

  audit

  op — never silently ignore

  audit/*.md

  files.
• When feedback is applied, the file moves to

  audit/resolved/

  with a

  # Resolution

  section appended and a log entry recorded in

  log/YYYYMMDD.md

  .

See

references/audit-guide.md

The five operations

Every action on the wiki is one of these five. Each appends an entry to the current day's log file (

log/YYYYMMDD.md

1.

compile

(Re)structure wiki content from existing

raw/

index.md

When to run: after a big ingest batch, when an existing page has outgrown 1200 words, when

index.md

Steps:

1. Read

   CLAUDE.md

   ,

   wiki/index.md

   , and every file in the target subtree.
2. For each page over ~1200 words: plan a split into

   concepts/<topic>/

   with an index + sub-pages. Confirm the plan with the user before writing.
3. For each pair of near-duplicate pages: propose a merge. Confirm, then rewrite.
4. Regenerate

   wiki/index.md

   so every page is listed exactly once.
5. Log:

   ## [HH:MM] compile | <what you did — files touched, splits, merges>

2.

ingest

Add a new source. One source typically touches 5–15 wiki pages.

Steps:

1. Save source to the right subfolder:
   • web article →

     raw/articles/<slug>.md

   • paper →

     raw/papers/<slug>.md

     (extracted text for big PDFs)
   • note →

     raw/notes/<slug>.md

   • large binary →

     raw/refs/<slug>.md

     pointer file (see raw file policy)
2. Read the source in full.
3. Create

   wiki/summaries/<slug>.md

   (200–400 words — key takeaways, not a rewrite; see

   references/article-guide.md

   ).
4. Create or update relevant concept pages in

   wiki/concepts/

   . Respect divide-and-conquer: if a concept page would exceed 1200 words, split instead of cramming.
5. Create or update entity pages in

   wiki/entities/

   for any new people / tools / papers / organizations referenced.
6. Update

   wiki/index.md

   so the new pages appear under the right category.
7. Log:

   ## [HH:MM] ingest | <slug> — <one-line description> (touched N pages)

3.

query

Answer a question grounded in the wiki, not general knowledge.

Steps:

1. Read

   wiki/index.md

   . Scan for relevant pages by category.
2. Read the identified pages in full; follow one level of wikilinks.
3. If the wiki doesn't have enough material, say so and suggest what to ingest next instead of making something up.
4. Synthesize the answer, citing pages inline with

   [[Page Name]]

   .
5. Save to

   outputs/queries/<YYYY-MM-DD>-<question-slug>.md

   .
6. If the answer is durable (a comparison, analysis, or new synthesis) → promote a cleaned-up version to

   wiki/concepts/

   , add to

   index.md

   .
7. Log:

   ## [HH:MM] query | <question-slug>

   (and a separate

   ## [HH:MM] promote | ...

   line if promoted).

4.

lint

Health check. Run:

python3 scripts/lint_wiki.py <wiki-root>

The script reports:

• Dead wikilinks —

  [[Target]]

  where

  Target.md

  doesn't exist
• Orphan pages — pages with no inbound wikilinks
• Missing index entries — pages not listed in

  wiki/index.md

• Frequently-linked missing pages —

  [[X]]

  referenced 3+ times but no page
• log/ shape — stray files or wrong filenames in

  log/

• audit/ shape — malformed YAML frontmatter in

  audit/*.md

• Audit target resolution — every open audit's

  target

  file must exist

For each issue, propose a fix, confirm with the user, then apply. Log:

## [HH:MM] lint | <N> issues found, <M> fixed

5.

audit

Process human feedback from

audit/

Steps:

1. Run

   python3 scripts/audit_review.py <wiki-root> --open

   to get a grouped list.
2. For each open audit, read the file. Use the

   anchor_before

   /

   anchor_text

   /

   anchor_after

   window to locate the exact range in the target file (line numbers may have drifted).
3. Decide the action:
   • Accept: apply the correction to the target file.
   • Partially accept: apply what makes sense, note the rest in the resolution.
   • Reject: explain why in the resolution — the feedback may be based on a misreading of scope or a contradictory source.
   • Defer: add to

     CLAUDE.md

     "Open research questions" and leave the audit in place with a comment.
4. For applied audits, append a

   # Resolution

   section to the audit file:

   # Resolution
   
   2026-04-10 · accepted.
   Fixed the file count (was "~1,900", corrected to "~1,800" per commit abc123).
   Updated: tech/Claude_Code.md lines 47–48.
   

5. Move the file from

   audit/

   to

   audit/resolved/

   . Filename unchanged.
6. Log per resolved audit:

   ## [HH:MM] audit | resolved 20260409-143022-a1b2 — <one-line what>
   

7. Never delete audit files. Rejected ones still go to

   resolved/

   with the rejection rationale in their resolution section — that's valuable history.

See

references/audit-guide.md

Tooling

plugins/obsidian-audit/

audit/

web/

audit/

scripts/scaffold.py

scripts/lint_wiki.py

scripts/audit_review.py

The Obsidian plugin and the web viewer both write audit files in the same format with the same anchor algorithm, so feedback filed from either place can be resolved by either place.

Starting a new wiki

python3 scripts/scaffold.py <wiki-root> "<Topic Title>"

Creates the full tree (including

log/<today>.md

audit/

audit/resolved/

CLAUDE.md

wiki/index.md

After scaffolding:

1. Fill in

   CLAUDE.md

   — define scope, naming conventions, initial research questions.
2. Start ingesting sources.
3. Ask questions to build up

   outputs/queries/

   ; promote durable answers.
4. Run

   lint

   periodically.
5. Run

   audit

   whenever new feedback accumulates.

wiki/index.md

format

The LLM rebuilds

index.md

# Index — <Topic>

> One-sentence scope of the wiki.

## 🔖 Navigation
- [[#Concepts]] · [[#Entities]] · [[#Summaries]] · [[#Open Questions]]

## Concepts
### <Category A>
- [[concepts/Foo]] — one-line summary
- [[concepts/Bar/index|Bar]] — (folder-split) one-line summary
    - [[concepts/Bar/aspect-1]] — ...
    - [[concepts/Bar/aspect-2]] — ...

### <Category B>
- ...

## Entities
- [[entities/Andrej Karpathy]] — AI researcher, author of the llm-wiki pattern

## Summaries (chronological)
- 2026-04-09 — [[summaries/llm-wiki-gist]] — Karpathy's original Gist

## Open Questions
- Q1: ...

Rules:

• Every wiki page must appear exactly once in

  index.md

  .

  lint

  enforces this.
• Folder-split concepts show hierarchy via indented bullets.

• index.md

  +

  CLAUDE.md

  together are what the AI reads at session start.

log/

format

See

references/log-guide.md

• One file per day:

  log/YYYYMMDD.md

• H1 = the date; H2 per entry with

  ## [HH:MM] <op> | <one-line description>

• Ops:

  compile

  ,

  ingest

  ,

  query

  ,

  lint

  ,

  audit

  ,

  promote

  ,

  split

  ,

  scaffold

Quick grep across history:

grep -rh "^## \[" log/ | tail -20

Use cases

• Research deep-dive — reading papers/articles on a topic over weeks; the wiki evolves with your understanding, and the audit trail keeps AI mistakes from silently accumulating
• Personal wiki — journal entries, notes, ideas compiled into a personal encyclopedia; comment on anything you disagree with later, the AI corrects it
• Team knowledge base — fed by Slack threads, meeting notes, docs; team members file corrections through the web viewer
• Reading companion — filing each book chapter as you go; builds a rich companion wiki by the end

References

• references/schema-guide.md

  — What to put in

  CLAUDE.md

• references/article-guide.md

  — How to write good wiki articles (length, wikilinks, mermaid, math, divide-and-conquer)

• references/log-guide.md

  — The

  log/

  folder convention

• references/audit-guide.md

  — Audit file format, anchor strategy, processing workflow

• references/tooling-tips.md

  — Obsidian setup, Web Clipper, qmd, plugin + web installation