Literate Programming Skill

CRITICAL: This skill MUST be activated BEFORE making any changes to .nw files!

You are an expert in literate programming using the noweb system.

Reference Files

This skill includes detailed references in

references/

noweb-commands.md

notangle

noweave

-R

-L

testing-patterns.md

test functions

pytest

after implementation

git-workflow.md

git

commit

generated files

multi-directory-projects.md

src/

doc/

tests/

MODULES

preamble.tex

\usepackage

memoir

When to Use This Skill

Correct Workflow

1. User asks to modify a .nw file
2. YOU ACTIVATE THIS SKILL IMMEDIATELY
3. You plan the changes with literate programming principles
4. You make the changes following the principles
5. You regenerate code with make/notangle

Anti-pattern (NEVER do this)

1. User asks to modify a .nw file
2. You directly edit the .nw file ← WRONG
3. Later review finds literate quality problems
4. You have to redo everything

Remember

• .nw files are NOT regular source code files
• They combine documentation and code for human readers
• Literate quality is AS IMPORTANT as code correctness
• Bad literate quality = failed task, even if code works

Planning Changes

When making changes to a .nw file:

1. Read the existing file to understand structure and narrative
2. Plan with literate programming in mind:
   • What is the "why" behind this change?
   • How does this fit into the existing narrative?
   • What new chunks are needed? What are their meaningful names?
   • Where in the pedagogical order should this be explained?
3. Design documentation BEFORE writing code:
   • Write prose explaining the problem and solution
   • Use subsections to structure complex explanations
4. Decompose code into well-named chunks:
   • Each chunk = one coherent concept
   • Names describe purpose, not syntax (like pseudocode)
5. Write the code chunks
6. Regenerate and test

Key principle: If you find yourself writing code comments to explain logic, that explanation belongs in the documentation chunks instead.

Reviewing Literate Programs

When reviewing, evaluate:

1. Narrative flow: Coherent story? Pedagogical order?
2. Variation theory: Contrasts used? "Whole, parts, whole" structure?
3. Chunk quality: Meaningful names? Focused on single concepts?
4. Explanation quality: Explains "why" not just "what"?
5. Test organization: Tests after implementation, not before?
6. Proper noweb syntax:

   [[code]]

   notation? Valid chunk references?

Core Philosophy

Literate programming (Knuth) has two goals:

1. Explain to human beings what we want a computer to do
2. Present concepts in order best for human understanding (psychological order, not compiler order)

Variation Theory

Apply

variation-theory

• Contrast: Show what something IS vs what it is NOT
• Separation: Start with whole (module outline), then parts (chunks)
• Generalization: Show pattern across different contexts
• Fusion: Integrate parts back into coherent whole

CRITICAL: Show concrete examples FIRST, then state general principles. Readers cannot discern a pattern without first experiencing variation.

Noweb File Format

Documentation Chunks

• Begin with

  @

  followed by space or newline
• Contain explanatory text (LaTeX, Markdown, etc.)
• Copied verbatim by noweave

Code Chunks

• Begin with

  <<chunk name>>=

  on a line by itself (column 1)
• End when another chunk begins or at end of file
• Reference other chunks using

  <<chunk name>>

• Multiple chunks with same name are concatenated

Syntax Rules

• Quote code in documentation using

  [[code]]

  (escapes LaTeX special chars)
• Escape:

  @<<

  for literal

  <<

  ,

  @@

  in column 1 for literal

  @

Writing Guidelines

1. Start with the human story - problem, approach, design decisions
2. Introduce concepts in pedagogical order - not compiler order
3. Use meaningful chunk names - 2-5 word summary of purpose (like pseudocode)
4. Decompose by concept, not syntax
5. Explain the "why" - don't just describe what the code does
6. Keep chunks focused - single, coherent idea each
7. Use bucket chunks - accumulate

   <<functions>>=

   and

   <<constants>>=

   throughout
8. Define constants for magic numbers - never hardcode values
9. Co-locate dependencies with features - feature's imports in feature's section
10. Keep lines under 80 characters - both prose and code

LaTeX Documentation Quality

Apply

latex-writing

Lists with bold labels: Use

\begin{description}

\item[Label]

\begin{itemize}

\item \textbf{Label}:

Code with manual escaping: Use

[[code]]

\texttt{...\_...}

Manual quotes: Use

\enquote{...}

"..."

Manual cross-references: Use

\cref{...}

Section~\ref{...}

Progressive Disclosure Pattern

When introducing high-level structure, use abstract placeholder chunks that defer specifics:

def cli_show(user_regex,
             <<options for filtering>>):
  <<implementation>>
@

[... later, explain each option ...]

\paragraph{The --all option}
<<options for filtering>>=
all: Annotated[bool, all_opt] = False,
@

Benefits: readable high-level structure, pedagogical ordering, maintainability.

Chunk Concatenation Patterns

Use multiple definitions when building up a parameter list pedagogically:

\subsection{Adding the diff flag}
<<args for diff>>=
diff=args.diff,
@

[... later ...]

\subsection{Fine-tuning thresholds}
<<args for diff>>=
threshold=args.threshold
@

Use separate chunks when contexts differ (different scopes):

<<args from command line>>=  # Has args object
diff=args.diff,
@

<<params for recursion>>=    # No args, only parameters
diff=diff,
@

Test Organization

CRITICAL: Tests appear AFTER implementation, not before.

See

references/testing-patterns.md

Key points:

• Distribute tests throughout file, near implementations
• Use single

  <<test functions>>

  chunk, concatenated
• Use

  from module import *

  in test file
• Frame tests pedagogically: "Let's verify this works..."

Multi-Directory Projects

For large projects (5+ .nw files), see

references/multi-directory-projects.md

Key structure:

project/
├── src/           # .nw files → .py + .tex
├── doc/           # Master document, preamble.tex
├── tests/         # Extracted test files
└── makefiles/     # Shared build rules (noweb.mk, subdir.mk)

LaTeX-Safe Chunk Names

Use

[[...]]

<<[[module_name.py]]>>=
def my_function():
    pass
@

Extract with:

notangle -R"[[module_name.py]]" file.nw > module_name.py

Best Practices Summary

1. Write documentation first - then add code
2. Keep lines under 80 characters
3. Check for unused chunks - run

   noroots

   to find typos
4. Keep tangled code in .gitignore - .nw is source of truth
5. NEVER commit generated files - .py and .tex from .nw are build artifacts
6. Test your tangles - ensure extracted code runs
7. Keep docstrings independent from LaTeX - no

   \cref

   in docstrings
8. Include table of contents - add

   \tableofcontents

   in documentation

Git Workflow

See

references/git-workflow.md

Core rules:

• Only commit .nw files to git
• Add generated files to .gitignore immediately
• Regenerate code with

  make

  after checkout/pull
• Never commit generated .py or .tex files

Noweb Commands Quick Reference

See

references/noweb-commands.md

# Tangling
notangle -R"[[module.py]]" file.nw > module.py
noroots file.nw                              # List root chunks

# Weaving
noweave -n -delay -x -t2 file.nw > file.tex  # For inclusion
noweave -latex -x file.nw > file.tex         # Standalone

When Literate Programming Is Valuable

• Complex algorithms requiring detailed explanation
• Educational code where understanding is paramount
• Code maintained by others
• Programs where design decisions need documentation
• Projects combining multiple languages/tools