RLM Prep

Prepare source content for RLM processing in one shot: discovers files, chunks each one, builds a searchable index, and writes a unified

manifest.json

rlm-search

fanout

Triggers

Alternate expressions and non-obvious activations:

• "index this codebase for search" → rlm-prep on directory
• "get this ready for RLM" → rlm-prep with defaults
• "prep the docs folder" → rlm-prep on

  docs/

• "build a chunk index" → rlm-prep with index output

Trigger Patterns Reference

rlm-prep src/

rlm-prep path/to/file.ts

--strategy fixed-count

--size 100

--output tmp/rlm/

--force

Behavior

When triggered:

1. Resolve source — determine whether the input is a single file or a directory. For directories, discover all supported file types (

   .ts

   ,

   .js

   ,

   .py

   ,

   .go

   ,

   .md

   ,

   .txt

   ,

   .yaml

   ,

   .json

   ,

   .sql

   , and others). Respect

   .gitignore

   patterns.

2. Check for existing prep — look for a manifest in the output directory. If found and

   --force

   is not set, report that prep already exists and offer to use it or re-run.

3. Chunk each file — apply the selected strategy per file. Each file produces its own subdirectory under

   chunks/

   , named after the file path (slashes replaced with underscores).

4. Build index — construct a searchable index (

   index.json

   ) with:
   • Chunk IDs mapped to file, line range, and boundary label
   • Content summaries (first non-blank line of each chunk)
   • File-level metadata (language, size, last-modified)

5. Write unified manifest — a single

   manifest.json

   at the output root that references all chunks across all files. This is what

   fanout

   and

   rlm-search

   consume.

6. Report result — print file count, total chunk count, index size, and output path.

Output Directory Structure

.aiwg/rlm-prep/<source-hash>/
├── manifest.json              # Unified chunk manifest (all files)
├── index.json                 # Searchable index with summaries
├── meta.json                  # Source path, strategy, timestamp
└── chunks/
    ├── src__auth__middleware.ts/
    │   ├── chunk-0001.txt
    │   ├── chunk-0002.txt
    │   └── chunk-0003.txt
    ├── src__auth__jwt.ts/
    │   ├── chunk-0001.txt
    │   └── chunk-0002.txt
    └── src__core__parser.ts/
        ├── chunk-0001.txt
        ├── chunk-0002.txt
        ├── chunk-0003.txt
        └── chunk-0004.txt

Manifest Format (multi-file)

{
  "source": "src/auth/",
  "source_hash": "sha256:a1b2c3d4...",
  "strategy": "semantic-boundary",
  "chunk_size": 200,
  "overlap": 20,
  "created_at": "2026-04-01T14:23:00Z",
  "files": 12,
  "total_chunks": 47,
  "output_dir": ".aiwg/rlm-prep/a1b2c3d4/",
  "chunks": [
    {
      "id": "src__auth__middleware.ts/chunk-0001",
      "file_source": "src/auth/middleware.ts",
      "chunk_file": ".aiwg/rlm-prep/a1b2c3d4/chunks/src__auth__middleware.ts/chunk-0001.txt",
      "start_line": 1,
      "end_line": 218,
      "boundary_label": "validateToken()"
    }
  ]
}

Parameters

• <file|dir>

  — Source file or directory to prepare (required)

• --output <dir>

  — Output directory (default:

  .aiwg/rlm-prep/<source-hash>/

  )

• --strategy semantic-boundary|fixed-count|adaptive

  — Chunking strategy (default:

  semantic-boundary

  )

• --size N

  — Target chunk size in lines (default:

  200

  )

• --overlap N

  — Overlap lines between adjacent chunks (default:

  20

  )

• --force

  — Re-prep even if a manifest already exists

Examples

Example 1: Prep a source directory

User: "prepare src/ for RLM processing"

Action:

aiwg rlm-prep src/

Response: "Prepped

src/

.aiwg/rlm-prep/a1b2c3d4/manifest.json

Example 2: Prep with smaller chunks for a dense codebase

User: "index the entire repo for RLM, use 100-line chunks"

Action:

aiwg rlm-prep . --size 100 --overlap 15

Response: "Prepped

.

.aiwg/rlm-prep/b3c4d5e6/manifest.json

Example 3: Prep a documentation set

User: "get the docs folder ready for recursive search"

Action:

aiwg rlm-prep docs/ --strategy fixed-count --size 150

Response: "Prepped

docs/

.aiwg/rlm-prep/c4d5e6f7/manifest.json

Example 4: Already prepped — user wants to force refresh

User: "re-prep the auth module, I've made changes"

Action:

aiwg rlm-prep src/auth/ --force

Response: "Re-prepped

src/auth/

.aiwg/rlm-prep/d5e6f7a8/manifest.json

Example 5: Check if already prepped

User: "is src/ already prepped for RLM?"

Action: Check

.aiwg/rlm-prep/

src/

Response: "Yes —

src/

--force

Clarification Prompts

If the user's intent is ambiguous:

• "Should I prep the entire directory or just a specific subdirectory?"
• "A previous prep exists from [date]. Should I use it or re-prep?"
• "Which strategy: split at natural boundaries (semantic-boundary), fixed line counts, or adaptive?"

References

• @$AIWG_ROOT/agentic/code/addons/rlm/skills/chunk/SKILL.md — Single-file chunking (used internally by rlm-prep)
• @$AIWG_ROOT/agentic/code/addons/rlm/skills/fanout/SKILL.md — Query the prepared manifest
• @$AIWG_ROOT/agentic/code/addons/rlm/skills/rlm-search/SKILL.md — Full pipeline that calls rlm-prep automatically
• @$AIWG_ROOT/agentic/code/addons/rlm/schemas/rlm-chunk-manifest.yaml — Manifest schema
• @$AIWG_ROOT/agentic/code/addons/aiwg-utils/rules/context-budget.md — Budget guidance for downstream fanout