Validate Context Engineering

Validate AGENTS.md quality and evolve it as the codebase changes. Good context engineering means AI agents always have accurate, current knowledge about the project.

When to Use

• After adding new files, modules, or packages to the project
• After renaming, moving, or deleting significant files
• After changing public APIs, architectural patterns, or conventions
• When onboarding a new contributor (human or AI) and want to verify context is current
• When

  on_context_check

  or

  on_pre_commit

  triggers fire
• Periodically (weekly or per-sprint) as a hygiene check
• NOT when making trivial changes (typo fixes, comment updates, formatting)
• NOT during active feature development — run this between features or at cycle boundaries

Process

Phase 1: Audit — Run Automated Checks

1. Run

   harness validate

   to check overall project health. Review any context-related warnings or errors in the output.

2. Run

   harness check-docs

   to detect documentation gaps, broken links, and stale references. Capture the full output for analysis.

3. Review AGENTS.md manually. Automated tools catch structural issues but miss semantic drift. Read each section and ask: "Is this still true?"

Graph-Enhanced Context (when available)

When a knowledge graph exists at

.harness/graph/

, use graph queries for faster, more accurate auditing:

• query_graph

  — find all undocumented code nodes (file nodes without

  documents

  edges), replacing manual cross-referencing

• search_similar

  — detect stale references in AGENTS.md by matching section text against current code entities

When a graph is available, it IS the source of truth for documentation coverage. Drift = stale or missing edges between code and doc nodes. Fall back to file-based commands if no graph is available.

Pipeline Context (when orchestrated)

When invoked by

harness-docs-pipeline

, check for a

pipeline

field in

.harness/handoff.json

:

• If

  pipeline

  field exists: read

  DocPipelineContext

  from it
  • Use

    pipeline.exclusions

    to skip findings that were already addressed in the FIX phase
  • Write

    GapFinding[]

    results back to

    pipeline.gapFindings

    in handoff.json
  • This enables dedup across FIX and AUDIT phases
• If

  pipeline

  field does not exist: behave exactly as today (standalone mode)

No changes to the skill's interface or output format — the pipeline field is purely additive.

Phase 2: Detect Gaps

Categorize findings into four types:

1. Undocumented files. New source files, modules, or packages that are not mentioned in AGENTS.md or any knowledge map section. These are the highest priority — an AI agent encountering these files has no context.

2. Broken links. References to files, functions, or URLs that no longer exist. These actively mislead agents.

3. Stale sections. Content that was accurate when written but no longer reflects reality. Examples: renamed functions still referenced by old name, removed features still described, changed conventions not updated.

4. Missing context. Sections that exist but lack critical information. A module is listed but its purpose, constraints, or relationships are not explained. An AI agent can find the file but does not understand why it exists or how to use it correctly.

Phase 3: Suggest Updates

For each gap, generate a specific suggestion:

• Undocumented files: Draft a new section or entry with the file path, purpose, key exports, and relationship to existing modules. Use the existing AGENTS.md style and structure.
• Broken links: Identify the correct target (renamed file, moved function) or recommend removal if the target was deleted.
• Stale sections: Draft replacement text that reflects current reality. Show the diff between old and new.
• Missing context: Draft additional content that fills the gap. Focus on what an AI agent needs to know: purpose, constraints, relationships, and gotchas.

Phase 4: Apply with Approval

1. Present all suggestions as a grouped list. Organize by section of AGENTS.md for easy review.

2. Apply approved changes. Update AGENTS.md with the approved suggestions. Preserve existing formatting and style.

3. Re-run

   harness check-docs

   to verify all fixes are clean. No new issues should be introduced.

4. Commit the update. Use a descriptive commit message that summarizes what was updated and why.

What Makes Good AGENTS.md Content

Good context engineering treats AGENTS.md as a dynamic knowledge base, not a static document.

• Accuracy over completeness. A small, accurate AGENTS.md is better than a comprehensive but stale one. Every statement must be verifiable against the current codebase.
• Purpose-first descriptions. Do not just list files. Explain WHY each module exists, what problem it solves, and what constraints apply to it.
• Relationship mapping. Show how modules connect. Which modules depend on which? What are the boundaries? An agent reading one section should understand how it fits into the whole.
• Gotchas and constraints. Document the non-obvious. What will break if someone does X? What patterns must be followed? What is forbidden and why?
• Change-friendly structure. Organize so that adding a new module means adding one section, not updating ten places. Use consistent formatting so automated tools can parse it.
• Actionable guidance. Every section should help an agent make correct decisions. "This module handles authentication" is less useful than "This module handles authentication. All auth changes must go through the AuthService class. Direct database access for auth data is forbidden — use the repository layer."

Harness Integration

• harness validate

  — Full project health check. Reports context gaps as part of overall validation.

• harness check-docs

  — Focused documentation audit. Detects broken links, missing references, stale sections, and undocumented files.

• harness fix-drift

  — Auto-fix simple drift issues (broken links, renamed references). Use after manual review confirms the fixes are correct.

Success Criteria

• harness check-docs

  passes with zero errors and zero warnings
• Every source file that contains public API or architectural significance is referenced in AGENTS.md
• All file paths and function names in AGENTS.md match the current codebase
• All links (internal and external) resolve correctly
• AGENTS.md sections accurately describe current module purposes, constraints, and relationships
• A new AI agent reading AGENTS.md can navigate the codebase and make correct decisions without additional guidance

Rationalizations to Reject

Rationalization	Why It Is Wrong
"The automated checks passed, so AGENTS.md is accurate"	Phase 1 says automated tools catch structural issues but miss semantic drift. Manual review of each section is required.
"This module is small and internal -- it does not need an AGENTS.md entry"	Undocumented files are the highest priority finding. Even small internal modules need at least a purpose statement.
"AGENTS.md is severely outdated -- let me rewrite it from scratch"	Do not attempt to fix everything at once when there are >20 issues. Prioritize: broken links first, then undocumented public APIs.
"I will add the file paths and leave the descriptions for later"	Purpose-first descriptions, relationship mapping, and gotchas are what make context engineering useful. A file listing without context is marginally better than nothing.

Examples

Example: New module added but not documented

Audit output from

harness check-docs

:

WARNING: Undocumented file detected: src/services/notification-service.ts
  - File contains 3 public exports: NotificationService, NotificationType, sendNotification
  - File is imported by 4 other modules
  - No AGENTS.md section references this file

Suggested update:

### Notification Service (`src/services/notification-service.ts`)

Handles all outbound notifications (email, Slack, webhook). All notification delivery
must go through `NotificationService` — direct use of transport libraries (nodemailer,
Slack SDK) outside this module is forbidden.

- `NotificationType` — enum of supported notification channels
- `sendNotification()` — primary entry point; routes to the correct transport
- Requires `NOTIFICATION_CONFIG` environment variables to be set
- Respects rate limits defined in `harness.config.json` under `notifications`

Apply: Add the section under the Services heading in AGENTS.md. Re-run

harness check-docs

to confirm the warning is resolved.

Example: Renamed function still referenced by old name

Audit output:

ERROR: Broken reference in AGENTS.md line 47: `calculateShipping()`
  - Function was renamed to `computeShippingCost()` in commit abc123
  - Located in src/services/shipping.ts

Fix: Replace

calculateShipping()

with

computeShippingCost()

in AGENTS.md. Verify no other references to the old name exist.

Escalation

• When AGENTS.md is severely outdated (>20 issues): Do not attempt to fix everything at once. Prioritize: broken links first, then undocumented public APIs, then stale descriptions. Batch the work across multiple commits.
• When you are unsure whether a section is stale: Check git blame for the section and compare against recent changes to the referenced files. If the section has not been updated since the referenced files changed, it is likely stale.
• When the project has no AGENTS.md: Escalate to the human. Creating an AGENTS.md from scratch is a significant decision about project structure and should be done intentionally, not automatically.