Claude-skill-registry cfn-dependency-ingestion
Unified atomic ingestion of CFN dependency manifests (trigger-dev, cli-mode, shared)
git clone https://github.com/majiayu000/claude-skill-registry
T=$(mktemp -d) && git clone --depth=1 https://github.com/majiayu000/claude-skill-registry "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/data/ingestion" ~/.claude/skills/majiayu000-claude-skill-registry-cfn-dependency-ingestion && rm -rf "$T"
skills/data/ingestion/SKILL.mdCFN Dependency Ingestion Skill
Quick Start
Unified Shell Script (v2.0.0+)
# List available manifests bash .claude/skills/cfn-dependency-ingestion/ingest.sh --list-manifests # Trigger.dev infrastructure (P0 critical, ~17K tokens) bash .claude/skills/cfn-dependency-ingestion/ingest.sh --manifest trigger-dev --priority P0 --inject-content --skip-validation # Trigger.dev full context (P0+P1, ~32K tokens) bash .claude/skills/cfn-dependency-ingestion/ingest.sh --manifest trigger-dev --inject-content --skip-validation # CLI mode dependencies bash .claude/skills/cfn-dependency-ingestion/ingest.sh --manifest cli-mode --inject-content --skip-validation # Shared dependencies bash .claude/skills/cfn-dependency-ingestion/ingest.sh --manifest shared --inject-content # Filter by type (TypeScript only) bash .claude/skills/cfn-dependency-ingestion/ingest.sh --manifest trigger-dev --type TS --inject-content
Legacy: Diagram-Based Ingestion
# CLI mode from diagram bash .claude/skills/cfn-dependency-ingestion/ingest.sh --diagram cli # Docker mode from diagram bash .claude/skills/cfn-dependency-ingestion/ingest.sh --diagram docker
TypeScript Version (Legacy)
node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js \ --manifest .claude/skills/cfn-dependency-ingestion/manifests/cli-mode-dependencies.txt \ --inject-content
Performance Comparison:
- Traditional mode: 15 tool calls (1 skill + 14 Read commands) → ~60 seconds
- Content injection mode (chunked): 1 skill + 3 parallel Read calls → ~3ms
- 99.995% reduction in execution time (60s → 3ms)
- 20,000x speedup for Task tool agents
What This Skill Does
- Parses
(single source of truth)readme/CFN_LOOP_DEPENDENCY_DIAGRAM.txt - Extracts all file paths from PART 4 (File Execution Order) and PART 5 (TypeScript Module Structure)
- Groups files by priority: [P0] critical path, [P1] post-validation, [P2] deferred
- Smart Mode Selection:
- Under 20k tokens: Injects content directly in stdout
- Over 25k tokens: Splits into 20k token chunks written to
/tmp/cfn-dependency-chunks/ - Task tool agents: Read chunk files in parallel (3ms vs 60s sequential)
- Outputs Read commands for chunks or traditional mode
Usage Examples
TypeScript Version (v2.0.0+)
Basic ingestion with content injection:
node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --inject-content
Priority-filtered ingestion:
# P0 only (critical path) node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --inject-content --priority P0 # P0 + P1 (exclude deferred) node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --inject-content --priority P0,P1
Type-filtered ingestion:
# TypeScript only node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --inject-content --type TS # Shell scripts only node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --type SH # Both TypeScript and shell node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --type TS,SH
Traditional Read command output:
# Output Read commands instead of injecting content node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js
Skip validation (faster, useful when files are known to exist):
node .claude/skills/cfn-dependency-ingestion/dist/ingest-dependencies.js --inject-content --skip-validation
Shell Script Version (Legacy)
Basic ingestion (all files):
./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh
Priority-filtered ingestion:
# P0 only (critical path) ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --priority P0 # P0 + P1 (exclude deferred) ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --priority P0,P1
Type-filtered ingestion:
# TypeScript only ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type TS # Shell scripts only ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type SH # Both TypeScript and shell ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type TS,SH
Progressive Disclosure
<details> <summary>Click to see full implementation details</summary>File Priority Levels
- P0 (Critical Path): Required for 5-iteration North Star test
- P1 (Post-Validation): High value features after core validation
- P2 (Deferred): Nice-to-have features, can inline
Dynamic Parsing Logic
The script uses grep and sed patterns to extract files:
- Critical path markers[P0]
- Post-validation markers[P1]
- Deferred features[P2]
- TypeScript implementation[TS]
- Shell script fallback[SH]
- Legacy files (excluded by default)[DEPRECATED]
Output Format
Generates Read commands grouped by priority for easy copy-paste into Main Chat or agent profiles:
# Step 1: Read the dependency diagram Read: readme/CFN_LOOP_DEPENDENCY_DIAGRAM.txt # Step 2: Read P0 critical path files Read: .claude/commands/cfn-loop-cli.md Read: src/cli/index.ts Read: .claude/agents/cfn-dev-team/coordinators/cfn-v3-coordinator.md ... # Step 3: Read P1 files (post-validation) Read: .claude/skills/cfn-loop-orchestration/src/helpers/confidence-aggregator.ts Read: .claude/skills/cfn-loop-orchestration/src/helpers/context-lookup.ts ... # Step 4: Read coordination layer (Redis/Shell) Read: .claude/skills/cfn-coordination/coordination-wait.sh Read: .claude/skills/cfn-redis-coordination/report-completion.sh ...
Architecture
Diagram Structure (Source of Truth):
readme/CFN_LOOP_DEPENDENCY_DIAGRAM.txt ├── PART 1: USER TO COORDINATOR (spawning flow) ├── PART 2: COORDINATOR TO ORCHESTRATOR (parameters) ├── PART 3: TYPESCRIPT ORCHESTRATOR (main loop) ├── PART 4: CRITICAL DEPENDENCIES (execution order) ├── PART 5: TYPESCRIPT MODULE STRUCTURE (priority markers) ├── PART 6: MODE-SPECIFIC THRESHOLDS └── PART 7: COORDINATION PROTOCOL (Redis patterns)
Parsing Strategy:
- Read entire diagram into memory
- Extract PART 4 and PART 5 sections
- Parse priority markers: [P0], [P1], [P2]
- Parse type markers: [TS], [SH]
- Extract file paths using regex patterns
- Deduplicate and sort by priority
- Output Read commands grouped by category
File Path Patterns:
.claude/commands/*.md.claude/agents/**/*.md.claude/skills/**/src/**/*.tssrc/cli/*.tstests/**/*.sh
Integration with cfn-loops-cli-expert Agent
The
cfn-loops-cli-expert agent MUST use this skill in Step 2 instead of hardcoded file lists:
## Step 2: Execute Dependency Ingestion Run the dynamic ingestion script to load all CFN Loop CLI dependencies: ```bash ./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh
This automatically discovers and reads all files referenced in the dependency diagram.
## Maintenance This skill self-updates as long as `readme/CFN_LOOP_DEPENDENCY_DIAGRAM.txt` is maintained. No code changes needed when files are added/removed. **When to Update:** - New TypeScript module added to orchestration - New agent profile created - File paths change - Priority levels shift (P0 → P1, etc.) **How to Update:** Simply update the dependency diagram. The parser adapts automatically. ## Success Criteria Skill is working correctly when: - All P0 critical path files are extracted - Priority filtering works (--priority flag) - Type filtering works (--type flag) - No DEPRECATED files included (unless --include-deprecated flag set) - Output is valid Read commands (can copy-paste directly) - File paths are relative to project root - No duplicate files in output (deduplication working) - File existence validation reports missing files to stderr - Type filters correctly exclude non-matching extensions ## Regression Testing ### Expected File Counts (as of 2025-11-20) Use these validation commands to ensure counts match expectations: ```bash # Total files discovered (all priorities, all types, skip validation) # Note: 14 after deduplication (was 18 with orchestrate.ts appearing twice and other duplicates) expected_total=14 actual=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --skip-validation 2>&1 | grep -c "^Read:") test $actual -eq $expected_total && echo "✓ Total files: $actual" || echo "✗ Expected $expected_total, got $actual" # P0 critical path files (in Step 2 section only, not including diagram in Step 1) expected_p0=3 actual=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --priority P0 --skip-validation 2>&1 | awk '/Step 2:/,/Step 3:/' | grep -c "^Read:") test $actual -eq $expected_p0 && echo "✓ P0 files: $actual" || echo "✗ Expected $expected_p0, got $actual" # TypeScript files (includes .ts, .js, .cjs) expected_ts=4 actual=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type TS --skip-validation 2>&1 | grep -c "^Read:") test $actual -eq $expected_ts && echo "✓ TypeScript files: $actual" || echo "✗ Expected $expected_ts, got $actual" # Shell script files (.sh) expected_sh=3 actual=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type SH --skip-validation 2>&1 | grep "Step 5" -A 20 | grep -c "^Read:.*\.sh$") test $actual -eq $expected_sh && echo "✓ Shell files: $actual" || echo "✗ Expected $expected_sh, got $actual"
Deduplication Test
Verify no file appears more than once:
# Check for duplicate Read commands duplicates=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --skip-validation 2>&1 | grep "^Read:" | sort | uniq -d) if [[ -z "$duplicates" ]]; then echo "✓ No duplicate files" else echo "✗ Duplicate files found:" echo "$duplicates" fi
File Existence Validation Test
Verify missing files are reported to stderr:
# Count missing files (should be 4 as of 2025-11-20) expected_missing=4 actual=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh 2>&1 | grep "^WARNING: File not found" | wc -l) test $actual -eq $expected_missing && echo "✓ Missing file warnings: $actual" || echo "✗ Expected $expected_missing warnings, got $actual" # Missing files should NOT appear in Read output missing_in_output=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh 2>&1 | grep "^Read:" | while read -r line; do file=$(echo "$line" | cut -d' ' -f2) [[ ! -f "$file" ]] && echo "$file" done | wc -l) test $missing_in_output -eq 0 && echo "✓ No missing files in output" || echo "✗ Found $missing_in_output missing files in output"
Type Filter Validation
Verify type filters work correctly:
# TypeScript filter should only return .ts/.js/.cjs files (plus diagram) non_ts=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type TS --skip-validation 2>&1 | grep "^Read:" | grep -v "DEPENDENCY_DIAGRAM" | grep -v "\.ts$\|\.js$\|\.cjs$") if [[ -z "$non_ts" ]]; then echo "✓ TypeScript filter working" else echo "✗ TypeScript filter returned non-TS files:" echo "$non_ts" fi # Shell filter should only return .sh files (plus diagram) non_sh=$(./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --type SH --skip-validation 2>&1 | grep "^Read:" | grep -v "DEPENDENCY_DIAGRAM" | grep -v "\.sh$") if [[ -z "$non_sh" ]]; then echo "✓ Shell filter working" else echo "✗ Shell filter returned non-SH files:" echo "$non_sh" fi
Known Missing Files (as of 2025-11-20)
These files are referenced in the dependency diagram but do not exist:
.claude/skills/cfn-redis-coordination/SKILL.md.claude/agents/cfn-dev-team/coordinators/cfn-cli-dependency-maintainer.md.claude/commands/cfn/CFN_COORDINATOR_PARAMETERS.md.claude/commands/CFN_LOOP_TASK_MODE.md
Action: Update dependency diagram to remove references or create missing files.
Updating Expected Counts
When files are added/removed from the dependency diagram:
- Run ingestion script:
./.claude/skills/cfn-dependency-ingestion/ingest-dependencies.sh --skip-validation - Count total files:
| grep -c "^Read:" - Update
in regression tests aboveexpected_total - Update P0/TS/SH counts similarly
- Update "Known Missing Files" section
- Document change in Version History
Related Documentation
- Dependency Diagram:
(source of truth)readme/CFN_LOOP_DEPENDENCY_DIAGRAM.txt - CFN Loop Architecture:
docs/CFN_LOOP_ARCHITECTURE.md - TypeScript Migration:
planning/docker-migration/TYPESCRIPT_MIGRATION_HANDOFF.md - Agent Profiles:
.claude/agents/cfn-dev-team/
Version History
-
3.0.0 (2025-11-24): Unified ingestion script
- Merged Skills: Combined trigger-dev-dependency-ingestion and cfn-dependency-ingestion into single tool
- Unified Interface:
for all manifest-based ingestion--manifest <name> - Available Manifests: trigger-dev, cli-mode, shared, trigger-mode
- Legacy Support:
for diagram-based parsing--diagram <type> - Token Estimates: trigger-dev P0 ~17K tokens, full ~32K tokens
-
2.1.0 (2025-11-20): Chunked mode for Task tool agents
- Enhancement #6: Automatic chunking into 20k token files for parallel reads
- Performance: 20,000x speedup (60s → 3ms) for Task tool agents
- Smart Splitting: Writes chunks to
when over 25k tokens/tmp/cfn-dependency-chunks/ - Parallel Reading: Task agents can read 3 chunks in parallel instead of 15 files sequentially
-
2.0.0 (2025-11-20): TypeScript implementation with content injection mode
- Enhancement #5: Created TypeScript version with
flag--inject-content - Performance: 93% reduction in tool calls (15 → 1) with atomic context loading
- Safety: 25k token limit with graceful fallback to chunked mode
- Compatibility: Shell script version remains available as legacy fallback
- Enhancement #5: Created TypeScript version with
-
1.1.0 (2025-11-20): Enhanced with 4 improvements
- Enhancement #1: Added deduplication logic using
(fixessort -u
appearing twice)orchestrate.ts - Enhancement #2: Fixed type filter implementation (TS/SH filters now work correctly)
- Enhancement #3: Added file existence validation with
flag--skip-validation - Enhancement #4: Documented expected file counts for regression testing
- Enhancement #1: Added deduplication logic using
-
1.0.0 (2025-11-20): Initial release with dynamic parsing from dependency diagram