Agentops flywheel
Knowledge flywheel health monitoring. Checks velocity, pool depths, staleness. Triggers: "flywheel status", "knowledge health", "is knowledge compounding".
install
source · Clone the upstream repo
git clone https://github.com/boshu2/agentops
Claude Code · Install into ~/.claude/skills/
T=$(mktemp -d) && git clone --depth=1 https://github.com/boshu2/agentops "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/flywheel" ~/.claude/skills/boshu2-agentops-flywheel-f38609 && rm -rf "$T"
manifest:
skills/flywheel/SKILL.mdsource content
Flywheel Skill
Monitor the knowledge flywheel health.
The Flywheel Model
Sessions → Transcripts → Forge → Pool → Promote → Knowledge ↑ │ └───────────────────────────────────────────────┘ Future sessions find it
Velocity = Rate of knowledge flowing through Friction = Bottlenecks slowing the flywheel
Execution Steps
Given
/flywheel:
Step 1: Measure Knowledge Pools
# Count top-level artifact files (avoid counting directories) LEARNINGS=$(find .agents/learnings -maxdepth 1 -type f 2>/dev/null | wc -l) PATTERNS=$(find .agents/patterns -maxdepth 1 -type f 2>/dev/null | wc -l) RESEARCH=$(find .agents/research -maxdepth 1 -type f 2>/dev/null | wc -l) RETROS=$(find .agents/retros -maxdepth 1 -type f 2>/dev/null | wc -l) echo "Learnings: $LEARNINGS" echo "Patterns: $PATTERNS" echo "Research: $RESEARCH" echo "Retros: $RETROS"
Step 2: Check Recent Activity
# Recent learnings (last 7 days) find .agents/learnings -maxdepth 1 -type f -mtime -7 2>/dev/null | wc -l # Recent research find .agents/research -maxdepth 1 -type f -mtime -7 2>/dev/null | wc -l
Step 3: Detect Staleness
# Old artifacts (> 30 days without modification) find .agents/ -name "*.md" -mtime +30 2>/dev/null | wc -l
Step 3.5: Check Cache Health
if command -v ao &>/dev/null; then # Get citation report (cache metrics) CITE_REPORT=$(ao metrics cite-report --json --days 30 2>/dev/null) if [ -n "$CITE_REPORT" ]; then HIT_RATE=$(echo "$CITE_REPORT" | jq -r '.hit_rate // "unknown"') UNCITED=$(echo "$CITE_REPORT" | jq -r '(.uncited_learnings // []) | length') STALE_90D=$(echo "$CITE_REPORT" | jq -r '.staleness["90d"] // 0') echo "Cache hit rate: $HIT_RATE" echo "Uncited learnings: $UNCITED" echo "Stale (90d uncited): $STALE_90D" fi else # ao-free fallback: compute approximate metrics from files echo "Cache health (ao-free fallback):" # Learnings modified in last 30 days (active pool) ACTIVE_30D=$(find .agents/learnings/ -name "*.md" -mtime -30 2>/dev/null | wc -l | tr -d ' ') echo "Active learnings (30d): $ACTIVE_30D" # Forge candidates awaiting promotion FORGE_PENDING=$(ls .agents/forge/*.md 2>/dev/null | wc -l | tr -d ' ') echo "Forge candidates pending: $FORGE_PENDING" # Citation tracking (if citations.jsonl exists) if [ -f .agents/ao/citations.jsonl ]; then CITATION_COUNT=$(wc -l < .agents/ao/citations.jsonl | tr -d ' ') UNIQUE_CITED=$(grep -o '"artifact_path":"[^"]*"' .agents/ao/citations.jsonl 2>/dev/null | sort -u | wc -l | tr -d ' ') echo "Total citations: $CITATION_COUNT" echo "Unique learnings cited: $UNIQUE_CITED" else echo "No citation data (citations.jsonl not found)" fi # Session outcomes (if outcomes.jsonl exists) if [ -f .agents/ao/outcomes.jsonl ]; then OUTCOME_COUNT=$(wc -l < .agents/ao/outcomes.jsonl | tr -d ' ') echo "Session outcomes recorded: $OUTCOME_COUNT" fi fi
Step 4: Check ao CLI Status
if command -v ao &>/dev/null; then ao metrics flywheel status 2>/dev/null || echo "ao metrics flywheel status unavailable" ao status 2>/dev/null || echo "ao status unavailable" ao maturity --scan 2>/dev/null || echo "ao maturity unavailable" ao anti-patterns 2>/dev/null || echo "ao anti-patterns unavailable" ao badge 2>/dev/null || echo "ao badge unavailable" # Knowledge maintenance ao dedup --merge 2>/dev/null || true ao contradict 2>/dev/null || true ao constraint review 2>/dev/null || true ao curate status 2>/dev/null || true ao metrics health 2>/dev/null || true ao metrics cite-report --days 30 2>/dev/null || true # Active pruning: archive stale, evict low-utility, and curate noisy uncited learnings ao maturity --expire --archive 2>/dev/null || true ao maturity --evict --archive 2>/dev/null || true ao maturity --curate --archive 2>/dev/null || true # Retrieval quality: use the representative live corpus when it exists if [ -d cli/cmd/ao/testdata/retrieval-bench-live ]; then ao retrieval-bench --live --corpus cli/cmd/ao/testdata/retrieval-bench-live --json 2>/dev/null || true fi else echo "ao CLI not available — using file-based metrics" # Pool inventory echo "Pool depths:" for pool in learnings patterns forge knowledge research retros; do COUNT=$(ls .agents/${pool}/*.md 2>/dev/null | wc -l | tr -d ' ') echo " $pool: $COUNT" done # Global patterns GLOBAL_COUNT=$(ls ~/.claude/patterns/*.md 2>/dev/null | wc -l | tr -d ' ') echo " global patterns: $GLOBAL_COUNT" # Check for promotion-ready learnings (see references/promotion-tiers.md) echo "See: skills/flywheel/references/promotion-tiers.md for tier definitions" fi
Step 4.5: Process Metrics (from skill telemetry)
If
.agents/ao/skill-telemetry.jsonl exists, use jq to extract: invocations by skill, average cycle time per skill, gate failure rates. Include in health report (Step 6) under ## Process Metrics.
Step 5: Validate Artifact Consistency
Cross-reference validation: scan knowledge artifacts for broken internal references. Use
scripts/artifact-consistency.sh (method documented in references/artifact-consistency.md).
Default allowlist lives at references/artifact-consistency-allowlist.txt; use --no-allowlist for a full raw audit.
Health indicator: >90% = Healthy, 70-90% = Warning, <70% = Critical.
Step 6: Write Health Report
Write to:
.agents/flywheel-status.md
# Knowledge Flywheel Health **Date:** YYYY-MM-DD ## Pool Depths | Pool | Count | Recent (7d) | |------|-------|-------------| | Learnings | <count> | <count> | | Patterns | <count> | <count> | | Research | <count> | <count> | | Retros | <count> | <count> | ## Velocity (Last 7 Days) - Sessions with extractions: <count> - New learnings: <count> - New patterns: <count> ## Artifact Consistency - References scanned: <count> - Broken references: <count> - Consistency score: <percentage>% - Status: <Healthy/Warning/Critical> ## Cache Health - Hit rate: <percentage>% - Uncited learnings: <count> - Stale (90d uncited): <count> - Status: <Healthy/Warning/Critical> ## Retrieval Quality - Live corpus coverage: <percentage or unavailable> - Live corpus learnings: <count or unavailable> - Status: <Healthy/Warning/Critical> ## Health Status <Healthy/Warning/Critical> ## Friction Points - <issue 1> - <issue 2> ## Recommendations 1. <recommendation> 2. <recommendation>
Step 7: Report to User
Tell the user:
- Overall flywheel health
- Knowledge pool depths
- Recent activity
- Any friction points
- Recommendations
Health Indicators
| Metric | Healthy | Warning | Critical |
|---|---|---|---|
| Learnings/week | 3+ | 1-2 | 0 |
| Stale artifacts | <20% | 20-50% | >50% |
| Research/plan ratio | >0.5 | 0.2-0.5 | <0.2 |
| Cache hit rate | >80% | 50-80% | <50% |
Golden Signals
Four golden signals (always shown) reveal whether knowledge is truly compounding or just accumulating noise.
ao flywheel status # table output with golden signals ao flywheel status --json # machine-readable
The Four Signals
| # | Signal | Question | Key Metric |
|---|---|---|---|
| 1 | Velocity Trend | Is σρ-δ increasing? | Linear regression slope of baseline velocities (7d/30d) |
| 2 | Citation Pipeline | Are citations useful? | % of feedback with reward > 0.6 |
| 3 | Research Closure | Is research being mined? | % orphaned research (no learning backlink) |
| 4 | Reuse Concentration | Is the whole pool active? | Gini coefficient of citation distribution |
Verdicts and Thresholds
| Signal | Healthy | Warning | Critical |
|---|---|---|---|
| Velocity Trend | compounding (slope > +0.01) | stagnant | decaying (slope < -0.01) |
| Citation Pipeline | reinforcing (>60% high-util) | inert (30-60%) | degrading (<30%) |
| Research Closure | mining (<=10% orphans) | — | hoarding (>=10% orphans) |
| Reuse Concentration | distributed (Gini<0.4, active>30%) | concentrated | dormant (Gini>0.7 or active<10%) |
Overall verdict: 3+ healthy = compounding, 3+ critical = decaying, mixed = accumulating.
Recommended Actions
| Verdict | Action |
|---|---|
| decaying | Run cycle, archive stale artifacts, increase citation via |
| accumulating | Review orphaned research (→ pipeline), improve forge quality |
| compounding | Maintain cadence. Consider capturing baselines () for trend tracking |
Cache Eviction
Read
references/cache-eviction.md for the full eviction pipeline (passive tracking → confidence decay → maturity scan → archive).
Key Rules
- Monitor regularly - flywheel needs attention; address bottlenecks early
- Feed the flywheel - run /retro and /post-mortem
- Prune stale knowledge - archive old artifacts
Examples
User says:
/flywheel — Counts pool depths, checks recent activity, validates artifact consistency, writes health report to .agents/flywheel-status.md.
Hook trigger: After /post-mortem — Compares current vs historical metrics, flags velocity drops and friction points.
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| All pool counts zero | directory missing or empty | Run or to seed knowledge pools |
| Velocity always zero | No recent extractions (last 7 days) | Run or to extract and index learnings |
| "ao CLI not available" | ao command not installed or not in PATH | Install ao CLI or use manual pool counting fallback |
| Stale artifacts >50% | Long time since last session or inactive repo | Run to audit and archive old artifacts |