Harness-engineering harness-supply-chain-audit

Harness Supply Chain Audit

install

source · Clone the upstream repo

git clone https://github.com/Intense-Visions/harness-engineering

Claude Code · Install into ~/.claude/skills/

T=$(mktemp -d) && git clone --depth=1 https://github.com/Intense-Visions/harness-engineering "$T" && mkdir -p ~/.claude/skills && cp -r "$T/agents/skills/claude-code/harness-supply-chain-audit" ~/.claude/skills/intense-visions-harness-engineering-harness-supply-chain-audit-d1af28 && rm -rf "$T"

manifest: agents/skills/claude-code/harness-supply-chain-audit/SKILL.md

source content

Harness Supply Chain Audit

6-factor dependency risk evaluation adapted from Trail of Bits security skill patterns. Surfaces dependency risk flags for human review — not automated verdicts.

When to Use

Before a major release to assess dependency risk
After adding new dependencies
During security audits or compliance reviews
When
```
on_milestone
```
trigger fires (part of release gate)
NOT as a replacement for
```
npm audit
```
— this complements it with risk signals beyond CVEs
NOT for license compliance (separate concern)

Iron Law

Present findings as flags for human review, never as verdicts. A dependency flagged as "high risk" may be entirely appropriate for a project. The skill surfaces signals; humans decide whether to act.

Process

Phase 1: INVENTORY — Build Dependency List

Resolve project root. Use the path argument or default to the current directory.
Detect lockfile. Check for the following in order:
- ```
package-lock.json
```
  (npm)
- ```
pnpm-lock.yaml
```
  (pnpm)
- ```
yarn.lock
```
  (yarn)
- If none found: report "No lockfile detected. Run
```
npm install
```
  first." and stop.

Parse direct dependencies from

package.json

Read
```
dependencies
```
and
```
devDependencies
```
Build a list:
```
{ name, version, isDev }
```

Parse transitive depth from lockfile:
- For
```
package-lock.json
```
  : read
```
packages
```
  keys to extract the dependency tree. Nesting depth of
```
node_modules/
```
  segments in keys indicates transitive depth.
- For
```
pnpm-lock.yaml
```
  : read
```
importers
```
  section for direct dependencies (keyed by workspace path, e.g.,
```
.
```
  for root). Each importer lists
```
dependencies
```
  and
```
devDependencies
```
  with version specifiers. Read
```
packages
```
  section for resolved versions — keys are package identifiers (e.g.,
```
/@scope/pkg@1.2.3
```
  ) with
```
resolution
```
  (tarball URL + integrity hash) and
```
dependencies
```
  sub-map for transitives.
- For
```
yarn.lock
```
  : parse block-format entries. Each block header is
```
"pkg@version-range":
```
  followed by indented fields:
```
version
```
  (resolved),
```
resolved
```
  (tarball URL),
```
integrity
```
  (hash), and
```
dependencies
```
  sub-block listing transitive deps as
```
"name" "version-range"
```
  pairs.
- Assign each package a depth (0 = direct, 1 = first-level transitive, etc.)
- Flag packages with depth > 5 for transitive risk evaluation

Build inventory table:

INVENTORY: <project-name>
Direct dependencies: N
Dev dependencies: N
Total packages (including transitives): N
Deep transitive packages (depth > 5): N

Proceed to EVALUATE.

Phase 2: EVALUATE — Score Dependencies on 6 Factors

For each direct dependency (and any transitive with depth > 5), score on 6 factors:

Network access required: npm registry (
https://registry.npmjs.org/<pkg>
) and GitHub API (
https://api.github.com/repos/<owner>/<repo>
).
If npm registry returns 404: mark as "unresolvable", flag for manual review, skip remaining factors
If GitHub API rate limits hit: score
maintenance-status
as "unknown", continue with other factors
If no GitHub repo link in package metadata: skip
maintenance-status
factor, note in report

Factor 1: Maintainer Concentration

Fetch:
```
GET https://registry.npmjs.org/<pkg>
```
Check:
```
maintainers
```
array length
Score:
- High risk: 1 maintainer (bus factor = 1)
- Medium risk: 2-3 maintainers
- Low risk: 4+ maintainers

Factor 2: Maintenance Status

Source: npm
```
time
```
field (last publish date) + GitHub API commit activity

npm:

GET https://registry.npmjs.org/<pkg>

→

time.modified

GitHub:

GET https://api.github.com/repos/<owner>/<repo>/commits?per_page=1

→ latest commit date

Score:
- High risk: Last publish > 12 months ago AND no GitHub commits in 6 months
- Medium risk: Last publish > 12 months ago OR no commits in 6 months (not both)
- Low risk: Active in both dimensions

Factor 3: Popularity Signal

Fetch:

GET https://api.npmjs.org/downloads/point/last-week/<pkg>

Score:
- High risk: < 1,000 weekly downloads
- Medium risk: 1,000–10,000 weekly downloads
- Low risk: > 10,000 weekly downloads
- Note: Low popularity is a signal, not a verdict — internal/niche packages are expected to be low

Factor 4: Install Scripts

Read:
```
node_modules/<pkg>/package.json
```
(or lockfile-resolved path) →
```
scripts
```
field

Check for:

preinstall

postinstall

install

preuninstall

postuninstall

Score:
- High risk: Any install script present
- Low risk: No install scripts
- Note: Some install scripts are legitimate (native addon compilation). Flag for review.

Factor 5: Known CVEs

Run:
```
npm audit --json
```
or
```
pnpm audit --json
```
Parse: map findings to their package name
Score:
- Critical: Any high/critical severity CVE
- Medium risk: Moderate severity CVE
- Low risk: No CVEs or low severity only

Factor 6: Transitive Risk

Source: Lockfile depth analysis from INVENTORY phase
Score:
- High risk: Depth > 5 AND subtree size > 20 transitive packages
- Medium risk: Depth > 5 OR subtree size > 20
- Low risk: Depth ≤ 5 and subtree size ≤ 20

Risk Scoring

Combine factor scores into an overall risk level:

Overall Risk	Condition
Critical	Factor 5 is Critical (any high/critical CVE)
High	2+ factors scored High, OR Factor 1 is High + Factor 2 is High
Medium	1 factor scored High, OR 3+ factors scored Medium
Low	All factors Low or at most 1 Medium

Phase 3: REPORT — Generate Risk Summary

Produce risk summary table sorted by overall risk (Critical first):

Supply Chain Audit: <project-name>
Date: <ISO date>
Packages evaluated: N direct + M deep transitives

┌─────────────────────┬──────────┬────────────┬─────────────┬────────────┬──────┬─────────────┐
│ Package             │ Version  │ Maintainers│ Last Publish│ Downloads  │ CVEs │ Overall Risk│
├─────────────────────┼──────────┼────────────┼─────────────┼────────────┼──────┼─────────────┤
│ example-pkg         │ 1.2.3    │ 1 (HIGH)   │ 18mo (HIGH) │ 500 (MED)  │ none │ HIGH        │
│ another-pkg         │ 2.0.0    │ 12         │ 2mo         │ 50k        │ 1 mod│ MEDIUM      │
└─────────────────────┴──────────┴────────────┴─────────────┴────────────┴──────┴─────────────┘

Detail section for Critical and High risk packages:

HIGH RISK: example-pkg@1.2.3
├── Maintainer concentration: 1 maintainer (bus factor = 1)
├── Maintenance status: Last publish 18 months ago, no commits in 12 months
├── Popularity: 500 weekly downloads
├── Install scripts: none
├── Known CVEs: none
└── Transitive risk: depth 2, subtree 4 packages
Recommendation: Consider replacing with a well-maintained alternative,
or pin the version and monitor for abandonment.

Install script warnings (any package with install scripts):

INSTALL SCRIPTS DETECTED:
- node-gyp@9.4.0: postinstall — native addon compilation (likely legitimate)
- suspicious-pkg@1.0.0: postinstall — review script contents before trusting

Summary line:

RESULT: 1 Critical, 2 High, 3 Medium, N Low — Review flagged items before release

Output: Print report to stdout. If
```
--output <file>
```
was passed, also write to that file.

Gates

Stop if no lockfile. Do not evaluate without a lockfile — results will be unreliable.
Present as flags, not verdicts. Never state "this package is unsafe." State "this package has signals that warrant review."
Do not block on API failures. If npm registry or GitHub API is unavailable, note which factors were skipped and continue with available data.

Harness Integration

harness validate
— Run after creating the skill files to verify they are properly placed.
Triggers:
```
on_milestone
```
fires this skill as part of the milestone completion checklist.
Depends on:
```
harness-security-scan
```
— run after mechanical scanning to complete the security picture.
Output: Stdout report, optionally written to file via
```
--output
```
. No state files written.

Evidence Requirements

When reporting findings, cite the source for each factor:

Maintainer data:
```
registry.npmjs.org/<pkg>
```
→
```
maintainers
```
field
Publish date:
```
registry.npmjs.org/<pkg>
```
→
```
time.modified
```

Downloads:

api.npmjs.org/downloads/point/last-week/<pkg>

Install scripts:
```
node_modules/<pkg>/package.json
```
→
```
scripts
```
CVEs:
```
npm audit --json
```
output
Depth: lockfile analysis

Do not assert risk scores without citing the specific data point that generated the score.

Success Criteria

Running
```
/harness:supply-chain-audit
```
on a project with dependencies outputs a risk table with all 6 factors scored
A dependency with a sole maintainer and no commits in 12 months scores "high risk"
A dependency with a
```
postinstall
```
script is flagged in the install scripts section
API failures produce "unknown" scores with a note, not errors that stop the audit
All findings are framed as flags for human review, not automated verdicts

Escalation

If a critical CVE is found: Surface immediately — do not bury it in the table. Recommend blocking the dependency update or requiring an immediate patch before merge.
If all maintainers are unresponsive: Flag the package as abandoned and recommend finding an alternative. Include download counts to help the user assess how widely adopted the package is.
If an install script has unknown behavior: Do not guess. State that the script requires manual review and link to the script source.
If the npm or GitHub API is unavailable: Note which factors were skipped with "unknown" scores. Do not fail the audit — partial results are better than none.
If the user asks for a verdict ("is this safe?"): Decline to give a binary answer. Supply chain risk is probabilistic. Present the risk signals and let the human decide.

Rationalizations to Reject

These are common rationalizations that sound reasonable but lead to incorrect results. When you catch yourself thinking any of these, stop and follow the documented process instead.

Rationalization	Why It Is Wrong
"This package has high risk signals but it is widely used, so it must be safe"	The Iron Law: present findings as flags for human review, never as verdicts. Popularity does not eliminate bus-factor risk or maintenance abandonment.
"The npm API returned an error for this package, so I will skip it and move on"	API failures produce "unknown" scores with a note, not skips. Partial results with noted gaps are always better than incomplete audits.
"The install script is probably just native addon compilation, so I do not need to flag it"	Every install script must be flagged in the report. "Probably legitimate" is exactly the assumption that supply chain attacks exploit.

Examples

Supply Chain Audit: my-project
Date: 2026-03-31
Packages evaluated: 24 direct + 3 deep transitives (depth > 5)

CRITICAL (1):
  lodash@4.17.20 — CVE-2021-23337 (high severity, unpatched)

HIGH (2):
  abandoned-util@0.9.1 — sole maintainer, last publish 22 months ago
  sketchy-helper@2.1.0 — sole maintainer, postinstall script detected

MEDIUM (3):
  small-lib@1.0.0 — 800 weekly downloads (low popularity signal)
  ...

LOW (18): no significant risk signals

INSTALL SCRIPTS:
  node-gyp@9.4.0 — postinstall (native compilation, likely legitimate)
  sketchy-helper@2.1.0 — postinstall (REVIEW: contents unknown)

RESULT: 1 Critical, 2 High, 3 Medium, 18 Low
Next steps: Update lodash to patch CVE. Review sketchy-helper postinstall script.
Consider alternatives to abandoned-util.

Example Output

Supply Chain Audit: my-project
Date: 2026-03-31
Packages evaluated: 24 direct + 3 deep transitives (depth > 5)

CRITICAL (1):
  lodash@4.17.20 — CVE-2021-23337 (high severity, unpatched)

HIGH (2):
  abandoned-util@0.9.1 — sole maintainer, last publish 22 months ago
  sketchy-helper@2.1.0 — sole maintainer, postinstall script detected

MEDIUM (3):
  small-lib@1.0.0 — 800 weekly downloads (low popularity signal)
  ...

LOW (18): no significant risk signals

INSTALL SCRIPTS:
  node-gyp@9.4.0 — postinstall (native compilation, likely legitimate)
  sketchy-helper@2.1.0 — postinstall (REVIEW: contents unknown)

RESULT: 1 Critical, 2 High, 3 Medium, 18 Low
Next steps: Update lodash to patch CVE. Review sketchy-helper postinstall script.
Consider alternatives to abandoned-util.