Reproducible Analysis

Maintain a machine-readable journal of every data operation so that humans, agents, and machines can independently verify the analysis end-to-end.

Core Principle

Every analysis should be independently reproducible: given the same input files, a third party should be able to replay the exact sequence of operations and arrive at bit-identical results for all deterministic steps.

Journal Format

Create a journal file named

<analysis-name>.journal.jsonl

Entry Schema

{"seq": 1, "ts": "2026-03-19T14:30:00Z", "op": "index", "tool": "mcp__qsv__qsv_index", "input": "sales.csv", "input_sha256": "a1b2c3...", "input_rows": 50000, "input_cols": 12, "params": {}, "output": "sales.csv.idx", "output_sha256": "d4e5f6...", "duration_ms": 45, "note": "Create index for fast access"}
{"seq": 2, "ts": "2026-03-19T14:30:01Z", "op": "stats", "tool": "mcp__qsv__qsv_stats", "input": "sales.csv", "input_sha256": "a1b2c3...", "params": {"cardinality": true, "stats_jsonl": true}, "output": "sales.stats.csv", "output_sha256": "f7a8b9...", "duration_ms": 320, "note": "Generate stats cache with cardinality"}

Required Fields

seq

ts

op

tool

mcp__qsv__qsv_stats

mcp__qsv__qsv_sqlp

init

complete

input

input_sha256

params

output

output_sha256

duration_ms

note

Optional Fields

input_rows

mcp__qsv__qsv_count

input_cols

mcp__qsv__qsv_headers

output_rows

output_cols

delta_rows

deterministic

ai_generated

sql

error

version

How to Compute Hashes

Use

mcp__qsv__qsv_sqlp

# Via shell (when available)
shasum -a 256 sales.csv | cut -d' ' -f1

# Via qsv sqlp (for CSV content hash)
# Hash the output file after each step

Alternatively, note the file size and row count as a lighter-weight fingerprint when hashing is impractical:

{"seq": 1, "input": "huge_file.csv", "input_fingerprint": {"rows": 5000000, "cols": 42, "bytes": 1073741824}, "note": "additional fields omitted for brevity"}

Journal Lifecycle

Starting a Journal

At the beginning of any analysis, create the journal and record the environment:

{"seq": 0, "ts": "2026-03-19T14:29:55Z", "op": "init", "tool": null, "input": null, "params": {"working_dir": "/path/to/data", "qsv_version": "0.142.0 (polars-0.46.0)", "platform": "darwin-aarch64"}, "output": "analysis.journal.jsonl", "note": "Initialize reproducibility journal"}

During Analysis

Log every data operation. For each step:

1. Record the entry after the operation completes (so you have the output hash and duration)
2. Include the

   note

   field explaining the analytical reasoning — this is what makes the journal useful to human reviewers
3. Mark

   deterministic: false

   for any AI-generated step (describegpt, chart selection, narrative)

Closing a Journal

At the end, write a summary entry:

{"seq": 99, "ts": "2026-03-19T15:10:00Z", "op": "complete", "tool": null, "input": "sales.csv", "input_sha256": "a1b2c3...", "params": {"total_steps": 98, "deterministic_steps": 95, "ai_steps": 3, "final_output": "analysis_report.md"}, "output": "analysis.journal.jsonl", "note": "Analysis complete. 95 of 98 steps are deterministic and independently reproducible."}

Verification Protocol

For Humans

1. Open the

   .journal.jsonl

   file
2. Review each

   note

   field to understand the analytical reasoning
3. Check that the sequence of operations makes logical sense
4. Verify

   input_sha256

   of the first entry matches your copy of the source data
5. Spot-check any step by re-running the

   tool

   with the recorded

   params

For Agents

1. Parse the

   .journal.jsonl

   file
2. Verify the

   seq

   0 entry to confirm environment compatibility (qsv version, platform)
3. For each entry where

   deterministic

   is true (or absent): a. Compute

   sha256

   of the input file — must match

   input_sha256

   b. Execute the

   tool

   with the recorded

   params

   c. Compute

   sha256

   of the output — must match

   output_sha256

   d. If mismatch, flag the step and stop
4. For entries where

   deterministic: false

   , skip hash verification but log that the step was AI-generated
5. Report:

   N of M deterministic steps verified, K AI-generated steps skipped

For Machines (CI/CD)

#!/bin/bash
# replay-journal.sh — replay and verify a journal
JOURNAL="$1"
FAILURES=0

jq -c 'select(.seq > 0 and .op != "complete" and (.deterministic // true))' "$JOURNAL" | while read -r entry; do
  SEQ=$(echo "$entry" | jq -r '.seq')
  INPUT=$(echo "$entry" | jq -r '.input')
  EXPECTED=$(echo "$entry" | jq -r '.output_sha256')

  # Verify input hash
  ACTUAL_INPUT_HASH=$(shasum -a 256 "$INPUT" | cut -d' ' -f1)
  INPUT_HASH=$(echo "$entry" | jq -r '.input_sha256')
  if [ "$ACTUAL_INPUT_HASH" != "$INPUT_HASH" ]; then
    echo "FAIL step $SEQ: input hash mismatch"
    FAILURES=$((FAILURES + 1))
    continue
  fi

  # Re-execute and verify output hash (tool-specific replay logic here)
  # ...

  echo "PASS step $SEQ"
done

echo "$FAILURES failures"
exit $FAILURES

Integration with Commands

When any

/data-*

/data-profile

/data-clean

/data-join

/csv-query

sql

/data-validate

/data-viz

deterministic: false

/data-describe

ai_generated: true

/data-convert

Integration with GenAI Disclaimer

The journal complements the

genai-disclaimer

• The journal records what was done and enables replay
• The disclaimer communicates which parts are AI-generated vs. deterministic
• Together, they provide full transparency for stakeholders

Use the journal's

deterministic

ai_generated

Best Practices

• Always hash inputs: The input hash is the anchor for reproducibility — without it, verification is impossible
• Log failures too: If a step fails and you retry with different parameters, log both attempts (the failure with an

  error

  field, then the successful retry)
• Include row count deltas:

  delta_rows

  makes it easy to spot where data was filtered, joined, or deduplicated
• Use relative paths: All file paths should be relative to the working directory so the journal is portable
• Version pin: Record

  qsv_version

  in the init entry — different versions may produce different stats precision
• One journal per analysis: Don't append unrelated analyses to the same journal file
• Commit journals to version control: They're small (a few KB) and provide audit trail