Counterfactual Reasoning

Simulate alternative realities. The logic of "what if" and decision evaluation.

Type Signature

Counterfactual : Actual → Intervention → Alternative → Comparison

Where:
  Actual       : Decision × Outcome → ActualWorld
  Intervention : ActualWorld × Δ → ModifiedPremise
  Alternative  : ModifiedPremise → ProjectedOutcome
  Comparison   : (ActualWorld, ProjectedOutcome) → DifferenceAnalysis

When to Use

Use counterfactual when:

• Evaluating past decisions ("Should we have...")
• Scenario planning ("What if X happens...")
• Comparing options not taken ("If we had chosen...")
• Strategic simulation ("If competitor does X...")
• Learning from outcomes ("Was our decision right?")

Don't use when:

• Executing known process → Use Causal
• Explaining observation → Use Abductive
• Resolving disagreement → Use Dialectical

Core Principles

Minimal Intervention

Change only what's necessary to test the hypothesis:

• Modify one variable at a time where possible
• Keep everything else constant (ceteris paribus)
• Trace downstream effects carefully

Probability Weighting

Alternative outcomes aren't certain:

• Assign probability to each projected outcome
• Consider multiple possible alternatives per intervention
• Avoid overconfidence in projections

Asymmetry Awareness

Counterfactual analysis has inherent biases:

• Hindsight makes alternatives seem clearer
• Survivors don't see paths that led to failure
• Confidence in projections often too high

Four-Stage Process

Stage 1: Actual World

Purpose: Document the decision made and observed outcome.

Components:

actual:
  decision:
    what: "The choice that was made"
    when: ISO8601
    who: "Decision maker(s)"
    context: "Circumstances at decision time"
    alternatives_considered: [string]  # At the time
    
  outcome:
    result: "What actually happened"
    metrics:
      - metric: "Measurable outcome"
        value: number
        expected: number  # What was predicted
    timeline: "How long to outcome"
    
  assessment:
    success_level: high | medium | low | failed
    surprise_level: 0.0-1.0  # How unexpected
    
  causal_chain:
    - step: "Decision led to X"
    - step: "X led to Y"
    - step: "Y produced outcome"

Example:

actual:
  decision:
    what: "Priced enterprise tier at $50K/year"
    when: "2024-06-01"
    who: "Founders"
    context: "First enterprise launch, no market data"
    alternatives_considered:
      - "$30K/year (lower barrier)"
      - "$75K/year (higher margin)"
      - "Usage-based pricing"
      
  outcome:
    result: "Closed 3 deals in 6 months, $150K ARR"
    metrics:
      - metric: "Deals closed"
        value: 3
        expected: 5
      - metric: "ARR"
        value: 150000
        expected: 250000
      - metric: "Sales cycle"
        value: 120  # days
        expected: 90
    timeline: "6 months"
    
  assessment:
    success_level: medium
    surprise_level: 0.4  # Somewhat below expectations
    
  causal_chain:
    - step: "$50K price point set"
    - step: "3/5 prospects required CFO approval at this level"
    - step: "CFO approval added 30 days to cycle"
    - step: "2 deals lost to budget cycle timing"

Stage 2: Intervention

Purpose: Define the alternative decision to evaluate.

Intervention Types:

Type	Description	Example
Price	Different pricing decision	"$30K instead of $50K"
Timing	Earlier or later action	"Launched 3 months earlier"
Strategy	Different strategic choice	"SMB-first instead of enterprise"
Resource	Different allocation	"Hired sales earlier"
Partner	Different relationship	"Partnered with X instead of Y"

Components:

intervention:
  what: "The alternative choice"
  
  change:
    variable: "What's being changed"
    from: "Actual value"
    to: "Alternative value"
    
  rationale:
    why_consider: "Why this alternative is worth evaluating"
    was_available: bool  # Was this actually an option at the time?
    
  assumptions:
    held_constant:
      - "What we assume stays the same"
    ripple_effects:
      - "Expected downstream changes"

Example:

intervention:
  what: "Price at $30K/year instead of $50K"
  
  change:
    variable: "Enterprise tier annual price"
    from: "$50,000"
    to: "$30,000"
    
  rationale:
    why_consider: "Test if lower price would have increased velocity"
    was_available: true  # This was considered at the time
    
  assumptions:
    held_constant:
      - "Same product features"
      - "Same sales team"
      - "Same market conditions"
      - "Same target customer profile"
    ripple_effects:
      - "Different approval threshold (manager vs CFO)"
      - "Potentially different customer expectations"
      - "Lower margin per deal"

Stage 3: Alternative Projection

Purpose: Project what would have happened under the intervention.

Projection Method:

1. Identify decision point - Where paths diverge
2. Trace causal chain - What changes downstream?
3. Estimate outcomes - With probability weights
4. Consider multiple scenarios - Best/worst/expected

Components:

alternative:
  scenarios:
    - name: "Expected case"
      probability: 0.6
      outcome:
        deals: 6  # vs actual 3
        arr: 180000  # vs actual 150000
        cycle: 75  # days, vs actual 120
      reasoning: "Lower price = faster approval, more deals, but lower $ each"
      
    - name: "Optimistic case"
      probability: 0.25
      outcome:
        deals: 8
        arr: 240000
        cycle: 60
      reasoning: "Volume effect stronger than expected"
      
    - name: "Pessimistic case"
      probability: 0.15
      outcome:
        deals: 4
        arr: 120000
        cycle: 90
      reasoning: "Lower price signals lower value, some prospects hesitate"
      
  weighted_outcome:
    deals: 6.0  # (6×0.6 + 8×0.25 + 4×0.15)
    arr: 178000
    cycle: 74
    
  causal_reasoning:
    - "At $30K, most prospects can approve at director level"
    - "Director approval takes ~45 days vs CFO 90+ days"
    - "Faster cycle = more deals in same period"
    - "But: lower price per deal = lower total ARR per deal"
    
  confidence: 0.65  # How confident in this projection
  
  key_uncertainties:
    - "Would lower price attract different (worse?) customers?"
    - "Would sales team close at same rate at lower price?"
    - "Would competitors have responded differently?"

Stage 4: Comparison

Purpose: Compare actual vs alternative, extract insights.

Components:

comparison:
  quantitative:
    - metric: "Deals"
      actual: 3
      alternative: 6.0
      difference: "+3 (100%)"
      direction: better
      
    - metric: "ARR"
      actual: 150000
      alternative: 178000
      difference: "+$28K (19%)"
      direction: better
      
    - metric: "Sales cycle"
      actual: 120
      alternative: 74
      difference: "-46 days (38%)"
      direction: better
      
    - metric: "ARR per deal"
      actual: 50000
      alternative: 29667
      difference: "-$20K (41%)"
      direction: worse
      
  qualitative:
    better_in_alternative:
      - "Faster sales velocity"
      - "Lower customer acquisition cost"
      - "More reference customers faster"
      
    worse_in_alternative:
      - "Lower margin per customer"
      - "Potentially lower perceived value"
      - "Less room for discounting"
      
  verdict:
    assessment: "Alternative likely better overall"
    confidence: 0.65
    caveat: "Lower price creates different customer dynamics long-term"
    
  insight:
    learning: "At this stage, velocity matters more than margin"
    applies_to: "Early enterprise sales with unproven product"
    recommendation: "Consider price reduction or tier restructuring"
    
  action_implication:
    retrospective: "Pricing decision was suboptimal but not catastrophic"
    prospective: "For next segment, start lower and raise after validation"

Quality Gates

Gate	Requirement	Failure Action
Actual documented	Outcome with metrics	Gather actual data
Intervention minimal	Single variable change	Simplify intervention
Scenarios weighted	Probabilities sum to 1.0	Adjust probabilities
Confidence bounded	State uncertainty explicitly	Add confidence intervals
Insight actionable	Clear learning for future	Extract practical lesson

Intervention Validity

Not all counterfactuals are useful:

Valid interventions:

• Was actually an option at the time
• Changes something controllable
• Has traceable downstream effects
• Provides actionable insight

Invalid interventions:

• "What if we had known X" (not available info)
• "What if competitor hadn't existed" (not controllable)
• "What if market was bigger" (not a decision)

Common Failure Modes

Failure	Symptom	Fix
Hindsight bias	Alternative seems obviously better	Account for what was knowable at decision time
Single scenario	Only one alternative considered	Generate multiple scenarios with probabilities
Overconfidence	High certainty in projections	Widen confidence intervals
Untraceable	Can't explain why alternative differs	Build explicit causal chain
Fantasy	Intervention wasn't actually available	Verify intervention was feasible

Multiple Interventions

For complex decisions, evaluate multiple alternatives:

interventions:
  - name: "Lower price ($30K)"
    outcome: {arr: 178000, deals: 6}
    
  - name: "Higher price ($75K)"
    outcome: {arr: 150000, deals: 2}
    
  - name: "Usage-based pricing"
    outcome: {arr: 200000, deals: 4}
    confidence: 0.5  # Higher uncertainty
    
comparison_matrix:
  best_arr: "Usage-based"
  best_velocity: "Lower price"
  best_margin: "Higher price"
  best_overall: "Lower price (velocity matters most at this stage)"

Output Contract

counterfactual_output:
  actual:
    decision: string
    outcome: {result: string, metrics: [Metric]}
    success_level: string
    
  intervention:
    what: string
    change: {variable: string, from: any, to: any}
    was_available: bool
    
  alternative:
    scenarios: [Scenario]
    weighted_outcome: {metric: value}
    confidence: float
    
  comparison:
    quantitative: [{metric: string, actual: any, alternative: any, direction: string}]
    verdict: string
    confidence: float
    
  insight:
    learning: string
    applies_to: string
    recommendation: string
    
  action:
    retrospective: string  # What does this mean for past decision
    prospective: string    # What does this mean for future decisions
    
  next:
    suggested_mode: ReasoningMode  # Usually causal
    canvas_updates: [string]
    experiments_to_run: [string]
    
  trace:
    interventions_evaluated: int
    confidence_average: float
    duration_ms: int

Example Execution

Context: "Should we have taken the Series A when offered 18 months ago?"

Stage 1 - Actual:

Decision: Declined $5M Series A at $20M valuation
Outcome: Bootstrapped to $600K ARR, now raising at $30M valuation
Success level: Medium-high (slower growth, higher ownership)

Stage 2 - Intervention:

What: Accepted $5M Series A
Change: Funding status from bootstrapped to funded
Was available: Yes, term sheet was on the table

Stage 3 - Alternative:

Scenarios:
  - Expected (60%): $1.5M ARR now, but 25% dilution
  - Optimistic (25%): $2M ARR, enterprise sales team
  - Pessimistic (15%): $800K ARR, burned capital on wrong bets

Weighted: $1.4M ARR, 75% ownership vs current $600K ARR, 100% ownership

Stage 4 - Comparison:

ARR: Alternative 133% higher
Ownership value: Alternative $31.5M (75% × $42M) vs Actual $30M (100% × $30M)
Net: Roughly equivalent in value, different risk profiles

Verdict: Decision was reasonable given risk tolerance
Insight: Bootstrapping is viable if willing to accept slower growth
Recommendation: Current path validated, continue unless growth accelerates