Vibecosystem test-strategy

Test pyramid decision matrix, coverage targets, when to write which test type, mock vs real dependency decisions, and test ROI analysis.

install
source · Clone the upstream repo
git clone https://github.com/vibeeval/vibecosystem
Claude Code · Install into ~/.claude/skills/
T=$(mktemp -d) && git clone --depth=1 https://github.com/vibeeval/vibecosystem "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/test-strategy" ~/.claude/skills/vibeeval-vibecosystem-test-strategy && rm -rf "$T"
manifest: skills/test-strategy/SKILL.md
source content

Test Strategy

Test Pyramid Ratio Guidance

        /\
       /e2e\         ~10% — critical user flows only
      /------\
     /  integ  \     ~20% — API contracts, DB interactions
    /------------\
   /    unit      \  ~70% — pure logic, transformations, edge cases
  /--------------\

Keep the pyramid right-side-up. Inverting it (too many e2e) leads to slow, flaky CI.

Decision Matrix: Task Type → Test Type

Task TypeTest TypeTool
Pure function / utilityUnitJest / Vitest
API endpointIntegrationSupertest / httpx
Critical user flowE2EPlaywright
Data transformationProperty-basedfast-check / Hypothesis
React/Vue componentComponentTesting Library
CLI commandIntegrationexeca + assertions
Database queryIntegration (real DB)jest + pg / pytest
Cron job / schedulerUnit (mocked time)Jest fakeTimers

Mock vs Real Dependency Decision Tree

Is it an external API (Stripe, Sendgrid, etc.)?
  → YES: Always mock. Use recorded fixtures or MSW.

Is it a database?
  → Unit test context: mock (in-memory store or jest.fn())
  → Integration test context: real DB (test container or local)

Is it the file system?
  → Mock with memfs or tmp dir, then clean up.

Is it time / Date.now()?
  → Always mock. Use Jest fakeTimers or freezegun (Python).

Is it a third-party SDK wrapper you wrote?
  → Skip testing the wrapper itself, test your code's behavior.

Coverage Targets by Project Type

Project TypeBranch CoverageNotes
Published library90%+Every exported function needs tests
Production app80%+Focus on critical paths
Internal tool70%+Happy path + main error cases
Prototype / spikeSkipThrow it away anyway
Generated codeSkipDon't test codegen output

Test Naming Conventions

Jest / Vitest (describe + it)

describe('calculateDiscount', () => {
  it('returns 10% for gold members', () => { ... })
  it('returns 0% when cart is empty', () => { ... })
  it('throws when discount rate exceeds 100', () => { ... })
})

Given-When-Then (BDD style)

describe('OrderService', () => {
  describe('given a confirmed order', () => {
    describe('when the user cancels', () => {
      it('then it transitions to CANCELLED state', () => { ... })
      it('then it sends a cancellation email', () => { ... })
    })
  })
})

When NOT to Test

  • Generated code (Prisma client, GraphQL types, protobuf outputs)
  • Third-party SDK wrappers with zero custom logic
  • Trivial getters/setters (
    getEmail() { return this.email }
    )
  • Config files
  • Framework boilerplate (Next.js
    _app.tsx
    , Express server bootstrap)

Test Isolation Strategies

Transaction rollback (PostgreSQL)

beforeEach(async () => {
  await db.query('BEGIN')
})

afterEach(async () => {
  await db.query('ROLLBACK')
})

Cleanup hooks

afterEach(() => {
  jest.clearAllMocks()        // clear call counts
  jest.resetAllMocks()        // reset return values
  jest.restoreAllMocks()      // restore spied originals
})

Test containers (real DB, isolated)

import { PostgreSqlContainer } from '@testcontainers/postgresql'

let container: StartedPostgreSqlContainer

beforeAll(async () => {
  container = await new PostgreSqlContainer().start()
  process.env.DATABASE_URL = container.getConnectionUri()
})

afterAll(async () => {
  await container.stop()
})

Flaky Test Triage

When a test is flaky (passes/fails non-deterministically):

  1. Check for shared mutable state (global variables, singleton caches)
  2. Check for missing
    await
    on async calls
  3. Check for time-dependent assertions (
    setTimeout
    ,
    Date.now()
    )
  4. Check for ordering dependencies (tests relying on previous test state)
  5. Add
    --runInBand
    to isolate and confirm

Mutation Testing (Stryker)

Mutation testing verifies that your tests actually catch bugs:

npx stryker run
// stryker.config.json
{
  "mutator": { "excludedMutations": ["StringLiteral"] },
  "thresholds": { "high": 80, "low": 60, "break": 50 },
  "reporters": ["html", "progress"]
}

Mutation score < 60% means tests pass without catching real logic errors. Focus on the surviving mutants — each one is an untested code path.

Test ROI Analysis

High ROI (write these first):

  • Business logic with branching conditions
  • Error handling paths
  • Data validation functions
  • State machine transitions

Low ROI (write last or skip):

  • Simple CRUD with no custom logic
  • Pass-through adapters
  • Logging statements
  • UI cosmetic details