Vibecosystem test-strategy
Test pyramid decision matrix, coverage targets, when to write which test type, mock vs real dependency decisions, and test ROI analysis.
install
source · Clone the upstream repo
git clone https://github.com/vibeeval/vibecosystem
Claude Code · Install into ~/.claude/skills/
T=$(mktemp -d) && git clone --depth=1 https://github.com/vibeeval/vibecosystem "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/test-strategy" ~/.claude/skills/vibeeval-vibecosystem-test-strategy && rm -rf "$T"
manifest:
skills/test-strategy/SKILL.mdsource content
Test Strategy
Test Pyramid Ratio Guidance
/\ /e2e\ ~10% — critical user flows only /------\ / integ \ ~20% — API contracts, DB interactions /------------\ / unit \ ~70% — pure logic, transformations, edge cases /--------------\
Keep the pyramid right-side-up. Inverting it (too many e2e) leads to slow, flaky CI.
Decision Matrix: Task Type → Test Type
| Task Type | Test Type | Tool |
|---|---|---|
| Pure function / utility | Unit | Jest / Vitest |
| API endpoint | Integration | Supertest / httpx |
| Critical user flow | E2E | Playwright |
| Data transformation | Property-based | fast-check / Hypothesis |
| React/Vue component | Component | Testing Library |
| CLI command | Integration | execa + assertions |
| Database query | Integration (real DB) | jest + pg / pytest |
| Cron job / scheduler | Unit (mocked time) | Jest fakeTimers |
Mock vs Real Dependency Decision Tree
Is it an external API (Stripe, Sendgrid, etc.)? → YES: Always mock. Use recorded fixtures or MSW. Is it a database? → Unit test context: mock (in-memory store or jest.fn()) → Integration test context: real DB (test container or local) Is it the file system? → Mock with memfs or tmp dir, then clean up. Is it time / Date.now()? → Always mock. Use Jest fakeTimers or freezegun (Python). Is it a third-party SDK wrapper you wrote? → Skip testing the wrapper itself, test your code's behavior.
Coverage Targets by Project Type
| Project Type | Branch Coverage | Notes |
|---|---|---|
| Published library | 90%+ | Every exported function needs tests |
| Production app | 80%+ | Focus on critical paths |
| Internal tool | 70%+ | Happy path + main error cases |
| Prototype / spike | Skip | Throw it away anyway |
| Generated code | Skip | Don't test codegen output |
Test Naming Conventions
Jest / Vitest (describe + it)
describe('calculateDiscount', () => { it('returns 10% for gold members', () => { ... }) it('returns 0% when cart is empty', () => { ... }) it('throws when discount rate exceeds 100', () => { ... }) })
Given-When-Then (BDD style)
describe('OrderService', () => { describe('given a confirmed order', () => { describe('when the user cancels', () => { it('then it transitions to CANCELLED state', () => { ... }) it('then it sends a cancellation email', () => { ... }) }) }) })
When NOT to Test
- Generated code (Prisma client, GraphQL types, protobuf outputs)
- Third-party SDK wrappers with zero custom logic
- Trivial getters/setters (
)getEmail() { return this.email } - Config files
- Framework boilerplate (Next.js
, Express server bootstrap)_app.tsx
Test Isolation Strategies
Transaction rollback (PostgreSQL)
beforeEach(async () => { await db.query('BEGIN') }) afterEach(async () => { await db.query('ROLLBACK') })
Cleanup hooks
afterEach(() => { jest.clearAllMocks() // clear call counts jest.resetAllMocks() // reset return values jest.restoreAllMocks() // restore spied originals })
Test containers (real DB, isolated)
import { PostgreSqlContainer } from '@testcontainers/postgresql' let container: StartedPostgreSqlContainer beforeAll(async () => { container = await new PostgreSqlContainer().start() process.env.DATABASE_URL = container.getConnectionUri() }) afterAll(async () => { await container.stop() })
Flaky Test Triage
When a test is flaky (passes/fails non-deterministically):
- Check for shared mutable state (global variables, singleton caches)
- Check for missing
on async callsawait - Check for time-dependent assertions (
,setTimeout
)Date.now() - Check for ordering dependencies (tests relying on previous test state)
- Add
to isolate and confirm--runInBand
Mutation Testing (Stryker)
Mutation testing verifies that your tests actually catch bugs:
npx stryker run
// stryker.config.json { "mutator": { "excludedMutations": ["StringLiteral"] }, "thresholds": { "high": 80, "low": 60, "break": 50 }, "reporters": ["html", "progress"] }
Mutation score < 60% means tests pass without catching real logic errors. Focus on the surviving mutants — each one is an untested code path.
Test ROI Analysis
High ROI (write these first):
- Business logic with branching conditions
- Error handling paths
- Data validation functions
- State machine transitions
Low ROI (write last or skip):
- Simple CRUD with no custom logic
- Pass-through adapters
- Logging statements
- UI cosmetic details