Awesome-omni-skill testing

Lingua's testing tools for transformation coverage and end-to-end SDK validation.

install

source · Clone the upstream repo

git clone https://github.com/diegosouzapw/awesome-omni-skill

Claude Code · Install into ~/.claude/skills/

T=$(mktemp -d) && git clone --depth=1 https://github.com/diegosouzapw/awesome-omni-skill "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/ai-agents/testing" ~/.claude/skills/diegosouzapw-awesome-omni-skill-testing && rm -rf "$T"

manifest: skills/ai-agents/testing/SKILL.md

source content

Lingua Testing

Two complementary testing approaches for validating cross-provider transformations.

Overview

Tool Purpose Uses API? When to Use

coverage-report

Field-level roundtrip coverage

Finding missing/changed fields

lingua-capture

test:transforms

Real SDK validation

Capture only

Validating against real APIs

Coverage Report (Rust)

Tests transformations roundtrip without calling APIs. Detects lost, added, or changed fields.

Quick Iteration (Compact Mode)

# After code changes - fast validation
cargo run --bin coverage-report -- -f compact

# Filter to specific providers
cargo run --bin coverage-report -- -f compact -p anthropic,responses

# Filter to specific test case
cargo run --bin coverage-report -- -f compact -t "reasoning*"

Full Investigation (Markdown Mode)

# Initial investigation with full error details
cargo run --bin coverage-report

# Save to bug file for planning
cargo run --bin coverage-report -- -p anthropic > .claude/anthropic_bugs.md

Reading Compact Output

Stats: 669/1704 (39.3%) [512+157lim] 1035fail

[P1] L:usage.prompt_cache_tokens (123)
  ant→ggl: cacheParam (response)...(+44)

```
L:
```
= Lost fields,
```
A:
```
= Added,
```
C:
```
= Changed
```
(123)
```
= affected test cases
Abbreviations:
```
oai
```
=ChatCompletions,
```
ant
```
=Anthropic,
```
rsp
```
=Responses,
```
ggl
```
=Google,
```
bed
```
=Bedrock

SDK Transforms Testing (TypeScript)

Validates against real provider SDKs with schema validation at every step.

Transformation Pairs

```
chat-completions ↔ anthropic
```
```
responses ↔ anthropic
```

Capture Responses

Calls real SDKs (requires API keys):

cd payloads

pnpm lingua-capture              # Capture missing only
pnpm lingua-capture --force      # Recapture all
pnpm lingua-capture toolCall     # Filter by name

Output:

transforms/{source}_to_{target}/{caseName}.json

Run Tests

Uses captured responses (no API calls):

cd payloads

pnpm test:transforms             # Run all
pnpm test:transforms:update      # Update snapshots
pnpm test:transforms:watch       # Watch mode

What It Validates

Request transform → validates output against target schema
Load captured response → validates against target response schema
Response transform back → validates against source schema
Snapshots → detects unintended output changes

Known Incompatibilities

Some transformations are impossible. Document in

transform_errors.json

{
  "responses_to_anthropic": {
    "reasoningRequestTruncated": "max_tokens (100) < budget_tokens (1024 min)"
  }
}

These tests pass silently.

Workflow

Adding a New Test Case

Add case to

payloads/cases/

newFeature: {
  "chat-completions": { model: "...", messages: [...] },
  anthropic: { model: "...", max_tokens: 1024, messages: [...] },
}

Check coverage:

cargo run --bin coverage-report -- -f compact -t newFeature

Capture SDK responses:

cd payloads && pnpm lingua-capture newFeature

Run transform tests:

pnpm test:transforms

Fixing a Transformation Bug

Investigate with full coverage report:

cargo run --bin coverage-report -- -p anthropic,responses > .claude/fix_bugs.md

Iterate with compact mode:

cargo run --bin coverage-report -- -f compact -p anthropic,responses

Verify SDK compatibility:

cd payloads && pnpm test:transforms

If snapshots changed intentionally:

pnpm test:transforms:update