Agent-Friendly Score - DevTool Evaluation

Evaluate developer tools and SaaS products for compatibility with AI coding assistants (Cursor, Claude, GitHub Copilot).

When to Use

• User asks "How agent-friendly is [tool]?"
• User wants to evaluate a devtool for AI compatibility
• User is building a devtool and wants to optimize for AI assistants
• User is comparing tools and AI compatibility matters

The Agent Era Context

85% of developers now use AI tools regularly. When a developer asks Cursor to "add email functionality," the AI picks the service, writes the integration, and runs the install command.

The New Funnel:

• Traditional: Marketing → Landing → Docs → Trial → Conversion
• Agent: Problem → AI suggestion →

  npm install

  → Subscription

Tools that AI can easily work with get recommended. Tools it can't work with become invisible.

Scoring Framework

Category 1: SDK & API Design (30 points)

Criterion	Points	How to Check
SDK available for major languages	10	Check docs for JS, Python, Go, etc.
Consistent, predictable API	10	Review API reference for patterns
Complete TypeScript definitions	5	Check npm package for .d.ts files
Clear error messages	5	Test error responses

Scoring guide:

• 25-30: Excellent - AI can generate correct code first try
• 15-24: Good - AI mostly succeeds, occasional fixes needed
• 0-14: Poor - AI struggles to generate working code

Category 2: Documentation Quality (25 points)

Criterion	Points	How to Check
Docs lead with working code	10	First thing on quickstart page
Copy-paste examples work	5	Try the first 3 examples
Parseable structure (H1→H2→H3)	5	View page source/outline
No login walls on docs	5	Access docs without account

Scoring guide:

• 20-25: AI can extract and apply correctly
• 10-19: AI needs some interpretation
• 0-9: AI will likely hallucinate or fail

Category 3: Training Data Presence (20 points)

Criterion	Points	How to Check
GitHub repos using this tool	8	Search GitHub for imports
Stack Overflow presence	6	Search SO for [tool] questions
Tutorial/blog coverage	6	Search "[tool] tutorial"

Scoring guide:

• 15-20: Strong training data signal
• 8-14: Moderate presence
• 0-7: AI may not know this tool well

Category 4: MCP Integration (15 points)

Criterion	Points	How to Check
Official MCP server exists	10	Check mcp.so, official docs
MCP server is maintained	3	Recent commits, version updates
MCP server is discoverable	2	Listed on MCP.so or npm

Scoring guide:

• 12-15: Full agent workflow integration
• 5-11: Partial integration
• 0-4: Not in agent workflow

Category 5: Time to Working (10 points)

Criterion	Points	How to Check
Install to "hello world" <5 min	5	Time yourself following quickstart
No complex onboarding	3	Can start without account?
Sensible defaults	2	Works without config?

Scoring guide:

• 8-10: Instant productivity
• 4-7: Reasonable setup
• 0-3: Significant friction

Evaluation Workflow

Step 1: Identify the Tool

Get from user:

• Tool name and URL
• Category (email, auth, database, etc.)
• Main competitor to compare against

Step 2: SDK Evaluation

Check official SDK:

# Check npm for TypeScript types
npm info [package] types

# Check for SDK in multiple languages
# Visit: github.com/[org] and look for SDK repos

Test API consistency:

• Are endpoints predictable? (e.g.,

  /users

  ,

  /users/:id

  )
• Are responses consistent?
• Are errors structured?

Step 3: Documentation Audit

Visit docs and check:

• First code example is within scroll view
• Examples include all necessary imports
• Examples actually work when copied
• Structure uses semantic headings
• No authentication required to view

Step 4: Training Data Check

Search GitHub:

"import { X } from '[package]'" language:JavaScript
"from [package] import" language:Python

Search Stack Overflow:

[tool] is:question

Step 5: MCP Check

Search for MCP server:

• https://mcp.so - search for tool name
• Official docs - search for "MCP" or "Model Context Protocol"
• GitHub - search "[tool] mcp server"

Step 6: Time Test

Follow quickstart:

1. Start timer
2. Follow official quickstart exactly
3. Stop when first API call succeeds
4. Record time and friction points

Output Template

# Agent-Friendly Score: [Tool Name]

## Overall Score: [X]/100

| Category         | Score | Max |
| ---------------- | ----- | --- |
| SDK & API Design | X     | 30  |
| Documentation    | X     | 25  |
| Training Data    | X     | 20  |
| MCP Integration  | X     | 15  |
| Time to Working  | X     | 10  |

## Grade: [A/B/C/D/F]

- A (85-100): AI will recommend and integrate correctly
- B (70-84): AI will usually succeed
- C (55-69): AI needs help, may hallucinate
- D (40-54): Significant AI compatibility issues
- F (<40): AI will struggle or avoid

## Breakdown

### SDK & API Design ([X]/30)

**Strengths:**

- [What works well]

**Gaps:**

- [What's missing]

### Documentation ([X]/25)

**Strengths:**

- [What works well]

**Gaps:**

- [What's missing]

### Training Data Presence ([X]/20)

- GitHub repos found: [X]
- Stack Overflow questions: [X]
- Tutorial coverage: [High/Medium/Low]

### MCP Integration ([X]/15)

- MCP server: [Official/Community/None]
- Status: [Active/Stale/N/A]

### Time to Working ([X]/10)

- Quickstart time: [X minutes]
- Friction points: [list]

## Recommendations

### Quick Wins (High Impact, Low Effort)

1. [Recommendation]
2. [Recommendation]

### Strategic Improvements

1. [Recommendation]
2. [Recommendation]

## Competitor Comparison

| Metric          | [Tool] | [Competitor] |
| --------------- | ------ | ------------ |
| Agent Score     | X/100  | Y/100        |
| MCP Server      | Yes/No | Yes/No       |
| Time to Working | X min  | Y min        |

## Verdict

[One paragraph summary of whether this tool is positioned for the agent era]

Benchmarks by Category

Email APIs

Tool	Typical Score	Notes
Resend	85-90	MCP, clean SDK, great docs
Postmark	80-85	MCP, enterprise-ready
SendGrid	60-70	No official MCP, legacy API

Authentication

Tool	Typical Score	Notes
Clerk	85-90	MCP, great DX
Auth0	75-80	MCP, but complex
Firebase Auth	80-85	MCP, Google ecosystem

Databases

Tool	Typical Score	Notes
Supabase	85-90	MCP, hosted, great DX
Neon	85-90	MCP, serverless
PlanetScale	75-80	MCP (read-only)

Use these as calibration when scoring.

Key Insight

The best technical product doesn't always win anymore. The most AI-accessible product wins. When an AI assistant can:

1. Understand your docs
2. Generate working code
3. Integrate via MCP

...you're in the conversation. When it can't, you're invisible to the fastest-growing developer segment.

---

By Kimmo Ihanus | kimmoihanus.com