PDF Miner Skill

Extract text and tables from PDF files using

pdfplumber

Prerequisites

python -m pip install pdfplumber

For OCR capabilities (scanned/image PDFs), also install:

python -m pip install pymupdf openai

Initial Setup for OCR

Before using

--ocr

1. Environment variables (recommended for temporary use):

   export OCR_API_KEY="your-openrouter-api-key"
   export OCR_MODEL="qwen/qwen3.6-plus:free"   # optional
   export OCR_BASE_URL="https://openrouter.ai/api/v1"   # optional
   

2. Config file (persistent, skill-specific):
   Create

   skills/skills/pdf-miner/config.json

   with:

   {
     "vision_api_key": "your-openrouter-api-key",
     "vision_model": "qwen/qwen3.6-plus:free",
     "vision_base_url": "https://openrouter.ai/api/v1"
   }
   

3. Command-line arguments (override per invocation):

   python scripts/extract_pdf.py scanned.pdf --ocr --ocr-api-key "sk-..." --ocr-model "stepfun/step-3.5-flash:free"
   

Usage

Run commands from this skill directory.

Basic Extraction

# Full extraction (text + tables)
python scripts/extract_pdf.py input.pdf

# Output to custom path
python scripts/extract_pdf.py input.pdf output.md

# Specific pages
python scripts/extract_pdf.py input.pdf --pages 1-5,10,15-20

# Text or tables only
python scripts/extract_pdf.py input.pdf --text-only
python scripts/extract_pdf.py input.pdf --tables-only
python scripts/extract_pdf.py input.pdf --tables-only --json

Advanced Modes

# Search: find pages containing keywords with context
python scripts/extract_pdf.py report.pdf --search "Vietnam export penetration"

# Metrics: extract lines with keywords + numeric values
python scripts/extract_pdf.py report.pdf --metrics "market size growth export penetration"

# TOC: extract table of contents / chapter structure (robust, multi-format)
python scripts/extract_pdf.py report.pdf --toc
# Optionally adjust sensitivity (default: 3 entries per page required)
python scripts/extract_pdf.py report.pdf --toc --toc-min-entries 2

# Diff: compare two PDFs, show pages unique to each
python scripts/extract_pdf.py old_version.pdf new_version.pdf --diff

# Chunk: split output into LLM-friendly chunks
python scripts/extract_pdf.py report.pdf --chunk             # single file, 8000 chars each
python scripts/extract_pdf.py report.pdf --chunk --max-chars 4000
python scripts/extract_pdf.py report.pdf --chunk --output-dir ./chunks   # separate files

# Clean headers/footers
python scripts/extract_pdf.py report.pdf --clean-headers

# Batch: process multiple PDFs
python scripts/extract_pdf.py file1.pdf file2.pdf file3.pdf --output-dir ./extracted

OCR for Scanned/Image PDFs (Automatic by Default)

OCR is automatically triggered for pages with very little extractable text (default threshold: 100 characters). This helps handle scanned or image-based PDFs without requiring the

--ocr

Usage Examples

# Automatic OCR (default behavior)
python scripts/extract_pdf.py scanned.pdf

# Force OCR on all pages (ignore text length)
python scripts/extract_pdf.py scanned.pdf --ocr

# Force OCR only on specific pages
python scripts/extract_pdf.py scanned.pdf --ocr --ocr-pages 1-5,10

# Adjust OCR quality (DPI)
python scripts/extract_pdf.py scanned.pdf --ocr --ocr-dpi 300

# Use a different vision model
python scripts/extract_pdf.py scanned.pdf --ocr --ocr-model "stepfun/step-3.5-flash:free"

# Disable automatic OCR detection (if you want pure extraction only)
python scripts/extract_pdf.py file.pdf --no-auto-ocr

# Change the low-text threshold (default 100 chars)
python scripts/extract_pdf.py file.pdf --ocr-threshold 200

Configuration

OCR requires a vision API key. See Initial Setup for OCR.

--ocr

--ocr-pages

--auto-ocr

--no-auto-ocr

--no-auto-ocr

--ocr-pages

--ocr

--ocr-threshold

--ocr-dpi

--ocr-api-key

--ocr-base-url

--ocr-model

Troubleshooting

OCR failed with "No API key"
→ Configure your API key in

config.json

OCR_API_KEY

OCR model rejects images
→ The configured model might not support vision. Choose a vision-capable model (e.g.,

qwen/qwen3.6-plus:free

stepfun/step-3.5-flash:free

Too many pages being OCR'd
→ Increase the threshold:

--ocr-threshold 300

--no-auto-ocr

--ocr-pages

Rate limit errors
→ Reduce concurrent OCR calls, switch to a paid model tier, or try a different provider.

Configuration Reference

OCR_API_KEY

OCR_API_KEY

config.json

vision_api_key

OCR_MODEL

qwen/qwen3.6-plus:free

OCR_MODEL

config.json

vision_model

OCR_BASE_URL

https://openrouter.ai/api/v1

OCR_BASE_URL

config.json

vision_base_url

Precedence: CLI argument > environment variable >

config.json

Tool Comparison

Modes Reference

--search "kw1 kw2"

--metrics "kw1 kw2"

--toc

--toc-min-entries N

--diff

--chunk

--max-chars N

--clean-headers

file1 file2 ...

--output-dir

Output Options

--output-dir ./dir

--chunk --output-dir

--context N

--max-chars N

--header-lines "a" "b"

Workflow

1. Download PDF (if URL)

import urllib.request
urllib.request.urlretrieve(url, "report.pdf")

2. Extract

Run from this skill directory:

cd <skill-directory>
python scripts/extract_pdf.py /path/to/report.pdf [options]

3. Read & Answer

Read the output

.md

4. Clean Up

Delete temporary PDF and

.md

Limitations

• Scanned/image-based PDFs: Cannot extract text without OCR. Install OCR dependencies and configure an API key.
• Embedded charts/graphs: Only text labels extracted, not chart data.
• Multi-column layouts: Use

  --layout

  flag for improved reading order via x_tolerance.
• TOC detection: Robust multi-format matching with validation. Very non-standard layouts may still require manual extraction.
• Diff: Uses text similarity (Jaccard on normalized lines), not page numbers. Threshold adjustable via

  --diff-threshold N

  (default 0.8).

Troubleshooting

OCR fails with "No API key"
→ Set

OCR_API_KEY

config.json

OCR model rejects images
→ The configured model may not support vision; either choose a vision-capable model (e.g.,

qwen/qwen3.6-plus:free

stepfun/step-3.5-flash:free

Rate limit errors
→ Reduce concurrent calls, switch to a paid tier, or try a different model provider.