Apify Actor Development

Build serverless Apify actors for web scraping, browser automation, and data extraction using Python.

Prerequisites & Setup (MANDATORY)

Before creating or modifying actors, verify that

apify

CLI is installed: Run

apify --help

.

If it is not installed, you can run:

curl -fsSL https://apify.com/install-cli.sh | bash

# Or (Mac): brew install apify-cli
# Or (Windows): irm https://apify.com/install-cli.ps1 | iex
# Or: npm install -g apify-cli

When the apify CLI is installed, check that it is logged in with:

apify info  # Should return your username

If it is not logged in, check if the APIFY_TOKEN environment variable is defined (if not, ask the user to generate one on https://console.apify.com/settings/integrations and then define APIFY_TOKEN with it).

Then run:

apify login -t $APIFY_TOKEN

Quick Start Workflow

Creating a New Actor

1. Copy template - Copy all files including hidden ones from the skill's

   assets/python-template/

   directory to your new actor directory. The template is located at

   {base_dir}/assets/python-template/

   where

   {base_dir}

   is the skill's base directory.
2. Setup pre-commit - Run

   uv run pre-commit install

   for automatic quality checks
3. Add dependencies - Use

   uv add package-name

   for each required dependency
4. Implement logic - Write the actor code in

   src/main.py

   (the

   src/__main__.py

   entry point is already set up)
5. Configure schemas - Update input/output schemas in

   .actor/input_schema.json

   and

   .actor/output_schema.json

6. Configure platform settings - Update

   .actor/actor.json

   with actor metadata
7. Write documentation - Create comprehensive

   .actor/ACTOR.md

   for the marketplace
8. Test locally - Run

   apify run

   to verify functionality
9. Deploy - Run

   apify push

   to deploy the actor on the Apify platform

CRITICAL REMINDERS:

• NEVER create

  requirements.txt

• NEVER use

  pip install

  or

  uv pip install

• ALWAYS use

  uv add

  to add dependencies
• ALWAYS use

  uv sync

  to install dependencies
• ALWAYS format with

  uv run ruff format .

  after file changes
• ALWAYS lint with

  uv run ruff check --fix .

  after file changes
• ALWAYS check the

  apify push

  output for build errors before considering deployment complete
• Input/output schemas should be updated when changing actor functionality

Core Concepts

Input/Output Pattern

Every actor follows this pattern:

1. Input: JSON from key-value store (defined by input schema)
2. Process: Actor logic extracts/transforms data
3. Output: Results pushed to dataset or key-value store

Storage Types

• Dataset: Structured data (arrays of objects) - use for scraping results and tabular data
• Key-Value Store: Arbitrary data (files, objects) - use for screenshots, PDFs, state, and binary files
• Request Queue: URLs to crawl - use for deep web crawling and multi-page scraping workflows

Project Structure

my-actor/
├── .actor/
│   ├── actor.json                    # Actor metadata
│   ├── input_schema.json             # Input schema
│   ├── output_schema.json            # Output schema
│   ├── ACTOR.md                      # PUBLIC marketplace documentation (CRITICAL)
│   └── datasets/
│       └── dataset_schema.json       # Dataset schema with views
├── src/ or package_name/             # Source code
│   ├── __init__.py
│   ├── __main__.py                   # Entry point for CLI (REQUIRED)
│   └── main.py                       # Main actor logic
├── tests/                            # Test files
│   └── test_*.py
├── .dockerignore                     # Docker build exclusions
├── .pre-commit-config.yaml           # Pre-commit hooks
├── Dockerfile                        # Container config
├── pyproject.toml                    # Python project config
├── uv.lock                          # Dependency lock file
└── README.md                         # Development docs

Common Patterns

See

references/python-sdk.md

for complete examples of:

• Simple HTTP scraping with BeautifulSoup
• Browser automation with Playwright and Selenium
• Deep crawling with Request Queue
• Proxy management and error handling
• Storage APIs (Dataset, Key-Value Store, Request Queue)

Input Schema Design

Input schemas use JSON Schema format to define and validate actor inputs. See

references/input-schema.md

for:

• Field types (string, number, boolean, array, object)
• Special editors (requestListSources, globs, pseudoUrls, proxy, json, textarea)
• Validation patterns (regex, length, range, required fields)
• Complete examples with best practices

Key principles:

• Always include descriptions and examples
• Provide examples for all fields
• Set sensible defaults for ease of use
• Use appropriate editors for better UX
• Add units for numeric fields (pages, seconds, MB)

Output Schema Design

Output schemas define where actors store outputs and provide templates for accessing that data. See

references/output-schema.md

for:

• Schema structure and template variables (links.apiDefaultDatasetUrl, links.apiDefaultKeyValueStoreUrl, etc.)
• Dataset and key-value store output configurations
• Multiple output types in a single actor
• Integration with Python code
• Complete examples with emojis and descriptions

Key principles:

• Define all outputs explicitly (even if empty)
• Use descriptive titles with emojis for visual clarity
• Include helpful descriptions for users and LLM integrations
• Match templates to actual storage locations in code

ACTOR.md Documentation (CRITICAL)

The

.actor/ACTOR.md

file is the public-facing documentation that users see in the Apify marketplace. This is your actor's main sales page and user guide.

Required sections:

1. Title & Description - Clear, compelling one-liner
2. What it does - Bullet points of key capabilities
3. Input - Example JSON with field explanations
4. Output - Example JSON showing expected results
5. Use Cases - Who benefits and why (with emojis)
6. Standby Mode (if applicable) - API usage examples
7. Tips & Best Practices - Performance and configuration guidance

See

assets/python-template/.actor/ACTOR.md

for a complete template.

Key principles:

• Write for non-technical users - assume no coding knowledge
• Use emojis to make sections scannable (🎯 🔍 ⚡ 🚀)
• Provide copy-paste ready code examples
• Show actual input/output samples, not schemas
• Highlight benefits and use cases clearly

Modifying Existing Actors

When modifying an existing actor:

1. Understand current logic - Read

   src/main.py

2. Check input schema - Review

   .actor/input_schema.json

   for expected inputs
3. Add dependencies with uv - Use

   uv add package-name

   (NEVER pip install)
4. Make code changes - Implement the requested features
5. Format code - Run

   uv run ruff format .

   (MANDATORY)
6. Lint code - Run

   uv run ruff check --fix .

   (MANDATORY)
7. Test changes locally - Use

   apify run

   before deploying
8. Update schema if needed - Add new fields to input schema
9. Deploy - Push changes with

   apify push

Debugging Actors

1. Test locally - Use

   apify run

   to test actor locally before deployment
2. Check storage - Inspect

   ./storage/

   directory for datasets, key-value stores, and request queues
3. Add logging - Use

   Actor.log.info()

   ,

   Actor.log.debug()

   ,

   Actor.log.error()

   (see SDK references)
4. View logs on platform - Check actor run logs in Apify Console for production issues

Best Practices

Code Quality

• Validate input - Always check required fields and formats with clear error messages
• Handle errors - Use try/catch with proper error logging and graceful degradation
• Structured logging - Use Actor.log with extra fields for better debugging
• Type hints - Add type annotations for better code clarity and IDE support
• Docstrings - Document functions and modules for maintainability
• Format with ruff - ALWAYS run

  uv run ruff format .

  before committing
• Lint with ruff - ALWAYS run

  uv run ruff check --fix .

  before deploying

Performance & Scalability

• Batch processing - Push data in batches (100-1000 items) for large datasets to reduce API calls
• Use proxies - Avoid IP blocking for web scraping with proxy configuration
• Resource limits - Set appropriate memory limits and timeouts in

  .actor/actor.json

• Optimize Docker - Use multi-stage builds, bytecode compilation, and minimal base images
• Consider Standby mode - For low-latency (<100ms), high-frequency use cases

Security & Configuration

• Environment variables - Never hardcode secrets; use

  Actor.config

  and environment variables
• Input validation - Use JSON Schema patterns, required fields, and runtime validation
• Run as non-root - Use

  myuser

  in Dockerfile for container security
• Minimize image size - Use

  .dockerignore

  to exclude unnecessary files and reduce build time

Development Workflow

• Testing - Write tests with pytest; use coverage and snapshot testing for reliability
• Pre-commit hooks - Use ruff and pre-commit for consistent code quality (MANDATORY)
• Use uv exclusively - NEVER use pip or requirements.txt; only use

  uv add

  and

  uv sync

  (MANDATORY)
• Lock dependencies - Always commit

  uv.lock

  for reproducible builds (MANDATORY)
• Test locally - Always test with

  apify run

  before deploying to catch issues early
• Dataset schemas - Define

  dataset_schema.json

  with views for better Apify Console UI
• CLI support - Add CLI entry points via

  __main__.py

  for local testing and development

Standby Mode (Real-time API)

Standby mode allows actors to run as persistent HTTP servers, providing instant responses without cold start delays.

Perfect for:

• Real-time APIs requiring <100ms response times
• Webhook endpoints that need immediate processing
• High-frequency requests (multiple requests per second)
• Integration with real-time services (Slack bots, chat applications, webhooks)
• Low-latency scraping APIs and on-demand data extraction

See

references/standby-mode.md

for complete implementation patterns, authentication, and examples.

References

Detailed documentation in

references/

:

• python-sdk.md

  - SDK patterns and complete code examples

• standby-mode.md

  - Real-time API implementation

• input-schema.md

  - Input validation and UI configuration

• output-schema.md

  - Output configuration and templates

Troubleshooting

If you need information not covered in this skill, use the WebFetch tool with https://docs.apify.com/llms.txt to access the complete official documentation.