SkyPilot Skill

SkyPilot is a unified framework to run AI workloads on any cloud, Slurm or Kubernetes. It provides a single interface to launch clusters, run jobs, and serve models across 25+ clouds (AWS, GCP, Azure, Coreweave, Nebius, Lambda, Together AI, RunPod, and more), Kubernetes clusters, and Slurm clusters.

When to Use SkyPilot

Use SkyPilot when you need to:

• Manage compute resources on any cloud, Slurm, or Kubernetes cluster
• Launch CPU/GPU/TPU (GB300, GB200, B200, H200, H100, etc.) on any cloud, Kubernetes or Slurm
• Run training, fine-tuning, or batch inference jobs
• Serve models with autoscaling and multi-cloud replicas (SkyServe)
• Run long-running jobs with automatic lifecycle management and recovery (managed jobs)
• Find the cheapest or most available GPU across clouds

Don't use SkyPilot for:

• Local-only workloads (use Docker/conda directly)

Capabilities: When to Use What

SkyPilot has three core abstractions. Use the right one for each stage of your workflow:

1. SkyPilot Clusters (

sky launch

sky exec

• Use during initial development, debugging, and experimentation
• Launch a cluster, SSH in or connect VSCode/Cursor (

  code --remote ssh-remote+CLUSTER

  ), iterate quickly
• Cluster stays up until you stop/down it or autostop triggers
• Best for: prototyping, debugging, short experiments

2. Managed Jobs (

sky jobs launch

• Use when submitting long-running jobs that should run unattended
• Manages the full lifecycle: provisioning, execution, recovery, and teardown
• Automatically recovers from spot preemptions, quota limits, and transient failures
• Works across clouds, Kubernetes, and Slurm (handles preemptions and quota)
• Best for: training runs, fine-tuning, hyperparameter sweeps, batch inference

3. SkyServe (

sky serve up

• Use when serving models at scale with autoscaling
• Start with

  sky launch

  + open port to test your serving setup, then use

  sky serve up

  to scale
• Provides load balancing, autoscaling, and multi-cloud replicas
• Best for: model serving endpoints, API services

Before You Start (Agent Bootstrap)

Bootstrap to confirm SkyPilot is installed, connected to an API server, and has cloud credentials. Once confirmed, skip straight to the user's task.

Step 1: Check installation and API server connectivity

sky api info

No SkyPilot API server is connected

Could not connect to SkyPilot API server

sky api login --relogin -e <endpoint>

command not found: sky

Install SkyPilot (only if

sky

pip install "skypilot[aws,gcp,kubernetes]"  # Pick clouds the user needs

Ask the user which clouds they need if unclear, then re-run

sky api info

Start or connect a server (only if "not running"):

Ask the user:

Do you have an existing SkyPilot API server to connect to, or should I start one locally?

• Connect to existing server:

  sky api login -e <API_SERVER_URL>

  — get the URL from the user.
• Start locally:

  sky api start

After either path, re-run

sky api info

Step 2: Check cloud credentials (only for fresh setups — skip if the server was already running)

sky check -o json

This shows which clouds are enabled or disabled. If the user's target cloud is not enabled, guide them through credential setup (see Troubleshooting).

Essential Commands

Use

-o json

Clusters — interactive development and debugging:

sky launch -c NAME task.yaml

sky exec NAME task.yaml

sky exec NAME task.yaml -d

sky status -o json

sky logs NAME

sky logs NAME --no-follow

sky logs NAME --tail 50

sky logs NAME --status

sky queue NAME -o json

sky stop NAME

sky start NAME

sky down NAME

sky gpus list -o json

Managed Jobs — long-running unattended workloads:

sky jobs launch task.yaml

sky jobs queue -o json

sky jobs logs JOB_ID

sky jobs cancel JOB_ID

SkyServe — model serving with autoscaling:

sky serve up serve.yaml -n NAME

sky serve status NAME

sky serve update NAME new.yaml

sky serve down NAME

For complete CLI reference, see CLI Reference.

Quick Start

# Launch a GPU cluster
sky launch -c mycluster --gpus H100 -- nvidia-smi

# Run a task from YAML
sky launch -c mycluster task.yaml

# SSH into cluster
ssh mycluster

# Connect VSCode or Cursor to the cluster for interactive development
code --remote ssh-remote+mycluster /home/user/sky_workdir
# or: cursor --remote ssh-remote+mycluster /home/user/sky_workdir

# Tear down
sky down mycluster

Task YAML Structure

The task YAML is SkyPilot's primary interface. All fields are optional.

# task.yaml
name: my-training-job

# Local directory to sync to remote ~/sky_workdir
workdir: .

# Number of nodes (for distributed training)
num_nodes: 1

resources:
  # GPU/TPU accelerators (SkyPilot auto-selects the cheapest cloud/region)
  accelerators: H200:8
  # Optional: pin to a specific cloud/region/infra
  # infra: aws  # or aws/us-east-1, k8s, ssh/my-pool
  # If infra is left out, SkyPilot automatically fails over across all
  # enabled clouds/regions to find the cheapest available option.
  # Use spot instances for cost savings
  use_spot: false
  # Disk size in GB
  disk_size: 256
  # Open ports for serving
  ports: 8080

# Environment variables (accessible in file_mounts, setup, and run)
envs:
  MODEL_NAME: my-model
  BATCH_SIZE: 32

# Setup: runs once on cluster creation, cached on reuse
setup: |
  pip install torch transformers

# Run: the main command
run: |
  python train.py --model $MODEL_NAME --batch-size $BATCH_SIZE

For complete YAML schema including file mounts, environment variables set by SkyPilot, and advanced fields, see YAML Specification.

GPU and Cloud Selection

IMPORTANT: Let SkyPilot choose the cloud and region. Do NOT manually pick a cloud/region/instance by parsing

sky gpus list

infra:

Default behavior (recommended): Just specify the GPU type. SkyPilot finds the cheapest cloud/region automatically:

resources:
  accelerators: H200:8  # SkyPilot picks the cheapest cloud/region with H200:8

If the user doesn't specify a GPU type, ask them what GPU they need (or what model/workload they're running so you can recommend one). Do NOT run

sky gpus list

any_of

# Let SkyPilot choose from multiple acceptable GPU types (cheapest wins)
resources:
  any_of:
    - accelerators: H100:8
    - accelerators: A100-80GB:8
    - accelerators: A100:8

Use

ordered

# Try H100 first on AWS, fall back to GCP, then A100
resources:
  ordered:
    - infra: aws/us-east-1
      accelerators: H100:8
    - infra: gcp/us-central1
      accelerators: H100:8
    - infra: aws/us-west-2
      accelerators: A100-80GB:8

Only set

infra:

resources:
  infra: aws             # User asked for AWS specifically
  accelerators: H100:8

Cluster Lifecycle

# Launch and run a task
sky launch -c mycluster task.yaml

# Launch with autostop at launch time (preferred: saves cost, no follow-up command needed)
sky launch -c mycluster task.yaml -i 30        # stop after 30 min idle
sky launch -c mycluster task.yaml -i 30 --down # tear down after 30 min idle

# Override or pass environment variables via CLI
sky launch -c mycluster task.yaml --env MODEL_NAME=llama3 --env BATCH_SIZE=64

# Re-run a different task on the same cluster (fast, skips provisioning)
sky exec mycluster another_task.yaml

# Run an inline command
sky exec mycluster -- python train.py --epochs 10

# Set autostop after launch (use if you forgot to set -i at launch time)
sky autostop mycluster -i 30        # stop after 30 min idle, preserving disk (can restart with sky start)
sky autostop mycluster -i 30 --down # tear down after 30 min idle (disk is deleted, cannot restart)

# Stop to save costs, restart later
sky stop mycluster
sky start mycluster

# Tear down completely
sky down mycluster

Workdir Sync Behavior

workdir:

~/sky_workdir

rsync

sky exec

To ensure a clean slate, SSH and wipe before

sky exec

ssh mycluster "rm -rf ~/sky_workdir"
sky exec mycluster task.yaml

Or clean inside

run:

run: |
  find ~/sky_workdir/build -name '*.o' -delete 2>/dev/null || true
  cd ~/sky_workdir && make

Managed Jobs

Use

sky jobs launch

# managed-job.yaml
name: training-job

resources:
  accelerators: A100:8

run: |
  python train.py --resume-from-checkpoint

# Launch as managed job
sky jobs launch managed-job.yaml

# Check status
sky jobs queue -o json

# Stream logs
sky jobs logs <job_id>

# Cancel
sky jobs cancel <job_id>

Checkpoint pattern: Your training script should save checkpoints to persistent storage (cloud bucket or volume) and resume from the latest checkpoint on restart. SkyPilot handles the cluster recovery; your script handles the state recovery.

SkyServe: Model Serving

# serve.yaml
resources:
  accelerators: A100:1
  ports: 8080

run: |
  python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Llama-3.1-8B-Instruct \
    --port 8080

service:
  readiness_probe: /v1/models
  replica_policy:
    min_replicas: 1
    max_replicas: 3
    target_qps_per_replica: 5

# Start service
sky serve up serve.yaml -n my-llm

# Check status / get endpoint
sky serve status my-llm
sky serve status my-llm --endpoint

# Update (rolling)
sky serve update my-llm new-serve.yaml

# Tear down
sky serve down my-llm

Common Workflows

Fine-Tuning Workflow

1. Write task YAML with

   setup

   (install deps) and

   run

   (training command)
2. Use

   file_mounts

   or

   workdir

   to sync code

3. sky launch -c train task.yaml

   to launch

4. sky logs train

   to monitor

5. sky exec train -- python eval.py

   to evaluate on same cluster

6. sky down train

   when done

Hyperparameter Sweep

1. Create parameterized YAML with

   envs

2. Launch multiple managed jobs:

   for lr in 1e-4 1e-5 1e-6; do
     sky jobs launch sweep.yaml --env LR=$lr --name sweep-lr-$lr
   done
   

3. Monitor with

   sky jobs queue -o json

Model Serving Deployment

1. Write serve YAML with

   service:

   section

2. sky serve up serve.yaml -n my-service

3. Get endpoint:

   sky serve status my-service --endpoint

4. Update model:

   sky serve update my-service updated.yaml

Parallel Experiment Submission

Use

sky exec -d

# Submit all experiments (detached, returns after job is queued)
for i in 1 2 3 4; do
  sky exec exp-vm-0$i task.yaml --env LR=1e-$i -d
done

# Get the latest job ID from a cluster
job_id=$(sky queue exp-vm-01 -o json \
  | python3 -c "import sys, json; jobs = json.load(sys.stdin).get('exp-vm-01', []); print(max(j['job_id'] for j in jobs) if jobs else '')")

# Wait for a specific job and fetch last 50 lines
sky logs exp-vm-01 $job_id --status && sky logs exp-vm-01 $job_id --tail 50

# Check all jobs across a cluster at once
sky queue exp-vm-01 -o json

Agent Feedback Loop

When using SkyPilot programmatically, follow this loop:

1. Validate:

   sky launch --dryrun task.yaml

   (check resource availability/cost)
2. Launch:

   sky launch -c mycluster task.yaml

3. Monitor:

   sky status -o json

   and

   sky queue mycluster -o json

4. Wait for completion:

   sky logs mycluster <JOB_ID>

   (streams logs so you can observe progress and react to stalls; blocks until job finishes; get JOB_ID from

   sky queue mycluster -o json

   ). For long-running jobs where you don't need intermediate output, use

   sky logs mycluster <JOB_ID> --status

   instead (blocks silently, exits 0 on success).
5. Inspect output:

   sky logs mycluster <JOB_ID> --no-follow

   or

   sky logs mycluster <JOB_ID> --tail 100

6. Debug:

   ssh mycluster

   (interactive)
7. Iterate:

   sky exec mycluster updated_task.yaml

   (run on existing cluster)
8. Cleanup:

   sky down mycluster

Never poll with

sleep

+

sky queue

— use

sky logs CLUSTER JOB_ID

to stream logs and block until done. Use

--status

if you only need the exit code, or

--tail N

to fetch recent output after completion.

Common Agent Mistakes

sky gpus list

accelerators:

sky launch

sky jobs launch

sky down

-i <minutes> --down

infra: aws

infra:

envs:

envs:

--env KEY=VAL

sky launch

-c <name>

-c

-o json

cloud:

region:

zone:

infra:

infra: aws/us-east-1

sleep

sky queue

sky logs CLUSTER JOB_ID --status

sky exec

~/sky_workdir

run:

--tail

sky logs CLUSTER JOB_ID --tail 50

Common Issues Quick Reference

any_of

sky exec

sky logs <cluster>

ssh <cluster>

sky down <cluster>

sky jobs launch

ports:

workdir

sky check -o json

References

For detailed reference documentation:

• CLI Reference — All commands and flags
• YAML Specification — Complete task YAML schema, file mounts, environment variables
• Python SDK — Programmatic API and SDK usage
• Advanced Patterns — Multi-cloud, distributed training, production patterns
• Troubleshooting — Error diagnosis and solutions
• Examples — Copy-paste task YAML examples