Vast.ai Hello World

Overview

Rent your first GPU instance on Vast.ai, run a PyTorch workload, and destroy the instance when done. Demonstrates the full lifecycle: search offers, create instance, connect via SSH, run a job, and tear down.

Prerequisites

• Completed

  vastai-install-auth

  setup
• Vast.ai account with credits ($1+ recommended for testing)
• SSH key uploaded to Vast.ai (cloud.vast.ai > Account > SSH Keys)

Instructions

Step 1: Search for Available GPUs (CLI)

# Find cheap single-GPU offers sorted by price
vastai search offers 'num_gpus=1 gpu_ram>=8 inet_down>100 reliability>0.95' \
  --order 'dph_total' --limit 5

# Output columns: ID, GPU, VRAM, $/hr, DLPerf, Reliability, Location

Step 2: Search for Available GPUs (REST API)

curl -s -H "Authorization: Bearer $VASTAI_API_KEY" \
  "https://cloud.vast.ai/api/v0/bundles/?q=%7B%22num_gpus%22%3A%7B%22eq%22%3A1%7D%2C%22gpu_ram%22%3A%7B%22gte%22%3A8%7D%2C%22reliability2%22%3A%7B%22gte%22%3A0.95%7D%2C%22rentable%22%3A%7B%22eq%22%3Atrue%7D%7D&order=dph_total&limit=5" \
  | jq '.offers[:3] | .[] | {id, gpu_name, num_gpus, gpu_ram, dph_total, reliability2}'

Step 3: Create an Instance (CLI)

# Replace OFFER_ID with the ID from search results
vastai create instance OFFER_ID \
  --image pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime \
  --disk 20 \
  --onstart-cmd "echo 'Instance ready'"

Step 4: Create an Instance (Python)

from vastai_client import VastClient

client = VastClient()

# Search for affordable RTX 4090 offers
offers = client.search_offers({
    "num_gpus": {"eq": 1},
    "gpu_name": {"eq": "RTX_4090"},
    "reliability2": {"gte": 0.95},
    "rentable": {"eq": True},
})

# Pick the cheapest offer
best = sorted(offers["offers"], key=lambda o: o["dph_total"])[0]
print(f"Best offer: {best['gpu_name']} at ${best['dph_total']:.3f}/hr (ID: {best['id']})")

# Create instance with PyTorch image
instance = client.create_instance(
    offer_id=best["id"],
    image="pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime",
    disk_gb=20,
    onstart="nvidia-smi && python -c 'import torch; print(torch.cuda.is_available())'",
)
print(f"Instance created: {instance}")

Step 5: Monitor and Connect

# Check instance status (wait for 'running')
vastai show instances --raw | jq '.[] | {id, actual_status, ssh_host, ssh_port}'

# Connect via SSH once running
ssh -p SSH_PORT root@SSH_HOST

# On the instance: verify GPU access
nvidia-smi
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

Step 6: Run a Test Workload

# test_gpu.py — run this ON the rented instance
import torch
import time

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Device: {device} ({torch.cuda.get_device_name(0)})")

# Simple matrix multiplication benchmark
size = 4096
a = torch.randn(size, size, device=device)
b = torch.randn(size, size, device=device)

torch.cuda.synchronize()
start = time.time()
c = torch.matmul(a, b)
torch.cuda.synchronize()
elapsed = time.time() - start

tflops = (2 * size**3) / elapsed / 1e12
print(f"Matrix multiply {size}x{size}: {elapsed:.3f}s ({tflops:.2f} TFLOPS)")
print("Hello World from Vast.ai!")

Step 7: Destroy the Instance

# IMPORTANT: Destroy to stop billing
vastai destroy instance INSTANCE_ID

# Verify it's gone
vastai show instances

Output

• GPU instance rented and running on Vast.ai
• SSH connection established to the remote GPU machine
• PyTorch workload executed successfully with GPU acceleration
• Instance destroyed (billing stopped)

Error Handling

No offers found

Insufficient funds

Instance failed to start

SSH connection refused

running

CUDA not available

Resources

• Vast.ai Search & Filter
• Creating Instances
• CLI Reference
• REST API Quickstart

Next Steps

Proceed to

vastai-local-dev-loop

Examples

Cheapest GPU test: Search with

vastai search offers 'num_gpus=1' --order 'dph_total' --limit 1

nvidia-smi

Specific GPU model: Filter for H100 with

gpu_name=H100_SXM

reliability>0.99