Motion Canvas Agent Plugin

Full programmatic control of a running Motion Canvas editor via HTTP. Seek to any frame, capture screenshots, inspect the scene graph, change resolution/fps/background, and trigger rendering — all without browser automation.

Architecture

Agent (curl/script)  ──HTTP──▶  Vite Server (agent-plugin.ts)  ──HMR WS──▶  Browser (agent-client.ts)
                                      │                                           │
                                      ▼                                           ▼
                                screenshots/                               Player + Renderer
                                (PNG files)                                + ProjectMetadata

Setup

1. Copy plugin files into your project

• assets/agent-plugin.ts

  → project root

• assets/agent-client.ts

  →

  src/

2. vite.config.ts

import {defineConfig} from 'vite';
import motionCanvas from '@motion-canvas/vite-plugin';
import {agentPlugin} from './agent-plugin';

export default defineConfig({
  plugins: [
    motionCanvas(),
    agentPlugin({screenshotDir: './screenshots'}),
  ],
});

3. project.ts

import {makeProject} from '@motion-canvas/core';
import {agentClient} from './agent-client';

export default makeProject({
  plugins: [agentClient()],
  scenes: [...],
});

4. Start + open browser

npm start
# Open http://localhost:9000 in a browser

HTTP API Reference

Base URL:

http://localhost:9000/__agent

Playback Control

GET

/status

{connected, frame, duration, fps, paused, sceneName, errors}

POST

/seek

{frame: 120}

{ok, frame}

POST

/next-frame

{ok, frame}

POST

/prev-frame

{ok, frame}

POST

/play

{ok}

POST

/pause

{ok}

Screenshots

POST

/screenshot

{name: "my-shot"}

{ok, path, frame}

POST

/screenshot-base64

{ok, data, frame}

Settings

GET

/settings

{background, range, size, previewFps, previewScale, renderingFps, renderingScale}

POST

/settings/background

{color: "#0F172A"}

{ok}

POST

/settings/range

{start: 0, end: 300}

{ok}

POST

/settings/size

{width: 1920, height: 1080}

{ok}

POST

/settings/preview-fps

{fps: 30}

{ok}

POST

/settings/preview-scale

{scale: 0.5}

{ok}

POST

/settings/rendering-fps

{fps: 60}

{ok}

POST

/settings/rendering-scale

{scale: 2}

{ok}

Rendering

POST

/render

{fps?, range?}

{ok, message}

POST

/render/abort

{ok, message}

Scene Info

GET

/scenes

{scenes: [{name}], currentScene}

GET

/scene-graph

{scene, graph}

GET

/threads

{scene, slides: [{name, id}]}

Errors

GET

/errors

{errors: [{message, stack}]}

POST

/clear-errors

{ok}

Scene Graph Format

GET /__agent/scene-graph

{
  "scene": "my-scene",
  "graph": {
    "type": "View2D",
    "key": "my-scene/View2D[1]",
    "position": {"x": 960, "y": 540},
    "size": {"width": 1920, "height": 1080},
    "children": [
      {
        "type": "Rect",
        "position": {"x": 0, "y": 0},
        "size": {"width": 400, "height": 300},
        "fill": "#1E293B",
        "children": [
          {
            "type": "Txt",
            "text": "Hello World"
          }
        ]
      }
    ]
  }
}

Each node includes:

type

key

position

size

opacity

fill

text

children

AI Agent Workflow

# 1. Check connection
curl -s localhost:9000/__agent/status

# 2. Set resolution
curl -s -X POST localhost:9000/__agent/settings/size \
  -H "Content-Type: application/json" -d '{"width": 1920, "height": 1080}'

# 3. Seek to a frame
curl -s -X POST localhost:9000/__agent/seek \
  -H "Content-Type: application/json" -d '{"frame": 60}'

# 4. Screenshot
curl -s -X POST localhost:9000/__agent/screenshot \
  -H "Content-Type: application/json" -d '{"name": "check-1"}'
# → Read ./screenshots/check-1.png

# 5. Inspect scene graph
curl -s localhost:9000/__agent/scene-graph

# 6. Check which scene we're in
curl -s localhost:9000/__agent/scenes

# 7. Check errors
curl -s localhost:9000/__agent/errors

# 8. Change background
curl -s -X POST localhost:9000/__agent/settings/background \
  -H "Content-Type: application/json" -d '{"color": "#000000"}'

# 9. Trigger render
curl -s -X POST localhost:9000/__agent/render

Security

• Localhost only: The agent API runs on the Vite dev server which binds to localhost by default. Do not expose it to the network (avoid

  --host 0.0.0.0

  in production)
• Screenshot paths: The screenshot name is sanitized — path separators and dots are stripped, and the resolved path is verified to stay inside the configured

  screenshotDir

• Scene graph text: Text content extracted from the scene graph is sanitized (control characters stripped, truncated to 100 chars). Treat scene graph data as untrusted when processing it

How It Works

• Runtime plugin (

  agentClient()

  ) uses

  Plugin.player()

  ,

  Plugin.renderer()

  , and

  Plugin.project()

  callbacks to capture the Player, Renderer, and Project instances
• Player provides seek, frame stepping, playback control, current scene, duration
• ProjectMetadata provides

  shared

  (background, range, size),

  preview

  (fps, scale), and

  rendering

  (fps, scale, exporter) settings via

  .get()

  /

  .set()

• Renderer provides

  render(settings)

  and

  abort()

  for export
• Scene graph is traversed via

  scene.getView().peekChildren()

  recursively, serializing type, position, size, fill, text, opacity
• Communication uses Vite's HMR WebSocket

Files

assets/agent-plugin.ts

assets/agent-client.ts

src/