BrowserOS test-ui

Test the BrowserOS agent extension UI by starting the dev environment and visually verifying changes via CDP. Covers the new tab page (left sidebar — Home, Scheduled Tasks, Settings, etc.) and the right side panel (chat interface). Use after making UI changes to apps/agent/.

install

source · Clone the upstream repo

git clone https://github.com/browseros-ai/BrowserOS

Claude Code · Install into ~/.claude/skills/

T=$(mktemp -d) && git clone --depth=1 https://github.com/browseros-ai/BrowserOS "$T" && mkdir -p ~/.claude/skills && cp -r "$T/packages/browseros-agent/.claude/skills/test-ui" ~/.claude/skills/browseros-ai-browseros-test-ui && rm -rf "$T"

manifest: packages/browseros-agent/.claude/skills/test-ui/SKILL.md

source content

Test Agent UI

Visually test the BrowserOS agent extension UI — both the new tab page (left sidebar) and the right side panel (chat) — by starting the dev environment and inspecting via CDP.

When to use

After making code changes to

apps/agent/

(the Chrome extension), use this skill to:

Verify new UI components render correctly
Check navigation between views works
Confirm layout/styling changes look right
Test interactive elements (buttons, inputs, forms)

Prerequisites

Go must be installed (
```
brew install go
```
) — the dev tool is written in Go
BrowserOS.app must be installed at
```
/Applications/BrowserOS.app/
```
The
```
scripts/dev/inspect-ui.ts
```
utility must exist (CDP inspector script)

Step 1: Start the dev environment

bun run dev:watch -- --new

This single command handles everything:

Builds the Go dev CLI tool
Picks random available ports (avoids conflicts)
Creates a fresh browser profile
Builds controller-ext
Runs GraphQL codegen if
```
apps/agent/generated/graphql/
```
doesn't exist
Starts the agent extension with WXT HMR (hot module replacement)
Waits for CDP to be ready
Starts the MCP server

Run it in the background and read the output to find the CDP port:

[info] Ports: CDP=9552 Server=9065 Extension=9929

The CDP port is randomized. You MUST extract it from the output and set it for all subsequent commands:

export BROWSEROS_CDP_PORT=<port from output>

Wait for these messages before proceeding:

```
[server] CDP ready
```
```
[server] HTTP server listening
```

Step 2: Discover targets

bun scripts/dev/inspect-ui.ts targets

You will see targets like:

```
[service_worker]
```
— extension background scripts (not directly testable for UI)

[page] chrome-extension://bflpfmnmnokmjhmgnolecpppdbdophmk/app.html#/...

— New tab page (left sidebar)

```
[page] sidepanel.html
```
— Right side panel (chat)

The two main testable surfaces:

app.html
— the new tab page with left sidebar (Home, Connect Apps, Scheduled Tasks, Skills, Memory, Soul, Settings)
sidepanel.html
— the right side panel chat interface

Step 3: Navigate to the main UI

A fresh profile opens the onboarding page (

app.html#/onboarding

). Navigate to the home page first:

bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/home'"

Verify with a snapshot (not screenshot — snapshot is faster and sufficient for structural checks):

bun scripts/dev/inspect-ui.ts snapshot app.html

Snapshot vs Screenshot

Prefer

snapshot

for most checks — it's fast, text-based, and tells you what elements exist, their text, and their IDs. Use it after every navigation or interaction to verify state.

Use

screenshot

only when you need visual verification — layout changes, CSS/styling, colors, images, or a final "does it look right" check. Screenshots are expensive (capture → save → read image).

Check	Use
Did the page navigate?	`snapshot` — look for new elements
Does my new component render?	`snapshot` — look for its text/role
Did a click change state?	`snapshot` — check element names/values
Is the layout correct?	`screenshot` — visual check needed
Do CSS changes look right?	`screenshot` — visual check needed
Final verification before committing	`screenshot` — one visual confirmation

Step 4: Test the new tab page (left sidebar)

Get element IDs

bun scripts/dev/inspect-ui.ts snapshot app.html

Output shows interactive elements with IDs:

[52] link "Home"
[57] link "Connect Apps"
[65] link "Scheduled Tasks"
[74] link "Skills"
[103] link "Settings"

Navigate via click or hash routing

Click-based (use element IDs from snapshot):

bun scripts/dev/inspect-ui.ts click app.html 65    # Click "Scheduled Tasks"

Hash routing (faster, no snapshot needed):

bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/settings'"
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/scheduled-tasks'"
bun scripts/dev/inspect-ui.ts eval app.html "window.location.hash = '#/home'"

Verify navigation

# Snapshot to confirm the page changed (fast, preferred)
bun scripts/dev/inspect-ui.ts snapshot app.html

# Screenshot only if you need to check visual layout
bun scripts/dev/inspect-ui.ts screenshot app.html /tmp/settings.png

CRITICAL: Re-snapshot after every navigation

React re-renders change element IDs. Always run snapshot again before clicking/filling after navigating to a new view. Using stale IDs will fail.

Step 5: Open and test the right side panel

The side panel starts disabled in a fresh profile. Open it using BrowserOS-specific APIs:

bun scripts/dev/inspect-ui.ts open-sidepanel

Wait 2 seconds for it to appear as a target, then:

bun scripts/dev/inspect-ui.ts screenshot sidepanel /tmp/panel.png
bun scripts/dev/inspect-ui.ts snapshot sidepanel

Interact with the side panel

# Get element IDs
bun scripts/dev/inspect-ui.ts snapshot sidepanel
# Output: [37] textbox "What should I do?"
#         [124] button "Send"
#         [60] link "Chat history"
#         [99] button "Agent Mode ON"

# Fill the chat input and press Enter to send
bun scripts/dev/inspect-ui.ts fill sidepanel 37 "Hello world"
bun scripts/dev/inspect-ui.ts press_key sidepanel Enter

# Or click the Send button
bun scripts/dev/inspect-ui.ts click sidepanel 124

# Wait for a response to appear
bun scripts/dev/inspect-ui.ts wait_for sidepanel text "response text"

# Scroll down to see more content
bun scripts/dev/inspect-ui.ts scroll sidepanel down 3

# Hover over an element to test hover states
bun scripts/dev/inspect-ui.ts hover sidepanel 99

# Snapshot to verify state changed (fast, preferred)
bun scripts/dev/inspect-ui.ts snapshot sidepanel

# Screenshot only for visual/layout verification
bun scripts/dev/inspect-ui.ts screenshot sidepanel /tmp/result.png

Step 6: Verify and iterate

The core loop

snapshot → identify element IDs → click/fill/press_key → snapshot → verify

Use

screenshot

only when visual layout verification is needed (CSS changes, final check).

After making code changes

Fix the code in
```
apps/agent/
```
WXT HMR will hot-reload the extension automatically (watch mode)
Wait 2-3 seconds for the reload to complete
Re-snapshot — element IDs WILL change after HMR reload
Verify the fix with snapshot (or screenshot if visual)

Check server logs

The dev server output (running in background) contains useful diagnostics:

```
[agent]
```
— WXT build/HMR status, compilation errors
```
[server]
```
— MCP server logs, tool execution, errors
```
[build]
```
— Extension build output

If the UI isn't rendering, check for build errors in the

[agent]

output.

Check for JavaScript errors

bun scripts/dev/inspect-ui.ts eval sidepanel "JSON.stringify(window.__errors || 'no errors')"

Or check the console for React errors:

bun scripts/dev/inspect-ui.ts eval app.html "document.querySelector('#root')?.innerHTML?.substring(0, 200)"

Verify API connectivity

The extension talks to the MCP server. Verify the server is reachable:

bun scripts/dev/inspect-ui.ts eval sidepanel "fetch('http://127.0.0.1:<serverPort>/health').then(r => r.ok).catch(() => false)"

Common issues

Symptom	Cause	Fix
Blank page after navigation	React render error	Check `eval` for JS errors
Element IDs don't match	Page re-rendered (HMR/navigation)	Re-run `snapshot` before interacting
`open-sidepanel` fails	Extension not fully loaded	Wait longer after dev server starts
Click does nothing	Element not visible (below fold)	Use `scroll` first, then re-snapshot
`wait_for` times out	Content hasn't loaded yet	Check server logs for API errors

Available commands reference

Command	Description
`targets`	List all CDP targets, marks extension pages with `[EXTENSION]`
`screenshot <target> [file]`	Capture PNG screenshot (default: `screenshot.png` )
`snapshot <target>`	Print accessibility tree with `[elementId] role "name"`
`click <target> <elementId>`	Click element by ID (3-tier coordinate fallback + JS click)
`fill <target> <elementId> <text>`	Focus element, clear, type text
`press_key <target> <key>`	Press key or combo: `Enter` , `Escape` , `Tab` , `Control+A` , `Meta+Shift+P`
`scroll <target> <dir> [amount]`	Scroll `up` / `down` / `left` / `right` , amount in ticks (default 3)
`hover <target> <elementId>`	Hover over element (for tooltips, hover states)
`select_option <target> <id> <val>`	Select dropdown option by value or visible text
`wait_for <target> text\|selector <v>`	Wait up to 10s for text or CSS selector to appear
`eval <target> <expression>`	Run JavaScript in the target's context
`open-sidepanel`	Enable and open the right side panel

<target>

is a URL substring (e.g.,

sidepanel

app.html

) or numeric index from

targets

output.

Known app.html routes

These can be used with

eval app.html "window.location.hash = '#/<route>'"

Route	View
`/home`	Home page with search bar and top sites
`/settings`	Settings (LLM providers, customization, workflows, MCP)
`/scheduled-tasks`	Scheduled Tasks management
`/onboarding`	Onboarding flow (first-run experience)

Gotchas learned from real testing

Ports are randomized with
```
--new
```
— always extract from dev server output
Fresh profile = onboarding page — navigate to
```
#/home
```
to see the main UI
Element IDs change after navigation — always re-snapshot before clicking
Side panel starts disabled —
```
open-sidepanel
```
handles the BrowserOS-specific enable + toggle API
Input.enable
does not exist — the CDP Input domain has no enable method (already handled in the script)
DOM.getDocument
required — must be called before DOM operations like
```
pushNodesByBackendIdsToFrontend
```
(already handled in the script)
Settings sub-navigation — the settings page has its own left sidebar (BrowserOS AI, Chat & Council Provider, Search Provider, Customize BrowserOS, BrowserOS as MCP, Workflows) — use snapshot + click to navigate within settings