Skill: Connect Data

Purpose

Guided wizard to connect a new dataset. Walks the user through selecting a connection type, configuring credentials, validating the connection, profiling the schema, and setting up the knowledge brain.

When to Use

• User says

  /connect-data

  or "connect my database" or "add a new dataset"
• First-run welcome suggests connecting data
• After

  /switch-dataset

  when the target dataset doesn't exist yet

Invocation

/connect-data

— start the connection wizard

/connect-data type=postgres

— skip type selection

Instructions

Step 1: Choose Connection Type

Present options:

1. CSV files — "I have CSV files in a local directory"
2. DuckDB — "I have a local DuckDB database file"
3. MotherDuck — "I have a MotherDuck cloud database"
4. PostgreSQL — "I have a PostgreSQL database"
5. BigQuery — "I have a Google BigQuery dataset"
6. Snowflake — "I have a Snowflake warehouse"

Step 2: Collect Connection Details

For CSV:

• Ask: "What's the path to your CSV directory? (relative to this repo)"
• Verify the directory exists and contains .csv files
• List found files and ask to confirm

For DuckDB:

• Ask: "Path to your .duckdb file?"
• Verify file exists
• Test connection with

  SELECT 1

For MotherDuck:

• Ask: "Database name and schema?"
• Note: "MotherDuck connects via MCP. Make sure your token is configured."

For PostgreSQL / BigQuery / Snowflake:

• Copy the appropriate template from

  connection_templates/

• Ask user to fill in required fields
• IMPORTANT: Never ask for or store passwords directly. Guide the user to use environment variables (e.g.,

  $PG_PASSWORD

  ).

Step 3: Create Dataset Brain

1. Generate a dataset_id from the display name (lowercase, hyphens)
2. Create

   .knowledge/datasets/{id}/

   directory
3. Write

   manifest.yaml

   from the connection template + user inputs
4. Create empty

   quirks.md

   with section headers
5. Create empty

   metrics/index.yaml

Step 4: Test Connection

Use

ConnectionManager

from

helpers/connection_manager.py

:

1. Instantiate with the new config
2. Call

   test_connection()

3. If fails: show error, offer to retry or edit config
4. If passes: proceed

Step 5: Profile Schema

1. Call

   list_tables()

   to enumerate tables
2. For each table: get column names and types via

   get_table_schema()

3. Generate

   schema.md

   using

   schema_to_markdown()

   from

   helpers/data_helpers.py

4. Write to

   .knowledge/datasets/{id}/schema.md

5. Offer to run full data profiling: "Want me to deep-profile this dataset?"

Step 6: Set Active

1. Update

   .knowledge/active.yaml

   to point to the new dataset
2. Confirm: "Connected! {display_name} is now your active dataset."
3. Show: table count, estimated row count, date range (if detected)
4. Suggest next steps:

   /explore

   to browse,

   /metrics

   to define metrics, or just ask a question

Rules

1. Never store credentials in plain text in manifest files
2. Always test the connection before declaring success
3. Always generate a schema.md — it's required for analysis
4. Create the full .knowledge/datasets/{id}/ tree even if profiling fails
5. If the user already has this dataset, ask before overwriting

Edge Cases

• Directory doesn't exist: Offer to create it
• No CSV files found: Check for other formats (.parquet, .json)
• Connection fails repeatedly: Suggest checking credentials, firewall, VPN
• Schema too large (>100 tables): Profile only, skip per-table details
• Dataset name collision: Append a number (e.g., "mydata-2")