Lumen Dashboards Skill

Overview

Lumen is a declarative framework for creating data dashboards through YAML specifications. Build interactive data exploration dashboards without writing code - just configuration.

What is Lumen?

Lumen provides a declarative approach to building data dashboards:

• No-code dashboards: Define everything in YAML
• Data pipelines: Sources → Transforms → Views
• Interactive exploration: Built-in filters and cross-filtering
• Component library: Reusable sources, transforms, views
• Live updates: Auto-reload and real-time data

Lumen vs Panel vs Lumen AI

Feature	Lumen Dashboards	Panel	Lumen AI
Approach	Declarative YAML	Imperative Python	Conversational
Code Required	No	Yes	No
Use Case	Fixed dashboards	Custom apps	Ad-hoc exploration
Flexibility	Medium	High	High
Development Speed	Very fast	Medium	Very fast

Use Lumen when:

• Building standard data exploration dashboards
• Working with non-programmers
• Want rapid prototyping with configuration
• Need reproducible dashboard specifications

Use Panel when:

• Need fine-grained control over components
• Building custom application logic
• Creating novel interactions

Use Lumen AI when:

• Users need ad-hoc exploration
• Questions vary unpredictably
• Enabling self-service analytics

Quick Start

Installation

pip install lumen

Your First Dashboard

File:

dashboard.yaml

sources:
  data:
    type: file
    tables:
      penguins: https://datasets.holoviz.org/penguins/v1/penguins.csv

pipelines:
  main:
    source: data
    table: penguins

    filters:
      - type: widget
        field: species

layouts:
  - title: Penguin Explorer
    views:
      - type: hvplot
        pipeline: main
        kind: scatter
        x: bill_length_mm
        y: bill_depth_mm
        by: species
        title: Bill Dimensions

Launch:

lumen serve dashboard.yaml --show

Core Concepts

1. Sources

Data sources provide tables for your dashboard.

Supported sources:

• File: CSV, Parquet, Excel, JSON
• Database: PostgreSQL, DuckDB, SQLite
• REST API: JSON endpoints
• Intake: Data catalogs

Quick example:

sources:
  mydata:
    type: file
    tables:
      sales: ./data/sales.csv

See: Data Sources Reference for comprehensive source configuration.

2. Pipelines

Pipelines define data flows: Source → Filters → Transforms → Views

Basic pipeline:

pipelines:
  sales_pipeline:
    source: mydata
    table: sales

    filters:
      - type: widget
        field: region

    transforms:
      - type: aggregate
        by: ['category']
        aggregate:
          total_sales: {revenue: sum}

Components:

• Filters: Interactive widgets for user input
• Transforms: Data manipulation (filter, aggregate, sort, SQL)
• Views: Visualizations and tables

3. Filters

Add interactive controls:

filters:
  # Dropdown select
  - type: widget
    field: category

  # Multi-select
  - type: widget
    field: region
    multiple: true

  # Date range
  - type: widget
    field: date
    widget: date_range_slider

  # Numeric slider
  - type: param
    parameter: min_revenue
    widget_type: FloatSlider
    start: 0
    end: 100000

4. Transforms

Process data in pipelines:

Common transforms:

• columns

  : Select specific columns

• query

  : Filter rows with pandas query

• aggregate

  : Group and aggregate

• sort

  : Sort data

• sql

  : Custom SQL queries

Example:

transforms:
  - type: columns
    columns: ['date', 'region', 'revenue']

  - type: query
    query: "revenue > 1000"

  - type: aggregate
    by: ['region']
    aggregate:
      total: {revenue: sum}
      avg: {revenue: mean}

See: Data Transforms Reference for all transform types.

5. Views

Visualize data:

View types:

• hvplot

  : Interactive plots (line, scatter, bar, etc.)

• table

  : Data tables

• indicator

  : KPI metrics

• vega

  : Vega-Lite specifications

• altair

  : Altair charts

• plotly

  : Plotly charts

Example:

views:
  - type: hvplot
    pipeline: main
    kind: line
    x: date
    y: revenue
    by: category

  - type: indicator
    pipeline: main
    field: total_revenue
    title: Total Sales
    format: '${value:,.0f}'

See: Views Reference for all view types and options.

6. Layouts

Arrange views on the page:

layouts:
  - title: Overview
    layout: [[0, 1, 2], [3], [4, 5]]  # Grid positions
    views:
      - type: indicator
        # View 0 config...

      - type: indicator
        # View 1 config...

      - type: hvplot
        # View 2 config...

Layout types:

• Grid:

  [[0, 1], [2, 3]]

• Tabs: Multiple layouts become tabs
• Responsive: Adapts to screen size

See: Layouts Reference for advanced layout patterns.

Common Patterns

Pattern 1: KPI Dashboard

sources:
  metrics:
    type: file
    tables:
      data: ./metrics.csv

pipelines:
  kpis:
    source: metrics
    table: data
    transforms:
      - type: aggregate
        aggregate:
          total_revenue: {revenue: sum}
          total_orders: {orders: sum}
          avg_order_value: {revenue: mean}

layouts:
  - title: KPIs
    layout: [[0, 1, 2]]
    views:
      - type: indicator
        pipeline: kpis
        field: total_revenue
        format: '${value:,.0f}'

      - type: indicator
        pipeline: kpis
        field: total_orders
        format: '{value:,.0f}'

      - type: indicator
        pipeline: kpis
        field: avg_order_value
        format: '${value:.2f}'

Pattern 2: Filtered Exploration

pipelines:
  explorer:
    source: mydata
    table: sales

    filters:
      - type: widget
        field: region
        label: Region

      - type: widget
        field: category
        label: Category
        multiple: true

      - type: widget
        field: date
        widget: date_range_slider

    views:
      - type: hvplot
        kind: scatter
        x: price
        y: quantity
        by: category

      - type: table
        page_size: 20

Pattern 3: Multi-Source Dashboard

sources:
  sales_db:
    type: postgres
    connection_string: postgresql://localhost/sales
    tables: [orders, customers]

  inventory_file:
    type: file
    tables:
      stock: ./inventory.csv

pipelines:
  sales_pipeline:
    source: sales_db
    table: orders

  inventory_pipeline:
    source: inventory_file
    table: stock

Pattern 4: Cross-Filtering

pipelines:
  main:
    source: data
    table: sales

    filters:
      - type: widget
        field: region

layouts:
  - title: Analysis
    views:
      # Clicking bar filters other views
      - type: hvplot
        pipeline: main
        kind: bar
        x: category
        y: revenue
        selection_group: category_filter

      # Responds to selection above
      - type: hvplot
        pipeline: main
        kind: scatter
        x: price
        y: quantity
        selection_group: category_filter

Pattern 5: SQL Transform

transforms:
  - type: sql
    query: |
      SELECT
        region,
        category,
        SUM(revenue) as total_revenue,
        COUNT(*) as order_count,
        AVG(revenue) as avg_order_value
      FROM table
      WHERE date >= '2024-01-01'
      GROUP BY region, category
      HAVING total_revenue > 10000
      ORDER BY total_revenue DESC

Python API

While Lumen is designed for YAML, you can also use Python:

from lumen.sources import FileSource
from lumen.pipeline import Pipeline
from lumen.views import hvPlotView
from lumen.dashboard import Dashboard
import panel as pn

# Create source
source = FileSource(tables={'sales': './data/sales.csv'})

# Create pipeline
pipeline = Pipeline(source=source, table='sales')

# Create view
view = hvPlotView(
    pipeline=pipeline,
    kind='scatter',
    x='price',
    y='quantity'
)

# Create dashboard
dashboard = Dashboard(
    pipelines={'main': pipeline},
    layouts=[view]
)

# Serve
dashboard.servable()

See: Python API Reference for detailed API usage.

Configuration

Global Config

config:
  title: My Dashboard
  theme: dark  # or 'default', 'material'
  sizing_mode: stretch_width
  logo: ./logo.png
  favicon: ./favicon.ico
  layout: column  # or 'grid', 'tabs'

Themes

config:
  theme: material
  theme_json:
    palette:
      primary: '#00aa41'
      secondary: '#616161'

Authentication

# Serve with auth
lumen serve dashboard.yaml \
  --oauth-provider=generic \
  --oauth-key=${OAUTH_KEY} \
  --oauth-secret=${OAUTH_SECRET}

Deployment

Development

# Local with auto-reload
lumen serve dashboard.yaml --autoreload --show

# Specific port
lumen serve dashboard.yaml --port 5007

Production

# Production server
panel serve dashboard.yaml \
  --port 80 \
  --num-procs 4 \
  --allow-websocket-origin=analytics.company.com

Docker

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY dashboard.yaml data/ ./
CMD ["lumen", "serve", "dashboard.yaml", "--port", "5006", "--address", "0.0.0.0"]

See: Deployment Guide for production deployment best practices.

Best Practices

1. Source Organization

# ✅ Good: Descriptive names
sources:
  sales_database:
    type: postgres
    tables: [orders, customers]

  inventory_files:
    type: file
    tables:
      stock: ./inventory.csv

# ❌ Bad: Generic names
sources:
  db1:
    type: postgres
  file1:
    type: file

2. Pipeline Reusability

# Define reusable pipelines
pipelines:
  base_sales:
    source: data
    table: sales
    filters:
      - type: widget
        field: region

  summary_sales:
    pipeline: base_sales  # Extends base_sales
    transforms:
      - type: aggregate
        by: ['category']
        aggregate:
          total: {revenue: sum}

3. Performance

# Limit data size for large tables
sources:
  bigdata:
    type: postgres
    tables:
      events: "SELECT * FROM events WHERE date >= '2024-01-01' LIMIT 100000"

4. User Experience

# Provide clear labels and formatting
filters:
  - type: widget
    field: region
    label: "Sales Region"  # Clear label

views:
  - type: indicator
    field: revenue
    title: "Total Revenue"
    format: '${value:,.0f}'  # Formatted display

Troubleshooting

Dashboard Won't Load

# Check YAML syntax
python -c "import yaml; yaml.safe_load(open('dashboard.yaml'))"

# Run with debug logging
lumen serve dashboard.yaml --log-level=debug

Data Not Showing

• Verify data source path/connection
• Check table names match YAML config
• Ensure columns referenced exist in data

Performance Issues

• Limit query results (use SQL WHERE clauses)
• Reduce number of rows displayed
• Use aggregation before visualization

See: Troubleshooting Guide for common issues.

Progressive Learning Path

Level 1: Basics

1. Create simple file-based dashboard
2. Add filters
3. Create basic views

Resources:

• Quick Start (this doc)
• Data Sources Reference

Level 2: Transforms

1. Filter and aggregate data
2. Use SQL transforms
3. Chain multiple transforms

Resources:

• Data Transforms Reference

Level 3: Advanced Layouts

1. Multi-page dashboards
2. Cross-filtering
3. Custom themes

Resources:

• Layouts Reference
• Views Reference

Level 4: Production

1. Database integration
2. Authentication
3. Deployment

Resources:

• Deployment Guide

Additional Resources

Documentation

• Data Sources Reference - All source types and configuration
• Data Transforms Reference - Complete transform reference
• Views Reference - All visualization types
• Layouts Reference - Layout patterns and organization
• Python API Reference - Programmatic dashboard creation
• Deployment Guide - Production deployment
• Examples - Complete dashboard examples
• Troubleshooting Guide - Common issues

External Links

• Lumen Documentation
• Lumen Gallery
• GitHub Repository
• Community Discourse

Use Cases

Business Intelligence

• Executive dashboards
• Sales analytics
• Financial reporting
• Operational metrics

Data Exploration

• Dataset overview
• Interactive filtering
• Drill-down analysis
• Comparative views

Real-Time Monitoring

• Live data feeds
• Alert dashboards
• System metrics
• Performance tracking

Reporting

• Scheduled reports
• Standardized views
• Shareable dashboards
• Embedded analytics

Summary

Lumen enables rapid dashboard development through declarative YAML specifications.

Strengths:

• No Python code required
• Fast development cycle
• Reproducible specifications
• Built-in interactivity
• Standard dashboard patterns

Ideal for:

• Fixed dashboard layouts
• Standard data patterns
• Non-programmer dashboard creators
• Rapid prototyping

Consider alternatives when:

• Need custom application logic → Panel Dashboards
• Need ad-hoc exploration → Lumen AI
• Building novel interactions → Panel Dashboards

Related Skills

• Lumen AI - Conversational data exploration
• Panel Dashboards - Custom Python dashboards
• Plotting Fundamentals - Quick plotting with hvPlot