Claude-skill-registry lobe-vidol

LobeVidol - Virtual idol creation platform with MMD dance support, VRM character customization, multi-provider LLM integration, and interactive 3D conversations

install

source · Clone the upstream repo

git clone https://github.com/majiayu000/claude-skill-registry

Claude Code · Install into ~/.claude/skills/

T=$(mktemp -d) && git clone --depth=1 https://github.com/majiayu000/claude-skill-registry "$T" && mkdir -p ~/.claude/skills && cp -r "$T/skills/data/lobe-vidol" ~/.claude/skills/majiayu000-claude-skill-registry-lobe-vidol && rm -rf "$T"

manifest: skills/data/lobe-vidol/SKILL.md

LobeVidol Skill

LobeVidol (Vidol Studio) is an open-source virtual idol creation platform that enables anyone to create, customize, and interact with 3D virtual characters. It combines conversational AI with MMD (MikuMikuDance) animation support, VRM model customization, and multi-provider LLM integration for immersive virtual idol experiences.

Key Value Proposition: Create virtual idols with exquisite UI design, MMD dance choreography, VRM character customization, and seamless AI-powered conversations - all through an accessible web interface or self-hosted deployment.

When to Use This Skill

Deploying LobeVidol via Vercel or self-hosted
Configuring AI model providers (OpenAI, Claude, Ollama, 20+ providers)
Creating custom virtual characters with VRM models
Setting up MMD dance choreography and stages
Configuring character personalities and touch responses
Integrating TTS/STT for voice conversations
Publishing characters or dances to the marketplace
Troubleshooting deployment or configuration issues

When NOT to Use This Skill

For LobeChat (different project - chat-focused, not idol-focused)
For general MMD editing (use MikuMikuDance directly)
For VRM model creation (use VRoid Studio or Blender)
For AI model API usage without virtual idols (use provider-specific skills)

Core Concepts

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                     LobeVidol / Vidol Studio                     │
│                     (Next.js + TypeScript)                       │
└─────────────────────────────────────────────────────────────────┘
                              │
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
        ▼                     ▼                     ▼
┌───────────────┐    ┌───────────────┐    ┌───────────────┐
│  AI Providers │    │   Characters  │    │    Dances     │
├───────────────┤    ├───────────────┤    ├───────────────┤
│ • OpenAI      │    │ • VRM models  │    │ • MMD/VMD     │
│ • Claude      │    │ • Personality │    │ • PMX stages  │
│ • Gemini      │    │ • Touch resp. │    │ • Choreograph │
│ • Ollama      │    │ • Voice (TTS) │    │ • Mixamo lib  │
│ • 20+ more    │    │ • Marketplace │    │ • Marketplace │
└───────────────┘    └───────────────┘    └───────────────┘
        │                     │                     │
        └─────────────────────┼─────────────────────┘
                              │
        ┌─────────────────────┼─────────────────────┐
        ▼                     ▼                     ▼
┌───────────────┐    ┌───────────────┐    ┌───────────────┐
│   3D Engine   │    │   Features    │    │  Deployment   │
├───────────────┤    ├───────────────┤    ├───────────────┤
│ • Three.js    │    │ • Text chat   │    │ • Vercel      │
│ • three-vrm   │    │ • Video chat  │    │ • Self-host   │
│ • mmd-parser  │    │ • TTS/STT     │    │ • PWA         │
│ • Animations  │    │ • Streaming   │    │ • Docker      │
└───────────────┘    └───────────────┘    └───────────────┘

Project Statistics

Metric	Value
GitHub Stars	887+
Forks	122+
Contributors	Multiple
Total Commits	865+
Primary Language	TypeScript (99.5%)
License	Apache 2.0

Supported AI Providers (22+)

Provider	Type	Notes
OpenAI	Cloud	GPT-4, GPT-3.5 series
Anthropic	Cloud	Claude 3.5, Claude 3
Google Gemini	Cloud	Gemini Pro, Flash
Azure OpenAI	Cloud	Enterprise option
Ollama	Local	Run models locally
Groq	Cloud	Fast inference
DeepSeek	Cloud	Coding focused
Moonshot AI	Cloud	Chinese provider
OpenRouter	Cloud	Multi-model gateway
Together AI	Cloud	Open source models
Perplexity	Cloud	Search-enhanced
Amazon Bedrock	Cloud	AWS managed
Baichuan	Cloud	Chinese provider
MiniMax	Cloud	Chinese provider
Zhipu AI	Cloud	Chinese provider
Tongyi Qianwen	Cloud	Alibaba Cloud
Tencent Hunyuan	Cloud	Chinese provider
iFlytek Xinghuo	Cloud	Chinese provider
Wenxin Qianfan	Cloud	Baidu
SiliconCloud	Cloud	Cloud service
Fireworks AI	Cloud	Fast inference
GitHub Models	Cloud	GitHub integration

Quick Start

Prerequisites

Node.js 18+ or Bun (recommended)
Git
API key for at least one AI provider

Installation

# Clone repository
git clone https://github.com/lobehub/lobe-vidol.git
cd lobe-vidol

# Install dependencies (Bun recommended)
bun install
# Or with npm
npm install

# Start development server
bun dev
# Or with npm
npm run dev

Access

Open

http://localhost:3000

in your browser.

Live Demo

Try without installation: https://vidol.lobehub.com

Configuration

Environment Variables

Create

.env.local

file:

# OpenAI Configuration
OPENAI_API_KEY=sk-xxx
OPENAI_PROXY_URL=  # Optional proxy

# Azure OpenAI (alternative)
AZURE_API_KEY=xxx
AZURE_ENDPOINT=https://xxx.openai.azure.com

# Anthropic Claude
ANTHROPIC_API_KEY=sk-ant-xxx

# Google Gemini
GOOGLE_API_KEY=xxx

# Ollama (local models)
OLLAMA_PROXY_URL=http://localhost:11434

# OpenRouter (multi-provider)
OPENROUTER_API_KEY=sk-or-xxx

# Voice Services
OPENAI_TTS_API_KEY=sk-xxx  # For TTS
EDGE_TTS_ENABLED=true      # Microsoft Edge TTS

Provider Configuration in UI

Click Settings (gear icon)
Navigate to "Language Model" section
Select provider from dropdown
Enter API key
Choose model variant
Test connection

Character Creation

VRM Model Support

LobeVidol uses VRM (Virtual Reality Model) format for 3D characters:

Getting VRM Models:

VRoid Hub: https://hub.vroid.com - Free community models
VRoid Studio: Create your own characters
Blender: Convert existing 3D models with VRM addon
Unity: Export VRM using UniVRM plugin

Character Configuration

// Character configuration structure
interface Character {
  // Basic Info
  name: string;
  avatar: string;          // Image URL
  description: string;
  greeting: string;        // Initial message
  gender: 'male' | 'female' | 'other';
  category: string;

  // AI Personality
  systemRole: string;      // LLM system prompt

  // Voice Settings
  tts: {
    enabled: boolean;
    voice: string;         // Voice ID
    speed: number;
    pitch: number;
  };

  // 3D Model
  model: {
    url: string;           // VRM file URL
    scale: number;
    position: [number, number, number];
  };

  // Touch Responses
  touch: {
    head: string[];        // Responses when head touched
    face: string[];
    arm: string[];
    chest: string[];
    leg: string[];
  };
}

System Role (Personality)

The

systemRole

field defines the character's personality:

You are Luna, a cheerful virtual idol who loves singing and dancing.

## Personality Traits
- Energetic and optimistic
- Loves discussing music and dance
- Speaks with enthusiasm and occasional cute expressions
- Always supportive and encouraging

## Speech Patterns
- Uses casual, friendly language
- Occasionally adds "~" at end of sentences
- Expresses emotions openly

## Background
- A rising virtual idol from Vidol Studio
- Dreams of performing at major virtual concerts
- Enjoys interacting with fans and learning new dances

Touch Response Configuration

Configure how characters respond to interaction:

{
  "touch": {
    "head": [
      "Ehehe~ That tickles!",
      "Are you petting me? How sweet~",
      "My hair took forever to style!"
    ],
    "face": [
      "W-what are you doing?",
      "My cheeks are getting warm...",
      "Do I have something on my face?"
    ],
    "arm": [
      "Want to hold hands?",
      "My arm? Is something there?",
      "Let's dance together!"
    ]
  }
}

Dance & Animation

MMD Support

LobeVidol supports MikuMikuDance (MMD) content:

Supported Formats:

VMD: Vocaloid Motion Data (dance animations)
PMX: Polygon Model eXtended (3D stages)
VRMA: VRM Animation files

Adding Dances

Prepare Files:
- Dance motion file (.vmd)
- Audio file (.mp3, .wav)
- Optional: Stage file (.pmx)
Upload via UI:
- Navigate to Dance section
- Click "Upload Dance"
- Select motion and audio files
- Configure stage (optional)
Test Dance:
- Preview with selected character
- Adjust timing if needed
- Save to library

Built-in Animation Libraries

Mixamo Integration:

Idle animations
Walking/running
Gestures and expressions
Poses for photos

Dance Presets:

Popular MMD choreography
Various music genres
Community contributions

Stage Configuration

interface Stage {
  name: string;
  model: string;      // PMX file URL
  scale: number;
  lighting: {
    ambient: string;  // Hex color
    directional: string;
    intensity: number;
  };
  background: {
    type: 'color' | 'image' | 'video';
    value: string;
  };
}

Voice Features

Text-to-Speech (TTS)

Supported TTS Providers:

Provider	Quality	Languages	Cost
OpenAI Audio	High	Multi	Paid
Microsoft Edge	High	Multi	Free
Browser TTS	Medium	Varies	Free

Configuration:

# OpenAI TTS
OPENAI_TTS_API_KEY=sk-xxx
OPENAI_TTS_MODEL=tts-1-hd

# Edge TTS (free)
EDGE_TTS_ENABLED=true
EDGE_TTS_VOICE=en-US-AriaNeural

Speech-to-Text (STT)

Enable voice input for conversations:

# OpenAI Whisper
OPENAI_STT_API_KEY=sk-xxx
OPENAI_STT_MODEL=whisper-1

# Browser STT (free)
BROWSER_STT_ENABLED=true

Voice Selection

// Available voices (Edge TTS examples)
const voices = {
  english: [
    'en-US-AriaNeural',      // Female, friendly
    'en-US-GuyNeural',       // Male, casual
    'en-GB-SoniaNeural',     // Female, British
  ],
  japanese: [
    'ja-JP-NanamiNeural',    // Female, natural
    'ja-JP-KeitaNeural',     // Male, natural
  ],
  chinese: [
    'zh-CN-XiaoxiaoNeural',  // Female, friendly
    'zh-CN-YunxiNeural',     // Male, narrator
  ]
};

Marketplace

Character Marketplace

Browse and download community characters:

Open Character Market in sidebar
Browse by category or search
Preview character (3D model, personality)
Download to local library

Publishing Characters

Create character with complete configuration
Click "Upload to Market"
Fill metadata (tags, description)
Submit for review
GitHub issue created automatically
Community review process
Merged into marketplace

Dance Marketplace

Access MMD dance resources:

Various choreography styles
Multiple music genres
Stage environments
Community contributions

Deployment

Vercel Deployment (Recommended)

Click "Deploy with Vercel"
Connect GitHub account
Configure environment variables
Deploy

Docker Deployment

# Dockerfile
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:18-alpine AS runner
WORKDIR /app
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public

EXPOSE 3000
CMD ["node", "server.js"]

# Build and run
docker build -t lobe-vidol .
docker run -p 3000:3000 \
  -e OPENAI_API_KEY=sk-xxx \
  lobe-vidol

Docker Compose

version: '3.8'
services:
  lobe-vidol:
    build: .
    ports:
      - "3000:3000"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    restart: unless-stopped

Self-Hosted Requirements

Resource	Minimum	Recommended
CPU	2 cores	4 cores
RAM	2 GB	4 GB
Storage	5 GB	20 GB
Network	Broadband	Low latency

PWA Support

LobeVidol supports Progressive Web App installation:

Desktop Installation

Open https://vidol.lobehub.com in Chrome/Edge
Click install icon in address bar
Confirm installation
Launch from desktop

Mobile Installation

Open in mobile browser (Chrome/Safari)
Tap "Share" or menu icon
Select "Add to Home Screen"
Launch from home screen

PWA Features

Offline capable (cached assets)
Native-like experience
Push notifications (future)
Background sync

API Reference

Chat Endpoint

// POST /api/chat
interface ChatRequest {
  messages: Array<{
    role: 'user' | 'assistant' | 'system';
    content: string;
  }>;
  model: string;
  character?: string;  // Character ID
  stream?: boolean;
}

interface ChatResponse {
  content: string;
  model: string;
  usage: {
    prompt_tokens: number;
    completion_tokens: number;
    total_tokens: number;
  };
}

Character API

// GET /api/characters
interface CharacterListResponse {
  characters: Character[];
  total: number;
}

// GET /api/characters/:id
interface CharacterResponse {
  character: Character;
}

// POST /api/characters
interface CreateCharacterRequest {
  name: string;
  systemRole: string;
  model: string;  // VRM URL
  // ... other fields
}

Troubleshooting

Common Issues

VRM Model Not Loading:

Error: Failed to load VRM model

Verify VRM file URL is accessible
Check CORS settings if hosted externally
Ensure VRM version compatibility (0.x or 1.0)

Dance Animation Issues:

Error: VMD file parsing failed

Check file encoding (Japanese characters)
Verify VMD format compatibility
Try re-exporting from MMD

API Connection Errors:

Error: Failed to connect to OpenAI

Verify API key is valid
Check proxy settings if behind firewall
Confirm model name is correct

TTS Not Working:

Error: TTS initialization failed

Enable Edge TTS as fallback
Check browser permissions for audio
Verify API key for OpenAI TTS

Performance Optimization

3D Rendering:

Reduce model polygon count
Disable shadows on mobile
Use lower resolution textures

Network:

Enable response streaming
Cache static assets
Use CDN for models

Best Practices

Character Design

Clear Personality: Define distinct traits in systemRole
Consistent Voice: Match TTS voice to character personality
Natural Responses: Include varied touch responses
Quality Model: Use optimized VRM with proper rigging

Performance

Optimize Models: Keep VRM under 20MB
Compress Textures: Use WebP or compressed PNG
Stream Responses: Enable for better UX
Cache Assets: Leverage browser caching

Security

API Keys: Never expose in client-side code
Rate Limiting: Implement for public deployments
Input Validation: Sanitize user inputs
CORS: Configure properly for custom domains

Resources

Official Links

Live Demo: https://vidol.lobehub.com
Documentation: https://docs.vidol.chat
GitHub: https://github.com/lobehub/lobe-vidol
Discord: https://discord.gg/AYFPHvv2jT

Related Tools

VRoid Hub: https://hub.vroid.com - VRM model library
VRoid Studio: https://vroid.com/studio - Character creator
MikuMikuDance: MMD animation software
Mixamo: https://mixamo.com - Animation library

Community Resources

Character marketplace contributions
Dance marketplace submissions
Discord community support
GitHub issues and discussions

Version History

Current: Development active
Tech Stack: Next.js, TypeScript, Three.js, three-vrm
License: Apache 2.0

Last Updated: 2026-01-13 Skill Version: 1.0.0 Source: https://github.com/lobehub/lobe-vidol