Vertex AI Media Master - Comprehensive Multimodal AI Operations

This Agent Skill provides comprehensive mastery of Google Vertex AI multimodal capabilities for video, audio, image, and text processing with focus on marketing applications.

Core Capabilities

🎥 Video Processing (Gemini 2.0/2.5)

• Video Understanding: Process videos up to 6 hours at low resolution or 2 hours at default resolution
• 2M Context Window: Gemini 2.5 Pro handles massive video content
• Audio Track Processing: Automatic audio transcription from video
• Multi-video Analysis: Process multiple videos in single request
• Video Summarization: Extract key moments, scenes, and insights
• Marketing Use Cases:
  • Analyze competitor video ads
  • Extract highlights from long-form content
  • Generate video summaries for social media
  • Transcribe and caption video content
  • Identify brand mentions and product placements

🎵 Audio Generation & Processing

• Lyria Model (2025): Native audio and music generation
• Speech-to-Text: Transcribe audio with speaker diarization
• Text-to-Speech: Generate natural voiceovers
• Music Composition: Background music for campaigns
• Audio Enhancement: Noise reduction and quality improvement
• Marketing Use Cases:
  • Generate podcast scripts and voiceovers
  • Create audio ads and radio spots
  • Produce background music for video campaigns
  • Transcribe customer interviews
  • Generate multilingual voiceovers

🖼️ Image Generation (Imagen 4 & Gemini 2.5 Flash Image)

• Imagen 4: Highest quality text-to-image generation
• Gemini 2.5 Flash Image: Interleaved image generation with text
• Style Transfer: Apply brand styles to generated images
• Product Visualization: Generate product mockups
• Campaign Assets: Create ad creatives and social media graphics
• Marketing Use Cases:
  • Generate personalized ad images (Adios solution)
  • Create social media graphics at scale
  • Produce product lifestyle images
  • Generate A/B test variations
  • Create branded campaign visuals

📢 Marketing Campaign Automation

• ViGenAiR: Convert long-form video ads to short formats automatically
• Adios: Generate personalized ad images tailored to audience context
• Campaign Asset Generation: Photos, soundtracks, voiceovers from prompts
• Content Pipeline: Email copy, blog posts, social media, PMax assets
• Catalog Enrichment: Multi-agent workflow for product onboarding
• Marketing Use Cases:
  • Automated campaign asset production
  • Personalized content at scale
  • Multi-channel content distribution
  • Product catalog enhancement
  • Visual merchandising automation

🔧 Technical Implementation

API Integration:

from google.cloud import aiplatform
from vertexai.preview.generative_models import GenerativeModel

# Initialize Vertex AI
aiplatform.init(project="your-project", location="us-central1")

# Gemini 2.5 Pro for video
model = GenerativeModel("gemini-2.5-pro")

# Process video with audio
response = model.generate_content([
    "Analyze this video and extract key marketing insights",
    video_file,  # Up to 6 hours
])

# Imagen 4 for image generation
from vertexai.preview.vision_models import ImageGenerationModel
imagen = ImageGenerationModel.from_pretrained("imagen-4")
images = imagen.generate_images(
    prompt="Professional product photo, studio lighting, white background",
    number_of_images=4
)

Gemini 2.5 Flash Image (Interleaved Generation):

# Generate images within text responses
model = GenerativeModel("gemini-2.5-flash-image")
response = model.generate_content([
    "Create a 5-step recipe with images for each step"
])
# Returns text + images interleaved

Audio Generation (Lyria):

from vertexai.preview.audio_models import AudioGenerationModel
lyria = AudioGenerationModel.from_pretrained("lyria")
audio = lyria.generate_audio(
    prompt="Upbeat background music for product launch video, 30 seconds",
    duration=30
)

📊 Marketing Workflow Automation

1. Multi-Channel Campaign Creation:

# Single prompt generates all assets
campaign = model.generate_content([
    """Create a product launch campaign for [product]:
    - Hero image (1920x1080)
    - 3 social media graphics (1080x1080)
    - 30-second video script
    - Background music description
    - Email marketing copy
    - Instagram caption"""
])

2. Video Repurposing Pipeline:

# Long-form to short-form conversion (ViGenAiR approach)
long_video = "gs://bucket/original-ad-60s.mp4"
response = model.generate_content([
    f"Extract 3 engaging 15-second clips from this video for TikTok/Reels",
    long_video
])
# Auto-generates format-specific versions

3. Personalized Ad Generation:

# Context-aware image generation (Adios approach)
for audience in audiences:
    ad_image = imagen.generate_images(
        prompt=f"Product ad for {product}, targeting {audience.demographics}, {audience.style_preference}",
        aspect_ratio="16:9"
    )

🎯 Best Practices for Jeremy

1. Project Setup:

# Set environment variables
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_APPLICATION_CREDENTIALS="path/to/service-account.json"

# Install SDK
pip install google-cloud-aiplatform[vision,audio] google-generativeai

2. Rate Limits & Quotas:

• Gemini 2.5 Pro: 2M tokens/min (video processing)
• Imagen 4: 100 images/min
• Monitor usage in Cloud Console

3. Cost Optimization:

• Use Gemini 2.5 Flash for faster, cheaper operations
• Batch image generation requests
• Cache video embeddings for repeated analysis
• Use low-resolution video setting when appropriate

4. Security & Compliance:

• Keep API keys in Secret Manager, never in code
• Use service accounts with minimal permissions
• Enable VPC Service Controls for data residency
• Log all API calls for audit trails

🚀 Advanced Marketing Use Cases

1. Campaign Performance Analysis:

# Analyze competitor campaigns
competitor_videos = ["gs://bucket/competitor1.mp4", "gs://bucket/competitor2.mp4"]
analysis = model.generate_content([
    "Compare these competitor videos: themes, messaging, CTAs, production quality",
    *competitor_videos
])

2. Content Localization:

# Generate multilingual campaigns
for lang in ["en", "es", "fr", "de", "ja"]:
    localized_content = model.generate_content([
        f"Translate and culturally adapt this campaign for {lang} market:",
        campaign_brief,
        hero_image
    ])

3. A/B Test Generation:

# Generate variations automatically
variations = []
for style in ["minimalist", "bold", "luxury", "playful"]:
    variation = imagen.generate_images(
        prompt=f"Product ad, {style} style, {brand_guidelines}",
        number_of_images=1
    )
    variations.append(variation)

📚 Reference Documentation

Official Documentation:

• Vertex AI Multimodal: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/overview
• Gemini 2.5 Pro: https://cloud.google.com/vertex-ai/generative-ai/docs/models
• Imagen 4: https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview
• Video Understanding: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding

Marketing Solutions:

• GenAI for Marketing: https://github.com/GoogleCloudPlatform/genai-for-marketing
• ViGenAiR (video repurposing)
• Adios (personalized ad images)

Pricing:

• Gemini 2.5 Pro: $3.50/1M input tokens, $10.50/1M output tokens
• Imagen 4: $0.04/image
• Video processing: Included in Gemini token pricing

When This Skill Activates

This skill automatically activates when you mention:

• Video processing, analysis, or understanding
• Audio generation, music composition, or voiceovers
• Image generation, ad creatives, or visual content
• Marketing campaigns, content automation, or asset production
• Gemini multimodal capabilities
• Vertex AI media operations
• Social media content, email marketing, or PMax campaigns

Integration with Other Tools

Google Cloud Services:

• Cloud Storage for media asset management
• BigQuery for campaign analytics
• Cloud Functions for automation triggers
• Vertex AI Pipelines for content workflows

Third-Party Integrations:

• Social media APIs (LinkedIn, Twitter, Instagram)
• Marketing automation platforms (HubSpot, Marketo)
• CMS integrations (WordPress, Contentful)
• DAM systems (Bynder, Cloudinary)

Success Metrics

Track These KPIs:

• Asset generation speed (baseline: 5 images/min)
• Content approval rate (target: >80%)
• Campaign personalization scale (target: 1000+ variants)
• Cost per asset (target: <$0.10/image)
• Time saved vs manual production (target: 90% reduction)

---

This skill makes Jeremy a Vertex AI multimodal expert with instant access to video processing, audio generation, image creation, and marketing automation capabilities.