Vibeship-spawner-skills model-optimization

id: model-optimization name: Model Optimization category: ai description: Use when reducing model size, improving inference speed, or deploying to edge devices - covers quantization, pruning, knowledge distillation, ONNX export, and TensorRT optimization

patterns: golden_rules: - rule: "Measure baseline first" reason: "Know what you're optimizing against" - rule: "Quantization before pruning" reason: "Quantization is usually higher ROI" - rule: "Validate on real data" reason: "Don't just check loss, check task metrics" - rule: "Profile on target hardware" reason: "Desktop GPU ≠ mobile ≠ server" - rule: "Combine techniques carefully" reason: "Order matters, some don't compose well" - rule: "Keep the original model" reason: "You may need to retrain"

optimization_techniques: quantization: description: "Reduce numerical precision" methods: - "FP32 → FP16/BF16: 2x memory savings" - "INT8 Dynamic: Weights quantized, activations at runtime" - "INT8 Static: Both quantized, needs calibration" - "GPTQ/AWQ: 4-bit LLM quantization" tools: ["torch.quantization", "bitsandbytes", "GPTQ"] pruning: description: "Remove parameters" types: unstructured: "Remove individual weights (needs sparse hardware)" structured: "Remove entire channels/filters (actual smaller model)" methods: ["Magnitude pruning", "Iterative pruning", "Lottery ticket"] distillation: description: "Train small model from large model outputs" components: - "Hard labels: Standard cross-entropy" - "Soft labels: KL divergence with temperature" - "Feature distillation: Match intermediate representations" export: description: "Convert to optimized runtime" formats: - "ONNX: Cross-framework compatibility" - "TensorRT: NVIDIA GPU optimization" - "OpenVINO: Intel optimization"

quantization_comparison: fp16_bf16: bits: 16 accuracy_loss: "~0%" speedup: "2x" use_case: "Training, inference" int8_dynamic: bits: 8 accuracy_loss: "1-2%" speedup: "2x" use_case: "CPU inference" int8_static: bits: 8 accuracy_loss: "1-3%" speedup: "2-4x" use_case: "Server inference" gptq_4bit: bits: 4 accuracy_loss: "2-5%" speedup: "3-4x" use_case: "LLM inference"

anti_patterns:

• pattern: "Quantizing untrained model" problem: "Poor accuracy" solution: "Train first, then quantize"
• pattern: "Over-pruning without fine-tuning" problem: "Accuracy collapse" solution: "Iterative prune + fine-tune"
• pattern: "Wrong quantization method" problem: "Suboptimal results" solution: "PTQ for inference, QAT for accuracy"
• pattern: "Ignoring target hardware" problem: "No speedup" solution: "Profile on actual deployment target"
• pattern: "Skipping calibration data" problem: "Poor quantization" solution: "Use representative dataset"
• pattern: "Combining techniques blindly" problem: "Degraded accuracy" solution: "Test each step, validate task metrics"

implementation_checklist: before_optimization: - "Baseline accuracy measured" - "Baseline latency/memory measured" - "Target constraints defined (latency, memory, accuracy)" - "Representative calibration data prepared" quantization: - "Choose appropriate precision (INT8, FP16, INT4)" - "PTQ vs QAT decision made" - "Calibration data representative" - "Task accuracy validated (not just loss)" pruning: - "Structured vs unstructured decision" - "Iterative pruning with fine-tuning" - "Target sparsity achievable" - "Hardware supports sparse operations (if unstructured)" deployment: - "ONNX export validated" - "Target runtime tested (ONNX Runtime, TensorRT)" - "End-to-end latency measured" - "Memory footprint verified"

handoffs:

• skill: distributed-training trigger: "training optimized model at scale"
• skill: llm-fine-tuning trigger: "fine-tuning before optimization"
• skill: transformer-architecture trigger: "architecture changes for optimization"

ecosystem: quantization_tools: - "torch.quantization - PyTorch native" - "bitsandbytes - GPU quantization" - "GPTQ - LLM 4-bit quantization" - "AWQ - Activation-aware quantization" export_tools: - "ONNX - Open format" - "TensorRT - NVIDIA optimization" - "OpenVINO - Intel optimization" - "Core ML - Apple devices" inference_runtimes: - "ONNX Runtime" - "TensorRT" - "vLLM" - "llama.cpp"

sources: tutorials: - "ONNX Runtime Optimization Guide" - "TensorRT Documentation" - "PyTorch Quantization Tutorial" papers: - "GPTQ: Accurate Post-Training Quantization for GPT" - "AWQ: Activation-aware Weight Quantization"