Databricks Model Serving

Deploy MLflow models and AI agents to scalable REST API endpoints.

Quick Decision: What Are You Deploying?

mlflow.sklearn.autolog()

mlflow.pyfunc.PythonModel

ResponsesAgent

Prerequisites

• DBR 16.1+ recommended (pre-installed GenAI packages)
• Unity Catalog enabled workspace
• Model Serving enabled

Foundation Model API Endpoints

ALWAYS use exact endpoint names from this table. NEVER guess or abbreviate.

Chat / Instruct Models

databricks-gpt-5-2

databricks-gpt-5-1

databricks-gpt-5-1-codex-max

databricks-gpt-5-1-codex-mini

databricks-gpt-5

databricks-gpt-5-mini

databricks-gpt-5-nano

databricks-gpt-oss-120b

databricks-gpt-oss-20b

databricks-claude-opus-4-6

databricks-claude-sonnet-4-6

databricks-claude-sonnet-4-5

databricks-claude-opus-4-5

databricks-claude-sonnet-4

databricks-claude-opus-4-1

databricks-claude-haiku-4-5

databricks-claude-3-7-sonnet

databricks-meta-llama-3-3-70b-instruct

databricks-meta-llama-3-1-405b-instruct

databricks-meta-llama-3-1-8b-instruct

databricks-llama-4-maverick

databricks-gemini-3-1-pro

databricks-gemini-3-pro

databricks-gemini-3-flash

databricks-gemini-2-5-pro

databricks-gemini-2-5-flash

databricks-gemma-3-12b

databricks-qwen3-next-80b-a3b-instruct

Embedding Models

databricks-gte-large-en

databricks-bge-large-en

databricks-qwen3-embedding-0-6b

Common Defaults

• Agent LLM:

  databricks-meta-llama-3-3-70b-instruct

  (good balance of quality/cost)
• Embedding:

  databricks-gte-large-en

• Code tasks:

  databricks-gpt-5-1-codex-mini

  or

  databricks-gpt-5-1-codex-max

These are pay-per-token endpoints available in every workspace. For production, consider provisioned throughput mode. See supported models.

Reference Files

Quick Start: Deploy a GenAI Agent

Step 1: Install Packages (in notebook or via MCP)

%pip install -U mlflow==3.6.0 databricks-langchain langgraph==0.3.4 databricks-agents pydantic
dbutils.library.restartPython()

Or via MCP:

execute_code(code="%pip install -U mlflow==3.6.0 databricks-langchain langgraph==0.3.4 databricks-agents pydantic")

Step 2: Create Agent File

Create

agent.py

ResponsesAgent

Step 3: Upload to Workspace

manage_workspace_files(
    action="upload",
    local_path="./my_agent",
    workspace_path="/Workspace/Users/you@company.com/my_agent"
)

Step 4: Test Agent

execute_code(
    file_path="./my_agent/test_agent.py",
    cluster_id="<cluster_id>"
)

Step 5: Log Model

execute_code(
    file_path="./my_agent/log_model.py",
    cluster_id="<cluster_id>"
)

Step 6: Deploy (Async via Job)

See 7-deployment.md for job-based deployment that doesn't timeout.

Step 7: Query Endpoint

manage_serving_endpoint(
    action="query",
    name="my-agent-endpoint",
    messages=[{"role": "user", "content": "Hello!"}]
)

Quick Start: Deploy a Classical ML Model

import mlflow
import mlflow.sklearn
from sklearn.linear_model import LogisticRegression

# Enable autolog with auto-registration
mlflow.sklearn.autolog(
    log_input_examples=True,
    registered_model_name="main.models.my_classifier"
)

# Train - model is logged and registered automatically
model = LogisticRegression()
model.fit(X_train, y_train)

Then deploy via UI or SDK. See 1-classical-ml.md.

MCP Tools

If MCP tools are not available, use the SDK/CLI examples in the reference files below.

Development & Testing

manage_workspace_files

execute_code

Deployment

manage_jobs

manage_job_runs

manage_job_runs

manage_serving_endpoint - Querying

get

list

query

Example usage:

# Check endpoint status
manage_serving_endpoint(action="get", name="my-agent-endpoint")

# List all endpoints
manage_serving_endpoint(action="list")

# Query a chat/agent endpoint
manage_serving_endpoint(
    action="query",
    name="my-agent-endpoint",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=500
)

# Query a traditional ML endpoint
manage_serving_endpoint(
    action="query",
    name="sklearn-classifier",
    dataframe_records=[{"age": 25, "income": 50000, "credit_score": 720}]
)

Common Workflows

Check Endpoint Status After Deployment

manage_serving_endpoint(action="get", name="my-agent-endpoint")

Returns:

{
    "name": "my-agent-endpoint",
    "state": "READY",
    "served_entities": [...]
}

Query a Chat/Agent Endpoint

manage_serving_endpoint(
    action="query",
    name="my-agent-endpoint",
    messages=[
        {"role": "user", "content": "What is Databricks?"}
    ],
    max_tokens=500
)

Query a Traditional ML Endpoint

manage_serving_endpoint(
    action="query",
    name="sklearn-classifier",
    dataframe_records=[
        {"age": 25, "income": 50000, "credit_score": 720}
    ]
)

Common Issues

self.create_text_output_item(text, id)

manage_serving_endpoint(action="get")

pip_requirements

resources

log_model

catalog.schema.model_name

Critical: ResponsesAgent Output Format

WRONG - raw dicts don't work:

return ResponsesAgentResponse(output=[{"role": "assistant", "content": "..."}])

CORRECT - use helper methods:

return ResponsesAgentResponse(
    output=[self.create_text_output_item(text="...", id="msg_1")]
)

Available helper methods:

• self.create_text_output_item(text, id)

  - text responses

• self.create_function_call_item(id, call_id, name, arguments)

  - tool calls

• self.create_function_call_output_item(call_id, output)

  - tool results

Related Skills

• databricks-agent-bricks - Pre-built agent tiles that deploy to model-serving endpoints
• databricks-vector-search - Create vector indexes used as retriever tools in agents
• databricks-genie - Genie Spaces can serve as agents in multi-agent setups
• databricks-mlflow-evaluation - Evaluate model and agent quality before deployment
• databricks-jobs - Job-based async deployment used for agent endpoints

Resources

• Model Serving Documentation
• MLflow 3 ResponsesAgent
• Agent Framework