LogQL Query Generator

Overview

Interactive workflow for generating production-ready LogQL queries. LogQL is Grafana Loki's query language—distributed grep with labels for filtering, plus aggregation and metrics capabilities.

When to Use This Skill

• Creating LogQL queries for log analysis, dashboards, or alerting
• Converting log analysis requirements into LogQL expressions
• Troubleshooting applications through log analysis
• Working with structured logs (JSON, logfmt)

Interactive Query Planning Workflow

CRITICAL: Always engage the user in collaborative planning before generating queries.

Stage 1: Understand the Goal

Gather requirements using AskUserQuestion:

1. Primary Goal: Error analysis, performance tracking, security monitoring, debugging, pattern detection?
2. Use Case: Dashboard, alerting rule, ad-hoc troubleshooting, metrics generation?
3. Context: Application/service, environment, time range, log format?

Stage 2: Identify Log Sources

1. Labels: What labels identify your logs? (

   job

   ,

   namespace

   ,

   app

   ,

   level

   ,

   service_name

   )
2. Log Format: JSON, logfmt, plain text, or custom?
3. Strategy: Use labels for stream selection (indexed), line filters for content (not indexed)

Stage 3: Determine Query Parameters

1. Query Type: Log query (return lines) or metric query (calculate values)?
2. Filtering: Stream selector

   {job="app"}

   , line filters

   |= "error"

   , label filters

   | status >= 500

3. Parsing:

   | json

   ,

   | logfmt

   ,

   | pattern "<ip> - <user>"

   ,

   | regexp "(?P<field>...)"

4. Aggregation:

   count_over_time()

   ,

   rate()

   ,

   sum by (label)

   ,

   quantile_over_time()

5. Time Range:

   [5m]

   ,

   [1h]

   ,

   [24h]

Stage 4: Present Plan & Confirm

Before generating code, present a plain-English plan:

## LogQL Query Plan

**Goal**: [Description]
**Query Structure**:
1. Select streams: `{label="value"}`
2. Filter lines: [operations]
3. Parse logs: [parser]
4. Aggregate: [function]

**Does this match your intentions?**

Use AskUserQuestion to confirm before proceeding.

Stage 4a: Consult Reference Files (REQUIRED for Complex Queries)

MANDATORY: Use the Read tool to explicitly read reference files during skill execution. Do NOT rely on prior knowledge or cached information.

BEFORE generating queries, when the query involves ANY of the following, you MUST use the Read tool:

Read examples/common_queries.logql

Read references/best_practices.md

Read references/best_practices.md

Read references/best_practices.md

Read examples/common_queries.logql

Read examples/common_queries.logql

How to consult (MUST use Read tool):

# Use the Read tool with these paths during skill execution:
Read(".claude/skills/logql-generator/examples/common_queries.logql")   # For query patterns
Read(".claude/skills/logql-generator/references/best_practices.md")    # For optimization and anti-patterns

Example workflow:

1. User requests alerting rule with topk aggregation
2. MUST call:

   Read examples/common_queries.logql

   to get topk patterns
3. MUST call:

   Read references/best_practices.md

   to get alerting best practices (#19-21, #39)
4. Then generate query using patterns from the files you just read

Why this matters: Reference files contain battle-tested patterns and edge cases not covered in the skill overview. Explicit consultation during each skill execution ensures you use the latest patterns and prevents syntax errors.

Stage 5: Generate Query

Best Practices

1. Specific Stream Selectors:

   {namespace="prod", app="api", level="error"}

   not just

   {namespace="prod"}

2. Filter Order: Line filter → parse → label filter (fastest to slowest)
3. Parser Performance: pattern > logfmt > json > regexp

Core Query Patterns

Log Filtering:

{job="app"} |= "error" |= "timeout"        # Contains both
{job="app"} |~ "error|fatal|critical"       # Regex match
{job="app"} != "debug"                      # Exclude

JSON/logfmt Parsing:

{app="api"} | json | level="error" | status_code >= 500
{app="app"} | logfmt | caller="database.go"

Pattern Extraction:

{job="nginx"} | pattern "<ip> - - [<_>] \"<method> <path>\" <status> <size>"

Metrics:

# Rate
rate({job="app"} | json | level="error" [5m])

# Count by label
sum by (app) (count_over_time({namespace="prod"} | json [5m]))

# Error percentage
sum(rate({app="api"} | json | level="error" [5m])) / sum(rate({app="api"}[5m])) * 100

# Latency percentiles
quantile_over_time(0.95, {app="api"} | json | unwrap duration [5m])

# Top N
topk(10, sum by (error_type) (count_over_time({job="app"} | json | level="error" [1h])))

Formatting:

{job="app"} | json | line_format "{{.level}}: {{.message}}"
{job="app"} | json | label_format env="{{.environment}}"

IP Filtering (prefer label filter after parsing for precision):

{job="nginx"} | logfmt | remote_addr = ip("192.168.4.0/24")

Stage 5a: Incremental Query Building (Educational/Debugging)

When to use this stage:

• User is learning LogQL
• Complex multi-stage queries
• Debugging query issues
• User explicitly requests step-by-step explanation

Present the query construction incrementally:

## Building Your Query Step-by-Step

### Step 1: Stream Selector (verify logs exist)
```logql
{app="api"}

Test this first to confirm logs are flowing

Step 2: Add Line Filter (fast pre-filtering)

{app="api"} |= "error"

Reduces data before parsing

Step 3: Add Parser (extract fields)

{app="api"} |= "error" | json

Now you can filter on extracted labels

Step 4: Add Label Filter (precise filtering)

{app="api"} |= "error" | json | level="error"

Final filter on parsed data

Step 5: Add Aggregation (if metric query)

sum(count_over_time({app="api"} |= "error" | json | level="error" [5m]))

Complete metric query

**Benefits of incremental building:**
1. Identify which step breaks (no results, parse errors)
2. Understand performance impact of each operation
3. Debug unexpected results by testing each stage
4. Learn LogQL query structure naturally

**Use AskUserQuestion** to offer incremental mode:
- Option: "Show step-by-step construction" vs "Show final query only"

### Stage 6: Provide Usage

1. **Final Query** with explanation
2. **How to Use**: Grafana panel, Loki alerting rules, `logcli query`, HTTP API
3. **Customization**: Labels to modify, thresholds to tune

## Advanced Techniques

### Multiple Parsers
```logql
{app="api"} | json | regexp "user_(?P<user_id>\\d+)"

Unwrap for Numeric Metrics

sum(sum_over_time({app="api"} | json | unwrap duration [5m]))

Pattern Match Operators (Loki 3.0+, 10x faster than regex)

{service_name=`app`} |> "<_> level=debug <_>"

Logical Operators

{app="api"} | json | (status_code >= 400 and status_code < 500) or level="error"

Offset Modifier

sum(rate({app="api"} | json | level="error" [5m])) - sum(rate({app="api"} | json | level="error" [5m] offset 1d))

Label Operations

{app="api"} | json | keep namespace, pod, level
{app="api"} | json | drop pod, instance

Note: LogQL has no

dedup

or

distinct

operators. Use metric aggregations like

sum by (field)

for programmatic deduplication.

Loki 3.x Key Features

Structured Metadata

High-cardinality data without indexing (trace_id, user_id, request_id):

# Filter AFTER stream selector, NOT in it
{app="api"} | trace_id="abc123" | json | level="error"

Query Acceleration (Bloom Filters)

Place structured metadata filters BEFORE parsers:

# ACCELERATED
{cluster="prod"} | detected_level="error" | logfmt | json
# NOT ACCELERATED
{cluster="prod"} | logfmt | json | detected_level="error"

approx_topk (Probabilistic)

approx_topk(10, sum by (endpoint) (rate({app="api"}[5m])))

vector() for Alerting

sum(count_over_time({app="api"} | json | level="error" [5m])) or vector(0)

Automatic Labels

• service_name: Auto-populated from container name
• detected_level: Auto-detected when

  discover_log_levels: true

  (stored as structured metadata)

Function Reference

Log Range Aggregations

rate(log-range)

count_over_time(log-range)

bytes_rate(log-range)

absent_over_time(log-range)

Unwrapped Range Aggregations

sum_over_time

avg_over_time

max_over_time

min_over_time

quantile_over_time(φ, range)

first_over_time

last_over_time

Aggregation Operators

sum

avg

min

max

count

stddev

topk

bottomk

approx_topk

sort

sort_desc

With grouping:

sum by (label1, label2)

sum without (label1)

Conversion Functions

duration_seconds(label)

bytes(label)

label_replace()

label_replace(rate({job="api"} |= "err" [1m]), "foo", "$1", "service", "(.*):.*")

Parser Reference

logfmt

| logfmt [--strict] [--keep-empty]

• --strict

  : Error on malformed entries

• --keep-empty

  : Keep standalone keys

JSON

| json                                           # All fields
| json method="request.method", status="response.status"  # Specific fields
| json servers[0], headers="request.headers[\"User-Agent\"]"  # Nested/array

Template Functions

Common functions for

line_format

label_format

String:

trim

upper

lower

replace

trunc

substr

printf

contains

hasPrefix

add

sub

mul

div

addf

subf

floor

ceil

round

date

now

unixEpoch

toDate

duration_seconds

regexReplaceAll

count

fromJson

default

int

float64

__line__

__timestamp__

See

examples/common_queries.logql

Alerting Rules

# Alert when error rate exceeds 5%
(sum(rate({app="api"} | json | level="error" [5m])) / sum(rate({app="api"}[5m]))) > 0.05

# With vector() to avoid "no data"
sum(rate({app="api"} | json | level="error" [5m])) or vector(0) > 10

Error Handling

Documentation Lookup

When to Fetch External Documentation (MANDATORY)

Trigger context7 MCP or WebSearch when the query involves ANY of these:

structured metadata

bloom filters

detected_level

approx_topk

approx_topk probabilistic

|>

!>

pattern match operator

vector()

vector function alerting

recording rules loki

grafana alloy loki

How to Use

context7 MCP (preferred - authoritative docs):

1. mcp__context7__resolve-library-id with libraryName="grafana loki"
2. mcp__context7__get-library-docs with context7CompatibleLibraryID and topic="[specific topic]"

WebSearch (fallback for latest features):

WebSearch query: "Grafana Loki LogQL [topic] documentation [year]"

Example Workflow

When user asks for "error tracking with trace correlation in Loki 3.x":

1. Recognize trigger: "Loki 3.x" + "trace" → structured metadata
2. Fetch docs:

   mcp__context7__get-library-docs

   with topic="structured metadata trace_id"
3. Apply patterns from docs to generate accurate query

Resources

• examples/common_queries.logql: Comprehensive query examples
• references/best_practices.md: 39+ LogQL best practices, performance optimization, anti-patterns

Guidelines

1. Always plan interactively - Present plain-English plan before generating
2. Use AskUserQuestion - Gather requirements and confirm plans
3. MUST use Read tool for complex queries - Explicitly call

   Read

   on

   examples/common_queries.logql

   and

   references/best_practices.md

   during skill execution for alerting rules, topk, percentiles, or performance-critical queries. Do NOT skip this step or rely on prior knowledge.
4. Fetch docs for advanced features - Use context7 MCP when Loki 3.x features, approx_topk, or unclear syntax is involved (see Documentation Lookup triggers)
5. Offer incremental building - For learning or debugging, present step-by-step query construction (see Stage 5a)
6. Explain queries - What it does, how to interpret results
7. Prioritize performance - Specific selectors, filter early, simpler parsers

Version Notes

• Loki 3.0+: Bloom filters, structured metadata, pattern match operators (

  |>

  ,

  !>

  )
• Loki 3.3+:

  approx_topk

  function
• Loki 3.5+: Promtail deprecated (use Grafana Alloy)
• Loki 3.6+: Horizontally scalable compactor, Loki UI as Grafana plugin

Deprecations: Promtail (use Alloy), BoltDB store (use TSDB with v13 schema)