How to catch cardinality explosions in the OpenTelemetry Collector and scrub sensitive data at the pipeline stage

Have you ever discovered — too late — that your metric costs suddenly tripled in production, or that user emails were being left in your logs? If you've ever watched Prometheus storage hit saturation in a single week after adding one user_id label, or spent time fixing dozens of places in application code to comply with GDPR, this post is for you.

OTTL (OpenTelemetry Transformation Language) and the Transform Processor let you solve both problems at the Collector pipeline level — without touching a single line of application code. You can reduce metric cardinality and scrub sensitive data like emails, card numbers, and SSNs before they ever reach your backend. We'll also cover patterns that landed upstream between 2025 and 2026.

Core Concepts

What Is OTTL

OTTL is a domain-specific language (DSL) for reading, modifying, and deleting telemetry data inside an OpenTelemetry Collector pipeline. It lives in the Go-based opentelemetry-collector-contrib repository, and the Transform Processor adopts it as its execution engine.

Each statement consists of two parts: an Editor function (data transformation) and a where condition (boolean expression).

# Basic syntax structure
transform:
  trace_statements:
    - context: span
      statements:
        - <EditorFunction> where <condition>

Think of "context" as "the unit of data I want to operate on" — a single span, a single data point, a single log record, and so on.

Contexts and Signal Types

The Transform Processor runs contexts separately per signal type.

When modifying resource attributes, use the resource context instead of the span context — it's far more efficient. Because contexts are nested, accessing resource.attributes from the span context repeats execution once per span.

Editor Functions at a Glance

keep_matching_keys and merge_maps are not covered directly in the scenarios below. See the official ottlfuncs README for usage context.

Why Cardinality Explosions Are Dangerous

What happens if you use method (5 values), status_code (5 values), pod_id (1,000 values), and user_id (1,000,000 values) simultaneously as metric labels?

5 × 5 × 1,000 × 1,000,000 = 25 billion unique time series. The default cardinality limit in the OpenTelemetry SDK is 2,000 per metric — push this into Prometheus or a SaaS backend and storage and query costs skyrocket exponentially.

The diagram below shows how the time series count changes when you remove two attributes (based on unique value counts).

flowchart LR
    subgraph Before Removal
        direction TB
        A1["method × status × pod_id × user_id"] --> B1["Time series count: 25 billion"]
    end
    subgraph After Removal
        direction TB
        A2["method × status"] --> B2["Time series count: 25"]
    end
    Before Removal -->|"delete_key pod_id, user_id"| After Removal

The numbers look extreme, but in practice label removal alone can produce orders-of-magnitude differences in storage footprint.

Which Processor to Use and When

Several processors can look similar at first, making it hard to know which one to reach for.

The key distinction: the Redaction Processor only handles attributes (key-value pairs) regardless of signal type, and cannot access the log body (whether a string or a map). If you need to scrub card numbers or passwords from inside a log body, the Transform Processor is required.

Practical Application

Scenario 1 — Reducing Metric Cardinality

This is the most common case. Simply removing per-instance unique IDs from metric labels can dramatically cut the number of time series.

processors:
  transform:
    metric_statements:
      - context: datapoint
        statements:
          # Remove unique instance attributes that cause cardinality explosions
          - delete_key(attributes, "container.id")
          - delete_key(attributes, "pod.uid")
          - delete_key(attributes, "server.instance_id")
          # Replace dynamic segments in URL paths with a wildcard
          - replace_pattern(attributes["http.route"], "/[0-9]+", "/{id}")

The last line is especially effective. Thousands of unique paths like /users/12345/orders and /users/67890/orders collapse into a single /users/{id}/orders.

Industry reports have cited this pattern alone cutting storage costs by up to 70%. In the author's own experience, removing a few unique-ID attributes consistently yields 30–40% savings.

Scenario 2 — Masking PII in Span Attributes

This covers cases where emails or phone numbers flow in as span attributes.

processors:
  transform:
    trace_statements:
      - context: span
        statements:
          # Tag PII first if user.email exists (for audit logs)
          - set(attributes["pii.detected"], true) where attributes["user.email"] != nil
          # Mask the local part of the email, preserve the domain
          # In YAML, $$ is interpreted as a literal $
          - replace_pattern(attributes["user.email"], "^(.+)@(.+)$", "****@$$2")
          # Mask phone number pattern
          - replace_pattern(attributes["phone"], "\\d{3}-\\d{4}-\\d{4}", "***-****-****")

There's a reason set comes first. Statements execute in the order they are written, so if you check user.email after masking, you'd be relying on the already-transformed value. If you need to check whether the original value existed, do it before masking.

Leaving the value as ****@company.com lets you identify which domain a user belongs to without pinpointing the individual — satisfying both debugging needs and compliance requirements.

Scenario 3 — Masking Credit Card Numbers and SSNs in Log Bodies

This is the defining reason to use the Transform Processor. You can apply OTTL directly to the log body string.

The example below assumes the log body is a plain string type. If the body is structured (e.g., parsed JSON), access individual fields using the body["field_name"] form.

processors:
  transform:
    log_statements:
      - context: log
        statements:
          # Mask card number pattern
          - replace_pattern(body, "card=\\d{13,19}", "card=***REDACTED***")
          # Mask SSN
          - replace_pattern(body, "\\b\\d{3}-\\d{2}-\\d{4}\\b", "***-**-****")
          # Mask password field
          - replace_pattern(body, "password=[^\\s&]+", "password=***")

The Redaction Processor cannot do this — it requires access to the body string itself, not just attributes.

Scenario 4 — SHA-256 Pseudonymization

Under GDPR, deletion isn't the only option. When you still need to trace the same user during debugging, hash-based pseudonymization is useful.

processors:
  transform:
    trace_statements:
      - context: span
        statements:
          # Replace user.id with a SHA-256 hash (including the uid- prefix)
          # In YAML, $$ is interpreted as a literal $
          - replace_pattern(attributes["user.id"], "(.+)", "$$1", SHA256, "uid-%s")

If you know the original user.id, you can track it via the same hash, while the backend stores only the hash.

Scenario 5 — A Three-Stage Integrated PII Defense Pipeline

Combining the individual scenarios above into a single pipeline produces the following structure, with each processor doing what it does best.

processors:
  # Stage 1: Delete known PII attribute keys
  attributes:
    actions:
      - key: user.password
        action: delete
      - key: credit_card_number
        action: delete
 
  # Stage 2: Block attribute values by pattern (allowlist approach)
  # Redaction Processor handles only attributes regardless of signal type; cannot access log bodies
  redaction:
    allow_all_keys: false
    allowed_keys:
      - http.method
      - http.status_code
      - service.name
    blocked_values:
      - "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b"
      - "\\b(?:\\d[ -]?){13,19}\\b"
 
  # Stage 3: Log body and complex conditional masking
  transform:
    log_statements:
      - context: log
        statements:
          - replace_pattern(body, "ssn=\\d{9}", "ssn=REDACTED")
 
service:
  pipelines:
    traces:
      processors: [attributes, redaction, transform]
    logs:
      processors: [attributes, transform]

Pros and Cons

Advantages

Disadvantages and Caveats

Common Mistakes in Practice

Mistake 1 — Modifying resource attributes from the span context

# Inefficient: executes once per span
- context: span
  statements:
    - delete_key(resource.attributes, "k8s.pod.uid")
 
# Efficient: executes once per resource
- context: resource
  statements:
    - delete_key(attributes, "k8s.pod.uid")

Mistake 2 — Referencing a deleted key later

statements:
  - delete_key(attributes, "user.email")
  # Already deleted, so the condition is always false
  - set(attributes["pii.detected"], true) where attributes["user.email"] != nil

Place set first and delete_key after.

Mistake 3 — Dropping entire signals with Filter to reduce cardinality

When the problem can be solved by removing attributes, solving it by dropping signals eliminates the debugging context entirely. The recommended approach is to reduce labels while keeping signals alive.

Closing Thoughts

Here's the summary:

• OTTL + Transform Processor is the most expressive tool for solving cardinality explosions and PII problems simultaneously from a single pipeline control point.
• The Redaction Processor is optimized for attribute allowlist blocking; the Transform Processor is optimized for log body masking and complex conditional transformations. The boundary between them is not signal type — it's the access target (attributes vs. body).
• Statement execution order, context selection, and regex performance are the three factors that determine operational quality.

A suggested adoption sequence:

1. Identify high-cardinality attributes in your current metric labels — find keys like container.id, pod.uid, request_id, and user_id, and start by removing them with delete_key. You'll see results immediately.

2. Trace the paths where PII leaks from logs and spans — temporarily attach a debug exporter to your Collector and sample actual attribute values. This will make it clear what patterns you need.

3. Deploy the three-stage pipeline (Attributes → Redaction → Transform) to staging first — validate basic syntax with a community playground, confirm rules against a sample of real traffic in staging, then roll out to production.

Into 2026, the Filter Processor has gained support for automatic OTTL context inference, and the Unroll Processor joined contrib in 2025. Together, these changes let you express record branching, expansion patterns, and filtering conditions more concisely. Learning OTTL syntax now means you'll absorb those changes naturally.

References

• OpenTelemetry Official — Transforming Telemetry
• OpenTelemetry Official — Handling Sensitive Data
• Transform Processor README (opentelemetry-collector-contrib)
• OTTL Functions README (ottlfuncs)
• Redaction Processor README (opentelemetry-collector-contrib)
• Dash0 — Mastering the OpenTelemetry Transformation Language (OTTL)
• Dash0 — Mastering the OpenTelemetry Transform Processor
• Dash0 — Scrubbing Sensitive Data from OpenTelemetry Logs, Traces & Metrics
• Better Stack — Redacting Sensitive Data with the OpenTelemetry Collector
• Better Stack — OTTL Recipes for Transforming OpenTelemetry Data
• oneuptime — Handle High-Cardinality Metrics in OpenTelemetry Without Blowing Your Budget
• oneuptime — How to Make Your OpenTelemetry Pipeline GDPR-Compliant
• oneuptime — How to Transform Log Bodies Using OTTL
• OpenTelemetry Blog — OTTL Context Inference Comes to the Filter Processor (2026)
• OpenTelemetry Blog — Contributing the Unroll Processor (2025)
• SigNoz Docs — OTTL OpenTelemetry Transformation Language
• ControlTheory — Control High Metric Cardinality with the OTel Collector Filter Processor
• Last9 — Redacting Sensitive Data in OpenTelemetry Collector
• Dynatrace Docs — Mask Sensitive Data with the OTel Collector
• Honeycomb Docs — Handle Sensitive Information with the OpenTelemetry Collector
• Grafana Alloy — otelcol.processor.transform Reference
• Splunk Observability — Transform Processor
• Sumo Logic Docs — String Hashing and Masking using Transform Processor and OTTL