Centralizing logs almost always ends with a storage bill that doubles every quarter. The usual cause is not the log volume itself: it is a tool choice that indexes the content of every line as if it were a full-text search engine, when most questions asked of logs start with “which service, over what time window” rather than a free-text search.

What Loki does not index, and why that changes everything

Elasticsearch indexes the content of every log line word by word: everything becomes searchable, at the cost of heavy ingestion CPU and disk space, often several times the size of the raw logs. Loki starts from a different principle, borrowed directly from Prometheus: it only indexes the labels attached to a log stream (the service name, the environment, the pod), never the text content of the lines themselves.

# Labels attached to a log stream by Promtail/Alloy
labels:
  app: payment-api
  namespace: production
  pod: payment-api-7d9f8c-x2k1p

Raw line content is compressed and stored as-is in chunks, on cheap object storage (S3, GCS, or a self-hosted equivalent like MinIO). A LogQL query always starts by filtering on labels to identify the small set of relevant chunks, then greps that reduced subset:

{app="payment-api", namespace="production"} |= "timeout"

That label-filtering step before any text search is what lets Loki cost a fraction of a fully-indexed system’s storage, for the same log volume.

The cardinality trap: labels are not search fields

The mistake that breaks Loki’s economics is treating labels like database columns and putting high-cardinality values in them: an HTTP request ID, a user ID, a precise timestamp. Every unique combination of label values creates a distinct stream, and Loki builds a separate index per stream. With a request_id as a label, every HTTP request creates its own stream: the index explodes, ingestion memory explodes with it, and the exact problem Loki was meant to avoid (an ever-growing index) comes back through the side door.

The practical rule: labels identify which log stream, not make every individual line searchable by some precise field. A request ID, a user agent, a transaction amount stay in the line’s content, filterable with |= or a LogQL expression, never as a label.

# Label to avoid: request_id="a3f9c2e1..." (unbounded cardinality)
# Better: keep request_id in the line's text, query it when needed
{app="payment-api"} | json | request_id="a3f9c2e1-..."

LogQL: structured at read time, not at write time

Loki accepts unstructured logs (plain text) as readily as structured ones (JSON), and structure parsing happens at query time rather than at ingestion. The | json operator in a LogQL query parses each line on the fly and exposes its fields as filterable attributes, without ever having to declare them as labels ahead of time:

{app=~"api-.*"} | json | duration > 500ms and status >= 500

This shifts the cost of parsing from ingestion (paid once, forever, even if nobody ever looks at those logs) to reading (paid only when someone actually asks the question). For a log volume that far exceeds what gets actively consulted day to day, that inversion is what keeps the economics workable.

Where this fits in an observability stack

Loki complements the golden signals Prometheus exposes rather than replacing them: metrics say a problem exists and since when, logs say what actually happened line by line at that exact moment. The same principle running through the rest of an observability stack applies here too: build a well-structured dashboard around the questions actually asked, not around whatever is technically possible to collect.

What to remember

Loki costs less because it only indexes labels, never line content; label filtering narrows the search space first, text search then runs on a small subset. The trap that erases this advantage is putting high-cardinality values into labels instead of into the line content. Building a log stack that stays usable and affordable as volume grows is part of what a reliability and observability engagement covers.