I spent the first two years of my career treating logs as the thing that fills up the disk. A console.log("here") sprinkled through the code, a timestamp, maybe the error message. Then I worked on a production incident where the only information I had was a spike in HTTP 500s and no useful logs, and spent three hours reading source code to figure out what had happened. After that I started treating observability as a first-class concern.
Observability is a bigger word than logging – it encompasses logs, metrics, and traces – but for most small teams starting from scratch, getting structured logging right is the highest-leverage first step. Metrics and tracing build on top of that foundation.
Structured logging, not free-text
The difference between useful logs and useless logs is usually structure. A free-text log line like:
2026-04-12 14:32:01 User 4821 completed checkoutis readable to a human but hard to query programmatically. You can’t group by user ID, filter to checkouts over a certain amount, or join with other events for that user. A structured log line:
{
"timestamp": "2026-04-12T14:32:01.234Z",
"level": "info",
"event": "checkout.completed",
"user_id": 4821,
"order_id": "ord_9f2a8b",
"amount_cents": 4999,
"currency": "USD",
"duration_ms": 143
}can be indexed, queried, aggregated, and alerted on. Every major log aggregation service (Datadog, Grafana Loki, AWS CloudWatch Logs Insights, Elastic) treats structured JSON as a first-class citizen. The query difference is significant: finding all checkout failures over $100 for a given user is a one-liner against structured data; against free text it’s a fragile regex.
In Node.js, pino is the go-to structured logger – fast, JSON-first, low overhead. In Python, the standard logging module can be configured for JSON output, or structlog gives you a nicer API. In Go, slog (added in Go 1.21) is now the standard library option.
# Node.js with pino
import pino from 'pino'
const logger = pino({
level: process.env.LOG_LEVEL ?? 'info',
base: { service: 'api', env: process.env.NODE_ENV },
})
logger.info({ userId: 4821, orderId: 'ord_9f2a8b', durationMs: 143 }, 'checkout.completed')What to log and what to skip
Log at the boundaries of your system: incoming HTTP requests and their responses, outgoing calls to external services, job starts and completions, significant state transitions in your domain. Don’t log inside tight loops or at the database query level by default – it’s noise and IO overhead.
A request log should capture: timestamp, method, path, status code, response time in milliseconds, request ID (more on this shortly), user ID if authenticated, and any error. That’s enough to reconstruct what happened for almost any incident.
Things to never log: passwords, tokens, full card numbers, PII in query parameters. It sounds obvious but it’s easy to accidentally capture these when you log the full request or serialize a user object. Audit what goes into your logs before they reach a third-party service.
Correlation IDs
When a request touches multiple services or spawns background jobs, you need a way to trace a user’s action across all the log lines it produced. A correlation ID (or trace ID) is a random identifier generated at the start of a request and propagated through every log line and every downstream call that request triggers.
# Express middleware to attach a request ID
import { randomUUID } from 'crypto'
app.use((req, res, next) => {
req.requestId = req.headers['x-request-id'] ?? randomUUID()
res.setHeader('x-request-id', req.requestId)
req.log = logger.child({ requestId: req.requestId, userId: req.user?.id })
next()
})With this in place, every log line from a request includes the same requestId. Finding all logs for a specific failed request is a single query: requestId = "abc123". When you hand off work to a background job, pass the requestId as part of the job payload so the worker logs it too.
Metrics: the signal layer
Logs tell you what happened. Metrics tell you what’s happening right now at a glance. The four golden signals (latency, traffic, errors, saturation) from the Google SRE book are a practical starting point.
For most applications, you want: HTTP request rate and error rate per endpoint, p50/p95/p99 response times, queue depth and job failure rate if you have background jobs, and basic infrastructure metrics (CPU, memory, database connection pool usage).
Prometheus + Grafana is the standard open-source stack for metrics. If you’re on a managed service, Datadog and New Relic give you metrics, logs, and traces integrated – the cost is justified once you’re spending engineering hours debugging production issues without them.
Error tracking
Unhandled exceptions need to go somewhere visible before a user reports them. Sentry is the default choice – free tier covers most small apps, the SDK integrates in a few lines, and the grouping algorithm is good at collapsing duplicate errors into a single issue.
import * as Sentry from '@sentry/node'
Sentry.init({
dsn: process.env.SENTRY_DSN,
environment: process.env.NODE_ENV,
tracesSampleRate: 0.1, // 10% of requests for performance monitoring
})Wire up Sentry before you go to production. The first time you see a user-affecting bug surface in Sentry an hour before the user reports it is when you understand why.
Log aggregation
Logs written to stdout on a container are ephemeral – they disappear with the container. In a multi-instance deployment you also need logs from all instances in one place. Ship logs to a centralized service: Grafana Loki (self-hosted), Logtail, Papertrail, or Datadog Logs.
The simplest integration is usually a logging driver in your container runtime that forwards stdout to the aggregation service, rather than adding SDK code to your application. Docker’s GELF or Fluentd logging drivers, or the CloudWatch Logs driver on AWS, handle this at the infrastructure level.
The OpenTelemetry project is worth watching – it’s becoming the standard for instrumenting applications to emit logs, metrics, and traces in a vendor-neutral format: opentelemetry.io/docs. The Google SRE book’s chapter on monitoring is freely available and still the best conceptual introduction: sre.google/sre-book.