Observability & Monitoring

You can't fix what you can't see. Modern systems require comprehensive observability across metrics, logs, traces, and custom telemetry to understand what's actually happening in production.

My expertise

I build production-grade observability systems that go beyond basic monitoring to provide deep insights into system behavior, performance bottlenecks, and business metrics.

What I instrument

The three pillars, plus one

Modern observability is built on metrics, logs, and traces — plus custom telemetry for domain-specific insights.

1. Metrics: What's Happening

Time-series data that answers "what is the system doing right now?"

• System Metrics: CPU, memory, disk I/O, network
• Application Metrics: Request rate, latency (p50/p95/p99), error rate
• Business Metrics: Sign-ups, checkouts, revenue, active users
• Custom Metrics: LLM tokens used, cache hit rate, queue depth

Tools: Prometheus, DataDog, CloudWatch, Grafana

2. Logs: What Happened

Event records that answer "why did this specific thing happen?"

• Structured Logging: JSON format for queryability
• Correlation IDs: Link logs across services
• Log Levels: DEBUG, INFO, WARN, ERROR, CRITICAL
• Sampling: Reduce volume while maintaining signal

Tools: ELK Stack, Loki, CloudWatch Logs, DataDog Logs

3. Traces: Where Time Went

Request flow that answers "where is time being spent?"

• Distributed Tracing: Follow requests across microservices
• Span Attributes: Enrich traces with context
• Latency Waterfall: Visualize where time is spent
• Trace Sampling: Balance cost with coverage

Tools: OpenTelemetry, Jaeger, Zipkin, DataDog APM

4. Custom Telemetry: Domain-Specific Insights

Specialized tracking for AI, ML, and business-critical flows

• LLM Tracking: Model, tokens, cost, latency, quality
• RAG Metrics: Retrieval accuracy, reranking effectiveness
• Cost Attribution: Per-customer, per-feature cost tracking
• Quality Scores: Manual evaluation, automated scoring

Case studies

Real-world observability implementations:

Observability patterns

Golden Signals (SRE)

The four metrics that matter most:

• Latency: How long requests take
• Traffic: How many requests
• Errors: How many requests fail
• Saturation: How close to capacity

RED Method (Services)

For request-driven services:

• Rate: Requests per second
• Errors: Number of failed requests
• Duration: Time per request (p50/p95/p99)

USE Method (Resources)

For resource monitoring:

• Utilization: % of time resource is busy
• Saturation: Amount of queued work
• Errors: Error count

Custom for AI Systems

For LLM/AI applications:

• Cost per Request: Track spending in real-time
• Quality Score: User feedback, automated eval
• Token Usage: Input/output tokens per request
• Cache Hit Rate: Reduce redundant LLM calls

Stack recommendations

For Startups (<$1M ARR)

Focus: Low cost, easy setup

• Metrics: Prometheus + Grafana (self-hosted) or CloudWatch
• Logs: CloudWatch Logs or Loki
• Traces: OpenTelemetry → Jaeger (self-hosted)
• Alerting: Grafana Alerts or PagerDuty free tier

Total Cost: $0-100/month

For Growth Companies ($1M-10M ARR)

Focus: Managed services, less ops overhead

• All-in-one: DataDog or New Relic
• Logs: DataDog Logs or ELK Cloud
• Traces: DataDog APM or Honeycomb
• Alerting: Built-in + PagerDuty

Total Cost: $500-2000/month

For Enterprise (>$10M ARR)

Focus: Scale, compliance, multi-region

• Metrics: DataDog, Prometheus (Thanos for long-term storage)
• Logs: Splunk, ELK, DataDog
• Traces: DataDog APM, custom OpenTelemetry pipeline
• Alerting: PagerDuty, OpsGenie

Total Cost: $5K-50K+/month

Related writing

• Observability for LLM apps: Logs, Traces, and Feedback — the signals that matter when LLM features fail silently, drift in quality, or get too expensive.
• My AI Delivery Principles: Reliability, Observability, ROI — how I keep AI projects reliable, observable, and tied to measurable business value.
• Benchmarks that matter: latency, throughput, dollars — which numbers matter when user experience, throughput, and cost are on the line.

All writing

Common questions

What's the difference between monitoring and observability?

Monitoring answers known questions: "Is the system up? Is latency high?" It's based on predefined metrics and alerts.

Observability answers unknown questions: "Why is this user's request slow?" It requires rich telemetry (metrics, logs, traces) that lets you explore and debug novel issues.

You need both — monitoring for known issues, observability for debugging novel problems.

How much does a good observability stack cost?

It varies by scale:

• Startup (self-hosted): $0-100/month
• Growth (managed): $500-2000/month (DataDog, New Relic)
• Enterprise: $5K-50K+/month depending on data volume

I help clients choose the right stack for their budget and scale, balancing cost with capability.

Do you help with setting up observability from scratch?

Yes. I can design and implement a complete observability stack including instrumentation strategy, tool selection, dashboard creation, alerting setup, and runbook documentation.

Typical timeline: 2-4 weeks for initial setup, with ongoing iteration as your system evolves.

What about AI/LLM-specific observability?

AI systems need custom telemetry beyond standard metrics. I implement cost tracking per request, prompt/completion logging, quality scoring, token usage monitoring, and latency analysis across multiple LLM providers.

This gives you visibility into what's actually happening (and what it's costing) in your AI systems.

Related topics: Production AI Systems · Performance Optimization · Case studies