Skip to content
Incident LabOpenTelemetry

Workshop

16 chapters · about 5 hours · every command says what it does, why, and what you should see.

You need Git and Docker (with Compose). uv is optional, for the tests.

Terminal
git clone https://github.com/jars-demo/otel-demo.git
cd otel-demo
docker compose up -d --build --wait
  1. 00What is Observability?Signals, telemetry, and the question "why is it broken?"15m
  2. 01Build the Distributed SystemStart the shop and the observability stack, place an order20m
  3. 02Instrument Your ApplicationSDK setup, library instrumentation and manual spans20m
  4. 03Understand TracesSpans, kinds, status, events and context propagation20m
  5. 04Add MetricsInstruments, RED metrics, histograms and percentiles20m
  6. 05Correlate LogsStructured logs with trace_id, from trace to log and back15m
  7. 06Meet the CollectorReceivers, processors, exporters and why a Collector15m
  8. 07Build Telemetry PipelinesEdit the pipeline, validate it, watch data flow20m
  9. 08Break the SystemBaselines, controlled faults and Toxiproxy15m
  10. 09Investigate an IncidentINC-001 and INC-002: from symptom to root cause25m
  11. 10Trace a Database ProblemINC-003: database spans, query text and DB metrics20m
  12. 11Diagnose Cascading LatencyINC-004 and INC-005: self time and the critical path20m
  13. 12Sampling and CostHead and tail sampling, cardinality, telemetry cost20m
  14. 13Docker DeploymentHow the lab is built, health checks, start, stop and reset15m
  15. 14Production ArchitectureAgents, gateways, security, SLOs and what changes in production20m
  16. 15Final Incident ChallengeINC-999: everything looks healthy. Find the root cause.30m