Database Query Tracing Through ORMs
A trace showed GET /orders/123 at 1.2s with 1.1s unaccounted in Express. Postgres slow query log was empty—180 queries at 6ms each. Sequelize lazy-loaded line items in a loop. ORM tracing makes those spans visible and N+1 patterns obvious in Jaeger.
OpenTelemetry database semconv
Use db.system, db.name, db.operation, sanitized db.statement, span kind CLIENT.
Instrumentation
Prisma: @prisma/instrumentation. SQLAlchemy: SQLAlchemyInstrumentor with SQL commenter. GORM: tracing.NewPlugin. Java: OTel Java agent JDBC instrumentation.
N+1 detection
Trace shape: >20 child DB spans with same statement template. Fix with eager load, DataLoader, or explicit IN (...) queries.
Pool wait spans
Instrument pool.acquire()—750ms pool wait looks like slow SELECT without separate span.
Guardrails
Never log bound parameters; normalize literals to ?; CI assert max DB spans per endpoint.
Connection pool spans
Database client instrumentation often misses pool wait—time blocked waiting for connection before query executes. Custom span around pool.acquire():
await tracer.startActiveSpan("db.pool.acquire", async (span) => {
const conn = await pool.connect();
span.end();
return conn;
});
Traces showing 800ms SELECT with 750ms pool wait need pool sizing not query indexes.
Read replica routing visibility
ORM middleware that routes reads to replicas should add span attribute db.role=replica|primary. Incidents where stale reads cause user confusion—trace shows read hit replica lagging 30 seconds behind primary.
Migration from ORM query logs
Teams enabling trace instrumentation should disable ORM SQL printf logging in production same release—duplicate IO and PII risk. Keep log_min_duration_statement on Postgres for DBA-side slow query capture as complement, not duplicate of every ORM query in app logs.
CI guardrails for span count
# pytest + opentelemetry test exporter
def test_list_orders_span_budget(client, span_exporter):
client.get("/orders")
db_spans = [s for s in span_exporter.get_finished_spans() if s.attributes.get("db.system")]
assert len(db_spans) <= 3, f"N+1 suspected: {len(db_spans)} db spans"
Fails PR when lazy loading regression adds loops—cheaper than production trace discovery.
Prepared statement and ORM cache effects
ORM L2 cache hits produce no DB span—traces show fast handler mysteriously. Add span attribute cache.hit=true on short path for debugging "sometimes slow" tickets. Without it, compare trace with missing DB spans vs many DB spans for same endpoint.
Sharding and cross-shard queries
ORM spanning shards may emit sequential spans to multiple hosts—trace shape looks like N+1 but is architectural. Label spans with db.shard=id for clarity in architecture reviews.
ORM tracing in microservice decomposition
During monolith extraction, identical repository methods may run in two services—trace comparison proves which deployment still emits N+1 patterns. Use span counts as migration gate: extracted service must not exceed span budget of monolith equivalent endpoint before cutover traffic shifts.
DBA collaboration improves when traces include db.system, db.name, and normalized statement—DBAs filter Tempo by slow span without application log access. Shared language reduces ping-pong during index recommendation tickets.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Document rollback paths and validate observability after every deploy affecting this surface.
Frequently asked questions
Why do ORMs hide database performance problems from traces?
Default HTTP instrumentation creates one span per request but batches ORM queries inside the handler. Without DB spans, traces show 800ms in controller code when 750ms was 200 SELECTs.
How do I detect N+1 queries with tracing?
Enable DB spans. One HTTP span with 50+ similar SELECT spans differing only by ID is classic N+1.
Should I put full SQL text in span attributes?
Use parameterized templates, never literal values—PII and high cardinality.
Hiring a senior Android / Flutter engineer?
I architect and ship production mobile software — Kotlin, Jetpack Compose, Flutter — for robotics, EV infrastructure, fintech, and real-time systems. Open to remote roles in Europe and the US.
Get in touch →