In this lesson, you will learn how to bridge the gap between application-level request spans and the granular performance of your SQL and NoSQL database operations.
When an incoming HTTP request triggers multiple database calls, relying solely on high-level request duration is insufficient for identifying performance bottlenecks. To achieve true observability, we must link database command execution directly into our distributed trace tree as child spans of the business logic.
Integrating Database Spans
Whether you are using a SQL client like pg or a NoSQL driver like mongodb, the goal is to wrap the execution method so that the database call appears as an explicit operation within your trace. Each database span should capture key attributes, such as the statement being executed, the table/collection accessed, and the execution time.
Consider this illustration using a manual instrumentation pattern for a query execution:
async function findUser(db, userId) {
const span = tracer.startSpan('db.query', { parent: activeSpan });
span.setAttribute('db.system', 'mongodb');
span.setAttribute('db.statement', 'find');
try {
const result = await db.collection('users').findOne({ _id: userId });
return result;
} catch (err) {
span.setStatus({ code: 'ERROR', message: err.message });
throw err;
} finally {
span.end();
}
}
Analyzing Performance
By nesting these calls, your trace visualizer will show exactly how much time the event loop spent waiting on the database versus performing application logic.
- For SQL: Ensure you capture the normalized query string (avoiding raw parameters to prevent high-cardinality issues). This allows you to group performance metrics by query pattern.
- For NoSQL: Treat command execution (e.g.,
find,aggregate,update) as the primary operation label.
By including these commands in your trace tree, you can pinpoint the specific query responsible for inflating your P99 latency. If a trace shows a large gap between the start of an app.service.process span and the finish of a db.query span, you know you are likely dealing with network overhead, connection pool queuing, or inefficient query execution that needs optimization.
Key Takeaway: Tracing database queries transforms opaque IO wait times into actionable performance data by explicitly logging command execution as discrete child spans within the request trace.