NodeJs app tracing with OpenTelemetry

This guide demonstrates how to instrument your NodeJs applications with OpenTelemetry and send traces to the TrueFoundry backend. We’ll cover installing the required packages, initializing the tracing, automatic HTTP instrumentation, adding custom attributes and spans, and configuring sampling and debugging options. By the end, you’ll have a clear blueprint for integrating OpenTelemetry tracing into your NodeJs services.

Installation

To start, add the OpenTelemetry SDK and necessary instrumentation packages to your Node.js project. You can install them with npm:

npm install @opentelemetry/sdk-node \
  @opentelemetry/api \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/sdk-trace-node \
  @opentelemetry/exporter-otlp-http \
  express

Initializing OpenTelemetry

Next, initialize the OpenTelemetry SDK in your application. This involves setting up a Tracer Provider (which manages tracers and spans) and OTLP exporter to send the traces to TrueFoundry Backend.

import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
});

sdk.start();

import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
});

sdk.start();

const { NodeSDK } = require('@opentelemetry/sdk-node');
const {
  getNodeAutoInstrumentations,
} = require('@opentelemetry/auto-instrumentations-node');

const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
});

sdk.start();

Automatic HTTP Instrumentation

Now that the OpenTelemetry SDK is set up, let’s instrument the Express server to automatically trace incoming requests. getNodeAutoInstrumentations() instruments express apps along with other apps.

At this point, all incoming HTTP requests are being traced automatically.

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [getNodeAutoInstrumentations()],
});

Adding Attributes to Spans

Automatic instrumentation captures basic request information, but you can add custom data to your traces using attributes. Attributes are key-value pairs that provide additional context about your operations. For example, in order service, you might add order.id to make traces more useful.

import { trace } from '@opentelemetry/api';

const span = trace.getActiveSpan();
if (span) {
    span.setAttribute('order.id', id as string);
}

Creating Custom Spans

Automatic instrumentation captures HTTP requests and external calls, but it doesn’t track your application’s internal logic. For important operations, you can manually create spans to trace specific parts of your code. A span represents a unit of work, and creating sub-spans helps you see detailed timing and context for key processes.

For example, if a request triggers a complex function or external call that isn’t automatically captured, you can create a span to trace that specific operation. Manual instrumentation fills these gaps by letting you track what happens inside your application, not just at the edges.

import { trace } from '@opentelemetry/api';

const db_tracer = trace.getTracer('order-server-db');

const span = db_tracer.startSpan('db.query');
if (span) {
    span.setAttribute('order.items', ['item1', 'item2', 'item3']);
    span.setAttribute('order.id', orderId);
    span.end();
}

Complete Application Example

Below is a comprehensive example that demonstrates all the OpenTelemetry concepts we’ve covered.

This application creates an order service Express server that sets up OpenTelemetry tracing with proper configuration, automatically instruments HTTP requests using auto-instrumentations, creates custom spans for database operations, and adds custom attributes to provide order-specific context.

import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();

import { trace } from '@opentelemetry/api';
import express, { Express, Request, Response } from 'express';

const PORT: number = parseInt(process.env.PORT || '8080');
const app: Express = express();

const db_tracer = trace.getTracer('order-server-db');

function mockFetchOrder(orderId: string): string {
    const span = db_tracer.startSpan('db.query');
    if (span) {
        span.setAttribute('order.items', ['item1', 'item2', 'item3']);
        span.setAttribute('order.id', orderId);
        span.end();
    }
    return `order.id: ${orderId}, items: ${['item1', 'item2', 'item3']}`;
}

app.get('/order-server/orders/:order_id', (req: Request, res: Response) => {
    const id = req.params.order_id;

    // span := oteltrace.SpanFromContext(r.Context())
    const span = trace.getActiveSpan();
    if (span) {
        span.setAttribute('order.id', id as string);
    }

    const order = mockFetchOrder(id as string);
    res.send(order);
});

app.listen(PORT, () => {
  console.log(`Listening for requests on http://localhost:${PORT}`);
});

process.on('SIGTERM', () => {
  sdk.shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();

import { trace } from '@opentelemetry/api';
import express, { Express, Request, Response } from 'express';

const PORT: number = parseInt(process.env.PORT || '8080');
const app: Express = express();

const db_tracer = trace.getTracer('order-server-db');

function mockFetchOrder(orderId: string): string {
    const span = db_tracer.startSpan('db.query');
    if (span) {
        span.setAttribute('order.items', ['item1', 'item2', 'item3']);
        span.setAttribute('order.id', orderId);
        span.end();
    }
    return `order.id: ${orderId}, items: ${['item1', 'item2', 'item3']}`;
}

app.get('/order-server/orders/:order_id', (req: Request, res: Response) => {
    const id = req.params.order_id;

    // span := oteltrace.SpanFromContext(r.Context())
    const span = trace.getActiveSpan();
    if (span) {
        span.setAttribute('order.id', id as string);
    }

    const order = mockFetchOrder(id as string);
    res.send(order);
});

app.listen(PORT, () => {
  console.log(`Listening for requests on http://localhost:${PORT}`);
});

process.on('SIGTERM', () => {
  sdk.shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

const { NodeSDK } = require('@opentelemetry/sdk-node');
const {
  getNodeAutoInstrumentations,
} = require('@opentelemetry/auto-instrumentations-node');

const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

process.env["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<enter_your_api_endpoint>"
process.env["OTEL_EXPORTER_OTLP_HEADERS"] = `Authorization=Bearer <enter_your_api_key>,TFY-Tracing-Project=<enter_your_trace_project>`

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk.start();

const express = require('express');
const { trace } = require('@opentelemetry/api');
const tracer = trace.getTracer('order-service');

const PORT = parseInt(process.env.PORT || '8080');
const app = express();

function mockFetchOrder(orderId){
    const span = tracer.startSpan('db.query');
    if (span) {
        span.setAttribute('order.items', ['item1', 'item2', 'item3']);
        span.setAttribute('order.id', orderId);
        span.end();
    }
    return `order.id: ${orderId}, items: ${['item1', 'item2', 'item3']}`;
}

app.get('/order-server/orders/:order_id', (req, res) => {
    const id = req.params.order_id;

    // span := oteltrace.SpanFromContext(r.Context())
    const span = trace.getActiveSpan();
    if (span) {
        span.setAttribute('order.id', id);
    }

    const order = mockFetchOrder(id);
    res.send(order);
});

app.listen(PORT, () => {
  console.log(`Listening for requests on http://localhost:${PORT}`);
});

Run your application and view logged trace

Run the application and make a request to test the tracing:

curl http://localhost:8080/order-server/orders/123

Advanced Configuration

Sampling

Tracing sampling is a crucial technique for managing the volume of trace data in production environments. By default, OpenTelemetry NodeJs traces every request, which works well for debugging or development but can become expensive and noisy in high-traffic production systems.

Sampling helps in several ways: it reduces noise in traces, helping you focus on important traces while maintaining visibility into your system. It also helps with cost management in terms of storage, processing, and network bandwidth, making it essential for production deployments.

Sampling Strategies

OpenTelemetry supports several built-in samplers, but in practice, two cover most use cases:

1. TraceIdRatioBased Sampler

Samples a fixed percentage of root traces. This sampler makes sampling decisions independently for each trace.

import { TraceIdRatioBasedSampler } from '@opentelemetry/sdk-trace-node';

const sdk = new NodeSDK({
    traceExporter: traceExporter,
    instrumentations: [getNodeAutoInstrumentations()],
    sampler: new TraceIdRatioBasedSampler(0.1),
  });

Pros: Simple to configure, predictable sampling rate, deterministic behavior

Cons: May create partial traces if child spans are sampled differently, especially when spans are spread across multiple microservices or services with different sampling configurations.

2. ParentBased Sampler (Recommended)

Samples a fixed percentage of root traces and ensures that child spans follow the parent’s sampling decision, maintaining complete trace integrity.

import { ParentBasedSampler, TraceIdRatioBasedSampler } from '@opentelemetry/sdk-trace-node';

const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [getNodeAutoInstrumentations()],
  sampler: new ParentBasedSampler({
    root: new TraceIdRatioBasedSampler(0.5)
  }),
});

Pros: Maintains trace integrity, prevents partial traces, ensures complete trace visibility when sampled

Cons: Slightly more complex configuration, but worth the additional setup for production environments

Troubleshooting

Partial Traces

If you see partial traces (missing spans in the middle of a trace), ensure you’re using ParentBased sampler:

// ❌ May create partial traces across services
const sampler = new TraceIdRatioBasedSampler(0.1);

// ✅ Maintains trace integrity across all services
const sampler = new ParentBasedSampler({
  root: new TraceIdRatioBasedSampler(0.1)
});

Too Much Data

If you’re still collecting too much data or experiencing high costs, reduce the sampling rate:

// Reduce from 10% to 5% for high-traffic environments
const sampler = new ParentBasedSampler({
  root: new TraceIdRatioBasedSampler(0.05)
});

// For very high traffic, consider 1% sampling
const sampler = new ParentBasedSampler({
  root: new TraceIdRatioBasedSampler(0.01)
});

Too Little Data

If you need more visibility for debugging or monitoring, increase the sampling rate:

// Increase from 10% to 25% for better visibility
const sampler = new ParentBasedSampler({
  root: new TraceIdRatioBasedSampler(0.25)
});

// For critical debugging, consider 50% or higher
const sampler = new ParentBasedSampler({
  root: new TraceIdRatioBasedSampler(0.5)
});

Local Debugging

For local development and debugging, you can use the console exporter to see traces in your terminal: Update setupOTelSDK function to create a console exporter and inject into NodeSDK

import { ConsoleSpanExporter, SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';

// ........

// Console exporter for local debugging
const consoleExporter = new ConsoleSpanExporter();

const sdk = new NodeSDK({
    traceExporter: new ConsoleSpanExporter(),
    instrumentations: [getNodeAutoInstrumentations()],
    // Add console exporter for debugging
    additionalSpanProcessors: [new SimpleSpanProcessor(consoleExporter)],
});

Using Instrumentation Libraries

OpenTelemetry provides instrumentation libraries for many popular frameworks and libraries in the Node.js ecosystem. The getNodeAutoInstrumentations() function we used earlier automatically includes all the instrumentation packages from the OpenTelemetry JavaScript Contrib repository. When you use getNodeAutoInstrumentations(), you get automatic instrumentation for popular frameworks like Express, Fastify, Koa, and many other libraries including HTTP clients, databases, and more.

LLM Deployment

LLM Finetuning

Prompt Management

LLM Tracing

NodeJs app tracing with OpenTelemetry

Installation

Initializing OpenTelemetry

Automatic HTTP Instrumentation

Adding Attributes to Spans

Creating Custom Spans

Complete Application Example

Run your application and view logged trace

Advanced Configuration

Sampling

Sampling Strategies

Troubleshooting

Local Debugging

Using Instrumentation Libraries

LLM Deployment

LLM Finetuning

Prompt Management

LLM Tracing

​Installation

​Initializing OpenTelemetry

​Automatic HTTP Instrumentation

​Adding Attributes to Spans

​Creating Custom Spans

​Complete Application Example

​Run your application and view logged trace

​Advanced Configuration

​Sampling

​Sampling Strategies

​Troubleshooting

​Local Debugging

​Using Instrumentation Libraries

Installation

Initializing OpenTelemetry

Automatic HTTP Instrumentation

Adding Attributes to Spans

Creating Custom Spans

Complete Application Example

Run your application and view logged trace

Advanced Configuration

Sampling

Sampling Strategies

Troubleshooting

Local Debugging

Using Instrumentation Libraries