[fastify] thousands of requests are grouped under the same trace ID

[punkpeye]

Environment

SaaS (https://sentry.io/)

Steps to Reproduce

I have pretty standard setup:

server.ts

const app = fastify({
  bodyLimit: 32 * 1_024 * 1_024,
  logger: config.FASTIFY_LOGGING,
  pluginTimeout: 60_000,
  return503OnClosing: false,
  trustProxy: true,
});

setupFastifyErrorHandler(app);

app.setErrorHandler((error, request, reply) => {
  captureException({
    error,
    extra: {
      request,
    },
    message: 'could not handle request',
  });

  return replyWithErrorPage(reply, error);
});

await app.register(checksFastifyPlugin, {
  getStatus: () => {
    return shutdownHandler.getStatus();
  },
  prefix: '/checks',
});

await app.register(ingestLogsFastifyPlugin);

instrument.ts

const integrations: NodeOptions['integrations'] = [];

if (config.SENTRY_PROFILE) {
  integrations.push(nodeProfilingIntegration());
}

integrations.push(
  Sentry.extraErrorDataIntegration({
    captureErrorCause: true,
    depth: 7,
  }),
);

const SENTRY_SAMPLE_RATE = 0.1;

const sentryOptions = {
  dsn: config.SENTRY_DSN,
  environment: config.ENVIRONMENT,
  integrations,
  maxValueLength: 1_024 * 10,
  normalizeDepth: 7,
  openTelemetryInstrumentations: [new OpenAIInstrumentation()],
  profilesSampleRate: config.SENTRY_PROFILE ? SENTRY_SAMPLE_RATE : 0,
  registerEsmLoaderHooks: config.SENTRY_TRACE,
  release: config.RELEASE_VERSION,
  skipOpenTelemetrySetup: !config.SENTRY_TRACE,
  tracesSampler: () => {
    if (!config.SENTRY_TRACE) {
      return false;
    }

    if (Math.random() > SENTRY_SAMPLE_RATE) {
      return false;
    }

    return true;
  },
} satisfies NodeOptions;

Sentry.init(sentryOptions);

/**
 * The difference between `beforeSend` and `addEventProcessor` is that `beforeSend` is called the last.
 * We have convert Request instance to Sentry request object here or else it will be stringified as {}.
 */
Sentry.addEventProcessor(async (event) => {
  const visitorSession = event.extra?.visitorSession as
    | PublicVisitorSession
    | undefined;

  // https://develop.sentry.dev/sdk/data-model/event-payloads/user/
  if (visitorSession?.userAccount) {
    event.user = {
      ...event.user,
      email: visitorSession.userAccount.emailAddress,
      id: visitorSession.userAccount.uid,
      username: visitorSession.userAccount.fullName,
    };
  }

  if (typeof event.extra?.ipAddress === 'string') {
    event.user = {
      ...event.user,
      ip_address: event.extra.ipAddress,
    };
  }

  if (event.extra?.request instanceof Request) {
    const request = event.extra.request;

    const headers = Object.fromEntries(request.headers.entries());

    delete headers.cookie;

    // https://develop.sentry.dev/sdk/data-model/event-payloads/request/
    event.request = {
      data: normalize(request.body) as unknown,
      headers: { user_agent: headers['user-agent'] },
      method: request.method,
      query_string: new URL(request.url).search.replace('?', ''),
      url: request.url,
    };

    event.transaction = `${request.method} ${request.url}`;
  }

  // TODO research when values would be more than 1
  if (event.exception?.values?.length === 1) {
    const exception = event.exception.values[0];

    // New frames are added to the end of the stacktrace.
    const frames = exception.stacktrace?.frames?.reverse();

    const route = frames
      ?.map((frame) => frame.filename)
      .find((filename) => filename?.includes('/app/routes/'));

    if (route) {
      // https://docs.sentry.io/platforms/javascript/enriching-events/fingerprinting/
      event.fingerprint = ['{{ default }}', route];
    }
  }

  return event;
});

Expected Result

I expect every traceId to be associated with a unique request.

Actual Result

Thousands of different requests are associated with the same traceId.

Here is one example:

Product Area

Issues

Link

No response

DSN

https://fc82aa9f808b13927ff29c56bda4d486@o4507975513735168.ingest.us.sentry.io/4507975514849280

Version

No response

[getsantry]

Assigning to @getsentry/support for routing ⏲️

[punkpeye]

@AbhiPrasad as per your request

#4784 (comment)

[AbhiPrasad]

@punkpeye could you share your Node.js, Fastify, and Sentry SDK versions? Going to transfer this to our JavaScript SDK repo so that we can track it as part of SDK tasks.

[getsantry]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀