Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Manually set the parentSpanId #89

Open
Fi3 opened this issue Jan 19, 2024 · 6 comments
Open

Manually set the parentSpanId #89

Fi3 opened this issue Jan 19, 2024 · 6 comments

Comments

@Fi3
Copy link

Fi3 commented Jan 19, 2024

Feature Request

Looking for a way to manually set the parentSpanId on span creation (someting like otel.name but for parentSpanId)

Motivation

I have several service that communicate via a custom binary protocol and I do not know how to pass context. A lot of messages have a request_id using it + the originator of the request + a rounded timestamp I should be able to have a consistent parentSpanId trough requests.

Very new to otel so maybe I'm missing something. What I'm looking for is the ability to have distributed traces with Jaeger witout using http.

@Fi3
Copy link
Author

Fi3 commented Jan 22, 2024

I also look for a way to set the span id. The idea is to deterministically set a span id (using my request id) so that in the service that I call (that know the request id) I know the parent span id and I can set is with set_parent. There is a way to do that? I can not find it.

@Fi3
Copy link
Author

Fi3 commented Jan 22, 2024

Looking at the code the most immediate way is to add in layer.rs a layer implementor that instead of using self.tracer.new_span_id() to set the span_id on on_new_span like OpenTelemetryLayer it use a field on the span.

Maybe instead of having a brand new implemetor we can just use OpenTelemetryLayer and look for a specific field. If we find it we use it as span id otherwise we fallback to the already existent method.

Does this approach make any sense?

In case I can open a PR

@jtescher
Copy link
Collaborator

You can't set span ids yourself in opentelemetry without implementing a custom id generator, however you can set parent span, which will associate the new span with the same trace which is generally advised. This would be an example of extracting a parent from headers or other kv data to associate the parent https://github.com/tokio-rs/tracing-opentelemetry/blob/v0.1.x/examples/opentelemetry-remote-context.rs#L32-L34

@Fi3
Copy link
Author

Fi3 commented Jan 23, 2024

From what I understood jaeger use parent child relations in order to render traces. Not having the possibility to pass additional data via the binary protocol that I'm using, I want to set span ids deterministically using request ids and timestamp. From what I understood this will require patching tracing-opentelemetry. I have 3 questions:

  1. To do that I need to patch tracing-opentelemetry or not?
  2. This approach have some risk or downside that I can not see?
  3. if I go with this approach are you interested in a PR that allow setting custom span ids?

I can not just use the span id as request id in a lot of cases.

@mladedav
Copy link
Contributor

I am not exactly sure, but I think you could try and create your own Tracer and call with_tracer on the OpenTelemetryLayer. It would let you create your own span IDs.

I think there are risks concerning the the uniqueness of span ID, e.g. if a client retries a request with the same request ID, you might end up with two spans with the same ID which could cause issues in your tracing backend.

Personally, I would advise you against this though, you should be able to pass the context to the child so that you can set the parent to the apriori generated span context.

@Fi3
Copy link
Author

Fi3 commented Feb 26, 2024

at the end I vendored tracing-opentelemetry an add a very little patch. I can have collisions and also request that should be grouped but end up not grouped. They are statistically very unlikely and this is acceptable, it do not mess with back-end and I do not need to have 100% accuracy.

I patched layer.rs adding:

struct IdVisitor {
    id: Option<otel::SpanId>,
}

And modifying:

impl<S, T> Layer<S> for OpenTelemetryLayer<S, T>
where
    S: Subscriber + for<'span> LookupSpan<'span>,
    T: otel::Tracer + PreSampledTracer + 'static,
{
        fn on_new_span(&self, attrs: &Attributes<'_>, id: &span::Id, ctx: Context<'_, S>) {
            ...
            let mut id_getter = IdVisitor { id: None };
            attrs.values().record(&mut id_getter);
            let span_id = match id_getter.id {
                Some(span_id) => span_id,
                None => self.tracer.new_span_id(),
            };
            ....
            let mut builder = self
                .tracer
                .span_builder(attrs.metadata().name())
                .with_start_time(crate::time::now())
                // Eagerly assign span id so children have stable parent id
                .with_span_id(span_id);
            ....   
}

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants