Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support kafka streams 3 #4236

Merged
merged 7 commits into from
Oct 5, 2021
Merged

Support kafka streams 3 #4236

merged 7 commits into from
Oct 5, 2021

Conversation

laurit
Copy link
Contributor

@laurit laurit commented Sep 29, 2021

Resolves #4174
Apparently tests didn't pass with kafka-streams 1.x perhaps we should also test some intermediate versions?

Copy link
Member

@trask trask left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's give a day or two for us to wrap up #4065 with @alesj, that PR has been through enough merge conflicts already 😭

Comment on lines 46 to 53
public static <K, V> Iterator<ConsumerRecord<K, V>> wrap(
Iterator<ConsumerRecord<K, V>> delegateIterator, @Nullable SpanContext receiveSpanContext) {
if (KafkaTracingWrapperUtil.wrappingEnabled()) {
return new TracingIterator<>(delegateIterator, receiveSpanContext);
}
return delegateIterator;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a similar use case that @mateuszrzeszutek recently ran into recently:

// this will forcefully suppress the kafka-clients CONSUMER instrumentation even though
// there's no current CONSUMER span
if (records instanceof KafkaConsumerIterableWrapper) {
records = ((KafkaConsumerIterableWrapper<?, ?>) records).unwrap();
}

would it work to use the same "workaround" for both? (either thread local suppression, or force unwrapping for both)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can use thread local for both, I'll just have to move the thread local to a different method that exists in all kafka-streams versions

Copy link
Member

@mateuszrzeszutek mateuszrzeszutek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait until #4065 is merged though (I think it should be this week)


package io.opentelemetry.javaagent.instrumentation.kafka;

public final class KafkaTracingWrapperUtil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could rename it to sth like KafkaClientsConsumerProcessTracing - since it only affects the CONSUMER operation = "process" part of kafka-clients instrumentation

@trask
Copy link
Member

trask commented Oct 3, 2021

@laurit fyi (so we don't duplicate) I'm resolving merge conflicts

@laurit
Copy link
Contributor Author

laurit commented Oct 3, 2021

@trask too late. I think that it might be better to put KafkaClientsConsumerProcessTracing into a bootstrap module as when kafka-clients and kafka-streams are in different class loaders they would each get their own copy of that class. The same applies to KafkaConsumerIterableWrapper and KafkaConsumerIteratorWrapper that are used from other instrumentations, probably should have a KafkaWrapper interface in bootstrap module.

@trask
Copy link
Member

trask commented Oct 3, 2021

No worries, feel free to force push over my changes


package io.opentelemetry.javaagent.bootstrap.kafka;

public final class KafkaClientsConsumerProcessTracing {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a brief comment about why this lives in the bootstrap class loader?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a comment, could you check whether it makes sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx 👍


package io.opentelemetry.javaagent.bootstrap.kafka;

public interface KafkaClientsConsumerProcessWrapper<T> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

* @param compileString An extra dependency in the gradle canonical form:
* '<group_id>:<artifact_id>:<version_id>'.
* @param dependencyNotation An extra dependency in the gradle canonical form:
* '<group_id>:<artifact_id>:<version_id>' or a project dependency project(':some-project').
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@trask trask merged commit 0cfc71c into open-telemetry:main Oct 5, 2021
@laurit laurit deleted the kafka-stream3 branch July 6, 2023 17:45
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support kafka-streams 3.0.0 (and above)
3 participants