ENGTAI-63552: adding filter eval for client spans #250

varkey98 · 2025-06-26T16:30:31Z

Adding blocking and header injection support for http and grpc clients. For DB clients not sure about it from product side, hence just adding sampling support for now

codecov · 2025-06-26T16:32:38Z

Codecov Report

Attention: Patch coverage is 62.16216% with 42 lines in your changes missing coverage. Please review.

Project coverage is 48.09%. Comparing base (95e853c) to head (3621fd5).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...nstrumentation/hypertrace/database/hypersql/sql.go	0.00%	12 Missing ⚠️
...k/instrumentation/google.golang.org/grpc/client.go	63.15%	6 Missing and 1 partial ⚠️
...n/hypertrace/google.golang.org/hypergrpc/client.go	0.00%	6 Missing ⚠️
...trumentation/hypertrace/net/hyperhttp/transport.go	0.00%	6 Missing ⚠️
sdk/instrumentation/net/http/transport.go	85.29%	4 Missing and 1 partial ⚠️
...umentation/hypertrace/database/hypersql/options.go	42.85%	4 Missing ⚠️
...rumentation/opentelemetry/database/hypersql/sql.go	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #250      +/-   ##
==========================================
+ Coverage   47.79%   48.09%   +0.29%     
==========================================
  Files          62       64       +2     
  Lines        3013     3144     +131     
==========================================
+ Hits         1440     1512      +72     
- Misses       1499     1553      +54     
- Partials       74       79       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

prodion23 · 2025-06-26T16:38:50Z

sdk/instrumentation/net/http/transport.go

@@ -58,6 +62,29 @@ func (rt *roundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
 		req.Body = io.NopCloser(bytes.NewBuffer(body))
 	}

+	filterResult := rt.filter.Evaluate(span)
+	if filterResult.Block {


is this for blocking client requests?

Yes, since we are anyways calling libtraceable, thought we can add blocking support as well

Hmm, I guess I'm wondering on use case, I understand that because we want to add sampling, we could add blocking support.
I think question is if we should.

This seems like it introduces ambiguity in our blocking feature as a whole. Up to this point, blocking has specifically been about preventing bad requests from making it into the system.

Since we don't currently propagate string taint from the original request to downstream client spans, it's unclear what would drive the decision to block here. The only scenarios I can think of are:

1.) The app loads something from a DB or config and decides to block the outbound request - which is totally unrelated to the original inbound request.
2.) We're trying to enforce some kind of policy based purely on app-generated or static context - which feels more like something the app should handle directly.

If we allow blocking here, we're essentially saying: "some internal logic or app state, something not tied to the inbound request caused us to block an outbound call."

Agreed. But we also have DLP based blocking which is basically to prevent data leaks and was made for client spans only, but at that time none of our agents did blocking eval on client spans and hence user said they'll put all of their egress under a proxy and instrument our agent there so that DLP based blocking will work.
Also, I was thinking that even if we dont turn it on today, we can add the blocking capability in code and later when we want to turn it on, we can enable blocking for client spans as well, since we have a config on libtraceable, which by default skips client spans for blocking eval.

But if you think even having the capability makes our feature ambiguous, then definitely removing this

Hmm, okay, but don't remove just because I don't think worth it -

i think worth identifying what we are trying to do..(I see you also did header injections) so is the purpose of these tickets to add sampling support, or add sampling support, header injections & blocking.
(my main intention was to ensure feature parity in the lang agents.., if we are adding these here, we need to make sure we do them in other lang agents)

I'd think this is a product question, if we see value in blocking client spans then sure..

Thanks, lets start a thread with protection team/pm involved as well

Sampling makes sense for client spans. Blocking I am not so sure. Let's get clarity from PM.

I have started a thread here: https://harness.slack.com/archives/C08SEG313E3/p1751005003984389

We kind of discussed it back when rolling dlp out, but since its a huge lift, never prioritised it. I am unsure if we should prioritise it now as well, but I thought I can add it here since its not a lot of extra changes from adding sampling support to get it supported in goagent.

prodion23 · 2025-06-27T14:01:42Z

maybe 2 more questions:
what is use case for the db call blocking, is it the same case as http clients, DLP?

& similarly can db spans be sampled today?

sdk/instrumentation/google.golang.org/grpc/client.go

varkey98 · 2025-06-30T05:24:18Z

maybe 2 more questions: what is use case for the db call blocking, is it the same case as http clients, DLP?

& similarly can db spans be sampled today?

From a product perspective, I dont think dbclient spans blocking makes a lot of sense. Sampling still makes sense to me

tim-mwangi · 2025-07-21T15:17:21Z

@varkey98 what's the ETA of this one?

varkey98 · 2025-07-21T18:54:12Z

@tim-mwangi I'll try to wrap it up tomorrow my daytime

tim-mwangi · 2025-07-21T19:22:03Z

@tim-mwangi I'll try to wrap it up tomorrow my daytime

Okay. Sounds good.

sdk/instrumentation/google.golang.org/grpc/client.go

tim-mwangi · 2025-07-22T13:55:53Z

examples/postgres-query/go.mod

@@ -6,7 +6,9 @@ toolchain go1.24.2

 replace github.com/hypertrace/goagent => ../..

-replace github.com/hypertrace/goagent/instrumentation/hypertrace/github.com/jackc/hyperpgx => ../../instrumentation/opentelemetry/github.com/jackc/hyperpgx
+replace github.com/hypertrace/goagent/instrumentation/hypertrace/github.com/jackc/hyperpgx => ../../instrumentation/hypertrace/github.com/jackc/hyperpgx


I need to check why we need this replace.

one replace was always there pointing to the opentelemetry one's go module. When I added options pattern I added it only to the hypertrace wrapper (for other instrumentations like the grpc one we only had it for our hypertrace wrapper). So for using it I switched the replace from the otel one to hypertrace. But stranger part was that replace was not working transitively, hence needed to add both.

Hopefully it does not give me trouble during the migration :)

varkey98 added 3 commits June 26, 2025 14:58

adding filter call to grpc

b6b4296

add code for http client

c780f19

separate imports

3e9b294

varkey98 requested review from prodion23, tim-mwangi and shashank11p June 26, 2025 16:30

varkey98 marked this pull request as draft June 26, 2025 16:31

prodion23 reviewed Jun 26, 2025

View reviewed changes

tim-mwangi reviewed Jun 27, 2025

View reviewed changes

sdk/instrumentation/google.golang.org/grpc/client.go Outdated Show resolved Hide resolved

add support for sql client

cec5b81

varkey98 added 3 commits July 22, 2025 00:29

adding sql client in options wrapper

a53951b

adding pgx support

96942cc

fix lint and add some more unit tests

a2de06a

varkey98 changed the title ~~draft: adding filter eval for client spans~~ ENGTAI-63552: adding filter eval for client spans Jul 21, 2025

varkey98 marked this pull request as ready for review July 21, 2025 19:21

remove unwanted heap allocation

31415ba

tim-mwangi reviewed Jul 21, 2025

View reviewed changes

sdk/instrumentation/google.golang.org/grpc/client.go Show resolved Hide resolved

varkey98 added 3 commits July 22, 2025 09:43

fixing unit tests

67017c2

adding options pattern to hypersql and hyperpgx

e88d487

fix example go module file

3621fd5

varkey98 requested review from prodion23 and tim-mwangi July 22, 2025 05:17

tim-mwangi reviewed Jul 22, 2025

View reviewed changes

tim-mwangi approved these changes Jul 22, 2025

View reviewed changes

varkey98 merged commit 9e56b19 into main Jul 22, 2025
6 of 7 checks passed

varkey98 deleted the jacob branch July 22, 2025 18:32

ENGTAI-63552: adding filter eval for client spans #250

ENGTAI-63552: adding filter eval for client spans #250

Uh oh!

Conversation

varkey98 commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

prodion23 Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

varkey98 Jun 26, 2025 • edited by prodion23 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prodion23 commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

varkey98 commented Jun 30, 2025

Uh oh!

tim-mwangi commented Jul 21, 2025

Uh oh!

varkey98 commented Jul 21, 2025

Uh oh!

tim-mwangi commented Jul 21, 2025

Uh oh!

Uh oh!

tim-mwangi Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

varkey98 commented Jun 26, 2025 •

edited

Loading

codecov bot commented Jun 26, 2025 •

edited

Loading

prodion23 Jun 26, 2025 •

edited

Loading

varkey98 Jun 26, 2025 •

edited by prodion23

Loading

prodion23 commented Jun 27, 2025 •

edited

Loading

tim-mwangi Jul 22, 2025 •

edited

Loading