Sanitize HTTP headers when the agent reads them #1286
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪Test errors
Expand to view the tests failures> Show only the first 10 test failures
|
russcam
approved these changes
May 3, 2021
| { | ||
| if (realError.Context.Request?.Headers != null && realError.ConfigSnapshot != null) | ||
| { | ||
| foreach (var key in realError.Context?.Request?.Headers?.Keys.ToList()) |
There was a problem hiding this comment.
Can the ToList() be removed, and avoid the allocation?
| foreach (var key in realError.Context?.Request?.Headers?.Keys.ToList()) | ||
| { | ||
| if (WildcardMatcher.IsAnyMatch(realError.ConfigSnapshot.SanitizeFieldNames, key) | ||
| && realError.Context.Request.Headers[key] != Consts.Redacted) |
There was a problem hiding this comment.
I wonder if this redacted check is worth making, because the value is going to be redacted whether it has already been redacted or not, so I think it could be skipped.
There was a problem hiding this comment.
Now you made me to measure this :) I wasn't totally sure either. tlds: yeah, it's better to remove.
The reason for the check and my thinking was that with this PR now we redact immediately when we read the headers, so in most cases, like with a default ASP.NET Core app, when this code runs values are already redacted, so this will only redact things with a public API usage scenario.
Anyways, I had this numbers - this is with 8 headers, all redacted.
| Method | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|----------------------- |---------:|--------:|--------:|-------:|------:|------:|----------:|
| CheckForRedactedString | 497.0 ns | 3.88 ns | 3.63 ns | 0.2785 | - | - | 1.71 KB |
| NoCheck | 502.5 ns | 4.76 ns | 4.22 ns | 0.2785 | - | - | 1.71 KB |
The difference is close to nothing.
But then I tested with 8 headers again, but only 2 of them were redacted (and this is more like a usual case - typically most headers aren't redacted, only a few of them).
| Method | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|----------------------- |---------:|---------:|---------:|-------:|------:|------:|----------:|
| CheckForRedactedString | 513.9 ns | 10.16 ns | 14.90 ns | 0.2785 | - | - | 1.71 KB |
| NoCheck | 500.2 ns | 9.54 ns | 10.61 ns | 0.2785 | - | - | 1.71 KB |
With this, there is really no benefit, so I remove the check.
Code: (NoCheck is the same, except the check is removed).
[Benchmark]
public void CheckForRedactedString()
{
var context = new Context()
{
Request = new Request("GET", new Url())
{
Headers = new Dictionary<string, string>
{
// these are values for the 1. measurement above, the 2. one ran with 6 values which we do not redacted.
{ "password", Consts.Redacted },
{ "atoken", Consts.Redacted },
{ "btoken", Consts.Redacted },
{ "ctoken", Consts.Redacted },
{ "dtoken", Consts.Redacted },
{ "etoken", Consts.Redacted },
{ "ftoken", Consts.Redacted },
{ "gtoken", Consts.Redacted },
}
}
};
var transaction = new Transaction(context, "foo", "bar", 0.0, 0l, "a", "a", "a", false, "ok", null);
if (transaction is Transaction realTransaction)
{
if (realTransaction.IsContextCreated && realTransaction.Context.Request?.Headers != null)
{
foreach (var key in realTransaction.Context?.Request?.Headers?.Keys)
{
if (WildcardMatcher.IsAnyMatch(realTransaction.ConfigSnapshot.SanitizeFieldNames, key)
&& string.CompareOrdinal(realTransaction.Context.Request.Headers[key], Consts.Redacted) != 0)
realTransaction.Context.Request.Headers[key] = Consts.Redacted;
}
}
}
}
| @@ -24,7 +24,8 @@ public ITransaction Filter(ITransaction transaction) | |||
| { | |||
| foreach (var key in realTransaction.Context?.Request?.Headers?.Keys.ToList()) | |||
| { | |||
| if (WildcardMatcher.IsAnyMatch(realTransaction.ConfigSnapshot.SanitizeFieldNames, key)) | |||
| if (WildcardMatcher.IsAnyMatch(realTransaction.ConfigSnapshot.SanitizeFieldNames, key) | |||
| && realTransaction.Context.Request.Headers[key] != Consts.Redacted) | |||
There was a problem hiding this comment.
Same as above, I wonder if this redacted check is worth making, because the value is going to be redacted whether it has already been redacted or not, so I think it could be skipped.
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪Test errors
Expand to view the tests failures> Show only the first 10 test failures
|
|
Is this the PR for https://discuss.elastic.co/t/elastic-apm-net-agent-1-10-0-security-update/274668 ? |
Yes, it is. |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes a few things around HTTP header sanitization