Jaeger/Zipkin Exporter Performance #1274
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CodeBlanch
commented
Sep 16, 2020
| /// </summary> | ||
| public TimeSpan TimeoutSeconds { get; set; } = TimeSpan.FromSeconds(10); |
There was a problem hiding this comment.
TimeoutSeconds wasn't actually being used. The ActivityProcessor now owns timeout, so I just removed it.
CodeBlanch
commented
Sep 16, 2020
Codecov Report
@@ Coverage Diff @@
## master #1274 +/- ##
==========================================
- Coverage 79.12% 79.12% -0.01%
==========================================
Files 215 215
Lines 6176 6175 -1
==========================================
- Hits 4887 4886 -1
Misses 1289 1289
|
It makes total sense to leverage the HTTP/TCP layer to do buffering. For smaller events we might consider turning off the Nagle's Algorithm (e.g. metrics). |
eddynaka
approved these changes
Sep 18, 2020
CodeBlanch
changed the title
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
A round of performance tweaks for Zipkin. It now uses the
Batch<Activitythroughout (instead of making a copy) and flushes small (4k) chunks to the Http stream as they are ready.The new version is slightly slower. If I make the chunk size bigger, it can be faster. But I think that is cheating for the sake of benchmarking. In a real-world scenario, we want to start sending bytes as soon as we can. Today we buffer everything into memory and then send it all at the end, that's why the memory usage is so high. I think lower chunks will actually make the export finish faster for real-world usage. I did the same thing (lower packet size) for JaegerExporter. Size reduced from 65000 to 4096. Socket default send buffer is 8192. For Jaeger we want to get the packets to the socket and let the networking algorithm handle the transmission, it has enough buffering we don't need our own 😄
/cc @reyang Curious what your thoughts are on this 4096 size change?
Benchmarks
Before:
After:
TODOs
CHANGELOG.mdupdated for non-trivial changes