Parse request method, URL, and protocol separately in web server logs. #1824
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aaronweeden
added
enhancement
This was referenced Feb 21, 2024
aaronweeden
force-pushed
the
log-file-split-request
branch
5 times, most recently
from
005cda5 to
34da864
Compare
aaronweeden
force-pushed
the
log-file-split-request
branch
2 times, most recently
from
5686d66 to
f76b815
Compare
|
The webserver log parser already supports requestMethod (%m) and url === requestPath (%U) format strings. Why don't we use these? |
jpwhite4
requested changes
May 16, 2024
aaronweeden
force-pushed
the
log-file-split-request
branch
4 times, most recently
from
355ad7b to
94a483b
Compare
aaronweeden
changed the title
aaronweeden
force-pushed
the
log-file-split-request
branch
from
94a483b to
642bbe1
Compare
@jpwhite4 Thanks, I have refactored it to use |
aaronweeden
force-pushed
the
log-file-split-request
branch
from
642bbe1 to
de2d2ed
Compare
jpwhite4
approved these changes
May 22, 2024
aaronweeden
force-pushed
the
log-file-split-request
branch
from
de2d2ed to
edbfc29
Compare
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR updates the web server log parser to split the request line (i.e.,
%r, e.g.,GET /index.html HTTP/1.1) into its three parts (i.e.,%m %U %H) so these can be parsed separately.Motivation and Context
This enables ubccr/xdmod-ondemand#50.
Tests performed
I added unit tests to make sure a log is correctly parsed whether the format is
%ror%m %U %H. I also tested ingesting logs on my port onxdmod-devwith the changes from ubccr/xdmod-ondemand#50 and the PRs on which it depends.Checklist: