feat: add incoming request logging filter #255
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of Changes
Added the PathRequestLoggingFilter to log incoming requests to a Path connector in OpenSearch (e.g. requests from Synchronicity) including the connector's response. Added support for masking out sensitive fields in the MDX model separately from the upstream request logger
LogValueMasker
so that masker regular expressions in one do not affect the other.Note that this requires these path-core changes as well.
Fixes # MC-3689
Public API Additions/Changes
Nothing public was modified; however, the FilterOrderSequence was updated to include the PathRequestLoggingFilter so any connector that picks up these changes will automatically start using the PathRequestLoggingFilter. To actually get the connector to output the logs changes like this will need to made to add an
IncomingLogstash
appender and reference it from the "sandbox,qa,integration,production" Spring profile. For local development purposes you can reference the appender from the "development" Spring profile.Downstream Consumer Impact
This has an indirect way of impacting any connector that picks up these changes so careful testing locally and after deploying to pre-prod is recommended.
How Has This Been Tested?
I added unit tests and also tested this out with a connector by adding a reference to the
IncomingLogstash
appender from the "development" Spring profile and ran the Arsenal test suite locally. Then I inspected the log for each incoming request to the Path connector initiated by Arsenal to confirm that the logstasher fields were assigned the expected values and that masking was leveraged where expected. I recommend another Path developer vets out these changes.Checklist: