Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(replays): Add handling for uncompressed payload types #1793

Closed
wants to merge 12 commits into from

Conversation

cmanallen
Copy link
Member

@cmanallen cmanallen commented Jan 30, 2023

Couple of things happening here:

  • Functional change allowing payloads which could not be decompressed to be move forward in the pipeline any way. This allows us to accept non-compressed payloads which would have otherwise been rejected.
  • The "protocol" related behavior. Specifically compression, decompression, header parsing, and header validation have been moved to their own module. This was done to separate the parser and the implementation details necessary to interact with the service.

@cmanallen cmanallen requested a review from a team January 30, 2023 17:43
@JoshFerge JoshFerge self-requested a review January 30, 2023 17:56
@jan-auer
Copy link
Member

I'd like to put this one on hold until we've made progress on the memory issue.

That said, instead of ignoring compression errors, please match on the magic bytes of the compression header. If you're reading a broken gzip compressed payload, you wanna know that there was an error decompressing it. Only if the payload is not compressed, you should pipe it through.

relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
// deserialize the body bytes as is.
match decompress(body, limit) {
Ok(buf) => Ok((headers, buf)),
Err(_) => Ok((headers, body.into())),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignoring errors here can and will lead to other errors downstream. Instead, please check if the body is compressed by inspecting magic bytes, and run decompression conditionally.

relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
relay-replays/src/recording/protocol.rs Outdated Show resolved Hide resolved
@jan-auer
Copy link
Member

jan-auer commented Feb 1, 2023

This has been incorporated into #1800. I'll be following up with some more module separation, for now we can close this. Thank you for the nudge!

@jan-auer jan-auer closed this Feb 1, 2023
@jan-auer jan-auer deleted the replays-recording-pii-compression-failures branch February 1, 2023 09:52
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants