Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix: change JSON patch implementation for memory improvement #1019

Merged
merged 1 commit into from
Mar 21, 2020

Conversation

dkundel
Copy link
Contributor

@dkundel dkundel commented Mar 20, 2020

I've done some digging into #985 and here's the outcome.

Actual Issue:
The issue seems to be in the algorithm that the rfc6902 library is using for Arrays. Since it's recursive it will run out of memory when it encounters an array that is too big.

Possible Solution:
I found in this issue chbrown/rfc6902#39 an alternative library (fast-json-patch) that could be used. It seems to work against my use case and according to these benchmarks it should be significantly more performant.

Breaking Change:
As you can see in the updated tests, there are two changes to the output format compared to the other implementation:

  1. Order of operations is different
  2. path for insertions in arrays ends with new index of item instead of -. For example: /pixels/99 instead of /pixels/-. I checked the actual standard and this behavior seems to be up to the actual standard: https://tools.ietf.org/html/rfc6902#appendix-A.2

Let me know what you think.

@dkundel dkundel force-pushed the dkundel/fix-memory-json-patch branch from 5c18f17 to 37a4a65 Compare March 20, 2020 21:57
@orta
Copy link
Member

orta commented Mar 21, 2020

I think this is a great call.

I'll call it a major semver bump (we've had them for trivial things before) - I think it'd be unlikely people are expecting the order to be determinate but at least this warns folks upfront

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants