Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

chore: hotfix v0.6.4 #67

Merged
merged 2 commits into from
Jul 25, 2024
Merged

chore: hotfix v0.6.4 #67

merged 2 commits into from
Jul 25, 2024

Conversation

alfetopito
Copy link
Contributor

Summary

Changes based off of https://github.com/cowprotocol/bff/releases/tag/v0.6.3 which is the current prod version.

Pointing to the new branch v0.6.

It fixes the issue with cached responses.

The requests are being cached, but the cached part is the stream, not the the actual payload.
Since the stream is consumed in the first reply, the subsequent tries in the cached version return nothing:

"err":{"type":"FastifyError","message":"Reply was already sent.","stack":"FastifyError: Reply was already sent.\n    at _Reply.Reply.send (/Users/alfeto/work/cow/bff/node_modules/fastify/lib/reply.js:127:26)\n    at /Users/alfeto/work/cow/bff/node_modules/@fastify/reply-from/index.js:184:14\n    at /Users/alfeto/work/cow/bff/node_modules/@fastify/reply-from/index.js:277:9\n    at /Users/alfeto/work/cow/bff/node_modules/@fastify/reply-from/lib/request.js:173:7\n    at RequestHandler.runInAsyncScope (node:async_hooks:206:9)\n    at RequestHandler.onHeaders (/Users/alfeto/work/cow/bff/node_modules/undici/lib/api/api-request.js:105:14)\n    at Request.onHeaders (/Users/alfeto/work/cow/bff/node_modules/undici/lib/core/request.js:233:27)\n    at Parser.onHeadersComplete (/Users/alfeto/work/cow/bff/node_modules/undici/lib/client.js:794:23)\n    at wasm_on_headers_complete (/Users/alfeto/work/cow/bff/node_modules/undici/lib/client.js:408:30)\n    at null.<anonymous> (wasm://wasm/00036ac6:1:1173)","code":"FST_ERR_REP_ALREADY_SENT","name":"FastifyError","statusCode":500},"msg":"Reply already sent"}

Note: in order to properly consume and cache the payload, I had to disable the compression, similar to what I did in my first attempt at this 55387a7

@alfetopito alfetopito self-assigned this Jul 25, 2024
@alfetopito alfetopito requested a review from a team July 25, 2024 11:35
Comment on lines +58 to +60
for await (const chunk of payload as ReadableStream) {
contents += chunk.toString(); // Process each chunk of data
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the fix: consume the payload stream once, so it can be reused in the response and stored in the cache.

Comment on lines +21 to +22
// Remove it so we can cache it properly
request.headers['accept-encoding'] = undefined;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed since I don't know how to handle decompressing and re-compressing the reply afterwards.

@alfetopito
Copy link
Contributor Author

Tested it on bff staging https://app.pulumi.com/cowprotocol/bff/staging/updates/178
Working as intended
image

Caching is working too (the important part) https://bff.barn.cow.fi/proxies/coingecko/simple/token_price/ethereum?contract_addresses=0x6B175474E89094C44Da98b954EedeAC495271d0F&vs_currencies=usd

Will merge it, please post merge review.

@alfetopito alfetopito merged commit 1d464a5 into v0.6 Jul 25, 2024
10 checks passed
@alfetopito alfetopito deleted the hotfix/v0.6.4 branch July 25, 2024 12:53
Copy link
Contributor

@anxolin anxolin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve, thanks for the fix :)

anxolin added a commit that referenced this pull request Jul 26, 2024
* chore: hotfix v0.6.4 (#67)

* fix: consume the payload stream before caching it

* fix: fix start script

* fix: use url as cache key which contains query params (#68)

* Make tests pass

---------

Co-authored-by: Leandro <alfetopito@users.noreply.github.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants