Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Replicator Stack #54

Closed
vasco-santos opened this issue Nov 29, 2022 · 0 comments · Fixed by #57
Closed

Replicator Stack #54

vasco-santos opened this issue Nov 29, 2022 · 0 comments · Fixed by #57

Comments

@vasco-santos
Copy link
Contributor

Based on #50
This was referenced Nov 29, 2022
Closed
Merged
vasco-santos added a commit that referenced this issue Dec 1, 2022
@vasco-santos @olizilla
feat: replicator stack (#57)
b91742d 
Adds replicator stack by moving its content out of CARPARK on the
direction to #50

Implementation details:
- replicator-stack has 2 queues, one to replicate CARs and other to
replicate .idx files. At first I was attempting to do both in one single
queue. Ended up moving to 2 queues because I would need to have handlers
to identify origin bucket and have a hardcoded mapping where I would see
if it was carpark/satnav and with that select the appropriate CF
destination bucket.
- while my decision for 2 queues was based on the above, it might
actually be the best approach to not have queues throttling flow on
copies from independent buckets.
- also important noticing that despite 2 queues, there is only one
handler function code. The lambda code is the same, but it is injected
with destination BUCKET_NAME according to the queue it is handling.

Closes #54

Co-authored-by: Oli Evans <oli@protocol.ai>
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: replicator stack storacha/w3infra
1 participant
@vasco-santos