Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Rename repo? Decouple repo? this is not upload-api #50

Closed
vasco-santos opened this issue Nov 28, 2022 · 2 comments · Fixed by #85
Closed

Rename repo? Decouple repo? this is not upload-api #50

vasco-santos opened this issue Nov 28, 2022 · 2 comments · Fixed by #85

Comments

@vasco-santos
Copy link
Contributor

writes-pipeline
@olizilla olizilla mentioned this issue Nov 28, 2022
Open
This was referenced Nov 29, 2022
Closed
Closed
Merged
vasco-santos added a commit that referenced this issue Dec 1, 2022
@vasco-santos @olizilla
feat: replicator stack (#57)
b91742d 
Adds replicator stack by moving its content out of CARPARK on the
direction to #50

Implementation details:
- replicator-stack has 2 queues, one to replicate CARs and other to
replicate .idx files. At first I was attempting to do both in one single
queue. Ended up moving to 2 queues because I would need to have handlers
to identify origin bucket and have a hardcoded mapping where I would see
if it was carpark/satnav and with that select the appropriate CF
destination bucket.
- while my decision for 2 queues was based on the above, it might
actually be the best approach to not have queues throttling flow on
copies from independent buckets.
- also important noticing that despite 2 queues, there is only one
handler function code. The lambda code is the same, but it is injected
with destination BUCKET_NAME according to the queue it is handling.

Closes #54

Co-authored-by: Oli Evans <oli@protocol.ai>
@gobengo
Copy link
Contributor

gobengo commented Dec 9, 2022

wait this isn't the upload-api?

@vasco-santos
Copy link
Contributor Author

We sat together and decided to move this way

Today

  • web3-storage/w3infra
    • upload-api
    • upload-db
    • carpark
    • satnav
    • eventbus
    • replicator
    • ucan-stream

Tomorrow

  • w3protocol

    • upload-client
    • upload-api
    • upload-api-cf

  • w3infra

    • http-gateway
    • upload-db
    • carpark
    • satnav
    • eventbus
    • replicator
    • ucan-stream

This was referenced Dec 9, 2022
Open
Merged
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: rename repo and rescope stacks storacha/w3infra
2 participants
@gobengo @vasco-santos