Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

migrate-from-w32023-to-mime #1

Closed
wants to merge 6 commits into from
Closed

migrate-from-w32023-to-mime #1

wants to merge 6 commits into from

Conversation

gobengo
Copy link
Contributor

@gobengo gobengo commented Jan 17, 2024

Motivation:

  • export the list entries from w3 list --json to an internet message stream
  • prototype the idea of encoding many uploads into one file using this format. not sure we need it but i may come back to it.

Done

  • old uploads as ndjson -> a multipart/mixed file - i.e. 'migrate to mime'
  • old uploads as ndjson -> no output, but fetches part CARs for each upload, decodes file, writes to local fs dir - i.e. 'migrate to local fs'

Usage

migrate-to-w3up on  migrate-to-rfc822 [?] is 📦 0.0.0 via ⬢ v20.10.0 
⚡ npx @web3-storage/w3 list --json | migrate-from-w32023-to-mime --fetchParts > bengo.web3.storage.rfc822
wrote /dev/stdout

migrate-to-w3up on  migrate-to-rfc822 [?] is 📦 0.0.0 via ⬢ v20.10.0 took 14.0s 
⚡ head -n10 bengo.web3.storage.rfc822                                                       
Content-Type: multipart/mixed; boundary=45rl8rend6w; type=multipart/mixed

--45rl8rend6w
content-disposition: attachment; filename="bengo-test-report.html.bafybeia6bzetwmjngw4laf64fcnqiobhw2g5shbpg2rfrnxr2ylbxwrkhy.ipfs"
content-type: application/vnd.web3.storage.car+json;version=2023.old.web3.storage
content-id: ipfs:bagaaieraqclftv4bxsjhag4qqkwhlb6t3bd2s2frvzzq6o5aliye6y3bo3rq

{
  "_id": "315318962451059099",
  "type": "Car",

@gobengo gobengo marked this pull request as ready for review January 17, 2024 00:21
@gobengo gobengo marked this pull request as draft January 18, 2024 23:15
@gobengo gobengo requested review from alanshaw and removed request for alanshaw January 18, 2024 23:15
@gobengo
Copy link
Contributor Author

gobengo commented Jan 19, 2024

blocked on testing against old.web3.storage until web3-storage/web3.storage#2347

@gobengo
Copy link
Contributor Author

gobengo commented Jan 23, 2024

continuing sans MIME stuff over here #3

@gobengo gobengo closed this Jan 23, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant