Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Create DiskArchiver for SPS #4

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Conversation

blinkdog
Copy link
Contributor

This PR creates a DiskArchiver component/module to replace the one in jadeite, for use at SPS.

Motivation:

  • The jadeite DiskArchiver works, but is relatively slow at finalizing disks, especially large disks.
  • This wasn't a big problem when the archival disks were ~2-6 TB, but now that we're deploying 16 TB disks, it could become an operational issue.
  • Moreover, the way metadata is created at the end puts all of the metadata in one large JSON object. Again, less of a problem when archival disks were ~2-6 TB, but now with 16 TB disks, it can become an operational issue. (Also; who wants to load a huge multi-GB single JSON object?)
  • The new DiskArchiver is intended to solve all of these issues, and also provide new data safety checks using hardware metadata.
  • In particular, we'd like to capture the hardware metadata at the time the disk is used for archival purposes. We'd also like to store that in a database, make it queryable, and ensure that a serial number is not reused "too soon" (i.e.: within the same retro disk season).

@blinkdog blinkdog self-assigned this Nov 14, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant