Skip to content

Add option --time-only #719

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

steinhh
Copy link

@steinhh steinhh commented Jan 20, 2025

We maintain an archive with Solar observations (46 million and counting), with the files originating at another site (Japan). Naturally, we use rsync to fetch the files. Some observations are stored uncompressed at the origin, but we would like to store them compressed. To actually save space (i.e., to prevent having to store both compressed and uncompressed versions), I have created a tiny patch that adds an option called --time-only:

  1. --time-only

    This modifies rsync's "quick check" algorithm for finding files that need
    to be transferred, changing it from the default of transferring files with
    either a changed size or a changed last-modified time to just looking for
    files that have a changed last-modified time, ignoring size changes. This
    is useful when remote files are uncompressed but a local copy should be
    stored compressed, togethe, with a zero-size stub to prevent re-transfers.
    This option is only useful when pulling files, as it requires post-transfer
    compression and truncation of files whilst preserving the original
    modification time of the stub. Thus the option is not transmitted to the
    remote side, also preventing the server from croaking on an unknown option.

I hope you will accept this extremely simple patch (only 3 code lines modified, two of which are declarations of the flag), which I have found very useful (saves me several terabytes of storage space!).

Sincerely,
Stein Haugan
Institute of Theoretical Astrophysics, University of Oslo

@pjotrek-b
Copy link

This sounds very useful!

I'm working with similar scenarios with synchronizing large scale archival media contents across sites.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants