You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Apr 8, 2024. It is now read-only.
I would encourage to leverage Alluxio technology, in particular the fuse mount feature, for synchronizing local file system with remote data repositories such as S3.
Let's suppose the data is very large and you do not want to fully copy it locally, you can store the data in S3 and read smartly from the Alluxio tiered cache via the local file system.
lazydata should then only check consistency of files, store and track the metadata attributes into git.
The text was updated successfully, but these errors were encountered:
I would encourage to leverage Alluxio technology, in particular the fuse mount feature, for synchronizing local file system with remote data repositories such as S3.
Let's suppose the data is very large and you do not want to fully copy it locally, you can store the data in S3 and read smartly from the Alluxio tiered cache via the local file system.
lazydata should then only check consistency of files, store and track the metadata attributes into git.
The text was updated successfully, but these errors were encountered: