You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
force, false by default. if true:1. not check meta when list files, 2. not return an error if found dup when committing paths.
purge, false by default. if true, try delete copied files after committing. note that deleting can only be a best effort.
since purge is best effort. we rely on meta (force=false) to make sure we will not commit the same file twice.
some times, we may need to reload some files ( or reuse paths?). so we need force=true.
checking and committing paths may be slow when there are many small files.
so we used to introduce a feature: #10855, [purge] and [force] are both enabled, will not update copied-files set.
this is obscure to user, for completeness and orthogonality, I think it is better to add another copy option for this. Like force, it is an Advanced option.
after copy support return copied files lists,#7730. users may be able to dedup the files by themselves.
The text was updated successfully, but these errors were encountered:
Summary
currently we support 2 options:
since purge is best effort. we rely on meta (force=false) to make sure we will not commit the same file twice.
some times, we may need to reload some files ( or reuse paths?). so we need force=true.
checking and committing paths may be slow when there are many small files.
so we used to introduce a feature: #10855,
[purge] and [force] are both enabled, will not update copied-files set.
this is obscure to user, for completeness and orthogonality, I think it is better to add another copy option for this. Like
force
, it is an Advanced option.after copy support return copied files lists,#7730. users may be able to dedup the files by themselves.
The text was updated successfully, but these errors were encountered: