Copy mode: files in different source directories end up in the same destination dir

[kcgthb]

Copying multiple files located in different source directories end up in the destination files being created in the same destination directory. The effective destination directory is the source dir of the first file in the list.

To reproduce this, consider the following source tree:

# tree /tmp/test/
/tmp/test/
├── dir1
│   └── file2
└── file1

1 directory, 2 files

Copying /tmp/test/file1 and /tmp/test/dir1/file2 to sh03-sn01 results in file1 and file2 being copied in the directory where file1 was on the source machine (/tmp/test/).

# clush -d -c /tmp/test/file1 /tmp/test/dir1/file2 -w sh03-sn01
DEBUG:root:clush: STARTING DEBUG
Changing max open files soft limit from 14096 to 8192
User interaction: True
Create STDIN worker: False
clush: nodeset=sh03-sn01 fanout=512 [timeout conn=15.0 cmd=0.0] copy sources=['/tmp/test/file1', '/tmp/test/dir1/file2'] dest=/tmp/test/
SCPCLIENT: scp -oConnectTimeout=15 /tmp/test/file1 [sh03-sn01]:/tmp/test/
SCPCLIENT: scp -oConnectTimeout=15 /tmp/test/dir1/file2 [sh03-sn01]:/tmp/test/
`/tmp/test/dir1/file2' -> sh03-sn01:`/tmp/test/'
`/tmp/test/file1' -> sh03-sn01:`/tmp/test/'
DEBUG:ClusterShell.Worker.EngineClient:<EnginePort at 0x139629430609104 (streams=(8, 9))>: dropped msg: (<function _abort at 0x7efe0333aa28>, (False,), {})

Running with -d shows the 2 scp commands being used:

SCPCLIENT: scp -oConnectTimeout=15 /tmp/test/file1 [sh03-sn01]:/tmp/test/
SCPCLIENT: scp -oConnectTimeout=15 /tmp/test/dir1/file2 [sh03-sn01]:/tmp/test/

Here's the resulting destination tree:

# ssh sh03-sn01 tree /tmp/test/
/tmp/test/
├── file1
└── file2

The expected result would be to get the same directory structure recreated on the destination machine, that is:

• file1 in /tmp/test
• file2 in /tmp/test/dir1

[thiell]

Thanks! Great find! The current code (1.9.1) supports multiple source files but if --dest is NOT specified, the dirname() of the first source file is used as the destination directory. This is a limitation that should be fixed:

clustershell/lib/ClusterShell/CLI/Clush.py

Line 1128 in 4f6df52

options.dest_path = join(dirname(abspath(args[0])), '')

We use a call to task.copy() per source file so the problem resides in Clush.py only.