RFC: --exec with placeholder corresponding to regex groups/ocurrences

[juarezr]

What

Allow the groups/occurrences matched by the regex pattern over the filename/path to be used as placeholders for commands executed with the flag --exec.

Note however that it's just a proof-of-concept yet. I just have written for checking if it could become accepted. The code is far from done but at least it's possible to run and play with any supported regex and the placeholder support.

So, please share your opinion.

Features

• Allow the use of regex group constructs for extracting specific pieces from the files/directories name.
• Allow capturing as many occurrences as matched by the regex pattern and use them as placeholders.
• Allow specifying a default text when the group/occurrence is not matched.

Uses cases:

Renaming or moving files and folders:

fd '(.+)( copy.*)(\.\w+)$' -x mv "{}" "{1}.{3}"

Adding prefixes, text, and extensions to file names:

fd '([\w-]+)(\.[\w-]+)?$' -t f -x echo "|| {} |groups| {//}/{1}{2:-.ext}"

Syntax

From the help:

-x, --exec <cmd>...
    Execute a command for each search result in parallel (use --threads=1 for sequential command execution). All positional
    arguments following --exec are considered to be arguments to the command - not to fd. It is therefore recommended to
    place the '-x'/'--exec' option last.
    The following placeholders are substituted before the command is executed:
      '{}':   path (of the current search result)
      '{/}':  basename
      '{//}': parent directory
      '{.}':  path without file extension
      '{/.}': basename without file extension
+     '{N}':  text matched by the N-th group in the first pattern occurrence. Text outside groups is discarded.
+     '{M.N}': text matched in the M-th pattern occurrence by the N-th group over the path or filename.
+     '{N:-D}': text matched by the optional occurrence/group or defaults to 'D' when not matched.
+   Obs:
+   - Using 0 for M/N substitutes by the text from all groups or all occurrences respectively.
    - If no placeholder is present, an implicit "{}" at the end is assumed.

Examples

cargo run -- '(\w+)\.(\w+)$' -x echo "|| {} |groups| {//}/{2}.{1}" | column -t

cargo run -- '([a-zA-Z0-9]+)([\._-]\w+)\.(\w+)$' -x echo "|| {} |groups| {//}/{1}.{3}" | column -t

cargo run -- '([\w-]+)(\.[\w-]+)?$' -t f -x echo "|| {} |groups| {//}/{1}{2:-.ext}" | column -t

cargo run -- '([\./])([\w\.]+)' -p  -x echo "|| {} |matches| {1.0} {2.0} {3.0} {4.0} {5.0} {6.0} {7.0} {8.0} {9.0} {10.0}" | column -t

cargo run -- '(\w+)([_-]+\w+)(\.\w+)?' -x echo "|/| {/} |0| {0} {0.1} {0.2} {0.3} |1| {1.0} {1.1} {1.2} {1.3}" | column -t

cargo run -- '([-_\.]+)?([a-zA-Z0-9])([a-zA-Z0-9]+)' -x echo "|/| {/} |0| {0} {0.1} {0.2} {0.3} |1| {1.0} {1.1} {1.2} {1.3} |2| {2.0} {2.1} {2.2} {2.3}" | column -t

cargo run -- '(/)(\w+)(\.\w+)?' -p -a -x echo "|| {} |groups| {0.1} {0.2} {0.3}" | column -t

[sharkdp]

So, please share your opinion.

Thank you for your contribution and especially for sharing not just an implementation but also use cases and examples. To be frank, I'm rather skeptical. But certainly open to discuss this.

Renaming or moving files and folders:

fd '(.+)( copy.*)(\.\w+)$' -x mv "{}" "{1}.{3}"

I typically use rename for a job like this. Could even be combined with fd. Haven't tested this, but something similar to

fd ' copy' -X replace ' copy.*\.' '.'

should do the same job... in an (arguably) more descriptive way?

fd '([\w-]+)(.[\w-]+)?$' -t f -x echo "|| {} |groups| {//}/{1}{2:-.ext}"

Adding prefixes, text, and extensions to file names:

fd '([\w-]+)(\.[\w-]+)?$' -t f -x echo "|| {} |groups| {//}/{1}{2:-.ext}"

Not exactly sure what that does, but adding prefixes, suffixes, and different extensions should already be possible with the existing placeholders, no?

[juarezr]

Thanks for considering to discuss it and sorry for my examples not being good enough to show the use cases.

I typically use rename for a job like this. Could even be combined with fd. Haven't tested this, but something similar to
fd ' copy' -X replace ' copy.*.' '.'
should do the same job... in an (arguably) more descriptive way?

You are right that there is alternatives for handling these use cases.
The feature I'm proposing is to have additional flexibility in fd for convenience.
What I'm looking for is to reduce the need of including another tool for solving the problem.

Also, the current tricks are still useful. :)

Not exactly sure what that does, but adding prefixes, suffixes, and different extensions should already be possible with the existing placeholders, no?

Partially:

• Only before and after the whole filename placeholders: {/} (basename) and {/.} (basename.ext).
• Only using fixed strings as -x parameters

But not being able to:

• Put anything directly in the middle of the filename.
• Replacing variable text in the middle of base and name parts of the filename
• Not exchanging positions of variable text
• Not choosing specific classes/slices of characters for deleting/transposing

See:

❱ fd 'fixed.*.txt' -x echo "rename fixed adequated {}"
rename fixed adequated ./files/one-fixed-name.txt
❱ cargo run -- '(.*)-(.*)-(.*).txt' -x echo "mv {} {//}/{1}-adequated-{3}.txt"
...
mv ./files/one-dynamic-title.txt ./files/one-adequated-title.txt
mv ./files/such-fixed-name.txt ./files/such-adequated-name.txt

For sure that would be possible to use additional tools to solve more complex and more dynamic cases like in this example.

For example, one can pipe fd with awk or sed and with xargs and with mv too for renaming very dynamic parts on a large list of files.

But having placeholders for substitution of regex matches in fd reduces the complexity of dealing with many more tools and their arguments. Also, fd gains much more power and flexibility for modifying any parts of filename and path as allowed by the possibilities of regex groups and matches.

I missed this when renaming some music files that had very messy file names. At that time I needed to make many complex changes like inverting the position of the artist with the song name as in this example:

❱ cargo run -- '([^-\n]+)(?:(\s+-\s+)(.+))?(.mp3)' -t f -x echo "{}_{//}/{3:-Unknown} - {1}.mp3" | column -t -s '_'
...
./music/I Still Haven't Found What I'm Looking For - U2.mp3  ./music/U2 - I Still Haven't Found What I'm Looking For.mp3
./music/Time to Love.mp3                                     ./music/Unknown - Time to Love.mp3
./music/We Will Rock You - Queen.mp3                         ./music/Queen - We Will Rock You.mp3
./music/Angie - Rolling Stones.mp3                           ./music/Rolling Stones - Angie.mp3
./music/I Will Follow - U2.mp3                               ./music/U2 - I Will Follow.mp3
./music/Sunday Bloody Sunday - U2.mp3                        ./music/U2 - Sunday Bloody Sunday.mp3
./music/We Are the Champions - Queen.mp3                     ./music/Queen - We Are the Champions.mp3
./music/Brown Sugar - Rolling Stones.mp3                     ./music/Rolling Stones - Brown Sugar.mp3
./music/Desire - U2.mp3                                      ./music/U2 - Desire.mp3
./music/One - U2.mp3                                         ./music/U2 - One.mp3
./music/I Want to Break Free - Queen.mp3                     ./music/Queen - I Want to Break Free.mp3
./music/Satisfaction - Rolling Stones.mp3                    ./music/Rolling Stones - Satisfaction.mp3
./music/With or Without You - U2.mp3                         ./music/U2 - With or Without You.mp3

I hope that these examples let the motivations become clearer. :)

[cclifford]

Hi, I just wanted to express my interest in a feature like this being included in fd. I use fd pretty heavily to select data files from a collection of files for processing. Often we store metadata as path components within our data repository (for example, what sensor it originates from). various scripts we use require this information, so we pass it in with commandline flags. Right now I use short inline bash scripts to parse the information out of the file path and pass it to the scripts, but this is pretty fragile and gross.

Anyway, I ran into this older PR researching to see if I could implement a similar feature. I though I would comment because I have a use case that is not file renaming.

[dsummersl]

This would be an awesome feature - every now and then I need to make a rename or move based on some pattern within files or directories. This would make such changes kinda trivial/magic!