CLAs vs options

[zbeekman]

Does FLAP have any notion of command line arguments vs command line options? See for example https://en.wikipedia.org/wiki/Getopt#Shell and the links from there, http://www.bahmanm.com/blogs/command-line-options-how-to-parse-in-bash-using-getopt, and man getopt.

I often want to have a program that takes some optional flags, that might have arguments, but then a list of files, directories, etc just passed as white space separated strings. Also the special -- denotes the end of options so you can pass an argument that looks like an option but is actually treated as an argument. (I use shell globbing and brace expansion to construct lists of stuff to process.) For example I have a code that will take ascii tecplot data files, each one corresponding to a flow snapshot, and average them across snapshots. The code might have some options to specify different behaviors but then takes a list of files to average as the rest of the arguments. After the options are processed there are an arbitrary number of positional arguments. Looking through the documentation I can’t figure out whether FLAP already supports this usage case, and if so, how one would tell FLAP to loop over the remaining arguments not corresponding to option flags/switches.

I would be invoking such a program like this:
process_things --working-dir=../path/to/data --no-zones -- *H 0{0..2}{0..9}{0,5}h
The stuff between process_things and -- (which in this case is pretty much optional unless the shell glob *H expands to something that looks like an option, say -fooH or something like that) are flags/options to modify the behavior. These are the type of things FLAP already handles and handles well. (The = syntax isn’t required; I don’t know whether FLAP supports this already or not, a space would be fine instead of =) Then after the program processes the options, and sets default values etc. a loop is run over the remaining CLAs which are expanded by the shell via globbing or brace expansion etc. or from the output of some other command like process_things -v --working-dir=../path/to/data/ -- $(command_to_generate_remaining_args) I have some code that kinda-sorta emulate getopt but FLAP does a lot of it much better.

[szaghi]

I am reading with smartphone, thus I cannot read your references, later at ofgice I will investigate in more details. Anyhow, most part of your example is supported.

• process_things --working-dir=../path/to/data --no-zones supported;
• process_things -- *H 0{0..2}{0..9}{0,5}h not yet supported.

Is the -- *H... data always the last? If yes, it can be implemented very quickly. Presently, the main limitation is the unsupported list of arguments with undefined number of arguments (only a priori sized list is currently supported).

For the special -- option, I will try to implement as soon as I understand it :-)

[szaghi]

Ok, I find 5 minutes...

If I understand right, you are distinguishing between:

• arguments: passing to CLI a list of values;
• options: passing to CLI a list of named options (each having an eventual arguments list or not).

Presently in FLAP there are:

• positional arguments: argument-values that must not be preceded by a named option, but must be passed into a specific position;
• named arguments: named options with an eventual following argument(s) value(s).

For example:

foo 2 7 8 --named 1 --verbose

means that foo has 3 positional arguments that must be passed at the first 3 positions, 1 named option accepting a value (--named 1) and 1 named option not requiring a value (--verbose).

If I understand your issue, you are searching for a new (for FLAP) logic that when the special option -- is found all the arguments following are interpreted as final positional arguments (in the FLAP nomenclature) meaning that the named-options part of the CLI finishes at --. Thus, the previous example can become:

foo 2 7 8 --named 1 --verbose -- file1.f90 ~/my_special_path/pearl_jam.mp3 the_caos.pdf

Is my guess right? If not, please give me a minimal working example of what you are searching for.

[zbeekman]

Yes, that is exactly what I am after some “final positional arguments” and there needs to be a way to iterate over them. This could be to query how many of them there are so that the user can extract them in a loop, or an iterator method allowing you to get the next one. Querying the number would be more useful for me, though, since the number doesn’t change once the program execution begins.

Typically the -- is optional, and only required when one of the “final positional arguments” would otherwise be interpreted as a CLA named argument.

[szaghi]

mmm,

for the iteration/numbers-counting there is no problem: it is very simple to return to the user an array (of strings) dimensioned at runtime over which the user can loop. On the contrary, if the -- flag is optional, the implementation could be more complex...

[zbeekman]

Well, I'd be perfectly happy for it to be mandatory

[szaghi]

@zbeekman Ok, I first will try with mandatory -- and then I will think how to make it not mandatory.

[zbeekman]

I like your new profile image! Is that a numerical schlieren of a simulation computed with your OFF code?

[szaghi]

Yes Sherlock!

It represents an instant of the starting transient of an highly under-expanded (I do not remember but should be Mach 6) jet. It should be a multispecies flow (jet lighter than ambient) simulated with a modified WENO-positivity-preserving scheme.

[szaghi]

Done (v0.6.0).

Now you can pass your options (named and/or positional) and finally put a -- list_of_trailing_garbage, e.g. the following should be supported:

foo   231 -i pippo -- foo.f90 bar/baz.f90 pluto-giove.dat 2 313
        ^      ^            ^
    positional | named | garbage

Indeed the garbage could (in theory) appears everywhere, not only as trailing garbage. To retrieve the eventual garbage you must use the new cli%get_varying() method, see https://github.com/szaghi/FLAP/wiki/CLI-Method-get_varying: the garbage values are stored into an allocatable array (characters, numbers and logical are supported) over which you can easily loop (see https://github.com/szaghi/FLAP/blob/master/src/Test_Driver.f90#L188). In case there is no garbage the cli%get_varying() method returns an not allocated array.

This is a new feature, thus a lot of bugs could have been introduced. ASAP you tested this, feel free to submerge me of issues for bug-fix :-)

Thank you @zbeekman

[szaghi]

Oh no, I just see you PRs now... this is a good time for using your merging suggestions...

[szaghi]

@zbeekman help!!!!

After trying to push the new v0.6.0 I gotten:

Counting objects: 26, done.
Delta compression using up to 24 threads.
Compressing objects: 100% (26/26), done.
Writing objects: 100% (26/26), 7.55 KiB | 0 bytes/s, done.
Total 26 (delta 17), reused 0 (delta 0)
To git@github.com:szaghi/FLAP.git
 * [new tag]         v0.6.0 -> v0.6.0
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'git@github.com:szaghi/FLAP.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

What I have to do?

[zbeekman]

@szaghi run git fetch origin followed by git diff master origin/master to see why your version of master is different than the most recent version. It’s possible a simple git co master git pull origin master git push —tags origin will fix the issue… but basically what has happened is your version of master is different that the version pushed to github, and resolving the difference isn’t as simple as adding additional local commits on top of the commits on origin/master.

[zbeekman]

sorry… on my system git co is an alias for git checkout so in my previous comment, replace git co with git checkout

[szaghi]

@zbeekman

thank you, I see checkout. However, the git diff list the effectively differences, but master is more up-to-date than origin/master. What happen if I do a git pull origin master? I lost my master updates? Why on the github there is the correct tag v0.6.0 with the updates?

[zbeekman]

OK, so you never showed me what command you actually ran when pushing your tag when you listed the output, but whatever push command you used tried to push both the new tag, v0.6.0 and the master branch. The tag was accepted, and looks like it is now ahead of origin/master But, the histories of master and origin/master have somehow diverged. Because of this it won’t let you push the master branch because it can’t figure out how to apply your changes. … let’s chat in gitter.im

[zbeekman]

circling back to the original topic, I took a look at the implementation and it looks good! My only objection is calling the stuff after -- garbage. It is often the most critical input to the entire program! This is a just a small complaint though. If I work up a PR with git submodules etc. I may reword some of your references to “garbage”

[szaghi]

:-) I think that garbage is used only into the test driver, the user can name it also critical :-)

Anyhow, implementing it I have added some new other features (nargs=+, nasrgs=*) in particular now there is the notion of hidden CLA (not yet documented, the -- is indeed an implicit hidden cla): such a CLA could be used into the development cicle when the devoper modify the CLI but like to keep the public interface unchanged, or it could be useful for other implicit option, or for your spy-usage :-)