Run same OnDemand Request with different parameters set in commandline

[the100rabh]

I am trying to run a jar file while get some data and put it on S3. So I am thinking of it as an onDemand Task.
Things like what data to get and some other parameters change everytime this task needs to run.
Is there a way to set these parameters while running the task and not just while deploying it. Or do I need to deploy a different request each time which to me seems like an extra step

[the100rabh]

Just adding to the my doubts here, When I pass commandline arguments to a one off task, following is the what can be seen in stdout for the task.

Forked command at 38876
[java -jar mytask.jar , 1 1000]
Command terminated with signal Aborted (pid: 38876)

This comma in what makes me think that this might be a bug. If comma is not present the code will run correctly I guess

[ssalinas]

Hey @the100rabh , there are a few nuances with the way mesos processes the commands for these tasks, and it has to do with shell mode. There are a few fields you could use to get this working the way you want and I'll try to outline that here.

On deploy you would probably want something like these fields set in your deploy:

'command': 'java',
'arguments': ['-jar', 'mytask.jar']

This on the deploy is specifying that the command sent to mesos is ['java', '-jar', 'mytask.jar']. Because they are specified as arguments instead of the command, mesos does not process this in shell mode. (i.e. they end up passed as that array, not as just a string)

Now for the extra arguments each time. When you POST to the /requests/request/{requestId}/run endpoint, you can specify the commandLineArgs field in your SingularityRunNowRequest object. These will then be added to the end of that command array when passed to mesos.

To explain the shell setting a bit more, whenever Singularity sees that arguments or commandLineArgs are set, it will set shell to false. So in you case this meant that you command java -jar mytask.jar became the first element of that command array, and the way that gets processed, means that mesos expects that first element of the array to be an executable, and the following array elements are args. So it was looking for an executable named java -jar mytask.jar not just java.

Hope this helps, feel free to reopen if you have any more questions.

[the100rabh]

@ssalinas I tried exactly what you mentioned above. Below is the output I got in STDOUT

Forked command at 3264
[java, -jar, mytask.jar, 1, 1000]
Command exited with status 1 (pid: 3264)

[the100rabh]

Just to add to what I mentioned above @ssalinas, I if include all the arguments in command it works wonderfully.
by giving in the Deploy request

"command":"java -jar mytask.jar 1 1000" 

Output of STDOUT

Forked command at 3393
sh -c 'java -jar mytask.jar 1 1000'

[ssalinas]

@the100rabh was there any helpful output in stderr that might provide more info? First thought it that it isn't finding java on your path since there is no sh -c in front of it here

[the100rabh]

@ssalinas I think the problem here is because of no sh -c being added while running the command along with arguments. It feels weird because things work out well if there are no arguments being passed, but fails when there are any arguments attached to this.

Is there anyway I can try and debug this at my end. Would love to dig deeper into this but dont know where to start looking. Any implementation or design docs would help me as well.

[ssalinas]

@the100rabh , when in shell false mode, (i.e. the array of commands) it can be a bit pickier about the path and finding the command to run. Can you try it with the exact path to java (like /usr/bin/java or wherever it may be on your machine)? The problem with the shell true mode (adding sh -c) is that mesos will then ignore any additional arguments that are passed, only the original command will be executed.

[the100rabh]

Tried that, same result

[the100rabh]

Another piece of info, not sure if that will be helpful or not. Even if default arguments are given, it gives the same result.

[ssalinas]

@the100rabh have you gotten this working, or are you still having issues here?

[ssalinas]

Closing this one since we haven't heard back in a while. If you are still having issues, feel free to reopen

[ediril]

Many years later, it looks like this issue is not resolved. I'm having the same problem.

I have a simple bash script which expects 2 command line arguments. If I don't use "arguments" and just construct the command in full, then it works. If I specify command and arguments separately like you suggested about, it does not work no matter what I do.

This is not a problem for "Run Once" requests, but additional arguments is a must-have for "On-Demand" requests.

So how do I pass in arguments to commands separately? For example, for a "Run-Once" request, I'd like the following to work (but it doesn't at the moment):

{
  "deploy": {
    "requestId": "sleep-once",
    "id": "6",
    "command": "/home/sleep_task.sh",
	"arguments": [ "23s", "some string" ],
    "shell": true,
  }
}

I think if I can get this to work, the same thing will apply to specifying additional command line arguments to "On-Demand" deployments

[ediril]

I think I'm starting to understand how this is supposed to work for the "Run-Once" request. The following worked:

    "command": "/home/sleep_task.sh",
    "arguments": [ 
		"/home/sleep_task.sh",
		"23s", 
		"is it working?" 
    ],
    "shell": false

So my problem was that I was not adding the executable to the arguments list. This is weird, but I can live with that.. (even providing "" as the first argument in the arguments list works)

Now I need something like this for the "On-Demand" request though. In that case, I'd like to specify the base command during "Deploy" and when I run, I'd like to provide additional arguments which are run dependent. How can I make that work?

[ediril]

I think I got it working for On-Demand as well with the following json data:

{
  "runId": "99",
  "commandLineArgs": [
     "",
     "35s", 
     "another argument" 
  ],
  "shell": false
}

and POST'ing to this endpoint: /api/requests/request/{requestId}/run

In both cases, the trick is to use "shell": false. This was a lucky guess on my part for the On-Demand request because shell wasn't listed as a possible field in this endpoint's documentation. However, it is mentioned in #953 that extra arguments are ignored if shell is not set to false.

[ssalinas]

^closing since the above mentioned it was working now