Add API endpoint to kill a workflow

[ashiklom]

Sometimes, we make boo boos (e.g., requesting 100 years of met download -- oops!). In those cases, it would be nice to have a shortcut to cancel a currently running workflow. Something like:

rpecanapi::kill_run(server, workflowid)
# Candidate API endpoint: <IP>/api/workflows/cancel?workflowid=XXXX

Currently, to cancel a run, you need to do some combination of:

• Terminate the R process in Portainer
• Purge the RabbitMQ queue
• Restart RabbitMQ

Would be nice to do this in a more convenient way.

[github-actions]

This issue is stale because it has been open 365 days with no activity.

[AritraDey-Dev]

Is this issue still on?
After going through docs and apps/api/R/workflows.R it seems to be not done yet.
Should I work on this?

[mdietze]

AFAIK no one has worked on this. That said, if you want to dive into the API I'd argue that an easier and more urgent need would be to break up the monolithic "workflow" (which calls web/workflow.R) into additional endpoints that breaks things up into smaller modules for each major step. This would allow much greater flexibility in PEcAn usage. For example, if one had a pecan settings that had all of the inputs already configured, one could in theory call an API that just calls the run.write.configs module and the start.model.runs module (or even break these into two separate steps) and then the use could use other existing APIs to grab the outputs. To get to that state where you start the model runs there could be separate API end points for preparing the meteorology, soils, initial conditions, phenology, parameter files, etc. Such an effort might dovetail nicely with the GSOC project about notebook-based execution, as well as the one about parallelization, but to be 100% honest I think this idea (both in it's original form and in my suggestions) is far less urgent a priority than diving directly into the core tasks of those projects.

[AritraDey-Dev]

Thanks for the explanation! Given the priorities, do you think it's worth starting some initial work on breaking up the workflow, or should I wait until the GSOC projects progress further?

[mdietze]

I'd say that breaking up the workflow via direct execution of R code (e.g. R Markdown) is a higher priority than breaking up the workflow via API (though I'd expect the latter could/should also include an example/template notebook). In terms of "waiting until GSOC project progress further" that would depend on whether you're talking about waiting for someone else to complete tasks you need, or waiting to start those tasks yourself. If the latter I don't see a reason to wait.

[AritraDey-Dev]

Got it! I'll start working on breaking up the workflow via direct R execution. Here’s my planned approach:

• Identify major workflow steps in web/workflow.R and separate them into distinct modules.
• Create independent R scripts or R Markdown files for each step (e.g., meteorology, soils, initial conditions, phenology, parameter files, etc.).
• Ensure modular execution, so users can run only the necessary steps without executing the full workflow.

Let me know if you have any modifications or suggestions before I proceed!

[mdietze]

For direct execution, I don't think you want the separate steps in separate scripts or Rmd files, that kinda defeats the purpose of a notebook-based approach. Plus the workflow is already separated into fairly distinct modules, such that a number of the proposed scripts may contain a single line of code. API-based execution may be another story (on the backend each I suspect we want each API call running a single function/module)

[AritraDey-Dev]

Understood! I'll work on breaking the workflow into modular API endpoints for each major step. This should make execution more flexible and allow users to call only the necessary steps. If you have any preferences on how you'd like these endpoints structured or any additional functionality you'd like to see, let me know!

[AritraDey-Dev]

@mdietze need your insights on this PR #3486 before i proceed for any change required.