Difficulty using Async/Await code

[TheTechmage]

Maybe I'm just missing it and it's discussed somewhere (I couldn't find it while googling/searching the docs), but I'm trying to use code that relies on asynchronous Python code. Here's an example of what I'm trying to achieve. The code/flow shown here is dummy code, I don't want to go too deep into the weeds if it won't work out.

I'm trying to create a message router using an async-based library. The idea is that messages come in, than the appropriate "register_route" function will be called if it matches the message. I was going to put while-true generators in the nodes so that they could continually pump out messages, but since I rely on async/await, it doesn't appear that the other nodes down the line know how to handle it. This can be seen by the <async_generator object register_route> string being shown in the text viewer.

So with that being said, I'd like to know. Is it even possible at this point in time to use async/await? If not, will there be any plans to do so?

I've got to say, great application! Love how you designed the nodes and how simple it is to string things together.

[KennedyRichard]

Hello, @TheTechmage ! Thank you for the kind words!

Is it even possible at this point in time to use async/await?

Unfortunately at the moment Nodezator only executes its graphs sequentially. There are a few reasons for that.

The first and most important is that Nodezator is a rather ambitious project: it aims at being an almost 1:1 visual representation for Python. That's why any graph in Nodezator can be exported to plain Python code with the click of a button. However, this poses its own challenges for us when it comes to non-sequential execution: we need to come up with a design/model that allows our exported code to also execute non-sequentially. And not only that: because Python has multiple non-sequential execution paradigms (threads, parallelism with processes, async, coroutines), Nodezator also has to be able to deal with those.

I wrote about this in detail in this discussion: #80

Another reason is that there are a few other urgent features coming to Nodezator before I implement asynchronous execution, like group nodes (subgraphs), input nodes, output nodes, undo/redo and more.

Yet another reason is that the challenge of designing a model/design to handle multiple non-sequential execution paradigms requires time and research, something that I need to do before I can start any sort of implementation. I actually already started doing such research.

This doesn't mean you strictly can't execute non-sequential code. Even though the execution is sequential, it might be possible to gather all the data in a single node where, for instance, you could block while executing the different operations in a thread pool until that part of the code is finished. However, I never tried anything of the sort, so this is only theory.

If not, will there be any plans to do so?

Yes, there are.

It is my desire to not only allow the graphs to execute non-sequentially, but even provide a relatively simple way to do so, so that users only need to worry about plugging their nodes and the app will handle the asynchronous/parallel execution for them, possibly using multiple paradigms depending on the nature of the operations.

This seems impossible or very difficult when it comes to Python, but it is not unheard of in other languages (like Erlang, where concurrency is trivial), so I think some generalizations may be possible that can make concurrency/parallelism easy in a graph. I think we might be able to come up with a multiparadigm model that can handle different kinds of concurrency/parallelism in a seamless way.

On top of that, I want the equivalent exported code to also execute seamlessly and employ multiple paradigms as needed in much the same way.

As I said, I already started researching, although I had to temporarily pause such research due to my work on other Nodezator features. I estimate it will be quite some time before non-sequential execution lands in Nodezator, however. Maybe a year, probably more. But I think it is worth the wait, as it will be an invaluable tool for the Python ecosystem, specially since Nodezator is in the public domain and free of charge.

---

Don't ever hesitate to ask, suggest, share your concerns/feedback/criticism. Let me know if you ever need anything else.

Peace.