Skip to content
This repository has been archived by the owner on Oct 30, 2019. It is now read-only.

Testing of asynchronous code #38

Closed
ashfordneil opened this issue Aug 2, 2018 · 4 comments
Closed

Testing of asynchronous code #38

ashfordneil opened this issue Aug 2, 2018 · 4 comments
Labels
WG async/await Issues relevant to the async subgroup

Comments

@ashfordneil
Copy link

Right now, rust's standard way of writing tests and the standard way of executing futures (tokio) are at odds with eachother. The test framework expects a panic on test failure - tokio catches panics and treats them as normal errors. The test framework tries to run multiple tests in parallel where it can - tokio immediately spawns one thread per CPU core when you run it. This makes testing asynchronous code pretty hard, and that's something we should fix.

I think to fix this, we need a simpler futures executor that doesn't include quite so many batteries. Instead, a run function that just runs like a regular function - without trying to do anything with threads or panic catching. Ideally, in the name of keeping test and prod environments similar, I would suggest we at least consider making tokio that simpler executor*. Also possible would be taking something like toykio further, or potentially even going the full way to supporting #[test] async fn ... in future.

*and separating all of the threading magic done by tokio out into a wrapper around a simpler core executor. This would be a big move, and I may be missing a whole bunch of reasons why its a bad idea, but on the bright side the extra flexibility could help people who

  • want really high performance, and need custom control over how things work across threads in their program
  • want a really light footprint for their code (nobody likes the app that spawns 8 threads just to transfer a file to another computer)
  • don't expect high enough load for their program that the effort to make their code threadsafe is justified for the performance gains it would bring
@carllerche
Copy link

The Tokio current_thread::Runtime spawns no threads.

@Nemo157
Copy link
Member

Nemo157 commented Aug 2, 2018

futures itself already provides a light weight executor that can be used for tests, you wouldn’t be able to use tokio types with it as they additionally require a reactor for wake notifications, but you can use in-memory pseudo-IO types with it. They’re a bit limited at the moment for test purposes as they always return Ready, I would expect at some point there will be a library for test purposes that provide in-memory types that act more like real IO types by actually returning NotReady occasionally.

The test framework expects a panic on test failure - tokio catches panics and treats them as normal errors.

This is not an issue anymore, as long as the top-level executor call returns these panics-as-errors to you, your test function can now return the error to signal a failure.

@Thomasdezeeuw
Copy link

Not related to Tokio, but creating a Future's Context is a bit hard in the context of testing. For both the LocalWaker and Executor you need something of a runtime. Maybe a future-test crate would be useful, it would provide a Context that easy is to use in (unit) tests. For example calls to wake would just add a notification of some kind to a collection. The Executor would do something along the same lines.

@Nemo157
Copy link
Member

Nemo157 commented Jun 14, 2019

In case there are subscribers to this that haven't seen it, quite a while back a futures-test-preview crate was added that contains common utilities for testing futures, e.g. constructing different kinds of Context or wrapping synchronous/in-memory async IO implementations in ones that attempt to emulate being async, see the blog post and the documentation.

# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
WG async/await Issues relevant to the async subgroup
Projects
None yet
Development

No branches or pull requests

5 participants