Big scenarios execution time

[AndrewRybka]

Hi,

It takes ~111 ms to run all the 3 scenarios from the ByFramework/xUnit example when I run it as a standalone project:

But when I move that example to one of my test projects with 600+ other tests, the execution time raises to 5+ seconds, and each scenario is executed for more than a second:

Is there a way to reduce scenario execution time in the second case?

Thanks

[mchaloupka]

Hi,

are you able to share a project where it happens so we can reproduce it?

Potentially, are you able to run a profiler to see in which method it is spending most time?

Thanks,
Milos

[mchaloupka]

Hi @AndrewRybka,

are you able to run a profiler to see what is the issue?

Thanks,
Milos

[AndrewRybka]

Hi Milos,

Sorry for the delay with the answer.

Unfortunately, I can't share the initial project.

But I generated a project with ~2K unit test with a single assert in each (XUnit facts, theories, and FsCheck properties) in an attempt to replicate the issue. With that project it takes 900-1000 ms to execute scenarios from the ByFramework/xUnit example. With each scenario executed for 200-400 ms. And when I add additional scenarios, the overall execution time grows proportionally. So it seems that the number of tests in the project affects the execution time of each scenario. Here is the project with 2K tests:
ByFramework.XUnit.zip

Also, I profiled the initial project while running only tests from the ByFramework/xUnit example. Here is the results overview:
snapshot.txt

I can send the whole profiling snapshot, created by dotTrace, via email.

Thanks

[plainionist]

i observe similar behavior: the more tests (bigger scenarios) i add the slow the execution gets. any insights meanwhile what the root cause could be and how to improve?

[mchaloupka]

I have created a pull request with improvements: #69. That should improve it significantly.

@AndrewRybka thanks for providing the example project, although it took me forever to get to it, I have used it to figure out what is the problem.

[plainionist]

The improvement in execution time is HUGE in my case - thx a lot!

[mchaloupka]

@plainionist Out of curiosity, can you share what was roughly the improvement you have seen and how much time it still takes?