Incorrect coverage report

[jgosmann]

Using py.test --cov-config .coveragerc --cov nengo -n 6 nengo a lot of lines that should be hit get reported as missed (like class and function definitions in a module). This might be related to #19 as the project has a conftest file importing other modules from the project.

Using coverage run --rcfile .coveragerc --source nengo -m py.test nengo instead a correct coverage report is generated, but this command does not support xdist.

[ionelmc]

It might be possible to measure it with a workaround like this: https://github.com/ionelmc/pytest-benchmark/blob/master/tox.ini#L32-L33 - it forces the engine used for subprocess measurements to load early.

[yunstanford]

Have this been fixed ? i am still getting this issue.

[ionelmc]

Not yet fixed. There is a workaroud tho.

[ionelmc]

Show an example and maybe we can figure it out.

[yunstanford]

you can checkout the last travis ci build of Sanic https://github.com/channelcat/sanic

[ionelmc]

Maybe your local checkout is dirty (have any .coverage/.coverage.* files?). Can we narrow it down to something smaller?

[ionelmc]

Ah yes, it's a pytest plugin, so plugin loading is currently unpredictable. See pytest-dev/pytest#935

[yunstanford]

yeah, you're right. i had .coverage files, after git clean -xfd, i can also reproduce locally.

[yunstanford]

How can we fix that ? should we load pytest-cov first ? what step matters actually when running pytest, pytest_configure ?

[ionelmc]

Well ... this being a problem in pytest, we can only talk about workarounds here.

[yunstanford]

yeah, any suggestion ? it works fine for me when i use uranium another python build system (https://github.com/yunstanford/pytest-sanic/blob/master/ubuild.py#L16). but it doesn't work with tox.

[ionelmc]

Afaik it's either the workaround I use in pytest-benchmark (forceload pytest-cov subprocess engine) or ronny's plugin trick (haven't tried it).