Decorate pytest_load_initial_conftests hook with @pytest.mark.trylast

[mkemmerling]

... or convince the pytest-cov maintainer to decorate their pytest_load_initial_conftests hook with @pytest.mark.tryfirst.

This issue refers to issue #175 (and pytest-dev/pytest-cov#19) which did not solve the problem for me that pytest_cov fails to cover module level code.

pytest_django 2.7.0 ans pytest_cov 1.8.0 both use a pytest_load_initial_conftests hook to setup Django and to start coverage measurement, respectively. But it is important that the hook in pytest_cov is executed before the hook in pytest_django. So the question is, which of the two hooks is executed first. I found that the hook in pytest_django is executed first, therefore coverage measurement is still broken. I suppose that this is platform dependant.

In more detail:

As usual I installed pytest_django and pytest_cov via pip in a virtual environment. pytest loads both plugins via their setuptools entry points. The order in which they are loaded depends on the scannning of the installed packages in the virtual environment. pytest's PluginManager uses setuptools iter_entry_points() function to iterate over all entry points. https://pythonhosted.org/setuptools/pkg_resources.html#convenience-api clarifies that "entry points are yielded from the active distributions in the order that the distributions appear on sys.path", but it does not say anything about the order of entry points from packages living in the same directory. Poking a little bit around I found that - at least under Python 3.4 - the packages seem to be looked up with os.listdir() which returns directory entries in arbitrary order. In my case pytest_cov was registered before pytest_django, its pytest_load_initial_conftests hook therefore executed after the one in pytest_django (the order is reversed by MultiCall.execute() which pops the hooks to be executed from the list of collected hooks).

Fortunately pytest allows to determine the order in which hooks are executed in some way with the @pytest.mark.tryfirst and @pytest.mark.trylast decorators (see http://pytest.org/latest/example/markers.html#registering-markers). Decorating the pytest_load_initial_conftests hook in pytest_django with the @pytest.mark.trylast decorator ensures that Django is setup after coverage measurement has been started.

Note that the pytest_load_initial_conftests hook in pytest_cov is decorated with an invalid @pytest.mark.try_last decorator (note the underscore) which has no effect. Considering the fact that coverage measurement should probably always be started as early as possible the better solution might be to decorate the pytest_load_initial_conftests hook in pytest_cov with the @pytest.mark.tryfirst decorator.

Sorry for this rather long posting ;-)

[mkemmerling]

I also filed an issue with pytest_cov: pytest-dev/pytest-cov#24.

[mkemmerling]

The pytest_load_initial_conftests hook in pytest_cov is now decorated with @pytest.mark.tryfirst (pytest-dev/pytest-cov#25, not yet released). This solves this issue :-)

[pelme]

Ok, good to see that it was fixed. :) I'm closing this issue now!