Zero coverage of process spawned by dogtail

[olumide-x]

Our team uses dogtail to GUI test our application but unfortunately we've never been able to get any coverage from these tests.

I've prepared a simple test demonstrating this scenario. (Create/download these files into the same directory.)

1. my_app.py -- a simple GTK Python application
2. test_my_app.py -- a dogtail script that launches the above GTK Python application
3. .coveragerc -- .coveragerc file.

Other steps I took to run the above scripts on Ubuntu LTS 22.04:

• I created a sitecustomize.py module in my local site packages directory

#/home/Olumide/.local/lib/python3.10/site-packages/sitecustomize.py
import coverage
coverage.process_startup()

• Create the COVERAGE_PROCESS_START environment variable as follows export COVERAGE_PROCESS_START="${PWD}"/.coveragerc
• From the directory containing the files, attempt to get coverage on my_app.py when automated by the dogtail script which is launched by pytest as follows: coverage run --rcfile=.coveragerc -m pytest test_my_app.py. (The dogtail script automates UI actions.)

=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.12, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/Olumide/tmp/test_dogtail_coverage
plugins: bdd-6.1.1
collected 0 items  

• Get coverage report coverage report
  Outcome (no coverage on my_app.py:

Name             Stmts   Miss  Cover
------------------------------------
my_app.py           18     18     0%
test_my_app.py       7      0   100%
------------------------------------
TOTAL               25     18    28%

I don't know if I am doing something wrong, or this is a missing feature or a bug.

dogtail is based on subprocess so I've added concurrency = multiprocessing to the .coveragerc file.

---

Here are some packages/steps that may be needed in order to setup GTK, GTK Python and dogtail (extracted from our pipeline)

- apt-get update && apt-get install dbus-x11 libcairo2-dev libgirepository1.0-dev libgtk-3-dev at-spi2-core python3-pyatspi python3-dogtail gnome-icon-theme gsettings-desktop-schemas -y
- pip install pycairo PyGObject

- eval $(dbus-launch --sh-syntax)
- gsettings set org.gnome.desktop.interface toolkit-accessibility true
# Poor man's fix. (pyatspi and dogtail install to dist-packages on Ubuntu. )
- ln -s /usr/lib/python3/dist-packages/pyatspi /usr/local/lib/python3.10/site-packages/pyatspi
- ln -s /usr/lib/python3/dist-packages/dogtail /usr/local/lib/python3.10/site-packages/dogtail

Caveat: Run on dogtail scripts on X-Windows. dogtail does not run well on Wayland.

(I posted a version of this question on stackflow just over a month ago but without a working example. Hopefully a reproducible example will help make the problem clearer.)

[olumide-x]

@nedbat @marcgibbons Are you able to advise on this issue?

[devdanzin]

my_app.py 17 0 100%

This seems to indicate you have 100% coverage of my_app.py?

[olumide-x]

my_app.py 17 0 100%

This seems to indicate you have 100% coverage of my_app.py?

That's a typo. I've just rerun the test and edited the OP with the correct stats.

[olumide-x]

Problem solved.

I was running the test with sudo, i.e.

sudo xvfb-run coverage run --rcfile=.coveragerc -m pytest test_my_app.py

As such the environment variable COVERAGE_PROCESS_START was not preserved.

sudo -E xvfb-run coverage run --rcfile=.coveragerc -m pytest test_my_app.py

Fixes the problem.