CI: Add job for pip check

[cdce8p]

Proposed change

This PR adds a new CI job which runs pip check if any requirements change.
I had the idea after @MartinHjelmare mentioned a dependency conflict in #64045 (review) which I hadn't seen at the time.

Due to existing dependency conflicts, this job will always fail at the moment. The output should nevertheless help not to add additional issues. The job is only run for PRs which include requirement changes. It doesn't make much sense anywhere else. Especially not with it failing by default.

Edit: I've reworked the logic to only fail if a new dependency conflict is added. More details here: #64058 (comment)

This could also be helpful for #59769

/CC: @MartinHjelmare @frenck

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[MartinHjelmare]

I'm hesitant about adding a job that we know will fail, but since it will only run for requirement bumps, I think it will still provide useful information.

I'd like to hear what Franck and others think.

[cdce8p]

I'm hesitant about adding a job that we know will fail, [...]

For now. At least my idea was that we work towards fixing the existing problems, since this will be necessary anyway for the new pip resolver in #59769. Over the last few days, I've already addressed a couple of them with one currently pending #64045.

AFAIK there are a few options we have to resolve them, some better some worse.

1. Ask the library maintainers to bump their supported dependency range. If possible, I believe this is by far the best one. For example, this is something for the websockets and async-timeout issues. The new versions are mostly compatible, so it's easy enough to support them.
2. If it isn't possible, we can check if there exists a version which is compatible with every dependency. Once found, it can be pinned as constraint range. That's something I've done for python-socketio in Pin python-engineio and python-socketio #64074.
3. In case nothing else works or a package is no longer maintained, it might unfortunately be necessary to remove the integration.
4. Not currently possible: Pip has a --no-deps install option. If we know a package is compatible but has conflicting dependencies and is just unmaintained, we could install the dependencies separately and install the package "without" dependencies. That would also work with the new resolver.
   At the moment, I'm not aware that we support this in Home Assistant though.

--
I'll just include my preliminary working list below for the remaining issues. Feel free to have a look for yourself.

Conflicting dependencies

Open PRs

• surepy 0.7.2 has requirement async-timeout<4.0.0,>=3.0.1, but you have async-timeout 4.0.2.
  Allow async-timeout 4.x too benleb/surepy#103

Waiting for release

• python-songpal 0.12 has requirement click<8,>=7, but you have click 8.0.3.
  Switch to click 8 rytilahti/python-songpal#91

Open Issues

• ibm-watson 5.2.2 has requirement websocket-client==1.1.0, but you have websocket-client 1.2.3.
  Pinned down websocket-client==1.1.0 watson-developer-cloud/python-sdk#810

Discontinued

• mycroftapi 2.0 has requirement websocket-client==0.44.0, but you have websocket-client 1.2.3.

Requires HA dependency update

• google-cloud-texttospeech 0.4.0 has requirement google-api-core[grpc]<2.0.0dev,>=1.6.0, but you have google-api-core 2.4.0.
  https://github.com/googleapis/python-texttospeech/blob/v2.9.0/CHANGELOG.md
  https://github.com/googleapis/python-texttospeech/blob/main/UPGRADING.md#200-migration-guide
  Version range to update google-cloud-texttospeech to 2.6.0 - 2.9.0

Long wait / possibly unmaintained

• pysmarty 0.8 has requirement pymodbus==1.5.2, but you have pymodbus 2.5.3.
  Requirement is pinned for pysmarty 0.8: pymodbus==1.5.2 z0mbieprocess/pysmarty#1
• pyicloud 0.10.2 has requirement click<=7.1.1,>=6.0, but you have click 8.0.3.
  pyicloud 0.10.2 has requirement tzlocal==2.0.0, but you have tzlocal 4.1.

Pending dependency update

• volvooncall 0.9.1 has requirement websockets<=8.1, but you have websockets 10.1.
  Migrate away from hbmqtt molobrakos/volvooncall#78
  Upgrade volvooncall to 0.9.2 #64942
  --

Current conflicts

volvooncall 0.9.1 has requirement websockets<=8.1, but you have websockets 10.1.
surepy 0.7.2 has requirement async-timeout<4.0.0,>=3.0.1, but you have async-timeout 4.0.2.
python-songpal 0.12 has requirement click<8,>=7, but you have click 8.0.3.
pysmarty 0.8 has requirement pymodbus==1.5.2, but you have pymodbus 2.5.3.
pyicloud 0.10.2 has requirement click<=7.1.1,>=6.0, but you have click 8.0.3.
pyicloud 0.10.2 has requirement tzlocal==2.0.0, but you have tzlocal 4.1.
mycroftapi 2.0 has requirement websocket-client==0.44.0, but you have websocket-client 1.2.3.
ibm-watson 5.2.2 has requirement websocket-client==1.1.0, but you have websocket-client 1.2.3.
google-cloud-texttospeech 0.4.0 has requirement google-api-core[grpc]<2.0.0dev,>=1.6.0, but you have google-api-core 2.5.0.

[cdce8p]

@MartinHjelmare I though about what you said about an always failing job a bit more. Maybe we could get away with comparing the result to a predefined constant / environment variable. I.e. we expect 15 conflicts, fail only if there are more than 15. That would also allow us to run it on changes to the dev, rc, and master branch since it is expected to succeed now.

There is a small issue about what to do if a conflict gets resolved. Ideally, and what I've implemented in the last commit, the expected conflicts variable should be updated. Otherwise it would be possible to introduce a new conflict in another PR. To notice this case, I've chosen to emit an error for it too.

Let me know what you think.

[MartinHjelmare]

Sounds good!

One limitation with pip check that we should be aware of, is that a new conflict won't necessarily be caused by the updated requirement. It could instead be due to an existing requirement that has a bad pinning strategy and the updated requirement just crossed that bad pinning limit.

[cdce8p]

One limitation with pip check that we should be aware of, is that a new conflict won't necessarily be caused by the updated requirement. It could instead be due to an existing requirement that has a bad pinning strategy and the updated requirement just crossed that bad pinning limit.

Good point! I agree. My hope is that pip check will at least help a bit in detecting and resolving those. Probably necessary to test it with some real PRs to know for sure. Feel free to tag me if you do encounter an issue. Maybe I can help with that.

[frenck]

I think this might work for now 👍

[cdce8p]

I would like to merge it now. We can always come back to it later for fine-tuning after we got some more experience with it on a day by day basis.