Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix conf-openblas on macOS arm64 hardware, hopefully without breakages #25196

Merged
merged 3 commits into from
Feb 17, 2024

Conversation

dra27
Copy link
Member

@dra27 dra27 commented Feb 7, 2024

Re-attempt at #25076, following the needed revert of the second part in #25195.

First commit is the same as #25076 - it changes macOS only, introducing a missing dependency (the package already used pkg-config without depending on its conf- package) and amending the call to pkg-config for Homebrew only to match Homebrew's recommendation.

The second commit addresses the unintended breakage in #25194 (for which apologies again!). There are two issues at play: setenv doesn't support filters (ocaml/opam#5787) and empty updates to environment variables crash opam, which is fixed in 2.1.5, but not in earlier releases (ocaml/opam#5350).

The fix I propose is to put the setenv field in a package of its own, and have conf-openblas depend on that package only for macOS with Homebrew. Note that conf-openblas-macOS-env will only install if conf-openblas installed, which means that we know that the environment variable update will not be empty, hence there is no need to check for the opam-version itself.

@dra27 dra27 linked an issue Feb 7, 2024 that may be closed by this pull request
@dra27
Copy link
Member Author

dra27 commented Feb 7, 2024

(draft because I don't at present have access to my Mac to test it)

@mseri
Copy link
Member

mseri commented Feb 7, 2024

I like this suggestion more than requiring opam-version >= 2.1.5. But I think either way it would be fine.

@mseri
Copy link
Member

mseri commented Feb 7, 2024

Oh this lint was not there the last time right? If so, nice addition

2024-02-07 14:39.15: Error in conf-openblas-macOS-env.1:
warning 56: It is discouraged for non-compiler packages to use 'setenv:'

@hannesm
Copy link
Member

hannesm commented Feb 7, 2024

would you mind to increase the version of "conf-openblas" to 0.2.2?

@dra27
Copy link
Member Author

dra27 commented Feb 7, 2024

The lint issue was there last time - although it's at present removed in opam 2.2, because the issue which caused it be a warning has itself disappeared (cf. https://github.com/ocaml/opam/blob/5fa0ac762f4c8e6e2a45272adc6ef1fa4c6dcf28/src/state/opamFileTools.ml#L655-L661)

@dra27
Copy link
Member Author

dra27 commented Feb 7, 2024

For both changes, @hannesm, or just for the second env-related change?

If for both, presumably the 0.2.1 should be marked as unavailable on macOS arm64 with Homebrew?

@mseri
Copy link
Member

mseri commented Feb 7, 2024

I would just move packages/conf-openblas/conf-openblas.0.2.1/opam -> packages/conf-openblas/conf-openblas.0.2.2/opam and not touch the other. When you run opam upgrade you'll get the latest version anyway

@dra27
Copy link
Member Author

dra27 commented Feb 7, 2024

OK, I don't mind doing it (and will push in a sec!), but it's that bit which feels strange for using a version number bump. It's a packaging error (we're not trying to install an older version of the thing, we're preserving older incorrect packaging instructions for the same thing), but we end up leaving opam-repository where if for any reason a solver on macOS happens to try 0.2.1 (lowerbounds, etc.), then it'll fail.

pkg-config requires additional configuration after installing the
openblas package.
@dra27 dra27 force-pushed the macOS-conf-openblas branch from 6cf5424 to d02bc49 Compare February 7, 2024 16:55
This has to be done using a separate package since setenv cannot be
filtered and opam 2.0.10/2.1.4 and earlier cannot handle a setenv with
an empty string in the environment update.
@dra27 dra27 force-pushed the macOS-conf-openblas branch from d02bc49 to 7b242f7 Compare February 7, 2024 16:56
@mseri
Copy link
Member

mseri commented Feb 7, 2024

You can make it unavailable for macos on arm64 with homebrew if that is completely uninstallable, I thought it was working but required manually setting the correct PKG_CONFIG_PATH all the time you wanted to install something

@mseri
Copy link
Member

mseri commented Feb 7, 2024

As per the version number, I agree that at some point we should set on a more precise policy but bumping the number would follow what we did so far for the other conf- packages when changing the build steps so I think it is better to try to stay consistent

The package only works if the user has pre-configured the environment -
better to switch to 0.2.2 instead, which is hardened.
@dra27 dra27 marked this pull request as ready for review February 12, 2024 10:41
@dra27
Copy link
Member Author

dra27 commented Feb 12, 2024

With conf-openblas.0.2.1 with macOS Homebrew, PKG_CONFIG_PATH needs to be set both when running opam install and also when installing/building/running anything which uses the openblas package. The first part - having to set PKG_CONFIG_PATH in order to install conf-openblas - I think is a bug in the conf-openblas package (so I just pushed a disabling of conf-openblas.0.2.1 for macOS+Homebrew). The second part is more of a "smell" - the conf-openblas package is meant to configure the openblas package to be ready for use, but it hadn't (because the PKG_CONFIG_PATH is left unconfigured) forcing dependent packages to need to a special case for Homebrew. The setenv part is then what deals with that.

@dra27
Copy link
Member Author

dra27 commented Feb 12, 2024

I've now tested and verified this on both Intel and ARM Mac hardware

@mseri mseri merged commit 8b5a0a3 into ocaml:master Feb 17, 2024
1 of 2 checks passed
@mseri
Copy link
Member

mseri commented Feb 17, 2024

Thanks

@dra27 dra27 deleted the macOS-conf-openblas branch February 17, 2024 21:49
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Installing conf-openblas 0.2.1 needs PKG_CONFIG_PATH extended.
3 participants