Issues with automake based opam packages on OpenSuse cause of default site configure file

[MSoegtropIMC]

OpenSuse 15.2. Leap has a default site configure script, that is it sets:

CONFIG_SITE=/usr/share/site/x86_64-unknown-linux-gnu

The referenced script sets among other things the libdir environment variable to a global system path unless it is already set. automake generated configure scripts always read a file pointed to by CONFIG_SITE and this way get a libdir variable which is not what it should be under opam.

This might usually not be problematic because many opam packages set the prefix argument to configure scripts, which takes precednece over libdir. A notable exception are Coq opam files since they usually use coqc -where as prefix (rather than ocamlc -where) and this default is sometimes embedded in the configure script rather than the opam file. In that case the system supplied libdir path might take precedence.

Anyway, I think opam should avoid that configure scripts get passed system libdir paths in any way. So I think opam should unset CONFIG_SITE. The alternative of setting libdir would not be favourable for Coq because, as mentioned, it needs a different libdir setting than ocamlc -where, so this would break many Coq opam packages.

Possibly this view is a bit Coq centric, so I am not sure one can say that this is an opam issue. There might be other possible solutions, but unsetting CONFIG_SITE would be (likely) the most favorable one for Coq.

I am not sure if any depext packages would need CONFIG_SITE to be set.

In the opam based Coq platform build scripts I unset CONFIG_SITE and this fixes the issues on OpenSuse.

[dra27]

I think this affects opam-repository, rather than opam itself? i.e. we want to change packages rather than opam? If you agree, I'll do a GH transfer of the issue.

[MSoegtropIMC]

I think this affects opam-repository, rather than opam itself?

No, the idea is that opam should sanitize the environment when executing package configure / build code. Opam has features like a sandbox but then passes an environment to build code which leads to paths outside of the sandbox.

[dra27]

Hmm, I fear one person's feature may be another person's poison! opam's sandbox is designed to protect your system, not control how the build runs. Once opam mandates doing this, what happens if you have a package which wants that to work?

[MSoegtropIMC]

Yes, I am also afraid that this is not so easy to solve. I opened this issue mostly to discuss the options and also to make you aware of the situation that packages might fail to build on some platforms because they have CONFIG_SITE set.

One option to avoid users running into this would be to test all packages that they don't suffer from this and work around the issue for packages which do.

Another option would be to have flags in the opam file what level of environment sanitization the package expects. E.g. one could reduce the environment passed through to the package build to the bare minimum but have a per package and/or per site opam option to pass through additional variables. But I am not sure if this leads to a better user experience since it makes things more complicated. Maybe in the end testing and working around the problem is the best option from a users perspective.

Btw.: I did extensive tests with CONFIG_SITE unset on various platforms (Ubuntu, Fedora, OpenSuse, CentOS, Windows (cygwin + MinGW), MacOS) and a medium size set of opam packages (~ 150) and didn't see issues. So requiring this doesn't seem to be common at least.

[dra27]

For a specific environment variable, it would be possible to depend on a "scrubbing" package, yes - the only nuisance is that once installed that scrubs it for everything. There would be potential for a list of environment variables to be scrubbed in an opam file - although new features of course are slower to depend on. You could invoke configure as sh -c CONFIG_SITE="" ./configure ... - but that's not without its own risks (although it can be safer to have a wrapper script of your own in the package and call that from the opam file - there are various packages in opam-repository with a build.sh and/or install.sh)

[MSoegtropIMC]

I think we still don't fully understand each other. What I propose is this:

• Opam generally limits the environment variables passed on to package build/install processes to a well defined and documented set of variables. This applies to all packages on all platforms and would be part of the specification of opam. Environment variables not in the positive list are unset.

• If a specific package feels it needs some additional variables this can be enabled with an option in the opam file of the package, which also serves as documentation about this.

• Also if a specific platform or switch, say a cross compilation switch, needs to pass on specific environment settings to all packages, this can be done by a switch / platform level configuration.

My assumption is that this enhances the stability / portability of the build process because it reduces the amount of uncontrolled influence. I think it might be worthwhile to try it and see how many packages break in CI.

I understand that there are various ways to get around the issues in case it is decided against this.

[dra27]

We mused on this again yesterday:

• In general, environment variable scrubbing isn't likely to happen soon: opam 2.1+ scrub all OPAM* variables introduced since 2.1.0, but that's as far as it goes. Any change to that would likely require opt-in from packages, which limits its usefulness.
• At present, individual packages can elect to scrub variables using the build-env field and an entire switch could elect to scrub a variable by installing a package with a set-env for that variable as part of the switch base/invariant. It could be worth adding a list of globally scrubbed build variables to .opam/config.
• opam has no operator for unsetting an environment variable, and it would be useful in this context to add one!

[MSoegtropIMC]

Thanks for the update! In case it is not agreeable to do this by default, I would very much appreciate a way to configure this for a specific switch, as you suggest. Of course I can also handle this at the Coq Platform script level, but I would like that people can call opam to install additional packages without running an environment sanitizing script upfront. As mentioned, I would prefer a positive list with exceptions.