Use a hard-coded tool dependency map (fixes #4125)

[snoyberg]

This is a fundamental change to how Stack discovers build tools, its inclusion should be discussed before we merge. We should also wait to see if it actually fixes #4125.

Note: Documentation fixes for https://docs.haskellstack.org/en/stable/ should target the "stable" branch, not master.

Please include the following checklist in your PR:

• Any changes that could be relevant to users have been recorded in the ChangeLog.md
• The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

[snoyberg]

Some design decision time. Stack originally (and through the 1.7 release) calculated build tools by looking at all of the packages in a snapshot, finding the executables they provide, and performing a reverse lookup for any build-tools values. This works, but:

• Requires eagerly parsing all cabal files in a snapshot, something we may want to try to avoid
• Has had some bugs in the past with overridden packages, e.g. stack 1.7.1 still has problems finding build tools ("The program 'happy' is required") #4125
• Is out of sync with cabal (see next paragraph)

Starting with Cabal 2.0 (or 2.2), there's a new feature in the cabal file format called build-tool-depends, which allows you to specify which package a tool comes from, bypassing the need for the reverse lookup. cabal-install handles the legacy build-tools field with a hard-coded list of packages like alex and happy. This PR does the same for Stack, though with a slightly difference list.

This may cause some existing builds to break, but that's highly doubtful, and can be worked around easily enough by switching to build-tool-depends and/or explicitly building the needed dependencies. I lean in favor of merging this because matching cabal behavior is a Good Thing, pushing towards the new build-tool-depends and more explicit deps is a Good Thing, and reducing complexity in our code base is a Good Thing 😄 .

I'd say let's give it another few days for someone to object, and to confirm it fixes #4125, and then we can merge.

[Ericson2314]

N.B. I made haskell/cabal#5412 based on the issues I had making build-tool-depends (and build-tools) the sole way to get access to binaries with Nixpkgs alone. I'm curious what Stack's experience with packages that, e.g. do build-depends: hspec-discovery and expect to get an executable in addition to a library.

[snoyberg]

I'm not quite sure of the question, but I'll give an answer related to other bugs I've seen. I think it's a mistake for Cabal to hide the hspec-discovery executable when build-depends: hspec-discovery is specified. It breaks compatibility with how cabal worked in the past, and AFAICT doesn't add any real determinism to the process, since we've already specified that hspec-discovery must be installed before the package in question. Stack does not attempt to do any of this hiding, and I would strongly advocate against adding in such a restriction.

[snoyberg]

As an update: this does in fact seem to solve #4125.

[simonmichael]

This sounds good to me. I have an addition for the list of tools:

• hsx2hs, provided by http://hackage.haskell.org/package/hsx2hs. Users include the http://hackage.haskell.org/package/darcsden package. (It is not mentioned in darcsden's cabal file, but should be.)

If a tool is not in the hard-coded list, does/should the install fail, or fall back to the old behaviour ?

[snoyberg]

I'll add that, no problem. If it's not in the hard-coded list, and the executable isn't already available with that name, Stack will print a warning, and the build itself will most likely fail. Falling back to the old behavior would require all of the complicated code I mentioned which is suffering from the bugs right now and preventing performance optimizations.

[Ericson2314]

@snoyberg [BTW, happy to talk about this elsewhere as I'm definitely veering off-topic.] Being able to only build the libraries components of build-depends is, I agree a small benefit. The real benefit is cross compilation: the tools must be built for the build platform, but libraries are built for the host platform. If everything is built together then either the libraries are for the wrong platform, or the tools won't run. If every build-depends package has its binaries built for the build platform, then we've doubled the build time for cross as a great number of packages are conservatively being built twice.

[snoyberg]

OK, that motivation makes a lot more sense, I hadn't heard it before. I still think it's best to leave Stack as-is for maximum backwards compatibility, I know in particular the hspec-discovery thing has caused breakage with new-build.