GHC's -package <pkg> can expose installed package not listed by ghc-pkg list <pkg>

[juhp]

EDIT (by @mpilgrem): The anomaly described below is likely due to the bug (from Stack's perspective) in GHC versions described at: https://gitlab.haskell.org/ghc/ghc/-/issues/25025

In short:

• if an installed package has a name that is in the format that Cabal (the library) uses to 'mung' sub-libraries, some versions of GHC treat that name as if it were the name of the Cabal package for the purpose's of GHC's -package option; and
• as a consequence, GHC's -package <pkg> option can, in practice, cause an installed package to be exposed that is not one listed by ghc-pkg list <pkg>.

Stack, on the other hand, relies on an installed package having a unique name specified by its name field and that GHC's -package <pkg> option will expose only a installed package listed by ghc-pkg list <pkg>.

---

So far noone else has seen this: so likely it is an anomaly only in my system.

General summary/comments

After updating my projects to latest lts-22 (lts >= 22.42), builds using attoparsec started to fail for me.

Steps to reproduce

(Not reproducible so far except on my laptop)

stack --resolver lts-22.43 build aeson

Expected

To build normally.

Actual

$ stack --resolver lts-22.43 build aeson
:
aeson                        > configure
aeson                        > Configuring aeson-2.1.2.1...
aeson                        > Error: Cabal-simple_LRiJ_wrZ_3.10.3.0_ghc-9.6.6: Encountered missing or
aeson                        > private dependencies:
aeson                        > attoparsec ==0.14.4
aeson                        > 
Completed 13 action(s).

Error: [S-7282]
       Stack failed to execute the build plan.
       
       While executing the build plan, Stack encountered the error:
       
       [S-7011]
       While building package aeson-2.1.2.1 (scroll up to its section to see the error) using:
       /var/home/petersen/.stack/setup-exe-cache/x86_64-linux-tinfo6/Cabal-simple_LRiJ_wrZ_3.10.3.0_ghc-9.6.6 --verbose=1 --builddir=.stack-work/dist/x86_64-linux-tinfo6/ghc-9.6.6 configure --with-ghc=/var/home/petersen/.stack/programs/x86_64-linux/ghc-tinfo6-9.6.6/bin/ghc --with-ghc-pkg=/usr/bin/ghc-pkg-9.6.6 --user --package-db=clear --package-db=global --package-db=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/pkgdb --libdir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/lib --bindir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/bin --datadir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/share --libexecdir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/libexec --sysconfdir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/etc --docdir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/doc/aeson-2.1.2.1 --htmldir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/doc/aeson-2.1.2.1 --haddockdir=/var/home/petersen/.stack/snapshots/x86_64-linux-tinfo6/18cf10429d6b1427d08f2a7a920a4c784cc1f0eaa6921458e36e04319528d1da/9.6.6/doc/aeson-2.1.2.1 --dependency=OneTuple=OneTuple-0.4.2-7od06fXUUXRBNk1V3VY0g5 --dependency=QuickCheck=QuickCheck-2.14.3-E9VttT8lnId93rGQPNsiU9 --dependency=attoparsec=attoparsec-0.14.4-GAKm7J8Gleq2k06HFjeIeY-attoparsec-internal --dependency=base=base-4.18.2.1 --dependency=base-compat-batteries=base-compat-batteries-0.13.1-5fbdeENWe9mF7TcjAFq4dk --dependency=bytestring=bytestring-0.11.5.3 --dependency=containers=containers-0.6.7 --dependency=data-fix=data-fix-0.3.4-2n4PElhIt1Q7w0n82lPfpl --dependency=deepseq=deepseq-1.4.8.1 --dependency=dlist=dlist-1.0-7vDlnn0Hdvg35SyXLwMaWr --dependency=exceptions=exceptions-0.10.7 --dependency=generically=generically-0.1.1-I9byc5Nil798plofO827gA --dependency=ghc-prim=ghc-prim-0.10.0 --dependency=hashable=hashable-1.4.4.0-GGHvrv8RGdcAwi4fOW9L5d --dependency=indexed-traversable=indexed-traversable-0.1.4-8j5HZpShpE5BqFup9Ojenr --dependency=primitive=primitive-0.8.0.0-G7z1XrhwN0bFkYsIqIr1QU --dependency=scientific=scientific-0.3.7.0-BAvZdJlkHDUEf0LOVEPz37 --dependency=semialign=semialign-1.3.1-7lhN9UdvYxAGbqZZCyRcNC --dependency=strict=strict-0.5-GMywuJToqFcduYfo019sj --dependency=tagged=tagged-0.8.8-Kzng2lnKElzJiyKd9g735c --dependency=template-haskell=template-haskell-2.20.0.0 --dependency=text=text-2.0.2 --dependency=text-short=text-short-0.1.6-2ZxoUm7jRrSBd8GMEi1Fas --dependency=th-abstraction=th-abstraction-0.5.0.0-HAFjiAO2nGN58SdxVZCnLH --dependency=these=these-1.2.1-KST5KzzcktxI0GEHGKxqQO --dependency=time=time-1.12.2 --dependency=time-compat=time-compat-1.9.6.1-DIvC8GbQSzcCziVynZc4vt --dependency=unordered-containers=unordered-containers-0.2.20-5m928IsagguARIfUwwmGLp --dependency=uuid-types=uuid-types-1.0.5.1-3QSWhjKdWiE4JA9TKvvlMc --dependency=vector=vector-0.13.1.0-Jdel1KiNlSEIXGg2MpN3IL --dependency=witherable=witherable-0.4.2-D4tJt8ldIw17tHtCEGjK6w --dependency=attoparsec:attoparsec-internal=attoparsec-0.14.4-GAKm7J8Gleq2k06HFjeIeY-attoparsec-internal -f-cffi -fordered-keymap --exact-configuration --ghc-option=-fhide-source-paths
       Process exited with code: ExitFailure 1

See https://gist.github.com/juhp/8cf70c3e00509347ff5e5f4bbfad74c5 for the full --verbose output

Highlighting a few points:

• - attoparsec-0.14.4 appears twice (probably once for attoparsec and once for attoparsec-internal?)
• for attoparsec there is only --dependency=attoparsec=attoparsec-0.14.4-GAKm7J8Gleq2k06HFjeIeY-attoparsec-internal (attoparsec:attoparsec-internal gets mapped to the same pkgid)

Stack version

2.15.7 and 3.1.1

Method of installation

Fedora Linux and local build

Platform

Fedora Linux

Misc

I am using a workaround: adding os-string-2.0.6 to my project stack.yaml files (the version of os-string in lts < 22.42)

[mpilgrem]

Looking at the gist, one thing that is odd is that the unofficial version of Stack 2.15.7 that you are using itself lists attoparsec-0.14.4 twice as a dependency that Stack has been compiled against. It seems to me possible that something may have corrupted the local write-only (snapshot) database of immutable packages.

EDIT: Actually, that is a red-herring. I see that where a package has a sub-library (for example, pantry or tar), its version gets listed twice by stack --version.

[juhp]

I tried with both --system-ghc and also removing (renaming) ~/.stack/snapshots/ and can still reproduce here.
I guess I can try to rename "harder" (like all ~/.stack)... (though looks to me that these caused everything to get rebuilt?)

Also just noting here too (as I mentioned in matrix) that I couldn't reproduce in a fresh Fedora container.

[juhp]

I uploaded the stack-3.1.1 output too for completeness: https://gist.github.com/juhp/e4f92e6df99524da0d7df72afe2470d4

[juhp]

Also I also tried in a container first building with lts-22.41 and then lts-22.43, but this also worked fine.

[mpilgrem]

I am wondering if this GHC bug is relevant:

• https://gitlab.haskell.org/ghc/ghc/-/issues/25025
• https://gitlab.haskell.org/ghc/ghc/-/issues/25035

[juhp]

Ah yes indeed - I had quite forgotten about that problem 😢
Bullseye I guess!

Is there a related stack issue?

[mpilgrem]

@juhp, I have edited this issue to make it Stack's related issue.

[juhp]

BTW I finally "took the (small) plunge" and tried moving (aka "deleting") ~/.stack/ out of the way and the problem still occurs for me, so I don't know any solution other than avoiding ghc-9.6.6 (other than my workaround ;o) 😢

So wonder what triggers this on my machine, since it appears not reproducible, or is it just bad luck?

---

Guess I forgot to write down my workaround, just in case anyway one should hit this somehow, in stack.yaml:

extra-deps:
  - os-string-2.0.6