Various Rule Failed, shown as "Type constructor or class is not in scope (GHC-76037)" — yet the project compiles

[ulidtko]

Hi! Here's a weird issue, as a treat for the connoisseurs 🤌

On a project that's using TH splices of the persistent library, I'm reliably getting an extra stubborn "typecheck error" which is not real:

The names flagged as errors, RoleID, UserId, are defined in a TH splice, $(persistFileWith ...) above in the same module. Full error as text:

Not in scope: type constructor or class ‘UserId’ typecheck(GHC-76037)

I'm saying it is "not real" because:

1. stack build compiles just fine, GHC itself is happy with the module. Making it hard to believe the displayed error being a genuine typecheck error coming from GHC.
2. Furthermore, haskell-language-server-wrapper succeeds when I invoke it directly on this module, Completed (1 file worked, 0 files failed).

As a triple-check, I can load this module in this project into Neovim equipped with an LSP client — and the module typechecks just fine as well, no fake not in scope errors whatsoever:

Thus as shown, HLS itself seems alright — therefore I'm filing this onto the VSCode extension. ⚠

These aren't making much sense to me, but the extension logs a good bunch of Rule Failed messages:

and 3 Rule Failed exceptions in devtools console as well:

At the very least, these Rule Failed exceptions should not be translating into GHC-76037 a.k.a. NotInScope diagnostics. I hope to be wrong here, but doing so is a big compromise to trustworthiness of the UI. 💀

Then as a secondary matter, the question remains, what's the cause of those exceptions in the first place.

Environment

OS: Ubuntu 24.04.2 LTS.

GHC, Stack, HLS — all from GHCup. I found multiple executables called haskell-language-server-wrapper on this system, but also am reasonably certain that the only one that ever gets invoked is the ~/.ghcup/bin/haskell-language-server-wrapper symlink.

HLS version: just rechecked on 2.10.0.0-p1. This has also been happening on 2.9.0.1 with exact same symptoms.

GHC version: repro'd on 9.6.6 and today on 9.6.7.

IDE: VSCodium today's latest 1.99.32704.

Extension: vscode-haskell 2.6.0.

Steps to reproduce

Edit: Not quite minimal, but an isolated shareable repro case worked out — https://github.com/ulidtko/hls-issue4583-repro

I only see this issue in one specific project; minimizing a repro case seems a bit daunting. Didn't try yet.

Would appreciate further hints on what to try in this direction — perhaps it's an already known issue that I just couldn't find.

Workarounds

1. Don't use VSCode 🤷 Works okay in NeoVIM for me.
2. Within VSCode... none known so far.

From a few months of observation: the error does occasionally go away — when something, somehow, gets into caches. I couldn't spot what & in which cache exactly. Places I'm wiping clean now for experiment precision:

• rm -rf ~/.cache/ghcide, with HLS stopped, of course;
• git clean -fxd in the project dir, which blows out the dust in .stack-work, dist-newstyle, etc.
• the readme mentions that the extension has its own cache, in my case that's ~/.config/VSCodium/User/globalStorage/haskell.haskell/ — and on my machine, it's there but always empty.

Debug logs

haskell-language-server-wrapper --debug — done, it produces a whole bunch of information I don't want to review & redact. It ends with Completed (1 file worked, 0 files failed) — i.e. there're no typecheck errors.

Re: extension specific log, already shown on the screenshots. There're pages upon pages, among which a lot of Rule Failed errors like this one:

Extension log sample:

[Trace - 2:25:14 PM] Sending request 'textDocument/semanticTokens/full - (60)'.
2025-05-06T12:25:14.525852Z | Debug | semanticTokens: SemanticTokensConfig_: STC {stFunction = SemanticTokenTypes_Function, stVariable = SemanticTokenTypes_Variable, stDataConstructor = SemanticTokenTypes_EnumMember, stTypeVariable = SemanticTokenTypes_TypeParameter, stClassMethod = SemanticTokenTypes_Method, stPatternSynonym = SemanticTokenTypes_Macro, stTypeConstructor = SemanticTokenTypes_Enum, stClass = SemanticTokenTypes_Class, stTypeSynonym = SemanticTokenTypes_Type, stTypeFamily = SemanticTokenTypes_Interface, stRecordField = SemanticTokenTypes_Property, stModule = SemanticTokenTypes_Namespace, stOperator = SemanticTokenTypes_Operator}
2025-05-06T12:25:14.526202Z | Warning | semanticTokens: SemanticTokens' dependency error: Rule Failed: GetHieAst
2025-05-06T12:25:14.526442Z | Debug | LOOKUP PERSISTENT FOR: GetSemanticTokens
2025-05-06T12:25:14.526628Z | Debug | Finished: SemanticTokens.semanticTokensFull Took: 0.00s
2025-05-06T12:25:14.526788Z | Debug | semanticTokens: Rule Failed: GetSemanticTokens
[Trace - 2:25:14 PM] Received response 'textDocument/semanticTokens/full - (60)' in 5ms. Request failed: semanticTokens: Rule Failed: GetSemanticTokens (-32803).
[Error - 2:25:14 PM] Request textDocument/semanticTokens/full failed.
  Message: semanticTokens: Rule Failed: GetSemanticTokens
  Code: -32803 
[Trace - 2:25:27 PM] Sending request 'textDocument/foldingRange - (61)'.
2025-05-06T12:25:27.492652Z | Debug | Finished: FoldingRange Took: 0.00s
2025-05-06T12:25:27.492763Z | Debug | codeRange: Rule Failed: GetCodeRange
[Trace - 2:25:27 PM] Received response 'textDocument/foldingRange - (61)' in 2ms. Request failed: codeRange: Rule Failed: GetCodeRange (-32803).
[Error - 2:25:27 PM] Request textDocument/foldingRange failed.
  Message: codeRange: Rule Failed: GetCodeRange
  Code: -32803 
2025-05-06T12:25:49.092374Z | Info | Live bytes: 525.30MB Heap size: 2218.79MB

Kindly LMK what else to check.

[fendor]

Hi thanks for the bug report!
This looks like an unusual issue :)

Even though you can only reproduce this issue with VSCode, it is pretty much impossible to be caused by the extension, iirc, we are not even installing an onDiagnostic callback.
Also, all the Rules failed are from HLS and not the extension, so it is even more unlikely to be an issue in the extension.
So, I will move it to the main repo to give it more exposure.

While I am not aware of any active bug reports, I do recall something similar happening in the past...

What I can imagine is that stack and HLS get confused when TH is involved, perhaps you can try to compile your project using cabal (and use cabal with HLS) and report whether that works? If it has the same issue, then we have to keep looking.

[ulidtko]

OK, thanks for acknowledgement @fendor 🙏

and use cabal with HLS

Is this doable with just switching the hie.yaml cradle to cabal?... I imagine, an amount of cabal.project writing will be needed to build this thing with cabal(-install) instead of stack. As it uses lots of stackage, and some vendored deps too.

Also, all the Rules failed are from HLS and not the extension

I think I understand that, yes 🤔 But how doesn't running haskell-language-server-wrapper perform a complete simulation of typecheck? Seems it does. Sure thing, the extension does extras beyong typecheck — any easy way to trace & simulate them?

[fendor]

Is this doable with just switching the hie.yaml cradle to cabal?

No, I am afraid, you'd have to write that cabal.project file 🙈

But how doesn't running haskell-language-server-wrapper perform a complete simulation of typecheck?

It looks like the rule that is failing is GetHieAst, not necessarily TypeCheck. The cli tool only reports an error if TypeCheck fails... You could modify HLS in ghcide/src/Development/IDE/Main.hs:413 to not ignore the error.

Could you share the logs of your cli run? Maybe it includes more error messages.

[ulidtko]

Hmmm 🤔 Well OK, some findings.

One, I just tried copying the project to /tmp/acme (to tidy the path in hls logs a little) — and what do you know, it didn't repro there!

In HLS log of the extension, there was clearly re-typechecking of the problematic module happening (with e.g. collecting record-fields off the TH output) — and vscode showed 0 issues, too.

@fendor is there some other cache I'm missing to clear?

Another: my mistake was, doing haskell-language-server --debug src/Module/With/Issue.hs — instead of the full-project haskell-language-server --debug .

This latter call:

1. did succeed once in my fresh project-dir /tmp/acme and extension succeeded there too;
2. does fail on second and further attempts, emitting the same diagnostics;
   • debug log: hls-th-caching-notinscope-errors.log.gz, module Model.Security is the one to grep
   • even on cold cache, after I do rm -rf ~/.cache/ghcide and immediately retry — keeps emitting the errors, hls-th-caching-cold-cache-notinscope-errors.log.gz

So indeed, it's a purely HLS issue, extension isn't relevant. You were right in transferring it.

[fendor]

Interesting, is anything special about the directory structure? E.g., spaces or unicode characters in the filepath?

is there some other cache I'm missing to clear?

No, I think you got them all. ~/.cache/hie-bios and ~/.cache/ghcide.

does fail on second and further attempts, emitting the same diagnostics;

Ok, that still sounds like we are doing something wrong with our handling of TH... cc @wz1000, might be interesting for you.

In your example, are RoleId and UserId generated via TH? Could you share a minified, or censored version of your module? Perhaps, we can reproduce the issue by starting from this module, and then adding TH definitions.

Small aside, is this project a mono repo? In the logs, it looks like there are around 6 executables, but no libs, does that sound possible? Also, are you using an hie.yaml? If no, could generate one with https://hackage.haskell.org/package/implicit-hie and share what the hie.yaml would look like?

[ulidtko]

spaces or unicode characters in the filepath?

Nope, none of that, all printable ascii. The "weirdest" thing I can think of here, is that the persistent model file is without extension; meaning /tmp/acme/src/Model/Security.hs loads, within a TH splice, the model file /tmp/acme/src-config/model/security.

Also... I'm not entirely sure, but I think I noticed this issue started roughly at the same time when a teammate had changed

import Database.Persist.Quasi (lowerCaseSettings)
import Database.Persist.TH (mkPersist, persistFileWith, share, sqlSettings)

share [mkPersist sqlSettings]
  $(persistFileWith lowerCaseSettings (modelsDir <> "/security"))

into

import Database.Persist.Quasi (lowerCaseSettings)
import Database.Persist.TH (discoverEntities, mkPersistWith, persistFileWith, share, sqlSettings)

share [mkPersistWith sqlSettings $(discoverEntities)]
  $(persistFileWith lowerCaseSettings (modelsDir <> "/security"))

to fix another thing. This discoverEntities splice does do things to the module scope — so... might as well be relevant.

In your example, are RoleId and UserId generated via TH?

Yes. Those are Persistent's EntityKey's for entities Role, User. IIRC, they're data-family instances... but definitely defined in a TH quotation.

Could you share a minified, or censored version of your module?

Potentially yes... but not easy. Hacking off 108 kLoC of haskell code won't be so easy I'm afraid 😔 Not enthusiastic of starting that. I might better try to work out a repro case from scratch (from a template), if I'm lucky something shareable might come out.

Small aside, is this project a mono repo?

Ahhh... no + yes? 😖 In terms of packages, the stack.yaml says packages: [.] i.e. it's a single-package project. The cabal-file (package.yaml) does define 3 executables, 2 test-suites, and 1 library (the usual, unnamed library component).

Also, are you using an hie.yaml?

Yes, a hand-written one.

Output of implicit-hie

cradle:
  stack:
    - path: "./src"
      component: "zdocs:lib"

    - path: "./apps/app-config-check/Main.hs"
      component: "zdocs:exe:zd-config-check"

    - path: "./apps/app-config-check/HelperTypes.hs"
      component: "zdocs:exe:zd-config-check"

    - path: "./apps/zd-lo-migrate/Main.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/App.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/Command/CountOnly.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/Command/ToByteA.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/Command/ToLargeObject.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/DB.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zd-lo-migrate/Orphans.hs"
      component: "zdocs:exe:zd-lo-migrate"

    - path: "./apps/zDocs/Main.hs"
      component: "zdocs:exe:zdocs"

    - path: "./tests/fast"
      component: "zdocs:test:testsuite-fast"

    - path: "./tests/integration"
      component: "zdocs:test:testsuite-integration"

— misses a few details, and gets a few pieces wrong too; hence hie.yaml hand-written

[fendor]

Unfortunately, I don't have any obvious ideas what might be going wrong. The smaller reproducer would probably help, or bisecting the changes on your project might also help us out.

[ulidtko]

Aye, I minimized a standalone repro 🕺

@fendor @wz1000 https://github.com/ulidtko/hls-issue4583-repro

src/Model.hs acts up, just the same as in my project here. Something funny with TH in that file 🤔
LMK if that's workable for you

[fendor]

Thanks, I can reproduce the issue!

[fendor]

It seems like this issue is stack specific, at least with cabal, I can't seem to reproduce it.
Moreover, it seems related to multiple home units, when I only open src/Model.hs, then nothing crashes. If I additionally open test/TestImport.hs, I start to see the error.
If I only open test/TestImport.hs, then I don't see any error.

Could you confirm this behaviour as well? Opening src/Model on its own doesn't have the error, but once you open test/TestImport.hs it does?
It seems like it needs to be a test component, opening any of the executables, doesn't show the error.

[ulidtko]

Yep, indeed!

Need to have a parallel buffer with a test component module open. Then it errors.

[fendor]

We did not forget this issue, I talked to @wz1000, and we conjecture this issue is due to HLS's somewhat broken support for stack projects with multiple home units.
If that's truly the case, then there is currently not much we can do, as we need to improve HLS and stack interaction. See the tracking ticket #6154 where we are making progress towards a better future.