Fix testdata hie.yaml generation

[jneira]

• From #399

But now I'm stumped as to how the tests are working in the first place even

@bubba @fendor @isovector me too, analyzing the test code and data:

• Most tests are being ignored so although there is no hie.yaml, they pass
• Working tests (eval, tactic, etc) have a hie.yaml in its subdirectory, despite they were intended to be ignored with

haskell-language-server/.gitignore

Lines 23 to 24 in 438f699

	# ignore hie.yaml's for testdata
	test/testdata/**/hie.yaml

I suppose they were added with git add --force. 😺

• Moreover the hie.yaml generated by setupBuildToolFiles will not work anymore, as they should include modules explicitly

So, i would ignore hie.yaml's but:

• setupBuildToolFiles has to include auto all modules in the directory where it is about to be created (not sure if it would be feasible)
• specific tests should call a version of setupDirectFilesIn with a new parameter with the modules to add in the direct cradle
  • and remove the generic setupBuildToolFiles, or add the modules for each test suite there

[isovector]

Yeah, I force added my hie.yaml after a few hours of debugging why CI was failing but everything worked fine locally.

[jneira]

@isovector hlint plugin tests was failing for the same reason but then i've remembered your issue/pr relatively early.
Finally i've added the hie.yaml too and i hope we can drop them at once.
Not sure if hie.yaml schema will change but we will have one point to change it.

[peterwicksstringfield]

We just deleted the "real" cradles (in #741 via #1230) that we use for actually developing hls (and good riddance, if hls can sort-of just-work without users making weird helper files that's great)

... so it seems like a good time to revive this issue.

As konn mentions (#741 (comment)) I think we need the ability to handwrite cradles for testing. So long as cradles exist we will want to test what happens when they are weird / malformed / handwritten. In particular, our testdata currently includes "fake" cradles like this:

cradle:
  direct:
    arguments:
      - "NeedsPragmas"

And:

cradle:
  direct:
    arguments:
      - "-XCPP"
      - "-DFLAG"
      - "ApplyRefact3"
      - "ApplyRefact2"

There's no way we are auto-generating that.

Do we want to leave the "fake" cradles in the testdata where they are and close this issue ... or do something else?

If we are worried about the "real" hls seeing the "fake" cradles and causing problems while we are developing, then one option would be to rename the "fake" cradles from "testdata/../hie.yaml" to "testdata/../test-hie.yaml", then just before the tests run, we copy "test-hie.yaml" back to "hie.yaml", then after the tests run we can delete all the "hie.yaml" again. (I partially implemented this and it seems like it works just fine.)

[jneira]

There's no way we are auto-generating that.

Do we want to leave the "fake" cradles in the testdata where they are and close this issue ... or do something else?

Well, we can always write code parameterized enough to generate the files with something like:

generateTestCradle :: Filepath -> [String]
generateTestCradle dir args = ...

and make it generate a direct cradle in dir. Then we can add auxiliary functions to generate common direct cradles. Test code will have to call that function with appropiate arguments for each case. When all tests use that function we could remove them all from vcs again.

If we are worried about the "real" hls seeing the "fake" cradles and causing problems while we are developing

Well, in my case i like to open the testdata dir and test manually in the editor before or while i am writing the test, it is not real code so they did not cause problems for me, do they for you?

[jneira]

Not sure if at this point this continue being relevant

[jneira]

It is not 🙂