Traversing the directory hierarchy is broken on Windows as of version 0.47 #1600
Comments
|
I am not sure whether I can fix this soon. Biggest problem for me is that I do not have access to a Windows machine on which I can test. Also the unit testing on the windows-look-a-like-filesystem is limited. Is anybody willing to help me out by either fixing the problem or run tests whenever I have a possible fix ready? |
It doesn't actually matter whether any of the unit tests is successful on your or my PC, be it Windows or Linux. The only source of truth should be the CI server, and, since you're using GitHub, you can easily have one. Only when each commit to the project gets tested on the CI infrastructure can you be sure that nothing gets broken. Please read on about Building and testing Java with Gradle, and also about Using a matrix for your jobs. There's also plenty of examples on how to use the |
|
Yes, that is already in place. But you don't have an actual filesystem in CI. So we use jimfs to simulate that. The windows version of it, lacks some functionality. So certain unit tests can not run on CI. |
|
@paul-dingemans, thank you for the clarification. Correct me if I'm wrong, but file system operations are allowed when using GitHub Actions — you can create/delete/append files, traverse directories etc. Probably, |
|
I would not expect it to be too heavy. I just have no experience with it. Most code in this project was written before I got involved. If you have a reference to a project where something like this is set up, I would be happy to use that. Otherwise, I have to investigate it myself. Of course, it might take longer as a result of that. So any help is appreciated. |
|
@paul-dingemans, I'll try to refactor a unit test and give you a clue how a default file system can be tested. In a short while. |
That would really be appreciated. |
…matching Globs always use a "/" as directory separator on all OS's. Input patterns containing a "\" on Windows OS are transformed to "/" as users on Windows more likely would assume that the "\" may be used. On WindowsOS, transform "\" in the filepath to "/" before comparing the filename with the regular expression (of the glob) which always uses "/" as separator. Refactor all logic which create globs based on an input path. - If a path (absolute or relative) point to a directory, that path is expanded to the default globs (*.kt, *.kts) in that specific directory or any of its subdirectories. - If a path (absolute or relative) does not point to a directory, e.g. it points to a file, or it is a pattern. See "**" replacement below. - On Windows OS patters containing a "*" (or "**") can not be resolved with default Paths utilities. In such case the given input pattern is handled as is. See "**" replacement below. Patterns that contain one or more occurrence of a "**" are split into multiple patterns so that files on that specific path and subdirectories will be matched. - For example, for path "some/path/**/*.kt" an additional pattern "some/path/*.kt" is generated to make sure that not only the "*.kt" files in a subdirectory of "some/path/" are found but also the "*.kt" in directory "some/path" as well. This is in sync with the "**" notation in a glob which should be interpreted as having zero or more intermediate subdirectories. - For example, for path "some/**/path/**/*.kt", multiple additional patterns are generated. As it contains two "**" patterns, 2 x 2 patterns are needed to match all possible combinations: - "some/**/path/**/*.kt" - "some/**/path/*.kt" - "some/path/**/*.kt" - "some/path/*.kt" Finally, on Windows OS more fixes are needed as the resulting globs may not contain any drive destinations as the start of the path. Such a drive destination is replaced with a "**". So "D:/some/path/*.kt" becomes "/some/path/*.kt". Note that the last glob representation is less strict than the original pattern as it could match on other drives that "D:/" as well. Extend trace logging. Closes pinterest#1600 Closes pinterest#1601
|
Hi @0x6675636b796f75676974687562 , As far as I can see, I have fixed the problem with path traversing and the ant-path parsing (#1601). I would be grateful if you could test latest snapshot (see https://pinterest.github.io/ktlint/install/snapshot-build/) to see whether the problems are resolved on Windows. |
@0x6675636b796f75676974687562 Andrey, |
|
@paul-dingemans, I'll test and get back to you tomorrow morning. |
|
Tested on Windows, works like a charm. Thank you @paul-dingemans. |
|
Not sure if you fixed that as well, but when using Little demonstration: ktlint_test.zip |
|
@CokieForever, please enclose the last argument in double quotes to make sure it doesn't get expanded by the shell. |
|
@unix-junkie I'm not sure what you mean, the argument is logged by ktlint and seems perfectly fine. Besides I'm on Windows like the OP, so shell expansion is not really a thing 😄 And even if by some strange phenomenon it was indeed expanded, it should still work, right? 🤔 For the sake of completeness I tried with double quotes anyway, and it made no difference. I also tried with absolute paths, also no difference. From what I can gather from the debug logs the issue is caused by the |
|
Are you sure that you are using version |
|
No sorry I was using 0.47.0, I hadn't realized 0.47.1 was out. I can see the error message with 0.47.1, and apparently we went back to WARN instead of ERROR as well. Does this mean the issue will not get fixed then? Using parent directories would be an effective workaround for #1288 😞 |
|
Nevermind, nevermind, apparently #1288 was fixed in 0.47.1 as well, or at least it works for me again 😄 |
|
@paul-dingemans I think the problem mentioned by @CokieForever still remains. The only difference in my case is - I am using Mac OS X machine. I have a I've tested ktlint The Here is the output of a debug logger: With respective warnings (based on the arguments provided): I have tested multiple variants of the arguments:
In each case - the starting point of the file walk was defined incorrectly and scanned only the files from the current directory ( Side note: The last working version in the setup described above - was |
I would assume that |
|
Same here, I just updated from It is quite easy to reproduce with bash: just run Hope this helps. |
|
@paul-dingemans - in this exact project, all the gradle files are implemented in groovy. So the assumption about I've also tried to use the kltint cli with no explicitely passed input globs. With exactly same result of no files found due to the fact, the traversal started in the module's directory not in the project's directory. @jokinol described how one can reproduce the problem. There is no difference whether you will run the ktlint cli from the command line or try to use gradle to actually call it. In addition to what @jokinol wrote, I experience this problem with Z Sheel (zsh) - so the guess about the problem being shell-dependent, might be unlucky guess. |
Tnx for clarifying this false assumption. I can now reproduce the problem and will create a new issue for fixing it. |
|
For some reason I was not able to execute But in So I can confirm that at least it works on my machine. |
|
@paul-dingemans - I've tested @jokinol - regarding the It's enough to define the attributes within the module dependencies {
// (...)
// The attribute workaround below solves the problems introduced by the updated shadow library.
// @see https://github.com/pinterest/ktlint/issues/1114
// @see https://github.com/JLLeitschuh/ktlint-gradle/issues/458
add("ktlint", "com.pinterest:ktlint:<ktlint-version>") {
attributes {
attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling, Bundling.EXTERNAL))
}
}
} |
|
Could you release 0.49.1 kindly please? I did some ugly hacks to my CI scripts and would like to remove those 😬 |
|
I cannot release on request as everybody want a release as soon as their problem is solved ;-) |
Expected Behavior
Running
ktlintw/o specifying any input files previously made it search for*.ktand*.ktsfiles, starting from the current directory, e. g.:This behaviour still holds when
ktlintis run from the command line on Linux or Mac OS X.Observed Behavior
Yet, using just the very same command line switches on Windows (i. e. when running
ktlintfrom Git Bash) results in no input files being found:This is a clear regression against version 0.46.1 and, apparently, has been caused by the recent changes which re-enable (sort of — see #1601) the parsing of Ant-like path patterns (e.g.:
**/*.kt) — which was broken in version 0.46.1.Steps to Reproduce
Your Environment
.editorconfigsettings: noneMINGW64_NT-10.0-19043 unit-725 3.3.4-341.x86_64 2022-05-09 11:56 UTC x86_64 MsysLinux unit-725 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 GNU/LinuxThe text was updated successfully, but these errors were encountered: