_InternalSwiftSyntaxParser error with 1.3.0

[jpsim]

Steps to reproduce:

$ brew install sourcery
$ sourcery --version
1.3.0
$ git clone http://github.com/realm/SwiftLint
$ cd SwiftLint
$ git checkout 92061e8
$ rm Source/SwiftLintFramework/Models/PrimaryRuleList.swift Tests/SwiftLintFrameworkTests/AutomaticRuleTests.generated.swift Tests/LinuxMain.swift
$ make sourcery 
sourcery --sources Source/SwiftLintFramework/Rules --templates .sourcery/PrimaryRuleList.stencil --output .sourcery
No config file provided or it does not exist. Using command line arguments.
Scanning sources...
error: The loaded '_InternalSwiftSyntaxParser' library is from a toolchain that is not compatible with this version of SwiftSyntax
The loaded '_InternalSwiftSyntaxParser' library is from a toolchain that is not compatible with this version of SwiftSyntaxmake: *** [Source/SwiftLintFramework/Models/PrimaryRuleList.swift] Error 3
$ xcode-select -p
/Applications/Xcode-12.4.app/Contents/Developer
$ xcodebuild -version
Xcode 12.4
Build version 12D4e

[krzysztofzablocki]

@jpsim you'll need Xcode in default path to be the right version, which one do you have there?

I'm going to look into bundling the dylib with the tool but need to read up how one can replace rpath to point to local file with spm, or maybe you have any ideas?

[jpsim]

you'll need Xcode in default path to be the right version, which one do you have there?

What do you mean by "default path"? You mean what's set to xcode-select? As I indicated in the first comment, that's set to /Applications/Xcode-12.4.app/Contents/Developer

[krzysztofzablocki]

@jpsim I don't really know much about the SPM building / dylib linking but I've exact same Xcode version but in /Applications/Xcode.app and people that tested tool before also had Xcode in non-default location and they had to symlink it to that default one

I run your steps on my machine and had this output

sourcery --sources Source/SwiftLintFramework/Rules --templates .sourcery/PrimaryRuleList.stencil --output .sourcery
No config file provided or it does not exist. Using command line arguments.
Scanning sources...
Found 356 types in 295 files, 295 changed from last run.
Loading templates...
Loaded 1 templates.
Generating code...
Finished.
Processing time 1.0439350605010986 seconds
mv .sourcery/PrimaryRuleList.generated.swift Source/SwiftLintFramework/Models/PrimaryRuleList.swift
sourcery --sources Source/SwiftLintFramework/Rules --templates .sourcery/AutomaticRuleTests.stencil --output .sourcery
No config file provided or it does not exist. Using command line arguments.
Scanning sources...
Found 356 types in 295 files, 0 changed from last run.
Loading templates...
Loaded 1 templates.
Generating code...
Finished.
Processing time 0.14694392681121826 seconds
mv .sourcery/AutomaticRuleTests.generated.swift Tests/SwiftLintFrameworkTests/AutomaticRuleTests.generated.swift
sourcery --sources Tests --exclude-sources Tests/SwiftLintFrameworkTests/Resources --templates .sourcery/LinuxMain.stencil --output .sourcery --force-parse generated
No config file provided or it does not exist. Using command line arguments.
Scanning sources...
Found 254 types in 88 files, 88 changed from last run.
Loading templates...
Loaded 1 templates.
Generating code...
Finished.
Processing time 0.17964303493499756 seconds
mv .sourcery/LinuxMain.generated.swift Tests/LinuxMain.swift

I really want to embed the dylib with the tool, did you had any luck with that or didn't need to for SourceKit?

[jpsim]

We don't embed the SourceKit dylib, and I think there might be some tricky legal/licensing issues with redistributing Apple binaries anyway.

SourceKitten dynamically resolves which SourceKit/clang libraries to load and accesses them via dlopen. It's one of my favourite implementation details about how it works: https://github.com/jpsim/SourceKitten#how-is-sourcekit-resolved source

However, that's not as easily done with Swift because of the complexity of the ABI.

FYI SwiftLint tried to ship SwiftSyntax integration over a year ago but we had to pull it due to portability concerns like this one.

See realm/SwiftLint#3105 for lots of details.

It's something that should be fixable though, I just haven't prioritized it.

I haven't encountered this issue with DrString, which also integrates SwiftSyntax, so maybe look at how it's integrated there compared to Sourcery?

[krzysztofzablocki]

oh that's helpful, they do deal with moving the dylib https://github.com/dduan/DrString/blob/main/Scripts/package-darwin.sh I'll take their learning and try to make a minor release tomorrow with dylib included with the tool

[krzysztofzablocki]

fixed in 1.3.1

[daniele-pizziconi]

I'm having the same issue with the newly released version of Xcode 12.5.

The loaded '_InternalSwiftSyntaxParser' library is from a toolchain that is not compatible with this version of SwiftSyntax

[krzysztofzablocki]

@daniele-pizziconi which version of Sourcery are you on, and how did you install it? This issue shouldn't happen on Brew or Binary distribution channels since I rewrite rPath

[danielPeloton]

I'm having the same problem with Xcode 12.5, and I'm handling Sourcery entirely through mint. Should we not be doing this?

[krzysztofzablocki]

@danielPeloton until we migrate fully to self-contained SPM via #935, you want to use brew or binary distributions because both of those rewrite rPath to be exact version we use and not rely on your machine setup, mint just runs swift package manager and that setup isn't properly handling dylib yet

[daniele-pizziconi]

Thank you Krzysztof for the heads up. I was indeed using directly 'swift build -c release'.
Is there any eta for that work you mentioned?

[krzysztofzablocki]

I'll try to find some time on the weekend to wrap up the open prs

[Lutzifer]

@krzysztofzablocki You know how to reach me, if you need a guinea pig for the mint-compatibility

[krzysztofzablocki]

unfortunately it seems this is not doable in pure spm package as far as I could tell, spm doesn't support linker flags when building through command line so mint won't work until Apple fixes it 😞

[jpsim]

@krzysztofzablocki Mint & SwiftPM support should be feasible. Here are some open source projects that link SwiftSyntax and work just fine with Mint:

• https://github.com/dduan/DrString
• https://github.com/SwiftDocOrg/swift-doc

You could look at those to compare how they integrate with SwiftSyntax with the approach taken by Sourcery to see how they may have addressed this issue.

[krzysztofzablocki]

they work the same way as Sourcery does, which means spm / mint distributed channel don't work properly if you have custom Xcode from the tool version where's brew always work because we have exactly same setup through install tool rewrite of rpath

[jpsim]

Hmmm, maybe you're right. I'm not running Big Sur so I can't install Xcode 12.5 to troubleshoot.

[hujunfeng]

Any chance that the new Extensible Build Tools and Command Plugins features in Swift 5.6 could fix the Mint support?

[Lutzifer]

We did not have this issue until Xcode 13.3.0 (using mint), it then appeared and was fixed with v1.8.0 😆

[jpsim]

You should use https://github.com/keith/StaticInternalSwiftSyntaxParser as SwiftLint now does.

[krzysztofzablocki]

@jpsim we did from 1.8.0

[jpsim]

Ah great, I should have checked. That explains why updating to 1.8.0 fixed the issue for @Lutzifer. Shouldn't happen again.

[Lutzifer]

Thx for all the work on this @jpsim @krzysztofzablocki