Fix resource bundle lookup #3463

ffried · 2021-04-30T13:29:55Z

Fix the preferred lookup location for resource bundles.

Motivation:

The preferred lookup location for resource bundles is next to the binary that is executed. This was previously achieved by searching next to Bundle.main.bundlePath. This change moved the execution of Bundle.main outside of the generated bundle accessor code - and thus uses Bundle.main of the compilation process.

Modifications:

Move the Bundle.main call back to the generated code.

Result:

The preferred resource bundle lookup will again be the location of the compiled binary.
Fixes https://bugs.swift.org/browse/SR-14555.

abertelrud

Thank you for fixing this! Just one question inline about a path separator.

abertelrud · 2021-04-30T16:10:25Z

Sources/Build/BuildPlan.swift

        stream <<< """
        import class Foundation.Bundle

        extension Foundation.Bundle {
            static var module: Bundle = {
-                let mainPath = "\(mainPath.asSwiftStringLiteralConstant)"
-                let buildPath = "\(bundlePath.asSwiftStringLiteralConstant)"
+                let mainPath = Bundle.main.bundlePath + "\(bundlePath.basename.asSwiftStringLiteralConstant)"


Is the bundlePath guaranteed to end with a path separator on all platforms, or might one need to be conditionally inserted here?

@abertelrud Good point, thanks! The old code also appended a slash. I've added this back!

@abertelrud Actually, I'm now using Bundle.main.bundleURL.appendingPathComponent. This moves the responsibility for path separators to Foundation.URL. Using a forward-slash might break Windows, since the path separator is a backslash there.
Hope this is ok?

Thanks @ffried, I think that's even better, especially for Windows compatibility.

abertelrud

Thanks, this looks good to me. @neonichu, WDYT?

This would be good to nominate for 5.5 once it's in main, I think.

tomerd · 2021-04-30T20:43:39Z

+1 cherry-picking to 5.5 make sense

tomerd · 2021-04-30T20:43:51Z

@swift-ci please smoke test

tomerd · 2021-04-30T20:44:42Z

are there any tests we need to add here to make sure this does regress in the future?

ffried · 2021-04-30T20:49:29Z

are there any tests we need to add here to make sure this does regress in the future?

@tomerd I added a test assertion that checks for Bundle.main being in the generated code. That's not bulletproof but still better than nothing I guess.

abertelrud · 2021-04-30T22:36:32Z

That's a good point about a regression test. One way to test the functionality here would be to have a test package that builds an executable that relies on the presence of the resource bundle, and then a functional test that builds it, copies the executable and its resource bundle somewhere else, and then removes the original built resource bundle (to make sure hardcoded paths break), and then runs the executable. It's a bit involved but that would be a useful test to add. I don't think we should prevent nominating for 5.5 for that test but we should really have one of those to detect any functional breakage in the future.

tomerd · 2021-04-30T22:51:27Z

it's a bit involved but that would be a useful test to add. I don't think we should prevent nominating for 5.5 for that test but we should really have one of those to detect any functional breakage in the future.

lets record in a radar?

abertelrud · 2021-04-30T23:04:51Z

it's a bit involved but that would be a useful test to add. I don't think we should prevent nominating for 5.5 for that test but we should really have one of those to detect any functional breakage in the future.

lets record in a radar?

Definitely, if that test isn't part of this PR, then we should absolutely have a JIRA on it, and get to it soon. I can create it, or @ffried would you want to create a JIRA to track adding such a test (or you might have other ideas for how best to test it)?

ffried · 2021-05-01T04:21:36Z

@abertelrud What you're describing sounds exactly like what we're doing in our docker deployments (two images, one for building, one for running). So that would be an awesome test case.
However, I have no clue how to set this up in the current test setup. So if it's ok with you guys, I'd like to let you create the Jira issue to further track this for now.

abertelrud · 2021-05-03T01:32:52Z

@abertelrud What you're describing sounds exactly like what we're doing in our docker deployments (two images, one for building, one for running). So that would be an awesome test case.
However, I have no clue how to set this up in the current test setup. So if it's ok with you guys, I'd like to let you create the Jira issue to further track this for now.

Sounds good — happy to create the JIRA. I’ll post back here when I’ve written it up. To be clear, the JIRA initially should spell out what needs to happen, but can leave some details of exactly how to implement it for later. We should be able to slot that into the functional tests without too much problem.

neonichu · 2021-05-03T21:19:28Z

@ffried Thanks for this PR, could you also do a cherry-pick for the release/5.5 branch?

* Fix resource bundle lookup * Add test check that `Bundle.main` is executed in the compiled binary * Ensure trailing slash for bundle path * Rely on Foundation.URL for path separators

ffried · 2021-05-04T05:27:34Z

@neonichu Thanks! Cherry-pick PR is #3467...

abertelrud · 2021-05-04T18:38:28Z

Created https://bugs.swift.org/browse/SR-14592 for the functional test. I used a package similar to that for manual testing of #3467 though the actual work is in writing the functional test that causes all of that to happen.

abertelrud · 2021-05-04T18:38:36Z

Thanks again for this fix!

* Fix resource bundle lookup * Add test check that `Bundle.main` is executed in the compiled binary * Ensure trailing slash for bundle path * Rely on Foundation.URL for path separators

ffried · 2021-05-05T07:39:36Z

@abertelrud Couldn't let go of this... 🙃 I've taken a shot at adding this test in #3472. Maybe it's not perfect yet, but I've tested it on the release/5.5 branch where the test was green (tested on macOS only, though).

Fix resource bundle lookup

188a212

ffried requested review from abertelrud, friedbunny, neonichu and tomerd as code owners April 30, 2021 13:29

Add test check that Bundle.main is executed in the compiled binary

c476125

ffried mentioned this pull request Apr 30, 2021

Build: quote the path string for bundle resources #2972

Merged

abertelrud reviewed Apr 30, 2021

View reviewed changes

ffried added 2 commits April 30, 2021 18:13

Ensure trailing slash for bundle path

ea58843

Rely on Foundation.URL for path separators

8e008ba

abertelrud approved these changes Apr 30, 2021

View reviewed changes

neonichu merged commit 708c5d6 into swiftlang:main May 3, 2021

ffried deleted the bugfix/resource_paths branch May 4, 2021 05:22

ffried mentioned this pull request May 4, 2021

[5.5] Fix resource bundle lookup (cherry-pick of #3463) #3467

Merged

ffried mentioned this pull request May 5, 2021

Add functional test for binary with moved resources #3472

Merged

MaxDesiatov mentioned this pull request Oct 24, 2021

Fix generateResourceAccessor WASI regression #3819

Merged

ffried mentioned this pull request Apr 30, 2021

[SR-14555] Bundle.module searches for bundle in swift binary location instead of Bundle.main of compiled binary #4432

Closed

abertelrud mentioned this pull request May 4, 2022

[SR-14592] Create functional test for movable executable with resource bundle in SwiftPM #4425

Open

Fix resource bundle lookup #3463

Fix resource bundle lookup #3463

Uh oh!

Conversation

ffried commented Apr 30, 2021

Motivation:

Modifications:

Result:

Uh oh!

abertelrud left a comment

Choose a reason for hiding this comment

Uh oh!

abertelrud Apr 30, 2021

Choose a reason for hiding this comment

Uh oh!

ffried Apr 30, 2021

Choose a reason for hiding this comment

Uh oh!

ffried Apr 30, 2021

Choose a reason for hiding this comment

Uh oh!

abertelrud Apr 30, 2021

Choose a reason for hiding this comment

Uh oh!

abertelrud left a comment

Choose a reason for hiding this comment

Uh oh!

tomerd commented Apr 30, 2021

Uh oh!

tomerd commented Apr 30, 2021

Uh oh!

tomerd commented Apr 30, 2021

Uh oh!

ffried commented Apr 30, 2021

Uh oh!

abertelrud commented Apr 30, 2021

Uh oh!

tomerd commented Apr 30, 2021

Uh oh!

abertelrud commented Apr 30, 2021

Uh oh!

ffried commented May 1, 2021

Uh oh!

abertelrud commented May 3, 2021

Uh oh!

neonichu commented May 3, 2021

Uh oh!

ffried commented May 4, 2021

Uh oh!

abertelrud commented May 4, 2021

Uh oh!

abertelrud commented May 4, 2021

Uh oh!

ffried commented May 5, 2021

Uh oh!

Uh oh!