cleanup_paths doesn't behave the same way when building 'macos Xcode' and 'macos app' (and maybe doesn't behave like the documentation implies)

[michelcrypt4d4mus]

Describe the bug

Unexpected Behavior

I have a pyproject.toml for an app with the formal name Illmatic App that was originally configured for macos app with a cleanup_paths like this:

cleanup_paths = [
    "{app.formal_name}.app/Contents/Resources/app_packages/bin",
    "{app.formal_name}.app/Contents/Resources/app_packages/buildscript",
]

Which worked great for briefcase build macos app.

I expected that this would be sufficient to also remove these dirs during briefcase build macos Xcode but that turned out to not be the case in a lot of unexpected ways. I had to add more totally unrelated paths that I didn't seem to be able to derive from the various metadata available to get macos Xcode to remove these from the bundle, currently landing on:

cleanup_paths = [
    # macos App
    "{app.formal_name}.app/Contents/Resources/app_packages/bin",
    "{app.formal_name}.app/Contents/Resources/app_packages/buildscript",
    # macos Xcode
    "app_packages.x86_64/bin",
    "app_packages.x86_64/buildscript",
    "app_packages.arm64/bin",
    "app_packages.arm64/buildscript",
    "IllmaticApp/app_packages/bin",
    "IllmaticApp/app_packages/buildscript",
]

Now tbh I don't really know if I need to delete those app_packages.x86_64 etc paths or if they end up in the bundle, but the most unexpected thing was that the macos Xcode build removed the space from Illmatic App and so I had to manually configure IllmaticApp/ as the root dir.

Unclear Documentation Generally

The documentation says:

The paths provided will be interpreted relative to the app bundle folder (e.g., the macOS/app/My App folder in the case of a macOS app).

But I found this not to be the case, or at least it's quite vague. As the documentation is written I would have expected to not have to start my cleanup_paths with "{app.formal_name}.app/. In other words I would have expected this to work:

cleanup_paths = [
    "Contents/Resources/app_packages/bin",
    "Contents/Resources/app_packages/buildscript",
]

And I definitely was not expecting the macos Xcode build dir to strip the spaces out of the formal name to create a root dir.

Steps to reproduce

1. Configure some cleanup_paths to remove executables (or whatever) from imported python packages
2. Run briefcase build macos app and briefcase build macos Xcode and look in the resulting bundles

Expected behavior

1. cleanup_paths should allow for a list of relative paths that will be cleared from a macOS app bundle regardless of which template is used.
2. Either cleanup_paths should actually work with paths relative to the app bundle or the documentation should be explicit about the fact that the root dir is the parent dir of the app bundle, not the actual app bundle.

Screenshots

No response

Environment

    Operating System: macos 13.6.4
    Python version: Python 3.12.3
    Software versions:
        Briefcase: 0.3.19.dev199+g92a01219

Logs

I added some manual logging to print() for my own reference what exactly were the paths being cleaned up; can recreate and add if it's helpful.

Additional context

No response

[freakboy3742]

So - it sounds like there's a bunch of confusion that has stemmed almost entirely from one outdated documentation reference, and some terminology that is potentially confusing (especially on macOS).

The reference to the macOS/app/My App folder describes the location that Briefcase used to generate content, before the introduction of a single build and dist folder in Briefcase 0.3.13, a little over a year ago. It should reference build/myapp/macos/app.

The "app bundle" is the term Briefcase uses for this build output folder, not the .app folder used by macOS.

The documentation for cleanup_paths should be updated to reflect those two details.

The removal of the space from "Illmatic App" is because it is using the class name derived from the formal name. This mirror what happens when you define a project in Xcode that has a space in its name.

[michelcrypt4d4mus]

that's all helpful info. i don't know enough about what really goes on under the hood with macos Xcode vs. macos app builds but i would expect that the final app bundle should be basically the same for both formats? am i wrong?

in which case why can't the same values in cleanup_paths work on both formats?

[freakboy3742]

that's all helpful info. i don't know enough about what really goes on under the hood with macos Xcode vs. macos app builds but i would expect that the final app bundle should be basically the same for both formats? am i wrong?

in which case why can't the same values in cleanup_paths work on both formats?

The final app will be the same (or only cosmetically different). However, as I indicated in my previous reply, that's not the app bundle. The "Bundle" in briefcase's terminology is the contents of build/<project>/<platform>/<backend>. The bundle has fundamentally different structure for the macOS app and macos Xcode templates.

So no - the same values won't (and can't) work for both formats.

Cleanup is performed as part of the create command. It exists because, having generated a template, and installed the support package, app code, and requirements, there are files that may need to be removed before the build process. Thus, it needs to have access to the entire "bundle" directory (and thus the entire project folder), not just the generated artefact that is output by the build - an artefact that on some platforms (including macOS) has been signed, and thus can't be modified.

[michelcrypt4d4mus]

ok that makes sense... i was definitely confused by the overloading of the word "bundle" because that's the same word Apple uses for the folder containing a macOS application. the documentation should probably make that clearer w/r/t macOS apps. in an ideal world i would suggest that briefcase change its terminology and not use the word "bundle" to denote a directory structure that is used to compile a macOS application "bundle". that would be immediately much clearer for new users.

at a higher level my expectation around reading documentation for something called cleanup_paths in a python packaging context is that it's an option you can use to remove files from the final application bundle, not just from the intermediate briefcase bundle. i can't really think of any reason to bother messing with the intermediate briefcase bundle - i mean after all isn't it supposed to just be ephemeral? why offer people a parameter to mess around with intermediate steps?

in macOS world removing files from the final .app bunlde is highly relevant because apple will reject your app for not removing or chmoding any executable file and many python packages include executable .py files. good example of a briefcase user running into this issue is #1655. it's at least somewhat relevant in windows / linux world too because the expectation would be that you can use cleanup_paths to slim down your final distribution package.

what is the motivating reason for having cleanup_paths apply to ephemeral build output instead of applying to the actual final product?

[freakboy3742]

what is the motivating reason for having cleanup_paths apply to ephemeral build output instead of applying to the actual final product?

The primary motivating use case is purging content in the standard library that isn't needed at runtime. If your app doesn't use bzip2, there's no point shipping the _bzip2.so that implements that library.

It can't be applied to the final build product because the final build product has been signed, and injecting a "cleanup" phase post-compile, pre-signing would require reasonably complex interactions with project files for Xcode, VisualStudio and Gradle, or a manual re-sign process after the app has been built.

The long game is to get a "minimal viable embedded Python" as part of the core CPython release process, with all the non-essential packages available on PyPI (so you would opt-in to needing bzip2, rather than the opt-out that is currently used). However, that's a much more complex (and deeply political) discussion that has been underway for (checks notes) at least 6 years.

[michelcrypt4d4mus]

or a manual re-sign process after the app has been built.

doesn't briefcase already manually re-sign the .app bundle? i feel like when i run build -vvvv i see a million lines about manual code signing. or am i mistaken?

[freakboy3742]

doesn't briefcase already manually re-sign the .app bundle? i feel like when i run build -vvvv i see a million lines about manual code signing. or am i mistaken?

Yes, but that's the initial signing, not re-signing. Xcode will sign the app it creates. If you then modify the bundle by deleting files, you would need to re-sign it.

[michelcrypt4d4mus]

huh ok that makes sense / i had no idea. i guess i would just make the comment that the more i've learned about apple development / xcode / signing / etc. the more it has become obvious that for any kind of complicated apple app bundle - my case with an embedded helper app would be like the most minimally "more complicated" - you basically just have to give up on xcode and take over things like code signing and packaging manually with scripts that call apple's command line tools. even on the apple developer forums that seems to be the overwhelming opinion.

if briefcase aspires to helping people build more complicated macOS apps it might be good to start thinking that way sooner rather than later. xcode will break your heart.

[michelcrypt4d4mus]

maybe "give up" is a strong word - you can still profitably use Xcode to call your manual signing / packaging / whatever scripts because Xcode will inject like 1000 env vars that tell you everything about the build - but i've at this never seen a single person on the apple developer forums ever suggest any solution other than "write a bash script". even some of the official apple documentation about stuff like embedding helper apps shows you how to write such a bash script and call it from xcodebuild.