x/mobile: gomobile bind is ignoring target architecture parameter

[iamcalledrob]

What version of Go are you using (go version)?

$ go version
go version go1.16.5 darwin/arm64

Does this issue reproduce with the latest release?

Yes (and only with the latest release)

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/rob/Library/Caches/go-build"
GOENV="/Users/rob/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/rob/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/rob/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.16.5"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3l/mj1p0_6x3dnb850mkd9ph8b00000gn/T/go-build3577254656=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I ran gomobile bind with a specific target and architecture (amd64 in this example)

Exact bind command:

gomobile bind -target ios/amd64 -v -o mobile.xcframework ./pkg/...

Gomobile version

golang.org/x/mobile v0.0.0-20210710064935-76c259c465ba

What did you expect to see?

I expected a framework to be built only for iOS with the amd64 architecture (i.e. for an intel simulator). I would expect to see something like this inside the xcframework:

$ ls -l mobile.xcframework 
drwxr-xr-x  4 rob  staff   128 Jul 14 15:55 ios-x86_64-simulator

What did you see instead?

The framework was built for all "iOS" platforms (macos, catalyst, simulator, iOS) and architectures (amd64, arm64).

$ ls -l mobile.xcframework 
-rw-r--r--  1 rob  staff  1767 Jul 14 15:54 Info.plist
drwxr-xr-x  4 rob  staff   128 Jul 14 15:59 ios-arm64
drwxr-xr-x  3 rob  staff    96 Jul 14 15:54 ios-arm64_x86_64-maccatalyst
drwxr-xr-x  4 rob  staff   128 Jul 14 15:55 ios-arm64_x86_64-simulator
drwxr-xr-x  4 rob  staff   128 Jul 14 15:58 macos-arm64_x86_64

The additional platforms are a result of the latest release (golang/mobile@76c259c).

I think this is a combination of both a bug and some missing functionality—i.e. being able to specify which platforms to build for in addition to which architectures.

There's a comment by another person on this release with more information on a specific bug regarding this: golang/mobile@76c259c#commitcomment-53412472

[iamcalledrob]

More context: It looks like a PR was made whilst I was writing this issue (!) which might include fixes for the bug and missing functionality golang/mobile#70

[ydnar]

@iamcalledrob hi there, golang/mobile#70 / http://golang.org/cl/334689 author here.

It turns out that while golang/mobile#65 worked for some use cases, it subtly broke others. Sorry!

@hajimehoshi asked if I could file an issue to discuss the changes in the new PR, but it turns out someone (you) already did.

So to summarize what I think the problem is, and some thoughts on how we might fix it:

• golang.org/x/mobile predates Go modules, GOOS=ios, and currently has some outdated defaults that aren’t supported by current versions of Xcode.
• cmd/gomobile: support macOS and Catalyst mobile#65 and its antecedents overloaded -target=ios to mean ios, iphonesimulator, macosx, and maccatalyst SDKs.
• Go 1.16 introduced GOOS=ios with a darwin build tag: build: add ios/amd64 as a GOOS/GOARCH #42100
• gomobile wasn’t aware of GOOS=ios, and continued to build with GOOS=darwin and an ios build tag. When combined with building additional SDKs (macosx, maccatalyst), subtle breakage could occur.

My goal with golang/mobile#70 was to address these subtle breakages, and update gomobile to support GOOS=ios as well as be able to correctly build for the user’s choice of platforms, rather than forcing a macos and maccatalyst build on every user.

• The new PR restores the functionality of gomobile -target=ios to build just the ios and iphonesimulator platforms.
• If you want all supported Apple platforms, use -target=darwin. I’m open to nuking this. @hajimehoshi what do you think?
• It expands the current list of supported platforms of android and ios, adding macos, macatalyst (UIKit on macOS), iossimulator. These can all be individually targeted, along with a specific arch if you want, e.g. iossimulator/amd64.
• It adds a handful of new build tags so Go code can discriminate between the different Apple platforms. The new build tags are macos, maccatalyst, and iossimulator. For instance, you can use the maccatalyst tag to build a Go file that accesses UIKit.

@iamcalledrob could you try your project with this branch? -target=ios/amd64 should do what you want now. Add this to your go.mod file and run go mod tidy:

replace golang.org/x/mobile => github.com/ydnar/gomobile goos-ios

[ydnar]

@hajimehoshi does the mobile repo have a macOS trybot?

[hajimehoshi]

CC @hyangah

The issue that a specified architecture is ignored (as the first comment says) is a regression. Also, we have multiple issues as @ydnar listed. I think we should revert the Catalyst change once, and then re-discuss the design. @ydnar What do you think?

If you want all supported Apple platforms, use -target=darwin. I’m open to nuking this. @hajimehoshi what do you think?

This seems good, as macOS is not iOS, but let me think more...

@hajimehoshi does the mobile repo have a macOS trybot?

I don't know about this, but I don't think there is.

[ydnar]

The issue that a specified architecture is ignored (as the first comment says) is a regression. Also, we have multiple issues as @ydnar listed. I think we should revert the Catalyst change once, and then re-discuss the design. @ydnar What do you think?

I’m OK reverting the change, and revising it on the new PR, since it preserves the earlier behavior of -target=ios building just for iOS and the simulator. That’s better than forcing undesired platforms (and any related build issues) on gomobile’s users.

If you want all supported Apple platforms, use -target=darwin. I’m open to nuking this. @hajimehoshi what do you think?

This seems good, as macOS is not iOS, but let me think more...

The alternative would be for developers to manually specify all the platforms they want to build for, e.g. -target=ios,macos, which seems OK.

@hajimehoshi does the mobile repo have a macOS trybot?

I don't know about this, but I don't think there is.

Given that this needs to run tests on macOS with Xcode (and a developer environment with a cert), it’s probably a good idea. That would have caught a few of the broken tests, some of which predate the Catalyst PR. Is it possible to add one?

[hajimehoshi]

Given that this needs to run tests on macOS with Xcode (and a developer environment with a cert), it’s probably a good idea. That would have caught a few of the broken tests, some of which predate the Catalyst PR. Is it possible to add one?

CC @dmitshur

[gopherbot]

Change https://golang.org/cl/334590 mentions this issue: Revert "cmd/gomobile: support macOS and Catalyst"

[hajimehoshi]

gomobile wasn’t aware of GOOS=ios, and continued to build with GOOS=darwin and an ios build tag. When combined with building additional SDKs (macosx, maccatalyst), subtle breakage could occur.

I'd like to know whether there are things Go should care for macOS Catalyst, like what Go does for iOS. Or, is it enough for Go to behave as if the OS is macOS or iOS for macOS Catalyst?

https://github.com/golang/go/search?q=filename%3A*_ios.go&type=code

[ydnar]

I found an issue in cgo that’s potentially blocking a real fix for this: #47228

[ydnar]

@hajimehoshi my current workaround for #47228 is to supply both ios and macos build tags when building for Catalyst.

[ydnar]

If you want all supported Apple platforms, use -target=darwin. I’m open to nuking this. @hajimehoshi what do you think?

This seems good, as macOS is not iOS, but let me think more...

The alternative would be for developers to manually specify all the platforms they want to build for, e.g. -target=ios,macos, which seems OK.

I removed target=darwin and target=iossimulator in current build of https://go-review.googlesource.com/c/mobile/+/334689

It also fixes OP’s bug, and builds with only a single target arch will now succeed.

[iamcalledrob]

@iamcalledrob could you try your project with this branch? -target=ios/amd64 should do what you want now. Add this to your go.mod file and run go mod tidy:

replace golang.org/x/mobile => github.com/ydnar/gomobile goos-ios

I can confirm that things are working perfectly with your new branch (amazing!). The biggest benefit here is the build times while developing—since a build can now be for a specific arch/platform, the build time is reduced from ~90s to ~13s.

For anyone else who might be following this, I had to also go get golang.org/x/mobile/cmd/gomobile to pick up the new tools :)

[iamcalledrob]

Following up, I'm unable to get builds with -target ios or -target ios/arm64 to work in the iOS simulator on my M1 mac.

From peeking at the code, it looks like there's an assumption made that ios/arm64 is a physical device, and ios/amd64 is simulator. M1 complicates that assumption a bit :)

[ydnar]

Following up, I'm unable to get builds with -target ios or -target ios/arm64 to work in the iOS simulator on my M1 mac.

From peeking at the code, it looks like there's an assumption made that ios/arm64 is a physical device, and ios/amd64 is simulator. M1 complicates that assumption a bit :)

Curious, are you able to build for iOS simulator with current master branch of golang/mobile?

I ran into an issue merging ios/arm64 and iphonesimulator/arm64 frameworks into a single XCFramework. Maybe the solution I chose (inhibit creating an iphonesimulator/arm64 slice) wasn’t right.

[iamcalledrob]

I'm also not able to build for iOS simulator with the master branch: building for iOS Simulator, but linking in object file built for iOS, for architecture arm64. It appears to be this problem: #45805?

When building from the github.com/ydnar/gomobile goos-ios branch, the error is a little different (missing arm64 architecture for iphonesimulator):
ld: warning: ignoring file /Users/rob/Library/Developer/{...}/Debug-iphonesimulator/Xplat.framework/Xplat, missing required architecture arm64 in file /Users/rob/Library/Developer/{...}Debug-iphonesimulator/Xplat.framework/Xplat (1 slices)

[ydnar]

I think I got it! Going to run a couple more tests and push a commit:

Edit: commit pushed. @iamcalledrob can you try with current HEAD?

• It re-adds an iossimulator platform (name TBD), which has its own configuration.
• The ios platform can only be built for arm64. If you try to build -target=ios/amd64 it will fail. You’ll need to supply -target=iossimulator/amd64 to get an x86_64 slice for the simulator.
• If you run gomobile build -target=ios it will implicitly expand ios to ios,iossimulator . This probably isn’t the right design, but it works.

Edit: building for iOS, iPhoneSimulator, macCatalyst, and macOS with arm64 and amd64 architectures in the same XCFramework now work. I’ve tested this with a simple app in a simulator:

[iamcalledrob]

Brilliant. I can confirm this now builds a working framework for the ios simulator on arm64!

[ydnar]

Brilliant. I can confirm this now builds a working framework for the ios simulator on arm64!

Terrific. I just pushed a couple updates. Target iphonesimulator is now iossimulator.

[Milerius]

Nice work @ydnar i hope it's will be merged soon <3

[ydnar]

@Milerius me too!

@hajimehoshi what do you think about reviewing the updated CL?

[hajimehoshi]

My understanding is that Go itself doesn't support Catalyst yet, then we should fix Go first. Is that correct?

[ydnar]

My understanding is that Go itself doesn't support Catalyst yet, then we should fix Go first. Is that correct?

This CL doesn’t require any upstream fixes to Go to land it. gomobile bind works for iOS + multiple platforms + architectures.

It’s possible to improve this later if Go is Catalyst-aware, potentially supporting gomobile build for a Catalyst target. My sense is the primary use case for x/mobile is creating libraries, rather than building whole apps.

[hajimehoshi]

Oh sure. I'll take a look.

Which CL are you talking about? https://go-review.googlesource.com/c/mobile/+/334689 still has Catalyst-related code.

[ydnar]

Oh sure. I'll take a look.

Which CL are you talking about? https://go-review.googlesource.com/c/mobile/+/334689 still has Catalyst-related code.

Yes, that one. Catalyst support is kind of the point.

• It can build for iOS and Simulator, and has a backwards-compatible interface with current master branch.
• It can build fat XCFrameworks for multiple SDKs and architectures.
• Catalyst support is a bonus, and from what I can tell, folks really want that feature. Its inclusion doesn’t break anything in the other supported platforms. Unlike my previous reverted CL, Catalyst support is entirely optional. Users have to request it specifically by specifying -target=maccatalyst.

[hajimehoshi]

As it is still impossible to support Catalyst by Go now, we should remove the code for Catalyst from the CL and leave TODO comments. Also please update the description of the CL. Let's revisit supporting Catalyst later.

[ydnar]

As it is still impossible to support Catalyst by Go now, we should remove the code for Catalyst from the CL and leave TODO comments. Also please update the description of the CL. Let's revisit supporting Catalyst later.

It’s not impossible. A Catalyst framework generated by gomobile in this CL works in a Catalyst app. Do you need an example?

[hajimehoshi]

I think it works in most cases but is not well tested by the Go team. The Go itself has iOS-specific code in the standard libraries, and we're not sure how Catalyst code should treat them. We might need more fix like Catalyst-specific code.

[ydnar]

I see your point. Given that x/mobile is outside of the Go core, and not subject to the same compatibility guarantee, what about marking Catalyst support as experimental?

The Go Mobile project is experimental. Use this at your own risk. While we are working hard to improve it, neither Google nor the Go team can provide end-user support.

There are 18 lines (10 SLOC) that mention "catalyst" in this CL, and it doesn’t interfere with the other platforms:

$ grep -i catalyst ./cmd/gomobile/*
./cmd/gomobile/bind_darwinapp.go:		// Catalyst support requires iOS 13+
./cmd/gomobile/bind_darwinapp.go:		if platform == "maccatalyst" && v < 13.0 {
./cmd/gomobile/bind_darwinapp.go:			return errors.New("catalyst requires -iosversion=13 or higher")
./cmd/gomobile/build.go:				// Catalyst support requires iOS 13+
./cmd/gomobile/build.go:				if platform == "maccatalyst" && v < 13.0 {
./cmd/gomobile/build.go:					return nil, errors.New("catalyst requires -iosversion=13 or higher")
./cmd/gomobile/build.go://    ios,iossimulator,maccatalyst
./cmd/gomobile/doc.go:	gomobile bind [-target android|ios|iossimulator|macos|maccatalyst] [-bootclasspath <path>] [-classpath <path>] [-o output] [build flags] [package]
./cmd/gomobile/doc.go:either android (the default) or ios, iossimulator, macos, maccatalyst.
./cmd/gomobile/doc.go:	gomobile build [-target android|ios|iossimulator|macos|maccatalyst] [-o output] [-bundleid bundleID] [build flags] [package]
./cmd/gomobile/doc.go:default) or ios, iossimulator, macos, maccatalyst.
./cmd/gomobile/env.go:var darwinPlatforms = []string{"ios", "iossimulator", "macos", "maccatalyst"}
./cmd/gomobile/env.go:	case "macos", "maccatalyst":
./cmd/gomobile/env.go:	case "macos", "maccatalyst":
./cmd/gomobile/env.go:	case "maccatalyst":
./cmd/gomobile/env.go:		// TODO(ydnar): remove tag "ios" when cgo supports Catalyst
./cmd/gomobile/env.go:		return []string{"ios", "macos", "maccatalyst"}
./cmd/gomobile/env.go:			case "maccatalyst":

In summary, I’m happy to split this CL into two parts if you think it’s necessary:

1. Fat XCFrameworks, macOS support, test fixes
2. +18 lines for Catalyst support

[hajimehoshi]

I see your point. Given that x/mobile is outside of the Go core, and not subject to the same compatibility guarantee, what about marking Catalyst support as experimental?

I slightly tend to against it, but the Go team might allow this. CC @hyangah

[nadimkobeissi]

I strongly support @ydnar's proposed fix here.

[nadimkobeissi]

@hajimehoshi @hyangah @ydnar What's the status on this? :-)

[ydnar]

@nadimkobeissi

@hajimehoshi @hyangah @ydnar What's the status on this? :-)

You can use this branch now with a replace directive in your go.mod file. Otherwise I think we’re waiting for a response from @hyangah.

[hyangah]

This whole project is highly experimental - so it's fine to include the catalyst handling code IMO.

[ydnar]

@nadimkobeissi @iamcalledrob golang.org/cl/334689 was merged, so Catalyst and macOS support should be working again.

[changkun]

This is fixed in CL334689.