runtime/cgo: Undefined symbols for architecture arm64: _darwin_arm_init_mach_exception_handler when building for Mac Catalyst

[ydnar]

What version of Go are you using (go version)?

$ go version
go version go1.16.5 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ydnar/Library/Caches/go-build"
GOENV="/Users/ydnar/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/ydnar/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ydnar/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.16.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16.5/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.5"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ydnar/development/alta/quic-go-mobile-example/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6g/1gng3zts0t39s_qbtt7p0wsc0000gn/T/go-build4118271977=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Build a cgo library using gomobile for Mac Catalyst (UIKit on macOS).

What did you expect to see?

A successful build into a framework, and then an XCFramework. (I’m working on #47212 to fix support for Catalyst in gomobile.)

What did you see instead?

Command

When running go build -buildmode-c-archive without -tags=ios, it fails with an error (Undefined symbols for architecture arm64).

The missing function is called here:
https://github.com/golang/go/blob/master/src/runtime/cgo/gcc_darwin_arm64.c

#if TARGET_OS_IPHONE
	darwin_arm_init_mach_exception_handler();
	darwin_arm_init_thread_exception_port();
	init_working_dir();
#endif

When building for Catalyst, TARGET_OS_IPHONE is defined, so it’s called, but darwin_arm_init_mach_exception_handler and darwin_arm_init_thread_exception_port aren’t declared.

When run with -tags=ios, it succeeds. I assume because this file gets compiled in:
https://github.com/golang/go/blob/master/src/runtime/cgo/gcc_signal_ios_nolldb.c

Why is this a problem? It seems confusing to supply the ios build tag for a macOS target.

Catalyst is essentially iOS-on-macOS, but it’s definitely macOS. It uses GOOS=darwin, and not GOOS=ios, and has access to both UIKit and AppKit. It has some other differences from iOS, like access to OpenGL instead of OpenGLES.

[lots of env omitted]
go build -x -buildmode=c-archive -o $WORK/EchoGo-maccatalyst-arm64.a ./gobind

Partial Output

TERM='dumb' /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -I /usr/local/Cellar/go/1.16.5/libexec/src/runtime/cgo -fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK/b082=/tmp/go-build -gno-record-gcc-switches -fno-common -o $WORK/b082/_cgo_.o $WORK/b082/_cgo_main.o $WORK/b082/_x001.o $WORK/b082/_x002.o $WORK/b082/_x003.o $WORK/b082/_x004.o $WORK/b082/_x005.o $WORK/b082/_x006.o $WORK/b082/_x007.o $WORK/b082/_x008.o $WORK/b082/_x009.o -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -target arm64-apple-ios13.0-macabi -fembed-bitcode -arch arm64 -framework CoreFoundation
# runtime/cgo
Undefined symbols for architecture arm64:
  "_darwin_arm_init_mach_exception_handler", referenced from:
      _x_cgo_init in _x004.o
  "_darwin_arm_init_thread_exception_port", referenced from:
      _threadentry in _x004.o
      _x_cgo_init in _x004.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

cc @hajimehoshi

[cherrymui]

Build a cgo library using gomobile for Mac Catalyst (UIKit on macOS).

It sounds like you are building for a platform (Mac Catalyst) which we have not supported before. If you are requesting a new platform to support, it may be better to open an issue to discuss that.

It seems you're using gomobile to build for macOS? Why? I don't think macOS is a mobile platform. Thanks.

[ydnar]

Build a cgo library using gomobile for Mac Catalyst (UIKit on macOS).

It sounds like you are building for a platform (Mac Catalyst) which we have not supported before. If you are requesting a new platform to support, it may be better to open an issue to discuss that.

It seems you're using gomobile to build for macOS? Why? I don't think macOS is a mobile platform. Thanks.

golang.org/x/mobile implements bindings for Objective-C and Java to Go. macOS is an Objective-C platform, and macOS is a supported Go platform. Catalyst is a variant of macOS.

[cherrymui]

As far as I can tell, x/mobile's documentation does not mention macOS nor Catalyst. So this is entering undocumented territory. And I also don't think Go runtime code was written with Catalyst support in mind, and we have no test for it.

I still suggest you open some discussion about adding support of a new platform.

cc @hyangah @eliasnaur

[ydnar]

As far as I can tell, x/mobile's documentation does not mention macOS nor Catalyst. So this is entering undocumented territory. And I also don't think Go runtime code was written with Catalyst support in mind, and we have no test for it.

I still suggest you open some discussion about adding support of a new platform.

That’s because I’m working on macOS and Catalyst support in x/mobile.

Where’s the best place/way to open a discussion about a new platform?

[rsc]

It sounds like Apple is treating Catalyst as an iOS variant, more or less. If there's just a little bit of glue in gomobile to make that work, and maybe a symbol or two in runtime/cgo, then it seems fine - at least for now - to keep using GOOS=ios and see how things go. If there are significant changes required in the main tree, then we should reconsider.

[ydnar]

Now that https://golang.org/cl/334689 is merged, gomobile supports Mac Catalyst, including Ivy. See https://golang.org/cl/350489.

It sounds like Apple is treating Catalyst as an iOS variant, more or less. If there's just a little bit of glue in gomobile to make that work, and maybe a symbol or two in runtime/cgo, then it seems fine - at least for now - to keep using GOOS=ios and see how things go. If there are significant changes required in the main tree, then we should reconsider.

The changes were pretty minimal to support Catalyst, namely GOOS=darwin with some specific build tags. I think we can revisit if folks want to add explicit support for Catalyst to Go.

[tmm1]

I ran into this issue as well, when porting an iOS app that uses a golang c-archive over to an M1 mac using -target aarch64-apple-ios-macabi

Here is what I observed:

• with GOOS=darwin CGO_CFLAGS="-target aarch64-apple-ios-macabi" go build -buildmode=c-archive -tags=ios

  • build succeeds
  • crash on boot: unexpected return pc for runtime.sigtramp

• removing -tags=ios

  • build failure as above for undefined symbols

I was able to fix these issues with b26d786

EDIT: Seems like GOOS=ios also works, without any patches to golang. I'm not sure if GOOS=darwin vs GOOS=ios is preferable for mac catalyst

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)