Running the Unity builder action on a self-hosted M1 Mac #336
Comments
davidmfinol
added
help wanted
|
Yes, it should now be possible to run on mac machines. However, it's unclear if it works across all macs, or just intel, or just apple silicon. @vinaysshenoy Have you tried it out? Please let us know if you are able to try it out and if it works or doesn't for you. |
|
We tried it out on an M1 Mac self-hosted runner, and it worked! We needed to do a little work around installing some software on the runner (like The difference is massive. We went from needing ~20m to build on the GitHub Ubuntu runners to ~3m on the M1 mac. This is with the Library and LFS caching on both runners. |
|
Awesome! Closing this issue then. |
|
Hey @vinaysshenoy Sorry to ping you here, can you remember what you had to do to make it work on self hosted runner ? Having big troubles here.. |
|
Hey! We actually stopped using GameCI and just started using Unity directly via the Editor Command line. The GameCI code was a big help in getting it setup. We set up an M1 Mac Mini with Unity on it and things are really decent now. Builds are significantly faster than the default GitHub runners, although the machine does require a little babysitting now and then with macOS updates. |
|
What troubles are you having, specifically? Maybe I can help. |
|
@vinaysshenoy are you be able to (partially) share your solution so we can learn from it? |
|
Yup, let me take a look at our setup and see what is shareable. |
|
@webbertakken @antoiner77 The way we have set it up is that the core builder workflow is extracted into a discrete file, which is then invoked in the primary workflow files with parameters specific to the platform we are building on. This is what our core unity build workflow looks like: name: Build Unity3D application
on:
workflow_call:
inputs:
buildTarget:
required: true
type: string
buildType:
required: true
type: string
buildOutputDir:
required: true
type: string
buildPlatformResources:
required: true
type: string
buildArtifactQualifier:
required: true
type: string
buildName:
required: true
type: string
secrets:
unityEmail:
required: true
unityPassword:
required: true
unitySerial:
required: true
qweebiApiToken:
required: true
acceleratorEndpoint:
required: true
outputs:
buildId:
description: "The generated build ID"
value: ${{ jobs.build_project.outputs.buildId }}
artifactName:
description: "The generated artifact name uploaded for packaging"
value: ${{ jobs.build_project.outputs.artifactName }}
buildVersion:
description: "The app version read from config"
value: ${{ jobs.build_project.outputs.buildVersion }}
jobs:
build_project:
name: Build the Qweebi MVP application
runs-on: [self-hosted, macOS]
outputs:
buildId: ${{ steps.record_step_outputs.outputs.buildId }}
artifactName: ${{ steps.record_step_outputs.outputs.artifactName }}
buildVersion: ${{ steps.record_step_outputs.outputs.buildVersion }}
env:
BUILD_TARGET: ${{ inputs.buildTarget }}
BUILD_NAME: ${{ inputs.buildName }}
BUILD_TYPE: ${{ inputs.buildType }}
BUILD_OUTPUT_DIR: ${{ inputs.buildOutputDir }}
BUILD_PLATFORM_RESOURCES: ${{ inputs.buildPlatformResources }}
BUILD_ARTIFACT_QUALIFIER: ${{ inputs.buildArtifactQualifier }}
BUILD_NUMBER: ${{ github.run_number }}
QWEEBI_API_TOKEN: ${{ secrets.qweebiApiToken }}
steps:
- name: Clean repository
run: |
git clean -fdx --exclude="*Library/" && git reset --hard HEAD
# Checkout the repository
- name: Checkout repository
uses: actions/checkout@v2
with:
lfs: true
clean: false
- name: Read app version from config
run: |
appVersion=$(jq -r '.appVersion' build-config/release/config.json)
commitHash=$(echo ${{ github.sha }} | cut -c 1-16)
buildId="$appVersion.$commitHash.${{ github.run_id }}"
echo "App version: $appVersion, Build ID: $buildId"
echo "APP_VERSION=$appVersion" >> $GITHUB_ENV
echo "BUILD_ID=$buildId" >> $GITHUB_ENV
echo "ARTIFACT_NAME=dist-$BUILD_ARTIFACT_QUALIFIER-$buildId" >> $GITHUB_ENV
# Prepare the project for release
- name: Prepare release build
if: ${{ env.BUILD_TYPE == 'release' }}
run: |
echo $(jq --arg buildId $BUILD_ID '. + { "buildId": $buildId }' build-config/release/config.json) > ./build-config/release/config.json
echo $(jq -s '.[0] * .[1]' qweebi-unity/Assets/Resources/Config/config.json build-config/release/config.json) > ./qweebi-unity/Assets/Resources/Config/config.json
jq '.' qweebi-unity/Assets/Resources/Config/config.json
# Decide custom build parameters based on build type
- name: Generate custom parameters
id: generate_custom_parameters
run: |
./build-config/scripts/generate_custom_build_parameters.sh $BUILD_TYPE
- name: Install required software
run: |
arch -arm64 brew install jq gh
- name: Set Unity3D editor path
run: |
./build-config/scripts/extract_unity_version.sh ./qweebi-unity/ProjectSettings/ProjectVersion.txt
- name: Build project
run: |
${UNITY_EDITOR_PATH}/Unity \
-batchmode \
-EnableCacheServer -cacheServerEndpoint ${{ secrets.acceleratorEndpoint }} -cacheServerEnableDownload -cacheServerEnableUpload \
-projectPath ./qweebi-unity \
-executeMethod UnityBuilderAction.BuildScript.Build -buildTarget $BUILD_TARGET \
-customBuildPath "$(pwd)/build/$BUILD_TARGET/$BUILD_NAME" \
${{ steps.generate_custom_parameters.outputs.custom_parameters }} \
-logFile - \
-quit
# Needed to retain the executable flag on the binary
- name: Zip distribution for signing
run: |
mkdir -p "$BUILD_OUTPUT_DIR/build_resources"
cp -r "./build/$BUILD_TARGET/." "$BUILD_OUTPUT_DIR"
cp -r "$BUILD_PLATFORM_RESOURCES/." "$BUILD_OUTPUT_DIR/build_resources"
zip -r "./qweebi-dist-unsigned.zip" "$BUILD_OUTPUT_DIR"
- name: Record workflow outputs
id: record_step_outputs
run: |
echo "::set-output name=buildId::$BUILD_ID"
echo "::set-output name=artifactName::$ARTIFACT_NAME"
echo "::set-output name=buildVersion::$APP_VERSION"
# Output (Retain only for 1 day since we want the final signed and notarized app to be the final artifact)
- name: Upload unsigned artifact for signing
uses: actions/upload-artifact@v2
with:
name: ${{ env.ARTIFACT_NAME }}
path: ./qweebi-dist-unsigned.zip
retention-days: 1The Here's our workflow script which we use to upload our builds to the App Store, as a reference: name: Build macOS application for App Store deployment
on:
workflow_dispatch:
inputs:
buildType:
description: 'Build Type'
required: true
default: 'release'
type: choice
options:
- development
- release
jobs:
build_project:
name: Build the Qweebi MVP application
uses: ./.github/workflows/_build_unity3d_app.yml
with:
buildTarget: StandaloneOSX
buildName: "Qweebi - Virtual Makerspace"
buildType: ${{ github.event.inputs.buildType }}
buildOutputDir: ./dist
buildPlatformResources: ./build-config/release/macOS
buildArtifactQualifier: macos
secrets:
unityEmail: ${{ secrets.UNITY_EMAIL }}
unityPassword: ${{ secrets.UNITY_PASSWORD }}
unitySerial: ${{ secrets.UNITY_SERIAL }}
qweebiApiToken: ${{ secrets.QWEEBI_API_TOKEN }}
acceleratorEndpoint: ${{ secrets.UNITY_ACCELERATOR_ENDPOINT }}
upload_app:
name: Upload the package to the App Store
needs: build_project
runs-on: [self-hosted, macOS]
if: ${{ github.event.inputs.buildType == 'release' }}
env:
MACOS_CERTIFICATE_PWD: ${{ secrets.MACOS_DISTRIBUTE_CERTIFICATE_PWD }}
KEYCHAIN_NAME: qweebi-codesign.keychain
KEYCHAIN_PWD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
ARTIFACT_NAME: ${{ needs.build_project.outputs.artifactName }}
BUILD_ID: ${{ needs.build_project.outputs.buildId }}
ALTOOL_USERNAME: ${{ secrets.MACOS_APPSTORE_ALTOOL_USERNAME }}
ALTOOL_PASSWORD: ${{ secrets.MACOS_APPSTORE_ALTOOL_PASSWORD }}
PROVISIONING_PROFILE_ID: <PROVISIONING PROFILE ID>
APPSTORE_TEAM_ID: <APPSTORE TEAM ID>
BUNDLE_CERT: <DIST CERT NAME>
INSTALLER_CERT: <INSTALLER CERT NAME>
steps:
- name: Download built application
uses: actions/download-artifact@v2
with:
name: ${{ env.ARTIFACT_NAME }}
- name: Extract unsigned application
run: |
unzip ./qweebi-dist-unsigned.zip -d ./dist_out
ls -a ./dist_out
- name: Download and extract signing certificates
run: |
wget -O certs.7z "${{ secrets.CERT_ARCHIVE_URL }}"
7z -p"${{ secrets.CERT_ARCHIVE_DECRYPTION_PASSWORD }}" x certs.7z -aoa
find ./certs
- name: Copy the provisioning profile to the system profiles directory
run: |
mkdir -p ~/Library/MobileDevice/Provisioning Profiles
mv "./certs/macOS/appstore.provisionprofile" ~/Library/MobileDevice/"Provisioning Profiles"/$PROVISIONING_PROFILE_ID.provisionprofile
- name: Prepare signing keychain
id: prepare-signing-keychain
run: |
security create-keychain -p $KEYCHAIN_PWD $KEYCHAIN_NAME
security list-keychains -d user -s "${KEYCHAIN_NAME}" $(security list-keychains -d user | tr -d '"')
security unlock-keychain -p $KEYCHAIN_PWD $KEYCHAIN_NAME
security import ./certs/macOS/appstore_dist.p12 -k $KEYCHAIN_NAME -P $MACOS_CERTIFICATE_PWD -T /usr/bin/productbuild -T /usr/bin/codesign -T /usr/bin/xcodebuild
security import ./certs/macOS/appstore_installer.p12 -k $KEYCHAIN_NAME -P $MACOS_CERTIFICATE_PWD -T /usr/bin/productbuild -T /usr/bin/codesign -T /usr/bin/xcodebuild
security set-key-partition-list -S apple-tool:,apple:,productsign:,codesign:,xcodebuild:,productbuild: -s -k $KEYCHAIN_PWD $KEYCHAIN_NAME
security default-keychain -s $KEYCHAIN_NAME
security find-identity -p codesigning -v
security default-keychain
- name: Build the XCode archive
run: |
cp ./dist_out/dist/build_resources/qweebi.entitlements ./dist_out/dist/"Qweebi - Virtual Makerspace"/
xcodebuild archive \
-project ./dist_out/dist/"Qweebi - Virtual Makerspace"/"Qweebi - Virtual Makerspace.xcodeproj" \
-scheme "Qweebi - Virtual Makerspace" \
-configuration "Release" \
-destination 'generic/platform=macOS' \
-archivePath ./dist_out/dist/build/"Qweebi - Virtual Makerspace.xcarchive" \
CODE_SIGN_ENTITLEMENTS=./qweebi.entitlements \
CODE_SIGNING_ALLOWED=YES \
CODE_SIGN_IDENTITY="$BUNDLE_CERT" \
CODE_SIGN_STYLE="Manual" \
CODE_SIGN_KEYCHAIN=$KEYCHAIN_NAME \
DEVELOPMENT_TEAM=$APPSTORE_TEAM_ID \
PROVISIONING_PROFILE_SPECIFIER="$PROVISIONING_PROFILE_ID" \
OTHER_CODE_SIGN_FLAGS="--options runtime --keychain $KEYCHAIN_NAME --verbose"
- name: Generate env variables
run: |
echo "PACKAGE_NAME=Qweebi - Virtual Makerspace.pkg" >> $GITHUB_ENV
echo "EXPORT_OPTIONS_PLIST=./dist_out/dist/build_resources/export.plist" >> $GITHUB_ENV
- name: Update export options plist
run: |
export PATH="/usr/libexec:$PATH"
plistbuddy -c "Add :teamID string $APPSTORE_TEAM_ID" $EXPORT_OPTIONS_PLIST
plistbuddy -c "Add :signingCertificate string \"$BUNDLE_CERT\"" $EXPORT_OPTIONS_PLIST
plistbuddy -c "Add :installerSigningCertificate string \"$INSTALLER_CERT\"" $EXPORT_OPTIONS_PLIST
plistbuddy -c "Add :provisioningProfiles dict" $EXPORT_OPTIONS_PLIST
plistbuddy -c "Add :provisioningProfiles:com.virtuallearning.makerspace string $PROVISIONING_PROFILE_ID" $EXPORT_OPTIONS_PLIST
plistbuddy -c "Print" $EXPORT_OPTIONS_PLIST
- name: Export archive
run: |
xcodebuild -exportArchive \
-archivePath ./dist_out/dist/build/"Qweebi - Virtual Makerspace.xcarchive" \
-exportOptionsPlist $EXPORT_OPTIONS_PLIST \
-exportPath ./dist_out/dist/build/
pkgutil --check-signature "./dist_out/dist/build/$PACKAGE_NAME"
- name: Upload archive to App Store
run: |
xcrun altool --upload-app -f "./dist_out/dist/build/$PACKAGE_NAME" -t osx -u $ALTOOL_USERNAME -p $ALTOOL_PASSWORD --output-format json
- name: Revert to default keychain
if: always() && steps.prepare-signing-keychain.outcome == 'success'
run: |
security default-keychain -s login.keychain
security delete-keychain $KEYCHAIN_NAMEFor completeness, here are the other shell scripts we use in the workflows:
#!/bin/bash
echo "Read Unity3D version from $1"
unityVersion=$(head -n1 $1 | cut -d ' ' -f 2 | xargs)
echo "Project Unity3D version: $unityVersion"
echo "UNITY_EDITOR_PATH=/Applications/Unity/Hub/Editor/$unityVersion/Unity.app/Contents/MacOS" >> $GITHUB_ENV
echo "Setting build type to $1"
if [ $1 == "release" ]
then
echo "custom_parameters=-buildVersion $APP_VERSION -buildNumber $BUILD_NUMBER -iOSProvisioningProfile $IOS_TEST_PROVISIONING_PROFILE_ID -qweebiApiToken $QWEEBI_API_TOKEN" >> $GITHUB_OUTPUT
else
echo "custom_parameters=-Development -buildVersion $APP_VERSION -buildNumber $BUILD_NUMBER -iOSProvisioningProfile $IOS_TEST_PROVISIONING_PROFILE_ID" >> $GITHUB_OUTPUT
fiWe switched to building an XCode archive instead of an app bundle for mac because it was just simpler to let XCode manage signing and notarization for us. This requires a bit more effort to implement since it's a bit invasive and deeply tied to the macos ecosystem and workflow, so we can move that conversation to a separate topic if it's something that might be useful since it's specific to macos built targets only. This has mostly been working well for us, but there are some annoyances with occasionally managing the keychain for the mac machine, as well as needing to manually restart it occasionally. Would not recommend going this route unless you have physical access to the device and can secure it. Let me know if there is anything else that I can help with. Note that we moved away from GameCI mostly because it was faster since we could have Unity preinstalled on a runner and didn't need to spend time downloading it all over again. Cannot think of any other reason at the moment. Although we have done a lot of stuff after that may not be possible with GameCI that is with a self hosted machine. Nothing jumps to my mind specifically at the moment. |
The stable release of the
unity-builderaction only supports building on Ubuntu runners, as far as I know. We currently have an M1 mac machine which we want to set up as a self-hosted runner for running our builds, partly because the GitHub runners are quite slow, and partly because we've started running out of space on their machines when we try to build our projects.With the merge of #326, does this mean that we can use the runner to build our projects on Mac machines? We do not need IL2CPP for now, as long we can continue to build Mono projects, that works for us as well.
The text was updated successfully, but these errors were encountered: