Implement AWS Fargate support [Large build support] #246
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
frostebite
changed the title
Codecov Report
@@ Coverage Diff @@
## main #246 +/- ##
===========================================
- Coverage 73.16% 60.52% -12.64%
===========================================
Files 19 20 +1
Lines 544 684 +140
Branches 97 134 +37
===========================================
+ Hits 398 414 +16
- Misses 145 269 +124
Partials 1 1
|
| "com.unity.modules.wind": "1.0.0", | ||
| "com.unity.modules.xr": "1.0.0" | ||
| } | ||
| "dependencies": {} |
There was a problem hiding this comment.
Just to make sure, is this intended?
There was a problem hiding this comment.
Yes. Not including these packages does not meaningfully change the ability to test building unity projects. But it does speed up the build time.
|
LGTM Please make sure |
Codecov Report
@@ Coverage Diff @@
## main #246 +/- ##
===========================================
- Coverage 73.16% 60.52% -12.64%
===========================================
Files 19 20 +1
Lines 544 684 +140
Branches 97 134 +37
===========================================
+ Hits 398 414 +16
- Misses 145 269 +124
Partials 1 1
|
szymon-kazmierczak
pushed a commit
to zendesk/unity-builder
that referenced
this pull request
Apr 24, 2021
* Implement AWS Fargate support * Update aws-tests workflow to include aws-ts-clean * support remoteBuildCpu and remoteBuildContainer parameters for aws * Syntax fix * remove package-lock add yarn.lock * yarn lock * if: github.event.pull_request.draft == false Co-authored-by: mdugdale <mark.dugdale@bossastudios.com>
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements support for AWS Fargate with Git LFS support.
What it does?
With this feature you can run builds on the AWS Cloud on demand. Most importantly you can run builds for very large projects.
Why we need it?
GitHub Actions run on build machines provided by GitHub. You can see info about the specifications of the build machines here:
https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
For game development and Unity projects the disc size in particular is quite small. You may well experience an error related to running out of disc space. You may also want to run the build on a server with more memory or CPU cores.
You can use this feature to send a build from any supported git platform to any of the supported remote build cluster types, now including AWS. This is an inexpensive and reliable way to scale to larger builds. There is virtually no limit to the disc size for the supported remote build cluster types.
To Do
Planned for later
Concept
You can run Game CI Unity builds on the supported platforms. You can also run the build on a remote build cluster such as Kubernetes or a Cloud Provider like AWS. This is a bit like self-hosting build servers except on demand.
The main reasons for doing this are reducing build machine costs, running builds too large for other options, nearly unlimited parallel builds and workflow improvements from further isolating the builds.
There can also be significant performance and cost improvements if you run several builds per commit (or any other event like a PR) because you never have to worry about blocking other builds with this approach.
Advantages
Disadvantage
Required Parameters
awsStackName : The base stack setup in the getting started section.
Container
remoteBuildMemory
remoteBuildCPU
Disc size is completely elastic there is no need to specify.
Build artifacts
Completed builds are zipped then uploaded to aws S3. Steam deployment is still work-in-progress.
Logs
Logs are streamed via Kinesis in buffers back to the GitHub runner to allow fast and stable log streaming, with no aws polling issues.
We also write the AWS Cloud Watch dashboard url to the GitHub output.
Cleaning up resources
Even in the event of an unexpected error we have a failsafe to make sure resources don't accidently remain active and charging you. This is accomplished with a lambda function that runs after a delay if the resources are still active.