Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Missing documentation/examples #3340

Open
mjudeikis opened this issue Jun 20, 2023 · 11 comments
Open

Missing documentation/examples #3340

mjudeikis opened this issue Jun 20, 2023 · 11 comments

Comments

@mjudeikis
Copy link

Would be nice to have few examples docs/examples how one could run all these pipelines outside VSTS and build their own images. For now this project looks like one-leg-opensource, where external consumer would have challenges to re-use it.

@alexeldeib
Copy link
Contributor

admittedly this repo is heavily tied to AKS implementation today. it forked from aks-engine which is/was a much more community driven project. these days we'd point people to Cluster API Provider for Azure, which uses image-builder for reference images:

https://github.com/kubernetes-sigs/cluster-api-provider-azure
https://github.com/kubernetes-sigs/image-builder/tree/main/images/capi/packer/azure

we definitely can use better docs either way, however :)

@mjudeikis
Copy link
Author

Any plans to unify image build process?

@alexeldeib
Copy link
Contributor

not at the moment. there are some quirks, e.g. CAPI builds VM image per version while we have one mega VM image with as much crammed in as possible.

what's your goal exactly?

@mjudeikis
Copy link
Author

Im just exploring what it would take to make bottlerocket running for AKS, so exploring all building blocks in the chain :)

@alexeldeib
Copy link
Contributor

Hmm. So a few thoughts.

Do you really want it in AKS, or Azure? I see you mentioned both CoreOS and AKS-Engine in other issues. If you're fine self-managing, CAPI/CAPZ would allow you to use any images today.

You still have to do the legwork to ensure the OS is compatible. I've done this in the past for Talos Linux and I can't imagine it would be much harder for bottlerocket. bottlerocket-os/bottlerocket#1114 (comment) is very helpful in that vein.

From the Azure side, you'd want to ensure you fill the platform functionality requirements provided by the guest agent (for example, Talos doesn't use one, Ubuntu and Azure Linux and most distros use https://github.com/Azure/WALinuxAgent):
https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/agent-linux
https://learn.microsoft.com/en-us/azure/virtual-machines/linux/no-agent

For AKS, I'm not sure investing to support Bottlerocket as a managed OS makes a ton of sense, especially with the backdrop of existing work on AzLinux/Mariner https://github.com/microsoft/CBL-Mariner.

But even that can't really stop you, you can join whatever you want to the AKS control plane, it's just not supported: https://gist.github.com/alexeldeib/2a788d64d3229fe578f552e2239ce934

So...to me the obvious thing would be someone takes Sam's helpful comment from bottlerocket, tries to standup plain VMs in Azure, and if all goes well you could build CAPI reference bottlerocket images for Azure and use those just like the old AKS-Engine days.

@mjudeikis
Copy link
Author

Ou, https://gist.github.com/alexeldeib/2a788d64d3229fe578f552e2239ce934 this is so nice write-up. I might be poking devil in the eye, but do you have something similar for windows nodes by any chance? :)

@alexeldeib
Copy link
Contributor

unfortunately I do not :( I am not as familiar with windows, although something similar should be possible

@AbelHu
Copy link
Member

AbelHu commented Jul 25, 2023

@mjudeikis
Copy link
Author

I can see you recently added need to MSI and windows_private_packages_url in your build.
does this means windows build are no more reproducible outside Microsoft infrastructure/CICD?

We would like to get windows self-managed hosts to join our AKS cluster using this.

@alexeldeib @junjiezhang1997 ?

@AbelHu
Copy link
Member

AbelHu commented Nov 10, 2023

I can see you recently added need to MSI and windows_private_packages_url in your build. does this means windows build are no more reproducible outside Microsoft infrastructure/CICD?

We would like to get windows self-managed hosts to join our AKS cluster using this.

@alexeldeib @junjiezhang1997 ?

@mjudeikis we need to test or build windows vhds for some urgent fixes. Those urls will be public finally.

@mjudeikis
Copy link
Author

Is it possible not to block build of these urls and msi is not available? Just don't use them if not provided.

This way builds works without them too.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants