-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
✨(catalog:ansible): Add ansible role to create a single-node Kubernet…
…es cluster on PVE (#205) Signed-off-by: Alexandre Nicolaie <xunleii@users.noreply.github.com>
- Loading branch information
Showing
14 changed files
with
566 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
# pve.kairos-cluster | ||
|
||
This role sets up and configures a Kairos cluster on Proxmox Virtual Environment (PVE). | ||
|
||
> \[!CAUTION] | ||
> This role has only been tested for a single-node cluster and may not work as | ||
> expected for multi-node clusters. | ||
## Requirements | ||
|
||
* Proxmox VE 8.3 or higher | ||
* Ansible 2.16 or higher | ||
* Python 3.6 or higher on the control node | ||
* `community.general` collection for additional modules | ||
* `proxmoxer` and `requests-toolbelt` Python packages on the control node | ||
|
||
## Role variables | ||
|
||
### Required variables | ||
|
||
* *`kairos_cluster_name`*: Name of the Kairos cluster. (**required**) | ||
* *`proxmox_host`*: Host address of the Proxmox server. (**required**) | ||
* *`proxmox_user`*: Username for Proxmox authentication. (**required**) | ||
|
||
### Optional variables | ||
|
||
* *`kairos_cloudinit_hostname_prefix`*: Prefix for the hostname name. (*default: `krs`*) | ||
* *`kairos_cloudinit_k3s`*: k3s configuration. See the [k3s documentation](https://kairos.io/docs/reference/configuration/#k3s-settings) for more information. | ||
* *`kairos_cloudinit_user_sshkeys`*: SSH keys assigned to the kairos user. (*default: `[]`*) | ||
* *`kairos_cpu_cores`*: Number of CPU cores to allocate to the VM. (*default: `4`*) | ||
* *`kairos_cpu_type`*: Type of CPU to use. (*default: `host`*) | ||
* *`kairos_disk_enable_iothread`*: Enable IO thread for disk. (*default: `false`*) | ||
* *`kairos_disk_size`*: Size of the main disk, in gigabyte. (*default: `32`*) | ||
* *`kairos_distribution`*: OS distribution on which Kairos is based (`alpine`, `debian`, `ubuntu`, ...). (*default: `debian-bookworm`*) | ||
* *`kairos_memory_balloon`*: Memory ballooning setting. `0` to disable ballooning. (*default: `0`*) | ||
* *`kairos_memory_shares`*: Memory shares setting when ballooning is enabled. (*default: `1024`*) | ||
* *`kairos_memory`*: Amount of memory to allocate to the VM, in megabyte. (*default: `4096`*) | ||
* *`kairos_network_bridge`*: Network bridge to use. (*default: `vmbr0`*) | ||
* *`kairos_network_mtu`*: Network MTU setting. (*optional*) | ||
* *`kairos_network_vlan`*: Network VLAN setting. (*optional*) | ||
* *`kairos_start_on_boot_order`*: Boot order when the hypervisor starts. (*optional*) | ||
* *`kairos_start_on_boot`*: Start automatically when the hypervisor starts. (*default: `false`*) | ||
* *`kairos_version`*: Version of Kairos. (*default: `latest`*) | ||
* *`kairos_vm_id`*: VM ID. (*default: the first available*) | ||
* *`proxmox_disk_storage`*: Proxmox storage name where the VM disk will be stored. (*default: `local-lvm`*) | ||
* *`proxmox_iso_storage`*: Proxmox storage name where the VM ISO will be stored. (*default: `local`*) | ||
* *`proxmox_password`*: Password for Proxmox. (*required if `proxmox_token_id` and `proxmox_token_secret` are not set*) | ||
* *`proxmox_token_id`*: Token ID for Proxmox. (*required if `proxmox_password` is not set*) | ||
* *`proxmox_token_secret`*: Token secret for Proxmox. (*required if `proxmox_password` is not set*) | ||
|
||
### Kairos bundles variables | ||
|
||
> \[!NOTE] | ||
> This role supports the installation of Kairos bundles. A bundle is a set of software packages | ||
> that can be installed on a Kairos cluster. The role will configure the cloud-init script to | ||
> configure Kairos with the specified bundles. | ||
* *`kairos_bundles_<bundle_name>.description`*: Description of the bundle that will be displayed in Proxmox, on the VM notes. (*optional*) | ||
* *`kairos_bundles_<bundle_name>.url`*: Bundle documentation URL that will be displayed in Proxmox, on the VM notes. (*optional*) | ||
* *`kairos_bundles_<bundle_name>.targets`*: List of URLs to download the bundle. (*required*) | ||
* *`kairos_bundles_<bundle_name>_config`*: Bundle configuration (*optional*). | ||
|
||
For example, to install the `flux` bundle, you can use the following configuration: | ||
|
||
```yaml | ||
kairos_bundles_flux: | ||
description: Open and extensible continuous delivery solution for Kubernetes. | ||
url: https://github.com/kairos-io/community-bundles/tree/main/flux | ||
targets: | ||
- run://ghcr.io/chezmoi-sh/kairos-bundles:flux-sync # sync FluxCD configuration | ||
kairos_bundles_flux_config: | ||
git: | ||
url: https://github.com/fluxcd/flux2-monitoring-example.git | ||
path: clusters/test | ||
branch: main | ||
``` | ||
... that will be added to the cloud-init script as follows: | ||
```yaml | ||
bundles: | ||
- targets: | ||
- run://ghcr.io/chezmoi-sh/kairos-bundles:flux-sync | ||
|
||
flux: | ||
git: | ||
url: https://github.com/fluxcd/flux2-monitoring-example.git | ||
path: clusters/test | ||
branch: main | ||
``` | ||
## Dependencies | ||
This role depends on the following collections: | ||
* `community.general` | ||
|
||
## License | ||
|
||
This role is licensed under the Apache License, Version 2.0. See [LICENSE](../../../LICENSE) for the full license text. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
--- | ||
# - Proxmox VE global configuration | ||
# proxmox_host: # Hostname or IP address of the Proxmox VE node | ||
# proxmox_user: # Username to authenticate with the Proxmox VE API | ||
# proxmox_password: # Password to authenticate with the Proxmox VE API | ||
# proxmox_token_id: # API token ID to authenticate with the Proxmox VE API | ||
# proxmox_token_secret: # API token secret to authenticate with the Proxmox VE API | ||
proxmox_iso_storage: local # Storage name where all ISO images are stored | ||
proxmox_disk_storage: local-lvm # Storage name where all VM disks are stored | ||
|
||
# - Proxmox VE VM configuration | ||
kairos_cpu_type: host # Best performance for single node Proxme VE or if all Proxmox VE nodes have the same CPU | ||
kairos_cpu_cores: 4 # 4 cores seems to be a good balance between performance and resource usage for most VMs | ||
kairos_memory: 4096 # 4 GiB of memory is a good starting point for a Kubernetes node | ||
kairos_memory_balloon: 0 # Disable memory ballooning by default | ||
kairos_memory_shares: 1024 # Give the VM a fair share of memory by default | ||
kairos_disk_size: 32 # 32 GiB of disk space is a good starting point for a Kubernetes node (image + data) | ||
kairos_disk_enable_iothread: false # Disable I/O threads for better performance (the use of aio=threads is a legacy low-performance "aio" mechanism for QEMU|https://forum.proxmox.com/threads/proxmox-ve-7-2-benchmark-aio-native-io_uring-and-iothreads.116755/) | ||
kairos_network_bridge: vmbr0 # Default bridge for most Proxmox VE installations | ||
# kairos_network_mtu: # No MTU set by default | ||
# kairos_network_vlan: # No VLAN set by default | ||
kairos_start_on_boot: false # Do not start the VM automatically when the Proxmox VE node boots | ||
# kairos_start_on_boot_order: # Not set by default | ||
# kairos_vm_id: # Not set by default, the next available VM ID will be used | ||
|
||
# - Kairos OS configuration | ||
# renovate: datasource=github-releases lookupName=kairos-io/kairos | ||
kairos_version: v3.2.4 | ||
kairos_distribution: debian-bookworm | ||
# kairos_cluster_name: # Not set by default | ||
|
||
# - Kairos community bundles configuration | ||
kairos_bundles_suc: # System-Upgrade-Controller (https://github.com/rancher/system-upgrade-controller) | ||
description: General-purpose, Kubernetes-native upgrade controller (for nodes) | ||
url: https://github.com/kairos-io/community-bundles/tree/main/system-upgrade-controller | ||
targets: | ||
- run://quay.io/kairos/community-bundles:system-upgrade-controller_latest | ||
|
||
# kairos_bundles_flux: # Flux CD (https://fluxcd.io/) | ||
# description: Open and extensible continuous delivery solution for Kubernetes. | ||
# url: https://github.com/kairos-io/community-bundles/tree/main/flux | ||
# targets: | ||
# - run://quay.io/kairos/community-bundles:flux_latest # bootstrap FluxCD | ||
# - run://ghcr.io/chezmoi-sh/kairos-bundles-dev/flux-sync:latest # sync FluxCD configuration | ||
# kairos_bundles_flux_config: # Flux CD (https://fluxcd.io/) | ||
# git: | ||
# url: ... | ||
# ... read the bundle documentation for more details | ||
|
||
# WARN: DO NOT CHANGE THE FOLLOWING VARIABLES UNLESS YOU KNOW WHAT YOU ARE DOING. | ||
# THESE VARIABLES ARE USED TO PREPARE ALL THE KAIROS CLOUD-INIT CONFIGURATION. | ||
x_kairos_bundles_names: "{{ lookup('ansible.builtin.varnames', '^kairos_bundles_(?!.*_config$).+') | split(',') | map('regex_replace', 'kairos_bundles_', '') | list }}" | ||
|
||
# - Kairos cloud-init configuration | ||
kairos_cloudinit_hostname_prefix: krs | ||
kairos_cloudinit_user_sshkeys: [] | ||
|
||
kairos_cloudinit_k3s: | ||
args: | ||
- --disable=traefik |
Empty file.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
galaxy_info: | ||
author: Alexandre Nicolaie <xunleii@users.noreply.github.com> | ||
description: Ansible role to deploy a Kairos cluster on Proxmox VE | ||
|
||
issue_tracker_url: https://github.com/chezmoi-sh/atlas/issues | ||
license: Apache-2.0 | ||
|
||
min_ansible_version: 2.1 | ||
|
||
# If this a Container Enabled role, provide the minimum Ansible Container version. | ||
# min_ansible_container_version: | ||
|
||
galaxy_tags: | ||
- k3s | ||
- kairos | ||
- kubernetes |
156 changes: 156 additions & 0 deletions
156
catalog/ansible/pve.kairos-cluster/tasks/create-pve-vm.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
--- | ||
- name: 🛠️ Create the cluster instance pool on PVE | ||
community.general.proxmox_pool: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
poolid: "{{ kairos_cluster_name }}" | ||
comment: Kairos cluster instance pool for the {{ kairos_cluster_name }} cluster | ||
|
||
- name: 🛠️ Initiate the Kairos VM on the Proxmox VE cluster | ||
community.general.proxmox_kvm: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
node: "{{ proxmox_node | default(pve_nodes | first) }}" | ||
state: present | ||
|
||
# Static VM configuration | ||
acpi: true # Enable ACPI to allow graceful shutdown from Proxmox VE | ||
agent: true # Enable QEMU Guest Agent for better integration with Proxmox VE (qemu-guest-agent must be installed during the VM creation) | ||
bios: seabios # Use SeaBIOS as the BIOS firmware (EFI is not required) | ||
boot: order=scsi0;ide0 # Boot order of the VM (Kairos VM will boot from the first hard disk and then the first CD-ROM drive) | ||
|
||
ostype: l26 # Operating system type (Linux 2.6+) | ||
sockets: 1 # Number of CPU sockets to emulate (don't need more than 1) | ||
pool: "{{ kairos_cluster_name }}" | ||
|
||
name: "{{ kairos_cloudinit_hostname_prefix }}0.{{ kairos_cluster_name }}" # Name of the VM | ||
description: | | ||
<div align='center'> | ||
<a href='https://kairos.io/' target='_blank' rel='noopener noreferrer'> | ||
<img src='https://raw.githubusercontent.com/kairos-io/kairos-docs/f4e35110ef5b52eaf8ba0727585828f7a0aaad48/static/favicons/android-chrome-192x192.png'/> | ||
</a> | ||
# kairos {{ kairos_version }} _({{ kairos_distribution }})_ | ||
### _for [{{ kairos_cluster_name }}] cluster_ | ||
</div> | ||
--- | ||
### Enabled community bundles | ||
{% for bundle in x_kairos_bundles_names %} | ||
{% set bundle_info = lookup('ansible.builtin.vars', 'kairos_bundles_' + bundle) %} | ||
{% if bundle_info.url is defined and bundle_info.description is defined %} | ||
- **[{{ bundle }}]({{ bundle_info.url }})**: {{ bundle_info.description }} | ||
{% else %} | ||
- **{{ bundle }}** | ||
{% endif %} | ||
{% endfor %} | ||
tags: | ||
- kairos | ||
- "{{ kairos_distribution }}" | ||
- "{{ kairos_version }}" | ||
|
||
# User-defined VM configuration | ||
cpu: "{{ kairos_cpu_type }}" # CPU type to emulate (default host) | ||
cores: "{{ kairos_cpu_cores }}" # Number of CPU cores to allocate to the VM (default 4) | ||
vcpus: "{{ kairos_cpu_cores }}" # Number of virtual CPUs to allocate to the VM (same as cores) | ||
|
||
memory: "{{ kairos_memory }}" # Amount of memory to allocate to the VM in MiB (default 4096) | ||
balloon: "{{ kairos_memory_balloon }}" # Enable memory ballooning to allow the VM to dynamically adjust its memory usage (0 = disabled) | ||
shares: "{{ kairos_memory_shares }}" # Relative priority of the VM's memory allocation compared to other VMs (The larger the number is, the more memory this VM gets.) (default 1024) | ||
|
||
onboot: "{{ kairos_start_on_boot }}" # Automatically start the VM when the Proxmox VE node boots (default false) | ||
startup: "{%- if kairos_start_on_boot_order | default(false) %}boot={{ kairos_start_on_boot_order }}{% else %}{{ omit }}{% endif %}" # Boot order on the Proxmox VE node | ||
vmid: "{{ kairos_vm_id | default(omit) }}" # ID of the VM (unique per Proxmox VE node) | ||
register: kairos_vm | ||
|
||
- name: 🔍 List all the ISO images on a Proxmox VE node | ||
community.general.proxmox_storage_contents_info: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
node: "{{ pve_nodes | first }}" | ||
content: iso | ||
storage: "{{ proxmox_iso_storage }}" | ||
register: pve_contents_info | ||
|
||
- name: 💾 Attach Kairos OS and CloudInit ISO images | ||
community.general.proxmox_disk: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
vmid: "{{ kairos_vm.vmid }}" | ||
disk: ide{{ idx }} | ||
iso_image: "{{ item }}" | ||
media: cdrom | ||
loop: | ||
- "{{ pve_contents_info.proxmox_storage_content | selectattr('volid', 'search', kairos_os_image_filename + '$') | map(attribute='volid') | first }}" | ||
- "{{ pve_contents_info.proxmox_storage_content | selectattr('volid', 'search', kairos_cloudinit_iso + '$') | map(attribute='volid') | first }}" | ||
loop_control: | ||
index_var: idx | ||
|
||
- name: 💾 Create the primary storage for Kairos VM on Proxmox VE | ||
community.general.proxmox_disk: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
vmid: "{{ kairos_vm.vmid }}" | ||
disk: scsi0 | ||
name: vm-{{ kairos_vm.vmid }}-disk-0 | ||
state: present | ||
storage: "{{ proxmox_disk_storage }}" | ||
|
||
backup: true # Add this disk to the backup list | ||
cache: none # Disable cache for the disk (safest option for data integrity) | ||
detect_zeroes: true # Automatically detect and discard zeroed blocks (default on Proxmox VE) | ||
discard: true # Enable discard (TRIM) support for the disk (default on Proxmox VE) | ||
format: raw # Use the raw format for the disk (default on Proxmox VE, required on LVM storage) | ||
media: disk | ||
replicate: true # Replicate the disk to all nodes in the cluster | ||
size: "{{ kairos_disk_size }}" # Size of the disk in GiB. | ||
ssd: true # Mark the disk as an SSD (for better performance) | ||
iothread: "{{ kairos_disk_enable_iothread }}" # Enable I/O threads for the disk (default false) | ||
|
||
- name: 📡 Attach the network interface to the Kairos VM | ||
community.general.proxmox_nic: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
interface: net0 | ||
vmid: "{{ kairos_vm.vmid }}" | ||
model: virtio | ||
|
||
bridge: "{{ kairos_network_bridge }}" # Bridge to attach the network interface to | ||
mtu: "{{ kairos_network_mtu | default(omit) }}" # Maximum Transmission Unit (MTU) for the network interface | ||
tag: "{{ kairos_network_vlan | default(omit) }}" # VLAN tag for the network interface | ||
|
||
- name: 🚀 Start the Kairos VM | ||
community.general.proxmox_kvm: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
vmid: "{{ kairos_vm.vmid }}" | ||
state: started |
45 changes: 45 additions & 0 deletions
45
catalog/ansible/pve.kairos-cluster/tasks/download-kairos-iso.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
--- | ||
- ansible.builtin.file: | ||
state: directory | ||
path: "{{ playbook_dir }}/.cache" | ||
|
||
# Step 1: Fetch the Kairos OS image URL | ||
- name: 📝 Fetch the Kairos release information | ||
ansible.builtin.uri: | ||
url: https://api.github.com/repos/kairos-io/kairos/releases/tags/{{ kairos_version }} | ||
headers: | ||
Accept: application/vnd.github+json | ||
return_content: yes | ||
register: kairos_release_info | ||
|
||
- name: 🔗 Set the Kairos OS image URL | ||
ansible.builtin.set_fact: | ||
kairos_os_image_url: "{{ item.browser_download_url | urldecode }}" | ||
loop: "{{ kairos_release_info.json.assets | sort(attribute='browser_download_url') }}" | ||
when: item.name is match('^kairos-{{ kairos_distribution }}-standard-amd64-generic-{{ kairos_version }}-k3s{{ kairos_k3s_version | default('') }}.+\.iso$') | ||
|
||
# Step 2: Download the Kairos OS image | ||
- name: 📥 Download the Kairos OS image | ||
ansible.builtin.get_url: | ||
url: "{{ kairos_os_image_url }}" | ||
dest: "{{ playbook_dir }}/.cache/{{ kairos_os_image_filename }}" | ||
checksum: sha256:{{ kairos_os_image_url }}.sha256 | ||
register: kairos_os_image_download | ||
|
||
# Step 3: Upload the Kairos OS image to the Proxmox VE cluster nodes | ||
- name: 📤 Upload the Kairos OS image to the Proxmox VE cluster nodes | ||
community.general.proxmox_template: | ||
api_host: "{{ proxmox_host }}" | ||
api_user: "{{ proxmox_user }}" | ||
api_password: "{{ proxmox_password | default(omit) }}" | ||
api_token_id: "{{ proxmox_token_id | default(omit) }}" | ||
api_token_secret: "{{ proxmox_token_secret | default(omit) }}" | ||
|
||
node: "{{ item }}" | ||
|
||
content_type: iso | ||
src: "{{ kairos_os_image_download.dest }}" | ||
storage: "{{ proxmox_iso_storage }}" | ||
loop: "{{ pve_nodes_requiring_kairos_os_image }}" | ||
async: 1800 # 30 minutes | ||
poll: 15 # poll every 15 seconds |
Oops, something went wrong.