Skip to content

Commit

Permalink
✨(catalog:ansible): Add ansible role to create a single-node Kubernet…
Browse files Browse the repository at this point in the history
…es cluster on PVE (#205)

Signed-off-by: Alexandre Nicolaie <xunleii@users.noreply.github.com>
  • Loading branch information
xunleii authored Dec 20, 2024
2 parents a0d6435 + 28d6279 commit f8da1cd
Show file tree
Hide file tree
Showing 14 changed files with 566 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .commitlintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,10 @@ const types = [
* List of allowed commit scopes.
*/
const scopes = [
{
name: "catalog:ansible - Anything related to the Ansible role catalog",
value: "catalog:ansible",
},
{
name: "catalog:crossplane - Anything related to the Crossplane resource catalog",
value: "catalog:crossplane",
Expand Down
100 changes: 100 additions & 0 deletions catalog/ansible/pve.kairos-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# pve.kairos-cluster

This role sets up and configures a Kairos cluster on Proxmox Virtual Environment (PVE).

> \[!CAUTION]
> This role has only been tested for a single-node cluster and may not work as
> expected for multi-node clusters.
## Requirements

* Proxmox VE 8.3 or higher
* Ansible 2.16 or higher
* Python 3.6 or higher on the control node
* `community.general` collection for additional modules
* `proxmoxer` and `requests-toolbelt` Python packages on the control node

## Role variables

### Required variables

* *`kairos_cluster_name`*: Name of the Kairos cluster. (**required**)
* *`proxmox_host`*: Host address of the Proxmox server. (**required**)
* *`proxmox_user`*: Username for Proxmox authentication. (**required**)

### Optional variables

* *`kairos_cloudinit_hostname_prefix`*: Prefix for the hostname name. (*default: `krs`*)
* *`kairos_cloudinit_k3s`*: k3s configuration. See the [k3s documentation](https://kairos.io/docs/reference/configuration/#k3s-settings) for more information.
* *`kairos_cloudinit_user_sshkeys`*: SSH keys assigned to the kairos user. (*default: `[]`*)
* *`kairos_cpu_cores`*: Number of CPU cores to allocate to the VM. (*default: `4`*)
* *`kairos_cpu_type`*: Type of CPU to use. (*default: `host`*)
* *`kairos_disk_enable_iothread`*: Enable IO thread for disk. (*default: `false`*)
* *`kairos_disk_size`*: Size of the main disk, in gigabyte. (*default: `32`*)
* *`kairos_distribution`*: OS distribution on which Kairos is based (`alpine`, `debian`, `ubuntu`, ...). (*default: `debian-bookworm`*)
* *`kairos_memory_balloon`*: Memory ballooning setting. `0` to disable ballooning. (*default: `0`*)
* *`kairos_memory_shares`*: Memory shares setting when ballooning is enabled. (*default: `1024`*)
* *`kairos_memory`*: Amount of memory to allocate to the VM, in megabyte. (*default: `4096`*)
* *`kairos_network_bridge`*: Network bridge to use. (*default: `vmbr0`*)
* *`kairos_network_mtu`*: Network MTU setting. (*optional*)
* *`kairos_network_vlan`*: Network VLAN setting. (*optional*)
* *`kairos_start_on_boot_order`*: Boot order when the hypervisor starts. (*optional*)
* *`kairos_start_on_boot`*: Start automatically when the hypervisor starts. (*default: `false`*)
* *`kairos_version`*: Version of Kairos. (*default: `latest`*)
* *`kairos_vm_id`*: VM ID. (*default: the first available*)
* *`proxmox_disk_storage`*: Proxmox storage name where the VM disk will be stored. (*default: `local-lvm`*)
* *`proxmox_iso_storage`*: Proxmox storage name where the VM ISO will be stored. (*default: `local`*)
* *`proxmox_password`*: Password for Proxmox. (*required if `proxmox_token_id` and `proxmox_token_secret` are not set*)
* *`proxmox_token_id`*: Token ID for Proxmox. (*required if `proxmox_password` is not set*)
* *`proxmox_token_secret`*: Token secret for Proxmox. (*required if `proxmox_password` is not set*)

### Kairos bundles variables

> \[!NOTE]
> This role supports the installation of Kairos bundles. A bundle is a set of software packages
> that can be installed on a Kairos cluster. The role will configure the cloud-init script to
> configure Kairos with the specified bundles.
* *`kairos_bundles_<bundle_name>.description`*: Description of the bundle that will be displayed in Proxmox, on the VM notes. (*optional*)
* *`kairos_bundles_<bundle_name>.url`*: Bundle documentation URL that will be displayed in Proxmox, on the VM notes. (*optional*)
* *`kairos_bundles_<bundle_name>.targets`*: List of URLs to download the bundle. (*required*)
* *`kairos_bundles_<bundle_name>_config`*: Bundle configuration (*optional*).

For example, to install the `flux` bundle, you can use the following configuration:

```yaml
kairos_bundles_flux:
description: Open and extensible continuous delivery solution for Kubernetes.
url: https://github.com/kairos-io/community-bundles/tree/main/flux
targets:
- run://ghcr.io/chezmoi-sh/kairos-bundles:flux-sync # sync FluxCD configuration
kairos_bundles_flux_config:
git:
url: https://github.com/fluxcd/flux2-monitoring-example.git
path: clusters/test
branch: main
```
... that will be added to the cloud-init script as follows:
```yaml
bundles:
- targets:
- run://ghcr.io/chezmoi-sh/kairos-bundles:flux-sync

flux:
git:
url: https://github.com/fluxcd/flux2-monitoring-example.git
path: clusters/test
branch: main
```
## Dependencies
This role depends on the following collections:
* `community.general`

## License

This role is licensed under the Apache License, Version 2.0. See [LICENSE](../../../LICENSE) for the full license text.
60 changes: 60 additions & 0 deletions catalog/ansible/pve.kairos-cluster/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
# - Proxmox VE global configuration
# proxmox_host: # Hostname or IP address of the Proxmox VE node
# proxmox_user: # Username to authenticate with the Proxmox VE API
# proxmox_password: # Password to authenticate with the Proxmox VE API
# proxmox_token_id: # API token ID to authenticate with the Proxmox VE API
# proxmox_token_secret: # API token secret to authenticate with the Proxmox VE API
proxmox_iso_storage: local # Storage name where all ISO images are stored
proxmox_disk_storage: local-lvm # Storage name where all VM disks are stored

# - Proxmox VE VM configuration
kairos_cpu_type: host # Best performance for single node Proxme VE or if all Proxmox VE nodes have the same CPU
kairos_cpu_cores: 4 # 4 cores seems to be a good balance between performance and resource usage for most VMs
kairos_memory: 4096 # 4 GiB of memory is a good starting point for a Kubernetes node
kairos_memory_balloon: 0 # Disable memory ballooning by default
kairos_memory_shares: 1024 # Give the VM a fair share of memory by default
kairos_disk_size: 32 # 32 GiB of disk space is a good starting point for a Kubernetes node (image + data)
kairos_disk_enable_iothread: false # Disable I/O threads for better performance (the use of aio=threads is a legacy low-performance "aio" mechanism for QEMU|https://forum.proxmox.com/threads/proxmox-ve-7-2-benchmark-aio-native-io_uring-and-iothreads.116755/)
kairos_network_bridge: vmbr0 # Default bridge for most Proxmox VE installations
# kairos_network_mtu: # No MTU set by default
# kairos_network_vlan: # No VLAN set by default
kairos_start_on_boot: false # Do not start the VM automatically when the Proxmox VE node boots
# kairos_start_on_boot_order: # Not set by default
# kairos_vm_id: # Not set by default, the next available VM ID will be used

# - Kairos OS configuration
# renovate: datasource=github-releases lookupName=kairos-io/kairos
kairos_version: v3.2.4
kairos_distribution: debian-bookworm
# kairos_cluster_name: # Not set by default

# - Kairos community bundles configuration
kairos_bundles_suc: # System-Upgrade-Controller (https://github.com/rancher/system-upgrade-controller)
description: General-purpose, Kubernetes-native upgrade controller (for nodes)
url: https://github.com/kairos-io/community-bundles/tree/main/system-upgrade-controller
targets:
- run://quay.io/kairos/community-bundles:system-upgrade-controller_latest

# kairos_bundles_flux: # Flux CD (https://fluxcd.io/)
# description: Open and extensible continuous delivery solution for Kubernetes.
# url: https://github.com/kairos-io/community-bundles/tree/main/flux
# targets:
# - run://quay.io/kairos/community-bundles:flux_latest # bootstrap FluxCD
# - run://ghcr.io/chezmoi-sh/kairos-bundles-dev/flux-sync:latest # sync FluxCD configuration
# kairos_bundles_flux_config: # Flux CD (https://fluxcd.io/)
# git:
# url: ...
# ... read the bundle documentation for more details

# WARN: DO NOT CHANGE THE FOLLOWING VARIABLES UNLESS YOU KNOW WHAT YOU ARE DOING.
# THESE VARIABLES ARE USED TO PREPARE ALL THE KAIROS CLOUD-INIT CONFIGURATION.
x_kairos_bundles_names: "{{ lookup('ansible.builtin.varnames', '^kairos_bundles_(?!.*_config$).+') | split(',') | map('regex_replace', 'kairos_bundles_', '') | list }}"

# - Kairos cloud-init configuration
kairos_cloudinit_hostname_prefix: krs
kairos_cloudinit_user_sshkeys: []

kairos_cloudinit_k3s:
args:
- --disable=traefik
Empty file.
Empty file.
16 changes: 16 additions & 0 deletions catalog/ansible/pve.kairos-cluster/meta/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
galaxy_info:
author: Alexandre Nicolaie <xunleii@users.noreply.github.com>
description: Ansible role to deploy a Kairos cluster on Proxmox VE

issue_tracker_url: https://github.com/chezmoi-sh/atlas/issues
license: Apache-2.0

min_ansible_version: 2.1

# If this a Container Enabled role, provide the minimum Ansible Container version.
# min_ansible_container_version:

galaxy_tags:
- k3s
- kairos
- kubernetes
156 changes: 156 additions & 0 deletions catalog/ansible/pve.kairos-cluster/tasks/create-pve-vm.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
- name: 🛠️ Create the cluster instance pool on PVE
community.general.proxmox_pool:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

poolid: "{{ kairos_cluster_name }}"
comment: Kairos cluster instance pool for the {{ kairos_cluster_name }} cluster

- name: 🛠️ Initiate the Kairos VM on the Proxmox VE cluster
community.general.proxmox_kvm:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

node: "{{ proxmox_node | default(pve_nodes | first) }}"
state: present

# Static VM configuration
acpi: true # Enable ACPI to allow graceful shutdown from Proxmox VE
agent: true # Enable QEMU Guest Agent for better integration with Proxmox VE (qemu-guest-agent must be installed during the VM creation)
bios: seabios # Use SeaBIOS as the BIOS firmware (EFI is not required)
boot: order=scsi0;ide0 # Boot order of the VM (Kairos VM will boot from the first hard disk and then the first CD-ROM drive)

ostype: l26 # Operating system type (Linux 2.6+)
sockets: 1 # Number of CPU sockets to emulate (don't need more than 1)
pool: "{{ kairos_cluster_name }}"

name: "{{ kairos_cloudinit_hostname_prefix }}0.{{ kairos_cluster_name }}" # Name of the VM
description: |
<div align='center'>
<a href='https://kairos.io/' target='_blank' rel='noopener noreferrer'>
<img src='https://raw.githubusercontent.com/kairos-io/kairos-docs/f4e35110ef5b52eaf8ba0727585828f7a0aaad48/static/favicons/android-chrome-192x192.png'/>
</a>
# kairos {{ kairos_version }} _({{ kairos_distribution }})_
### _for [{{ kairos_cluster_name }}] cluster_
</div>
---
### Enabled community bundles
{% for bundle in x_kairos_bundles_names %}
{% set bundle_info = lookup('ansible.builtin.vars', 'kairos_bundles_' + bundle) %}
{% if bundle_info.url is defined and bundle_info.description is defined %}
- **[{{ bundle }}]({{ bundle_info.url }})**: {{ bundle_info.description }}
{% else %}
- **{{ bundle }}**
{% endif %}
{% endfor %}
tags:
- kairos
- "{{ kairos_distribution }}"
- "{{ kairos_version }}"

# User-defined VM configuration
cpu: "{{ kairos_cpu_type }}" # CPU type to emulate (default host)
cores: "{{ kairos_cpu_cores }}" # Number of CPU cores to allocate to the VM (default 4)
vcpus: "{{ kairos_cpu_cores }}" # Number of virtual CPUs to allocate to the VM (same as cores)

memory: "{{ kairos_memory }}" # Amount of memory to allocate to the VM in MiB (default 4096)
balloon: "{{ kairos_memory_balloon }}" # Enable memory ballooning to allow the VM to dynamically adjust its memory usage (0 = disabled)
shares: "{{ kairos_memory_shares }}" # Relative priority of the VM's memory allocation compared to other VMs (The larger the number is, the more memory this VM gets.) (default 1024)

onboot: "{{ kairos_start_on_boot }}" # Automatically start the VM when the Proxmox VE node boots (default false)
startup: "{%- if kairos_start_on_boot_order | default(false) %}boot={{ kairos_start_on_boot_order }}{% else %}{{ omit }}{% endif %}" # Boot order on the Proxmox VE node
vmid: "{{ kairos_vm_id | default(omit) }}" # ID of the VM (unique per Proxmox VE node)
register: kairos_vm

- name: 🔍 List all the ISO images on a Proxmox VE node
community.general.proxmox_storage_contents_info:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

node: "{{ pve_nodes | first }}"
content: iso
storage: "{{ proxmox_iso_storage }}"
register: pve_contents_info

- name: 💾 Attach Kairos OS and CloudInit ISO images
community.general.proxmox_disk:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

vmid: "{{ kairos_vm.vmid }}"
disk: ide{{ idx }}
iso_image: "{{ item }}"
media: cdrom
loop:
- "{{ pve_contents_info.proxmox_storage_content | selectattr('volid', 'search', kairos_os_image_filename + '$') | map(attribute='volid') | first }}"
- "{{ pve_contents_info.proxmox_storage_content | selectattr('volid', 'search', kairos_cloudinit_iso + '$') | map(attribute='volid') | first }}"
loop_control:
index_var: idx

- name: 💾 Create the primary storage for Kairos VM on Proxmox VE
community.general.proxmox_disk:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

vmid: "{{ kairos_vm.vmid }}"
disk: scsi0
name: vm-{{ kairos_vm.vmid }}-disk-0
state: present
storage: "{{ proxmox_disk_storage }}"

backup: true # Add this disk to the backup list
cache: none # Disable cache for the disk (safest option for data integrity)
detect_zeroes: true # Automatically detect and discard zeroed blocks (default on Proxmox VE)
discard: true # Enable discard (TRIM) support for the disk (default on Proxmox VE)
format: raw # Use the raw format for the disk (default on Proxmox VE, required on LVM storage)
media: disk
replicate: true # Replicate the disk to all nodes in the cluster
size: "{{ kairos_disk_size }}" # Size of the disk in GiB.
ssd: true # Mark the disk as an SSD (for better performance)
iothread: "{{ kairos_disk_enable_iothread }}" # Enable I/O threads for the disk (default false)

- name: 📡 Attach the network interface to the Kairos VM
community.general.proxmox_nic:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

interface: net0
vmid: "{{ kairos_vm.vmid }}"
model: virtio

bridge: "{{ kairos_network_bridge }}" # Bridge to attach the network interface to
mtu: "{{ kairos_network_mtu | default(omit) }}" # Maximum Transmission Unit (MTU) for the network interface
tag: "{{ kairos_network_vlan | default(omit) }}" # VLAN tag for the network interface

- name: 🚀 Start the Kairos VM
community.general.proxmox_kvm:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

vmid: "{{ kairos_vm.vmid }}"
state: started
45 changes: 45 additions & 0 deletions catalog/ansible/pve.kairos-cluster/tasks/download-kairos-iso.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
- ansible.builtin.file:
state: directory
path: "{{ playbook_dir }}/.cache"

# Step 1: Fetch the Kairos OS image URL
- name: 📝 Fetch the Kairos release information
ansible.builtin.uri:
url: https://api.github.com/repos/kairos-io/kairos/releases/tags/{{ kairos_version }}
headers:
Accept: application/vnd.github+json
return_content: yes
register: kairos_release_info

- name: 🔗 Set the Kairos OS image URL
ansible.builtin.set_fact:
kairos_os_image_url: "{{ item.browser_download_url | urldecode }}"
loop: "{{ kairos_release_info.json.assets | sort(attribute='browser_download_url') }}"
when: item.name is match('^kairos-{{ kairos_distribution }}-standard-amd64-generic-{{ kairos_version }}-k3s{{ kairos_k3s_version | default('') }}.+\.iso$')

# Step 2: Download the Kairos OS image
- name: 📥 Download the Kairos OS image
ansible.builtin.get_url:
url: "{{ kairos_os_image_url }}"
dest: "{{ playbook_dir }}/.cache/{{ kairos_os_image_filename }}"
checksum: sha256:{{ kairos_os_image_url }}.sha256
register: kairos_os_image_download

# Step 3: Upload the Kairos OS image to the Proxmox VE cluster nodes
- name: 📤 Upload the Kairos OS image to the Proxmox VE cluster nodes
community.general.proxmox_template:
api_host: "{{ proxmox_host }}"
api_user: "{{ proxmox_user }}"
api_password: "{{ proxmox_password | default(omit) }}"
api_token_id: "{{ proxmox_token_id | default(omit) }}"
api_token_secret: "{{ proxmox_token_secret | default(omit) }}"

node: "{{ item }}"

content_type: iso
src: "{{ kairos_os_image_download.dest }}"
storage: "{{ proxmox_iso_storage }}"
loop: "{{ pve_nodes_requiring_kairos_os_image }}"
async: 1800 # 30 minutes
poll: 15 # poll every 15 seconds
Loading

0 comments on commit f8da1cd

Please # to comment.