Learning Cloud-Init

cloud-init
hosting

Local-References:¶

Cloud-Init - What is it?¶

If you've booted pretty much any recent linux distro you may have noticed Cloud-Init running at the start. Most major distrobutions include it these days. But what is it? What's it do?

Cloud init's main job is to initialize new machines.

Scenario¶

Proxmox is running, or whatever your favorite hypervisor is. If you started out like me, you downloaded iso's for distrobutions and would provision a new vm through the UI. Then you'd go launch from the iso and install your fresh new OS to your fresh new VM. This is all fine and dandy, but now what if you want to set this machine up? Well that's easy, you log into it and set it up right?

Of course that works, now say you want to do a kubernetes cluster. Now you need at minimum 3 VM's. Sure you could follow the process of provisioning each vm, logging in to each vm, installing the dependencies, setting each one up with a unique hostname and ip address, configuring it to start on boot, for each one. Now you want 5 VM's in this cluster, or 15, or 30. Doing this each time becomes extremely tedious, and that's where cloud-init comes in.

How it works¶

Cloud-init is installed in the vm's operating system, not every operating system comes with it by default however. Alpine for example doesn't include it by default, and you have to set it up (it's really easy ) When the OS boots, cloud-init will look in various locations for a cloud-init file. The host (eg Proxmox) can mount a drive containing the cloud-init file into the vm, so the vm can use that file to read from on boot. It performs various tasks, such as creating users, setting ssh public keys, generating ssh private keys, and much, much much more. Besides having a plethora of built in plugins that just work as long as the dependencies for that plugin are installed, cloud-init allows us to run any CLI command we can dream up.

For my VM's I want them to all get a gitlab-runner installed on them, and to assign those runners to a project in my gitlab. So I created a script in gitlab, and used nginx proxy to create a shortcut to the raw file. Then in the cloud-init script, I told it to download that file with wget, then run it passing it a few arguments. And now when my fresh VM booted for the first time, it downloaded the runner and registered itself with gitlab, no more manually installing the gitlab runners for me.

That's all well and good, but Proxmox's UI doesn't do that?¶

The default cloud-init in proxmox is very limited. It allows the setting of the User, password, DNS, hostname, ip address, and sshkey. Sure these are handy things to set, and a great starting point for learning how to use cloud-init. However, cloud-init can be so much more.

Example cloud-init.yml¶

This used to be a single file on here, but that wasn't really working fully. Now I defined three files.

networking.yml
user.yml
vendor.yml

networking.yml defines a netplan, it should be fully compatible with netplan syntax.¶

#cloud-conig

network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      dhcp4: true
      dhcp6: true

user.yml defines the¶

user(s): (The password hashed in the example is "password")
hostname

#cloud-config

hostname: terraform-test

users:
  - name: deadc0de
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    password: "$5$wfdJCSfEQ.resMUn$2GwNZbXnKmZfoHMJT1660YO0tn74etND/wVNrcBxR51"
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBi7A81T7smfUrtyqDjg8kRjiuNu6KmS/CGVBMOn0WAPg/k5D4uAZT3CsO/MrpwFyx5Zx+wFd82Y+e68WRzqV2gsNszCUiG+7BEWD+ArDMUf/zbj7vafR4xzm8f9bPVRmV9PPqjnauZadAcwEP7rGHa8n8Eun8khB/cyfkRU3K/ziE7vhVCku82ECsYr5vsHs9+M+Q6j/IXoKFD9blBdqVgwUR6NjvKmpIo2kqe2f64mKrE0x2F95KWsWKjVu0ugwjYrpwmLmQFJYr4xBa+XAlwL9K99rJQrcKWUskiupbtYs0OgQPEnYamqQjLgB0qe4DD9bB4N/6NZMioVA24oXx deadc0de@deadc0de-PC

vendor.yml defines the scripts I want to run on first boot:¶

#cloud-config

bootcmd:
  - wget -O runner.sh https://cloud-runner.centerionware.com
  - bash runner.sh 56 glpat-api-key dreu-odoo
  #- [ cloud-init-per, once, mymkfs, mkfs, /dev/vdb ]

But how do I use it?!¶

Setup storage¶

First off, you need a place to store the cloud init .yml files, they don't need to be named 'cloud-init', but should end with the extension .yml. In my case, I created the folder /etc/pve/snippets on one of my proxmox hosts, since the /etc/pve folder and all it's contents are automatically mirrored between all the hosts in a proxmox cluster. This is the perfect place to store these, because all proxmox nodes need to have access to the cloud-init files if you wish to migrate a VM that's using one to another node in the cluster.

Next, you'll want to add this new directory to proxmox as a storage provider for snippets. I just used the UI, went to datacenter, storage, add, directory, /etc/pve/snippets , and named it cloud-init. If you name yours something else (EX snippets), use that in place of cloud-init: in the cicustom field (EX: snippets:snippets/your-files.yml vs cloud-init:snippets/your-files.yml)

Cloud-init Drive¶

For cloud-init to work, proxmox needs a cloud-init drive attached to the VM that will use it. With the UI it's as simple as going to the Hardware section of the chosen vm, clicking add, and clicking the cloud-init option. It shouldn't much matter what options you pick. Most documentation though says it should be mounted as an IDE drive (as a virtual cd-rom drive). It doesn't seem to create a problem to simply add this cloud-init drive to the template definition, then when it's cloned the clones already have one set up.

Method A¶

Now you've created the storage for your cloud-init's, I assume you put one in there, we can move to the next step.. Using it.

Create your VM by cloning a template with cloud-init enabled, for this example I cloned one and set it's ID to 9000.

qm set 9000 --cicustom "user=cloud-init:snippets/user.yml,network=cloud-init:snippets/network.yml,vendor=cloud-init:snippets/vendor.yml"

And that should be it, boot your VM, as long as it's got a cloud-init drive attached, it should work. The hard part is debugging issues that go wrong here.

Method B - The happy future way¶

Terraform

Debugging¶

This can actually be quite difficult, especially if working with a fully locked down cloud image that has the root account locked and no default account created - that expects the default account with sudo access to be created by cloud-init. You'll have to get creative here. Since we can watch the console in proxmox, I used a screen recorder (OBS Studio) to capture the booting of the vm's, because things happen so quickly I can't read it while it's scrolling past. But I can record it, then open the recording in VLC and step through frame by frame to actually read the output.

There are of course other options, don't lock down the cloud image so much so you can login and get logs from the system the normal way. But what fun is that?

To verify the syntax at least of your .yml files you can use any vm with cloud-init installed, it doesn't even have to be running, and run: cloud-init schema --config-file my-cloud-init.yml

Be sure to check the proxmox documentation here https://pve.proxmox.com/wiki/Cloud-Init_Support