Perfect Proxmox Template with Cloud Image and Cloud Init
ฝัง
- เผยแพร่เมื่อ 18 มิ.ย. 2024
- Using Cloud Images and Cloud Init with Proxmox is easy, fast, efficient, and fun! Cloud Images are small images that are certified cloud ready that have Cloud Init preinstalled and ready to accept a Cloud Config. Cloud Images and Cloud Init also work with Proxmox and if you combine the two you have a perfect, small, efficient, optimized clone template to provision machines with your ssh keys and network settings. So join me as we discuss, set up, and configure Proxmox with Cloud Images and Cloud Init.
Video Notes: technotim.live/posts/cloud-in...
Support me on Patreon: / technotim
Sponsor me on GitHub: github.com/sponsors/timothyst...
Subscribe on Twitch: / technotim
Become a TH-cam member: / @technotim
Merch Shop 🛍️: l.technotim.live/shop
Gear Recommendations: l.technotim.live/gear
Get Help in Our Discord Community: l.technotim.live/discord
2nd channel: / @technotimtalks
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 - What is Cloud Init?
01:07 - Configure Cloud Image
04:09 - Configure Cloud Init
06:40 - Create a Template
08:07 - Cloning a Machine
09:47 - SSH Intro cloned machine
09:59 - Troubleshooting
10:37 - What do I think about Cloud Images & Cloud Init
11:25 - Stream Highlight - "from 0 to Kubernetes in 3.6 minutes"
#Proxmox #HomeLab #Cloud
"Head Space" is from Harris Heller's album "Gold"
l.technotim.live/sb-music-lic...
Thank you for watching! - วิทยาศาสตร์และเทคโนโลยี
I haven't the foggiest idea if I mist anything, so let me know if I did ☁
Well you did misspell missed ;-)
@@MikeCarr07 I once watched a film about clouds but it was way over my head. :)
Perhaps resizing the boot drive to be larger than ~2 1/4 GB?
The one thing I got caught out by, is that the 2.2GB boot image doesn't give you too much room to install anything
You can add an extra 10 gigs to the image with:-
qm resize 8000 scsi0 +10G
Fantastic addition. The major thing now is to add or continiue on is at least a bit of hardening, topic that is fairly missed. Other then that bravo!
Okay, now I can officially begin my Saturday
Just used these today actually! Messing around with setting up my environment using Ansible, and cloud-init is great for this as well.
This is a great video tutorial on how to create a Proxmox template with Cloud Image and Cloud Init. I found it very helpful and informative. The presenter is very clear and concise in his explanations, and he does a great job of walking through the steps involved. I would definitely recommend this video to anyone who is interested in learning more about Proxmox
Thanks Tim, never done it before and worked perfectly.
A perfect video explaining basics of cloud-init cofiguration and minute details of proxmox internals. And the constructive comments in the document blog page has helped me get the jammy cloud image created. Almost perfect, one minor manual step to remove machine id after cloning VM from template. It could be taken care of while deploying using terraform. thank you Tim, for all your efforts in educating us. I didn't make a first stop at your site and paid the penalty of many hours loss.
Very useful and worked a treat. I followed some other guide and it didn't work, as always yours is very clear and easy to follow thank you
This is great, works like a charm thank you
This is exactly the method I personally use to create my templates and create Vms very quickly. Thanks for popularizing this practice Tim ;)
Glad it was helpful!
Your videos are legit really good. There can literally never be too much home lab content and yours is super palitable and informative.
Thank you so much!
Yeah he really explains things clearly without going long into the explanations making you want to speed the video up to 1.5x like I do some other channels. There's a reason I always come to this channel
@TechnoTim hey can you share the link to download .img images
thanks in advance
This was great! thanks Tim
ok, wild coincidence. i was revisiting doing cloud init with proxmox on the very same day right before you uploaded this. now i have a handy guide to help me out. thanks dude :)
Enjoy!
awesome - very useful! Planning to use this template with Terraform next.
Great stuff Tim. This is exactly what I wanted to set up at my studio/homelab!
Glad it was helpful!
I’ve been looking forward to this topic! Another great video!
Glad you enjoyed it!
This man single-handedly giving me a career path
Most clear video about the subject i found it in internet, now a must see video for new employees in our enterprise.
Another amazing tutorial - Straight o the point yet detailed. Man you are an asset
Thank you!
I just installed proxmox and was looking to do something like this. Thanks a lot for this
No problem 👍
Great video Tim!
I would generally suggest to use the 'releases' builds as you get the latest LTS build inside there (22.04), instead of choosing the daily builds like in the video.
Also please note that these images do not contain the same packages as the standard server, since some features are not relevant in the Cloud, like printer drivers and such. There is a manifest file next to the img where you can see the packages installed. I would suggest booting the image and making sure everything you need is there before setting it up as template.
Note that some of the images don't use the 'linux-generic' kernel which can cause issues down the line when trying to isntall packages such as 'nfs-server' for example.
And lastly, the VMs created by the template have a disk the same size as the image by default, which is probably not desired in most cases. However, these cloud images are usually designed to auto expand the disk upon boot, so with Proxmox you should be able to use the "Resize disk" button in the Hardware section to set the desired size.
Thanks for the demo and info, have a great day TT
Thank you, you too!
Another Great Video!!! 👏 I have found this way is a lot easier than using Packer to create my templates. I did run into a few snags. When creating the VM from the command line, I see that the SCSI Controller is set to the Default (LSI 53C895A) and when you import the disk image it is added as an Unused Hard Disk 0. So when you run the next command to add the disk to the vm it fails. All that is needed is to change the controller to VirtIO SCSI and then change the Unused Hard Disk 0 setting to make it a SCSI disk and then everything is fine. 😀 Now on to learn Ansible.👍
This is great! Will set this up first thing tomorrow.
Thx!
Hope you enjoy it!
always saving my life brother... thanks for this video... cheers from costa rica
Ya know I went back and watched one of your older videos yesterday for something rancher related. Crazy how much your background has changed haha
It's changed quite a bit! Everything has! Thanks for noticing!
Great video. This will save some time when pushing out new VM's 👍
Glad I found you! Great stuff.
Just what I was looking for! Now we need a Terraform/Proxmox video 👀
This is much more efficient than the docs. Thanks.
Cool! I'm doing the same, but without UI, just bash script to batch configure it with correct VLAN and IP. Of course would be simpler to do it with ansible pointed to the proxmox host
Thank you this help's me will thank you again !
I like cloud init a lot. Pitched it at work for a new way to deploy consul clusters. Only real draw backs I took away from it was to keep it simple. If your init gets to complex it can take several minutes to execute and do all the things, It can als be a little difficult to debug. Good you touched on it. Next video should be about service mesh!
Agreed! I handle everything after it get's an IP address with Ansible
Yet again another great tutorial. Thanks
Thank you!
very cool, thank you!
While following the video i put these commands into bash scripts to make it super easy to repeat if i mess it up. Some things were different but easy to adjust to my version.
I really love the implementation of cloud-init like this, saves SO much time.
edit2: the vm's show up as "ubuntu" on first boot, but will advertise the correct hostname after a reboot.
Thank you!
I’m an extremely picky and impatient open source home labber. I made it through this entire video. Really excellent work. All online instructors should watch this. Thorough, proper explanations, simple easy to follow commands without rushing. Beautiful! I didn’t even have to copy and paste. Paused and typed! (I don’t recommend that.) I just ssh’ed into a brand new Ubuntu server with no password! And it’s on my ZFS. Ready to be replicated and added to HA and backed up if I start caring about it. That’s so fast! And I can do it with other builds! The commands should be the same except for the img URL! That’s really powerful stuff. I have to stop and think about what I could do with that! I need to bake docker compose and portioner into the img. How would I do that without booting it? I’ll have to work on that. Thank you Tim for this excellent video.
Wow! Thank you so much! I am glad you like it!!!
@@TechnoTim FYI I found it best to leave out your serial terminal command and leave it on default. I’m on Proxmox 8.0.3.
Did i heard Ansible ?!
Really good video again thanks ^^
Thanks, super useful.
Thanks for this!
Great stuff. I love cloud init. One other thing you could do is use cloud init to kick off an initial ansible script that did the after provisioning bootstrapping too.
That's a great idea!
This is perfect. I also adjust the disk to my needs per VM. Besides installing qemu-guest-agent as one of the pkgs in my Ansible script I added the cloud-initramfs-growroot pkg, so when I do make the disk larger I just reboot the VM to get the file system expanded.
Twinsies!
@@TechnoTim So is this the content for next weeks video? Loved this one...but feels like the ansible stuff should be part 2!
yea disk and disk resizing wasnt covered, please do a short video on it! interested in cloud-initramfs-growroot pkg
@@mitchross2852 Before you fire it up first time, click on the disk then resize, it's a button above. The amount you put in will be added to the existing size will not be the new size.
@@enderst81 thanks for this!!
This is a good way to understand how to use cloudinit cloudinit with proxmox. But if you want to provision the best way i found around there is using terraform with proxmox provider :-).
Hey Tim, great video 👍 Before creating the template, I would recommend adding in the qemu-guest-agent to the cloud image with the following command: virt-customize -a {{IMG}} --run-command 'apt-get update && apt-get upgrade -y && apt-get install qemu-guest-agent -y'
Thank you!
Didn't know about virt-customize. Really handy for setting up VM templates. Saves me the hassle of cleaning up mashine-id and ssh-keys etc
Hey just wanted to note that this does something you wouldn't want with your image which is creating those machine-id that he mentions. Be aware of that, cheers.
You can run the command (change the image to whatever you're using, 22.04 LTS is out so I'm building using jammy): virt-customize -a jammy-server-cloudimg-amd64.img --install qemu-guest-agent --truncate /etc/machine-id
@@pewter77 Is this run inside the image or on proxmox?
Nice one. Been doing this with the terraform provider for proxmox for some time but recently been switching to Flatcar Container Linux and their ignition config system. It's a bit hacky getting up and running with images (gotta use args) but pxe booting and installing is magic (with matchbox and terraform provider)
Hi mate, do you have a link for a how-to that describe what you did?
didnt even know i could do this, it will save me alot of time because for some reason i reset all my proxmox nodes monthly
this is awesome and mind blowing. tks tim
Glad you enjoyed it!
@@TechnoTim yeah im from brazil, im devops and app engeneering. I spent most of my time using Hyper-V but now with proxmox everything changes. Thanks for the excellent tutorial.
I'm also running Cloud Images in Proxmox and they are set up through Ansible. Love it! On all my virtual machines i have a second drive which contains all my data so the Cloud Image drive is just a "throw away" drive, just like docker images.
To switch between different versions of ubuntu i only need to change 1 line in an ansible file and run 2 ansible scripts.
Nice!
Your video save me !! Thanks
Tim, great content!. Question: if I want to add some custom info, such as: adding a repository by default, modifying .bashrc, creating some folders, adding a tar.gz file inside a folder... Cloudinit would be the option?
Really good video, I'd just add a qemu-img resize command to make the image 32G or something that is more usable for a default VM setup
Nice and easy. I always just clone one I setup how I like it. Guess it is time for me to finally start learning Ansible
You can do it!
Until now i've just been using a base Ubuntu-Server machine as a template. I think i'll look at the cloud image and see if i can use that for a more efficient home docker stack.
Damn, I literally setup my cloudinit images 4 days ago. It's simple, but I'd prefer if I had this video for that.
This is what I do but using terraform or/and ansible depending my needs.
Easy to do, fast and you can optimize / customize the cloud init to fit your needs.
Do you have any recommendations for content on integrating terraform and ansible with cloud init images?
Doing the lords work
Getting cloud init working in proxmox is step 1 of 2 for full infrastructure as code using Terraform which has a proxmox module that relies on images having cloud init support
Very helpful! One question: if I want to update the template with a new image can I just replace the image in storage? Or do I need to do something to the VM?
Great content, thank you.
Thank you so much
Thanks Tim! Just a remark, I installed a cloud debian 11 template. After a few attempts I realized that dhcp was installed in the image by default! I I verified this in the .json file of the debian cloud repository. Indeed when I added a dhcp to the template, every vm generated by the clone had two ip addresses! My lesson, better to know what exactly you put it in making templates. Thanks anyway for the video.
Thanks bro
Super helpful!
Glad it was helpful!
You mentioned showing the post-deployment configuration (ex: QEMU agent) via Ansible later. Is that coming in another video?
(I know you have a getting started with Ansible video already, but I was hoping you'd go into more details on how you use it in your lab)
That's included here too! th-cam.com/video/w9eCU4bGgjQ/w-d-xo.html along with all of my playbooks, one of which includes qemu-guest-agent!
Do you have any plans to install OpenShift cluster on proxmox? I love your videos.
Hey Tim! Great video. When following the video to setup a cloud-init template I am see an issue when starting my cloneed VM. When the VM boots is also boots me into the grub resue prompt. Any help would be greatly appreciated.
Nice, is their a way to set a script to run in all of the vms and or host to edit the hard drive sizes, install 'my default" packages, ect. (on the template side so everything is done when its previsioned)
love me some cloud-init
me too!
I used cloud init + terraform in my kvm hypervisor it help me to build my VM when it first boot
Awesome!!! thank you so much!
Does deleting the machine-id also cause other identifiers, such as the host ssh keys, to be regenerated?
Great video. I was curious if when a new LTS comes out you should just create a new template or if there is a way to update your template with the new LTS?
I would just create a new template! Thank you!
Hey Tim, probably a late comment, but in your docs instead of removing the file: /etc/machine-id just remove its content ( I use echo -n > /etc/machine-id), that works every time!
For me this helps a lot with automations, I can grab later the ip of the machine with the proxmox api. Lot easier than doing some other hack!
Great vid,
will You create same video about LXC template?
I tried doing the same thing a while ago, but using ansible only... it resulted in unbootable/non configured VMs 😓 ansible's proxmox_kwm module is not good for this tasks! One thing I can suggest is that the qm set commands can be written as a single one (I keep seeing them separated on all the guides I found online, but the documentation says it can be done as a whole)
Interesting, I'm aware cli is quicker but I presume this can be done via the webui also?
Hello Tim, I'm interested on cleaning the UUIDs, I went to your docs link but i get error msg: "Web server is down" , is there another site with that information?
I’ve had issues where the VMs keep getting different IPs from the dhcp server every time they are restarted even though the DHCP has a lease of 364 days
been using this feature for a couple of months, however I find this problem when I'm building it for making my home-assistant. Somehow using cloudinit and building the supervised version will create problem with network. Somehow the HA script changes its network into 2 multiIP and did not work well. Unsure what happen with it. Running it on Manual Debian install did not cause any problem with network, so wondering why? Maybe you have a solutions?
recent find after using these images for a bit, if you plan on using terraform to provision vms in proxmox, having the guest agent in the template will allow terraform to clone these cloud init images successfully . a quick way to do that is to run the following commands after downloading the cloud init image using wget: sudo apt install libguestfs-tools -y
virt-customize -a ubuntu-22.04-minimal-cloudimg-amd64.img --install qemu-guest-agent
How do you pre-install qemu guest agent into cloudinit image before creating template out of it?
Recent proxmox actually have a download-from-link feature in the UI where you would normally upload the iso, which is a very practical addition.
Would you consider making a video on doing this entire setup via Terraform? I've been playing with it via the API, and it seems like an elegant way to setup K8s clusters.
I've written Ansible to spin up K8s using k3os and their takeover script. It's pretty wild to start with an Ubuntu template and end up with however many k3os machines as you want all clustered together. I even create DNS entries in Pihole and DHCP reservations on my Mikrotik router.
That said, I only did it that way because I use Ansible for my day job and I'm familiar with it. I do think terraform is probably the better way to do it. Terraform even has a Proxmox plugin for it.
@@FlexibleToast Thanks, I normally use k0s with k0sctl, which is relatively easy to add a node to a cluster when you can ssh to it. I'd love to see Tim's take on a setup like this. I guess there's many roads to rome this way.
@@TweakMDS I've never heard of k0s. The k0sctl kind of reminds me of openSUSE Kubic and kubicctl. Which is really awesome, but more resource heavy than k3s. I really wish OKD supported Proxmox for IPI installations.
@@FlexibleToast I'm running both k3s and k0s, and for my use cases, k0s has the edge, especially ease of install / adding nodes and upgrading with k0sctl.
@@FlexibleToast so much on resource usage XD , but okd is freaking awesome, especially with ipi. Proxmox really needs a Kubernetes cloud controller and CSI driver. That would be dope.
Ive been planning on deploying okd4 in my home lab... and waiting on some hardware.
CloudInit CD is shared between every VM?
I want to have multiple CloudInit Template in my cluster
I currently use packer and a preseed.cfg to build my own "clouinit" Debian Images. That way I can bake in all the packages I need and more.
Nice! I keep mine vanilla and lean and then and run ansible afterwards depending on the role! So many options!
Yooooooooooo. I just setup MinIO and it's amazing if you can give some thoughts on it or something I think it would be awesome
Great video! What I am missing is on what are those small vm's used. They have a fixed disk storage and were 70ish% used. Are they meant to use with microservices with data in a NAS/SAN? What would you run in them? I am honestly asking 😊😊. Thanks!
They can be used for anything that you need for Ubuntu - a Docker host, k3s nodes, or really any Ubuntu vm! Just expand the disk after a clone :)
Also cloud init has built-in option to expand the disk on reboot when you change size on proxmox.
How do we increase the size of the cloud image with something like ansible or terraform?
seems like no matter what i do the vm wont accept the login detials in the cloud init tab, even tried default ubuntu logins and non seem to work
Can i edit the Cloud Image that it installs a other Language ?
Well, I had a different idea of cloud-init than just providing default user, keys and network info - but rather the idea of specifying a run-once script based on a YAML config like the User-data in EC2 instances in AWS. I get that with MTU on my Mac - so I was initially excited to see Proxmox supported Cloud-Init until I realized how limited it was. Also, I thought server Ubuntu image already came with cloud-init.
There is a custom parameter in Proxmox. You can specify the path to the yaml file in it. But it doesn't work at all...
very intetresting, what about doing the same on esxi?
Sorry how do we change the drive size?
Tack!
Wow! Thank you!
Hello Tim u are great , can do a how monitor proxmox nodes?
Not sure whether this is related to Proxmox version (8), but this doesn't work for me when I create a template without password (SSH Key only). It always gives me publickey persmission denied, even though I do have everything configured correctly (been doing this in public clouds for a while now). So I imagine there might be more tweaking to do with a custom cloud init yaml file.
Yeah, I ran into this as well!!! You end up with a VM you can ONLY use on the Proxmox GUI.
What am I doing wrong here???
this method doesn't seem to allow to control disk size.
got this to work with the fedora cloud image, no issues. Can we run desktop environments on these or are we better off just installing from the ISO?
I think it will also work with desktop, if not you might need to install cloud-init first!
Can you elaborate on how to bake in Qemu-agent and other tools to the VM. I have found steps, but they include using virt-customize which is not an option with proxmox. Searching for a resolution to that leads to install libguestfs-tools which is not compatible with proxmox.
He said he's baked-in the guest agent before but ran into issues when cloning. I assume he is referring to what he mentioned before that "if you boot the machine before turning it into a template, you will have identical uuid and machine id when cloning". This means you will need to take additional steps of editing the uuid and machine id, to ensure they are unique, for every clone you create.
can you share which video i should check into about the Ansible to automate that qemu-guest-agent installation?
th-cam.com/video/w9eCU4bGgjQ/w-d-xo.html
@@TechnoTim thank you, 👍👍👍
Follow up, show the same functionality in Harvester :)
For small footprint I use turnkey-core based LXC containers (I've made a template which I clone when needed). It's supposed to be Ansible-able, too, but I haven't tried it yet (one of these days...).
How do these Cloud Images compare, resource-wise?
LXC is definitely going to be smaller but doesn't play well with everything. These are optimized for the cloud and since a majority of my vms run containers cloud images are the best option.
@@TechnoTim Thanks for the reply.
In my particular case I use them simply as Docker hosts within Proxmox (since Proxmox doesn't run Docker directly) that I can spin up quite quickly. Each runs a small particular set of Docker containers. One LXC for utility/productivity tools (wiki, CyberChef etc.), one for Grafana & InfluxDB, always one or two that I can mess with and easily reset them (I believe the technical term for that is "learning"), and so on. I've had no compatibility nor functionality issues so far.
does this also work with UEFI? why do most tutorials on this subject use legacy bios firmware for the VMs?
Because most cloud images are prepared for legacy BIOS.