I did a migration from ESXi to Proxmox just over two years ago (before the Broadcom massacre occurred). The switch wasn't even planned, as I'd bought new server gear who's disk controller was not supported by ESXi (my fault for not researching) and just jumped into the Proxmox deep end. I was shocked at how easy and rather painlessly I was able to migrate nearly every VM (mix of FreeBSD and Linux) to the new hypervisor. Adding in ZFS for local storage and a built-in backup solution to send those snapshots to my TrueNAS was like magic! Hope the new setup works out for you in the long run as well as it has for me!
@@2GuysTek I enjoyed it a lot... and for my part, I'm still waiting that my management will take a decision avout VMware. My lab will follow the decision and if we stay by VMware, I'll take VMUG again. I just don't have the time for 2 virtualizations technologies but I'll keep a nested instance of the other one, I never close doors!
As someone who’s setting up open source and self hosted small business solutions for myself, I absolutely adore Proxmox. Like these people behind it are the best. I have yet to see another hypervisor being so user friendly while being free for us non-corporate customers. I just hope the IOMMU supports hardware HSM passthroughs one day though.
@@walterscott9750 with amount of money I’m saving in just software licensing by using proxmox and not vmware or citrix, I’m fine by paying something to them in the future. They would be definitely deserving that money.
I'm surprised no one has brought up the issue of thin provisioning when migrating. I was surprised to find out that even when choosing qcow2, the disks came out the other side as a full size disk. You have to import 'the other way' to get that to work.
Removing the VMWare Tools after migration should work if you use the "old" Windows Software Center. Migration Speed is limited due to the VMWare settings for speed throttling for the backup feature (which Proxmox is using for the GUI-migration). You can also do live-migration over a NFS share. But this takes a few additional manual stept. On the other side you will learn a bit about the tech and the speed limitation for migration should be gone, too since you just mount the vmdk in Proxmox from the NFS and move it to the final storage. Also you shouldn't forget the The subsequent work after migration to optimize the performance of your VMs (virtio drivers for disk-drives and adding the quemu-agents. But i guess this is the topic of one of your following videos ;) Wish you lots of fun with Proxmox. I already have :)
@3:30 - can you show how you created the 5 network under SDN ???? - did you created the SDN under Datacenter or on the node itself? why i am asking is, if you get another Proxmox server will the SDN be copied over to the new Node?
For "folders" you want to use resource pools. You create pools of guests and then you can change the view to show the pools. Also you want to use virtio nics in your guests unless you have a specific reason to use something else.
Just migrated a couple of Windows Server VMs from VMware to Proxmox with no major issues apart from the waiting time. Got 600 megabits per second on average despite using gigabit ethernet. Had to delete the old interface from the VMs to use the previous IP address. When migrating using the wizard, it copies the exact UUID from the source machine and you have the option to take the same MAC address of your network interface from the source into the new machine. Don’t really know if it was a factor, but my machines’ licenses got activated without having to do anything else, and the activation count in volume license did not increase. Very impressed
I think the licensing hangup for me was likely because of my non-volume licensing license (say that 10-times fast!) This was a stand alone licensed copy of Windows 11 pro that has been updated from Windows 10 pro back in the day. It wasn't a surprise that it happened. I anticipate that in an enterprise environment with a KMS your licensing results would be different. Thank you for verifying that!
Thanks, Informative video run through of a migration. What is your experience with a ESXi VM with a HBA in passthrough mode? Something like the migration of a Linux storage VM having raw device access to the on-board SATA controller or PCIe HBA providing disks for a ZFS storage pool , would be interesting to see. Wonder of this get carried over without a hitch. My gut feeling pushes me towards exporting the ZFS pool and removing the passthrough controller prior to the migration but I might be mistaken.
about the light mode: try to work at evening/night where everything is dark around you and your monitor. it will burn your eyes or maybe you will get headache. and if you think you are only working at day or you have always lights around yourself turned on, think about other people watching your videos at night! btw having the monitopr not being to display bright content saves much energy. thatswhy also big monitoring walls at operations centers have a black background...
Yep. For a long time ProxMox didn't have the dark mode. Someone had to make a custom script to modify the web server files. Now it's built-in which is awesome.
Is 1GB/s that slow; Esxi and KVM use different disk formats? The disk has to be read, converted and re-made. I'd expect it to be tied to the CPU of the proxmox (and yours is certainly a beast!), but i saw another comment on here about how it is also depending on quickly Esxi allows the virtual disk file to be read.... Commercially, the company I work for, produce VMs and state support for Openstack. But from what i hear product management are going to open it up to 'all KVM platforms' (kind of impossible to QA every Linux platform out there IMO), but I guess it's to ensure we support customers doing exactly like you did.
We’ve seen vSphere/ESXi to CloudStack (KVM) migration in large production environments using the new CloudStack VMware migration tool, where one org migrate about 800-900 large VMs.
There were a variety of reasons I officially decided to move to Proxmox, and this is something that probably should have been touched on in the video. 1) Out of all of the FOSS hypervisors in the market right now, Proxmox is getting the most attention and visibility. Just look no further than Veeam's decision to officially support it 2) The development speed around Proxmox feels (at least) to be more active and aggressive and having it running on a more updated Debian kernel, getting regular patches, etc. is important to me. 3) As @asbestinuS also mentioned, native support for ZFS was another big benefit. Being able to leverage ZFS natively cuts down on RAID hardware which, as long as you have the RAM for ZFS, is a big benefit. I'll never say that I'm going Proxmox for good though, and there's still a ton of things that I don't really like about Proxmox (I'm working on a rant video about this, so sorry about that in advance! ;-). But the reality is that the shakeup of VMware really did open up a world of possibility for the future of virtualization and I still think we have at least a year or two before we see clear winners in the on-premises virtualization world.
what, if i have a VSAN Cluster, and all vm´s are located on the vsan and i have no option to migrate the vm´s to another storage that is supported from proxmox, what can be my migration way?
I heard omnissa supporting nutanix now. What other hypervisors it supports as of today ? What’s in roadmap? Have you seen proxmox linked clones? Is it a competition to omnissa in any way?
@@VictorEstrada I think you’ll see a lot more investment in the Blast development. In the past, VMware syphoned off a lot of our earnings to support other projects. Horizon and W1 had their development funding limited.
I am still evaluating a VMWare replacement for our enterprise. Since we are more interested in VM's and not containers/cloud... I am currently down to Proxmox and oVirt/Oracle VM Manager.
I'm not entirely sure what the next steps will be. I'm very interested in going HA with Proxmox, but the dual-node host I was running ESXi on is getting old by todays standards.
I expected requiring reactivation of Windows. Thankfully I should be able to activate again. I also expected the renaming of network adapters in Linux. Thank you for sharing your migration experience
Sir, can you tell onething if i want to use one 1 tb Hdd but some portion for promox and other free space for other purpose, how to do this or any way to use a pendrive for only proxmox server.
bu uyarıyı alıyorum VMware için; macOS 14 is not a supported guest operating system for Workstation 16.x virtual machines. Some features of it might not work properly unless its virtual hardware is upgraded.
It'll work, importing VMs directly from vCenter. But VMware does some weird traffic routing which leads to even slower migration speeds. They expose the disks over HTTP and all Proxmox does is basically download them, speeds are a direct result of how fast stuff comes out of VMware.
@@hanneslaimer8851 I meant vcenter itself. I've seen issues with the 14 or something disks vcenter creates... They don't migrate over or hit some kvm limitation... Probably by design
@@homehome4822 I had the same issue - Just to keep it around, I created a virtual ESXi VM on my promox and moved the vcenter VM to it. I doubt I will need it as it has been 4 months since I switched now.
@@da99beast great minds think alike. That was my workaround too. With everyone on TH-cam showing off proxmox's new V2V migration tool I was hoping for a way to get vcenter off exi.
Interesting and frighting. My question is, what caused the loss of the activation. The change of the virtual disk UUID or the change of hypervisor? Since 2009 I use Virtualbox and I have collected ~70 VMs. I have all Windows releases from 1.04 (1987) till 11 Pro (2024) in a VM. My oldest VM still in use is Windows XP Home installed and activated in March 2010. It survived 2 Virtualbox owners; 3 desktops and 4 CPUs and it never lost its activation. The same is true for almost all of my Windows VMs. They are still activated, only Vista lost its activation and I had the feeling, that Vista activation was more strict and not really adapted for the VM area yet. Note that Windows XP's control panel -> system shows that I run a Ryzen CPU, so Windows knows the host CPU. When I run the VM on my laptop, it nicely indicated the i5 CPU of my laptop.
My suspicion is that it was due to the quantity of changes that occurred in the migration. There's no 'official' document from Microsoft that outlines what hardware changes force a reactivation, but in this migration a lot changed, like CPU type, disk serial and ID, storage controller type, etc. Either way, if you're having success staying within Virtualbox for your VMs, you're probably not going to have to worry about it. But, if you decide to migrate those VMs to Proxmox or something else, you're likely going to have the same issues.
Three new ESXi hosts in the last month for the home lab. Eighty Dell Sapphire Rapids ESXi hosts coming in at work. As Rick Astley sang, "Never gonna give you up."
Not even for the load balancing necessarily, though, that would be nice. Just simple quality of life features like the ability of freeing up a host for maintenance without touching every single VM by hand. So yea I miss DRS
@@thetux0815 After you configure HA you can put the host in maint mode from the CLI and it will evacuate the host without manually migrating a single vm :)
Oh someone that likes light mode!? 🤔 Well what about your clothes and hat? They are not in "light mode".🤪 And just wait when you can't turn off "light mode" on your beard.😟
I did a migration from ESXi to Proxmox just over two years ago (before the Broadcom massacre occurred). The switch wasn't even planned, as I'd bought new server gear who's disk controller was not supported by ESXi (my fault for not researching) and just jumped into the Proxmox deep end. I was shocked at how easy and rather painlessly I was able to migrate nearly every VM (mix of FreeBSD and Linux) to the new hypervisor. Adding in ZFS for local storage and a built-in backup solution to send those snapshots to my TrueNAS was like magic! Hope the new setup works out for you in the long run as well as it has for me!
The end... I had to laugh!!! 🤣
Rest in peace poor vCenter
I’m glad you enjoyed the bit! Wasn’t sure how it was going to land! 😂
@@2GuysTek I enjoyed it a lot... and for my part, I'm still waiting that my management will take a decision avout VMware.
My lab will follow the decision and if we stay by VMware, I'll take VMUG again.
I just don't have the time for 2 virtualizations technologies but I'll keep a nested instance of the other one, I never close doors!
And that makes total sense. In the day job, we’re moving away from VMware, to what is yet to be decided.
@@2GuysTek either way, I'll be happy... so I'm not worried about the outcome!😊
Vcenter doesn't deserve what's happening to it, f u Broadcom, rip vcsa 🙏🕊️
As someone who’s setting up open source and self hosted small business solutions for myself, I absolutely adore Proxmox. Like these people behind it are the best. I have yet to see another hypervisor being so user friendly while being free for us non-corporate customers. I just hope the IOMMU supports hardware HSM passthroughs one day though.
Free is the given word.. and nothing is free… eventually you pay for it
@@walterscott9750 with amount of money I’m saving in just software licensing by using proxmox and not vmware or citrix, I’m fine by paying something to them in the future. They would be definitely deserving that money.
Excellent video Rich! Love the migration and going through the steps you took! Keep up your amazing content!
I'm surprised no one has brought up the issue of thin provisioning when migrating. I was surprised to find out that even when choosing qcow2, the disks came out the other side as a full size disk. You have to import 'the other way' to get that to work.
You do a great job with your videos. Thank you for the excellent and timely content.
Removing the VMWare Tools after migration should work if you use the "old" Windows Software Center.
Migration Speed is limited due to the VMWare settings for speed throttling for the backup feature (which Proxmox is using for the GUI-migration).
You can also do live-migration over a NFS share. But this takes a few additional manual stept. On the other side you will learn a bit about the tech and the speed limitation for migration should be gone, too since you just mount the vmdk in Proxmox from the NFS and move it to the final storage.
Also you shouldn't forget the The subsequent work after migration to optimize the performance of your VMs (virtio drivers for disk-drives and adding the quemu-agents. But i guess this is the topic of one of your following videos ;)
Wish you lots of fun with Proxmox. I already have :)
@3:30 - can you show how you created the 5 network under SDN ????
- did you created the SDN under Datacenter or on the node itself?
why i am asking is, if you get another Proxmox server will the SDN be copied over to the new Node?
I like that your iOT VLAN is 666!
so does Lucifer
For "folders" you want to use resource pools. You create pools of guests and then you can change the view to show the pools. Also you want to use virtio nics in your guests unless you have a specific reason to use something else.
Tah I think not many people knows about and use those different views. Problem in the Pools view is that you cant se Hypervisor stats.
@@hule8899 Just switch between views. Rarely would you need both at once.
Welcom to the prox-side
and Dark Mode is the best
Switched from ESXi to PROXMOX about 5 years ago... no regrets....
Revo uninstaller in advanced mode works well to remove a forgotten VMWare tools on Windows
Appreciate the tip!
The different OS types in the VM config during import are used for prefilling sensible defaults.
Great video, hahaha the emotional ending love it :)
About time, good choice!
Just migrated a couple of Windows Server VMs from VMware to Proxmox with no major issues apart from the waiting time. Got 600 megabits per second on average despite using gigabit ethernet. Had to delete the old interface from the VMs to use the previous IP address.
When migrating using the wizard, it copies the exact UUID from the source machine and you have the option to take the same MAC address of your network interface from the source into the new machine. Don’t really know if it was a factor, but my machines’ licenses got activated without having to do anything else, and the activation count in volume license did not increase. Very impressed
I think the licensing hangup for me was likely because of my non-volume licensing license (say that 10-times fast!) This was a stand alone licensed copy of Windows 11 pro that has been updated from Windows 10 pro back in the day. It wasn't a surprise that it happened. I anticipate that in an enterprise environment with a KMS your licensing results would be different. Thank you for verifying that!
Thanks,
Informative video run through of a migration.
What is your experience with a ESXi VM with a HBA in passthrough mode?
Something like the migration of a Linux storage VM having raw device access to the on-board SATA controller or PCIe HBA providing disks for a ZFS storage pool , would be interesting to see. Wonder of this get carried over without a hitch.
My gut feeling pushes me towards exporting the ZFS pool and removing the passthrough controller prior to the migration but I might be mistaken.
about the light mode: try to work at evening/night where everything is dark around you and your monitor. it will burn your eyes or maybe you will get headache. and if you think you are only working at day or you have always lights around yourself turned on, think about other people watching your videos at night! btw having the monitopr not being to display bright content saves much energy. thatswhy also big monitoring walls at operations centers have a black background...
Yep. For a long time ProxMox didn't have the dark mode. Someone had to make a custom script to modify the web server files. Now it's built-in which is awesome.
Did the same couple of months ago, you probably chose ProxMox because of the hardware support/newer kernel. That was my problem with XCP-NG.
Is 1GB/s that slow; Esxi and KVM use different disk formats? The disk has to be read, converted and re-made. I'd expect it to be tied to the CPU of the proxmox (and yours is certainly a beast!), but i saw another comment on here about how it is also depending on quickly Esxi allows the virtual disk file to be read....
Commercially, the company I work for, produce VMs and state support for Openstack. But from what i hear product management are going to open it up to 'all KVM platforms' (kind of impossible to QA every Linux platform out there IMO), but I guess it's to ensure we support customers doing exactly like you did.
Congratulations. You've migrated to the right virtualization platform.
We’ve seen vSphere/ESXi to CloudStack (KVM) migration in large production environments using the new CloudStack VMware migration tool, where one org migrate about 800-900 large VMs.
Why did you choose Proxmox over Xcp-ng?
I am also curious for the what was the deciding factor was
@@brockwilkie6022 I don't know but for me a big factor is ZFS which Proxmox nativly supports
There were a variety of reasons I officially decided to move to Proxmox, and this is something that probably should have been touched on in the video.
1) Out of all of the FOSS hypervisors in the market right now, Proxmox is getting the most attention and visibility. Just look no further than Veeam's decision to officially support it
2) The development speed around Proxmox feels (at least) to be more active and aggressive and having it running on a more updated Debian kernel, getting regular patches, etc. is important to me.
3) As @asbestinuS also mentioned, native support for ZFS was another big benefit. Being able to leverage ZFS natively cuts down on RAID hardware which, as long as you have the RAM for ZFS, is a big benefit.
I'll never say that I'm going Proxmox for good though, and there's still a ton of things that I don't really like about Proxmox (I'm working on a rant video about this, so sorry about that in advance! ;-). But the reality is that the shakeup of VMware really did open up a world of possibility for the future of virtualization and I still think we have at least a year or two before we see clear winners in the on-premises virtualization world.
Because Proxmox Virtual Environment, Proxmox Mail Gateway, and Proxmox Backup Server outclass XCP-ng.
what, if i have a VSAN Cluster, and all vm´s are located on the vsan and i have no option to migrate the vm´s to another storage that is supported from proxmox, what can be my migration way?
I work for Omnissa and hope our VDI product, Horizon, eventually supports Proxmox. But I know other hypervisors are a higher priority.
I heard omnissa supporting nutanix now. What other hypervisors it supports as of today ? What’s in roadmap? Have you seen proxmox linked clones? Is it a competition to omnissa in any way?
I'm the VDI admin where I work. You guys should concentrate on multimedia redirection, and improving the compute experience.
@@Mallikarjunaish Sorry brother, I’m not allowed to say.. But announcements are coming relatively soon.
@@VictorEstrada I think you’ll see a lot more investment in the Blast development. In the past, VMware syphoned off a lot of our earnings to support other projects. Horizon and W1 had their development funding limited.
I am still evaluating a VMWare replacement for our enterprise. Since we are more interested in VM's and not containers/cloud... I am currently down to Proxmox and oVirt/Oracle VM Manager.
Are you going yo convert the esx nodes to proxmox nodes for some HA?
I'm not entirely sure what the next steps will be. I'm very interested in going HA with Proxmox, but the dual-node host I was running ESXi on is getting old by todays standards.
I've done manual VM migrations from Xen and ESXi to proxmox and it's not that bad but it's nice to have a point and click method of doing it.
you could have tried Veeam as it supports proxmox now and restored backups to it.
I expected requiring reactivation of Windows. Thankfully I should be able to activate again. I also expected the renaming of network adapters in Linux. Thank you for sharing your migration experience
Thanks for the info. Are you sure you and Kevin Smith aren't related?
In regards of windows licensing you can migrate the SMBIOS UUID from the VM to Proxmox. It can solve some licensing issues.
Any tips on how?
@@derekbush6340 Options menu on the VM in Proxmox
Sir, can you tell onething if i want to use one 1 tb Hdd but some portion for promox and other free space for other purpose, how to do this or any way to use a pendrive for only proxmox server.
Windows Server licensing does the same exact thing. Also happens with XCP-ng
bu uyarıyı alıyorum VMware için;
macOS 14 is not a supported guest operating system for Workstation 16.x virtual machines. Some features of it might not work properly unless its virtual hardware is upgraded.
Use pools if you want VMs in folders.
Dude you missed Dark theme!!! common!
lol
Have you tried migrating a vcenter vm?
I have not, it still existing in the offline ESXi stack. What's your thinking?
It'll work, importing VMs directly from vCenter. But VMware does some weird traffic routing which leads to even slower migration speeds. They expose the disks over HTTP and all Proxmox does is basically download them, speeds are a direct result of how fast stuff comes out of VMware.
@@hanneslaimer8851 I meant vcenter itself. I've seen issues with the 14 or something disks vcenter creates... They don't migrate over or hit some kvm limitation... Probably by design
@@homehome4822 I had the same issue - Just to keep it around, I created a virtual ESXi VM on my promox and moved the vcenter VM to it. I doubt I will need it as it has been 4 months since I switched now.
@@da99beast great minds think alike. That was my workaround too. With everyone on TH-cam showing off proxmox's new V2V migration tool I was hoping for a way to get vcenter off exi.
Interesting and frighting. My question is, what caused the loss of the activation. The change of the virtual disk UUID or the change of hypervisor?
Since 2009 I use Virtualbox and I have collected ~70 VMs. I have all Windows releases from 1.04 (1987) till 11 Pro (2024) in a VM. My oldest VM still in use is Windows XP Home installed and activated in March 2010. It survived 2 Virtualbox owners; 3 desktops and 4 CPUs and it never lost its activation. The same is true for almost all of my Windows VMs. They are still activated, only Vista lost its activation and I had the feeling, that Vista activation was more strict and not really adapted for the VM area yet.
Note that Windows XP's control panel -> system shows that I run a Ryzen CPU, so Windows knows the host CPU. When I run the VM on my laptop, it nicely indicated the i5 CPU of my laptop.
My suspicion is that it was due to the quantity of changes that occurred in the migration. There's no 'official' document from Microsoft that outlines what hardware changes force a reactivation, but in this migration a lot changed, like CPU type, disk serial and ID, storage controller type, etc. Either way, if you're having success staying within Virtualbox for your VMs, you're probably not going to have to worry about it. But, if you decide to migrate those VMs to Proxmox or something else, you're likely going to have the same issues.
Dark Mode All The Things!
What about LXD ... snap lxd ..
Nice
I rather do the migration via CLI using SSHFS and qm disk import, than use thw WEB UI.
I just like the old school way.
Duno, but works better for me.
you ideally should have 4 nodes for vsan
What took you so long? 🙂
12:50 disagree. unsubscribed 😂
I have been using Proxmox for years.....
It's not late.
Three new ESXi hosts in the last month for the home lab. Eighty Dell Sapphire Rapids ESXi hosts coming in at work. As Rick Astley sang, "Never gonna give you up."
Biggest thing holding me back is that there is no DRS like load balancing yet.
This really is a glaring hole in Proxmox's feature set. 100% agree!
Not even for the load balancing necessarily, though, that would be nice. Just simple quality of life features like the ability of freeing up a host for maintenance without touching every single VM by hand. So yea I miss DRS
@@thetux0815 After you configure HA you can put the host in maint mode from the CLI and it will evacuate the host without manually migrating a single vm :)
Until Proxmox supports Citrix VaaD it’s a no go
Dark mode is better
Respectfully disagree! 🤣
vmware gone for good unless broadcome come to their mind and fix that
Looks like another one is out called Pextra th-cam.com/video/n3rVCMH-MDM/w-d-xo.htmlfeature=shared I am trying to get the ISO. Anyone tried it?
LOL you have more RAM than HDD space.
Oh someone that likes light mode!? 🤔 Well what about your clothes and hat? They are not in "light mode".🤪 And just wait when you can't turn off "light mode" on your beard.😟