K3s + Rancher + Longhorn is a great setup. I was able to make a test K3s cluster watching your videos and Jim's Garage. I have learned so much from your videos, they are so well done.
Hey Christian, since I've had many problems with longhorn, mainly perfomance and kubernetes upgrade (evicting volumes to other nodes) related, I found the drbd operator called piraeus. In my opinion it is vastly superior and makes use of proven tech like drbd for replication and either lvm or zfs for the backend storage. I hope you can give it a try and maybe compare the two. Piraeus could really use more attention as it is pretty easy to use as well and much faster.
Really love the idea of Longhorn as it's so easy to setup. But for me i ran into quota issues. That the volumes were out of space even though they were not. Apprantly when working with WAL files like prometheus uses the Used diskspace on disk and in longhorn differs by quite a lot due to the "Cleanup" Not replicating to the disk usage in longhorn. Which for me was a 2 GB metrics file that took up a 50GB Longhorn PV. and Prometheus crashed due to "Not enough diskspace. Now using Rook-Ceph for that reason.
@burads yeah I've had a number of other issues as well that I did not mention. Like problems with volume replication lagging behind and replicas failing at times. Also on bigger volumes draining nodes is nearly impossible and takes ages and resyncing failed replicas always resulted in the full volume having to be copied again, which is really annoying.
Hey Christian, what does Longhorn use in backend for storage? Is it ISCSI, NFS, NBD, ... ? It seems like it creates three big files, one on each node, exposes it over ISCSI, creates three-way raid1 md device, creates filesystem on top of it and then mounts it on host that hosts the container.
Hey! Great question, from my understanding, it is leveraging its own built-in mechanisms rather than traditional protocols like iSCSI or NFS, but I'm not sure.
Used Longhorn quite a bit in a homelab, and had/have nothing but a problems with it, in any scenario that involves more than few GB of data. Too lazy to switch to something like OpenEBS, CEPH, DirectPV or Piraeus at the moment though 🙂 Basically, anything usable rather than Longhorn.
@@christianlempa it could be, but the fact remains that it handles those situations poorly. Perhaps also my lack of in-depth knowledge about the system plays a role. Never the less, both their docs, and their resilience could be better 😀
So what if you also have proxmox ceph and running kubernetes? I like to apply both best practices for HA and have some redundancy. What if you run K8s on top of Promox with ceph and longhorn..? Doesn't that sound very very redundant? What is your architecture advice here?
That's a good question, and honestly, I'm not sure what I would do here. I think it doesn't make much sense to replicate the volumes when the whole disk is replicated in Ceph. I need some time to think about it and play around with Ceph.
What if I have a single-node cluster? I'd like it to be as simple as can be, but have e.g. ZFS volumes or datasets be created in the ZFS storage on my Proxmox host. I am trying my luck with proxmox-csi-plugin, but I don't think I am happy with it...
Under the hood its the same tech. K8s is a management layer on top for very large distributed apps, cool tech. But most businesses don't ever need it. Wait a few years until the videos come with the title "you probably don't need k8s" after the hype dies down (which is already happening). Edit; not to write off the tutorial christian, very thorough and well explained! If you think or want to work in mega scale services, this is valuable.
![เจ็บที่ยังฮัก - ม่อน วรวิทย์ [ Official Mv ] จอนนี่มิวสิค](http://i.ytimg.com/vi/KALIGSRSjFs/mqdefault.jpg)
K3s + Rancher + Longhorn is a great setup.
I was able to make a test K3s cluster watching your videos and Jim's Garage.
I have learned so much from your videos, they are so well done.
That is so nice! Thank you so much for the feedback :)
Really loved the video. I recently deployed longhorn in my cluster and this video made me realize how many mistakes I made. 😅
Thank you so much! :)
This is great! Thanks for taking the time to make such valuable and comprehensive content 👍
Glad you enjoyed it!
excellent tutorial as always.. looking forward to the upcoming traefik and certmanager tutorial
Thank you so much! 😊 it’s almost done so I’m gonna expect it to publish next week
@@christianlempa I can't wait. ;)))
Hey Christian,
since I've had many problems with longhorn, mainly perfomance and kubernetes upgrade (evicting volumes to other nodes) related, I found the drbd operator called piraeus. In my opinion it is vastly superior and makes use of proven tech like drbd for replication and either lvm or zfs for the backend storage. I hope you can give it a try and maybe compare the two. Piraeus could really use more attention as it is pretty easy to use as well and much faster.
Really love the idea of Longhorn as it's so easy to setup. But for me i ran into quota issues. That the volumes were out of space even though they were not. Apprantly when working with WAL files like prometheus uses the Used diskspace on disk and in longhorn differs by quite a lot due to the "Cleanup" Not replicating to the disk usage in longhorn. Which for me was a 2 GB metrics file that took up a 50GB Longhorn PV. and Prometheus crashed due to "Not enough diskspace.
Now using Rook-Ceph for that reason.
@burads yeah I've had a number of other issues as well that I did not mention. Like problems with volume replication lagging behind and replicas failing at times. Also on bigger volumes draining nodes is nearly impossible and takes ages and resyncing failed replicas always resulted in the full volume having to be copied again, which is really annoying.
But maybe they fixed a couple of those issues. Though performance is always going to be bad because of the architecture
I would be interested in a video about using Velero for backups in combination with Longhorn!
Great vid :)
Thanks! Never heard of it, but I will check it out ;)
Great Video, especially for the backup / restore part and the example with a helm chart!
Thank you! :)
Excellent tutorial, wonderful explanation as always
Thank you so much! 😍
Using Longhorn in kubernetes since 4 years doing better in every release
Nice!
Hello Christian I was just wondering what is the terminal client that you are using? Thanks
Hey Christian, what does Longhorn use in backend for storage? Is it ISCSI, NFS, NBD, ... ? It seems like it creates three big files, one on each node, exposes it over ISCSI, creates three-way raid1 md device, creates filesystem on top of it and then mounts it on host that hosts the container.
Hey! Great question, from my understanding, it is leveraging its own built-in mechanisms rather than traditional protocols like iSCSI or NFS, but I'm not sure.
excellent tutoriial! keep up the good work :)
May I ask what's the confguration of your shell environment ?
Thank you so much! There's a video about it: th-cam.com/video/NfggT5enF4o/w-d-xo.html
Used Longhorn quite a bit in a homelab, and had/have nothing but a problems with it, in any scenario that involves more than few GB of data. Too lazy to switch to something like OpenEBS, CEPH, DirectPV or Piraeus at the moment though 🙂 Basically, anything usable rather than Longhorn.
Maybe the network connection and the disks are not fast enough to replicate that amount of data.
@@christianlempa it could be, but the fact remains that it handles those situations poorly. Perhaps also my lack of in-depth knowledge about the system plays a role. Never the less, both their docs, and their resilience could be better 😀
So what if you also have proxmox ceph and running kubernetes? I like to apply both best practices for HA and have some redundancy. What if you run K8s on top of Promox with ceph and longhorn..? Doesn't that sound very very redundant? What is your architecture advice here?
Also backup remotely. .. 321 backup rule
That's a good question, and honestly, I'm not sure what I would do here. I think it doesn't make much sense to replicate the volumes when the whole disk is replicated in Ceph. I need some time to think about it and play around with Ceph.
What if I have a single-node cluster? I'd like it to be as simple as can be, but have e.g. ZFS volumes or datasets be created in the ZFS storage on my Proxmox host.
I am trying my luck with proxmox-csi-plugin, but I don't think I am happy with it...
Hm good question, I'm not sure if it works great for a single node cluster, it's recommended anyway to run a minimum of 3 nodes in Kubernetes.
i like rook ceph, but longhorn seems nice too
Nice, maybe I'll check it out at some point.
12:00 install add-on on Chrome or safari? You mean firefox or a firefox fork, right?
No firefox sucks ;)
Do a tutorial about your terminal please.
There is: th-cam.com/video/NfggT5enF4o/w-d-xo.html
How do you delete longhorn when it's stuck deleting lol
It depends on which resources are stuck, maybe let's follow up on Discord ;)
Gigabytes ✖
G B bites ✔
I hope I did it right :D
Isn't longhorn windows vista? Is this name even legal?
I don’t think it’s a tm :D
No, it is this big bovine creature. 😅
@@FredrikRambrisunderrated reply 🤣
Docker still better
For easy home setups. Yes
For fun and/or HA setups. No
Use whatever works for you tho
Under the hood its the same tech. K8s is a management layer on top for very large distributed apps, cool tech. But most businesses don't ever need it. Wait a few years until the videos come with the title "you probably don't need k8s" after the hype dies down (which is already happening). Edit; not to write off the tutorial christian, very thorough and well explained! If you think or want to work in mega scale services, this is valuable.
For small deployment yeah, wait until you need to manage multiple instances and manage system upgrades. Basic docker engine has limited capabilities