Your vedios are really helpful and i love the the way you explain it with real time use cases. Would request you to cover helm charts from scratch to complex. Lets say how to create helm charts, how to convert our own applications to helm charts etc etc. I would love to have some series of vedios on this with the ongoing standard use cases which currently organisations are doing. Thanks! keep the great work going going. Happy learning!!
I think I said it in one of you first videos but again, you are the best DevOps educator on youtube. Clearly you've got experience that many others don't. Your vids aren't some robotic read-out of the official docs for views, so I can always find something new. Great style, great structure. You definitely deserve more recognition but it's getting there and certainly it will get there!
I will rate you 100% . The content was too notch and the explanation was just what I need now. I wanted to deploy a postgres database on k8s, so I sort for resources on data persistent and I couldn't find any that is more explanatory. Your sample that resides in github was top-notch.
Thanks to you, I learned a lot of "production / professional" ways to understand and implement those strategies. None of your videos get bored, I even watch some vid 5-6 times to strengthen my knowledge until I completely understand everything. Thank you for your supports.
I don't know what I would do without your channel. You have an amazing skill to explain complicated topics in such a wonderful, easy-to-understand way. Keep going!! A video with Tekton would be great, if I can propose something.
Your video on the topic is excellent!! I would love to see more examples of statefulsets because your video has raised some critical issues about cluster failure which couldnt be automatically resolved by the kubernetes engine and may require manually fixing the cluster setup. Can you please show possible solutions on fixing cluster issues that cannot be automatically resolved by the k8s engine?
StatefulSets are an interesting beast. When I first ran into them, I didn't know what to make of it : I've always thought of k8s as stateless-oriented and StatefulSets felt like an anti-pattern. Once I looked into clustered / high availability databases and the on-going work in that space, it made a lot more sense (I'm a big fan of Stolon, if you're into distributed Postgres). K8s is really a great engine to distribute *stuff*, it doesn't have to be microservices.
This is an awesome series, but I wanted to point out an issue with the command @22:28. The command adds an extra ":6379" at the end of the string which is recorded in the env var. The output from: kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ' in my case was: "10.244.0.6:6379 10.244.0.8:6379 10.244.0.10:6379 10.244.0.12:6379 10.244.0.14:6379 10.244.0.16:6379 :6379" This causes the next cluster create command to fail. Took me a little bit to understand why the command was failing if anyone else runs into the issue.
The way you explain concepts and then finish them with real world examples is what most engineers wish to see. Redis, RabbitMQ I have watched both and request you to deploy more such complex applications. Also, if you can tell us what could be the best setup for cluster monitoring and how to approach to solve real world cluster down scenarios, that would be great.
Outstanding video mate. I'd say you are a welcome addition to our sunny shores but you chose VIC instead of WA :P Just teasing Melbourne rocks, lived there for 4 years myself as a young lad.
just rewatched this... thinking I'm picking up on the complexity of stateful sets and ... a stretched environment like multiple AZ's as present on AWS EKS... any chance you've done a video, where say 3 or more stateful sets are pined to nodes (AWS EKS Nodegroups). this can then be used against the Redis deployment and PostgreSQL deployment ?
My Redis series does this already. A statefulset will pin a pod to the same node's volume claim. Unless that node disappeared, the pod will always stay on that node. If a rolling node upgrade occurs which my roll new VMs, you may lose data ( depending how the cloud provider does updates)
@@MarcelDempers understand the stateful pod will gravitate back to where the pvc is, but i want to pin/direct the pod before that, I have a EKS cluster spread across 3 x az's, and additionally the cluster have node groups for apps and nodegroups for db... i want to direct the stateful set to the database node groups, and additionally push each of the Redis masters onto a node in each of the az's.
another good videos on Kubernetes.. can you please make video to capture logs of application in cabernets in ELK with cabana or any other tools to view app logs
I would like to add a correction, please correct me if I'm wrong. At 12:18 you said in deployment scaling the pods scale all at same time but I think with rolling updates we can choose % of scaling pods at a time.
Hello Marcel, Can you make a video based on DevOps security aspects ?.. Like all things to be considered when we think about security in CICD? Like as a hiring manager what all u think is important and relevant.
In your example you used ClusterIP type of service. Is headless service not one of requirements for statefulset? Or I can still use ClusterIP? Good video anyway, thanks.
Hi At first would like to thank you for the wonderful videos...I am having request... Could you please make a detailed video on how MySQL High availability works and how to configure it on k8s. Also detailed video from. Basic to Advance on K8S Networking.. Thanks a lot in Advance..Stay Safe..!!
definitely helped explain stateful sets and pv's snd pvc's... but now to map to real world... 2 AZ's on AWS with Prometheus deployed, with a ALB in front ? if the Prometheus servers were running on the same node I'd say lets use deployments, that way either node would have access to all the data, but then you going to have duplication where they scrape the same target. lets make it simpler. one prometheus server, running active in AZ1.. but it needs to be protected to be able to fail over to AZ2... snd as I discovered EFS not allowed.
Hey, do you have a video describing how to make an EBS based StatefulSet in AWS? It seems that there's an issue reguarding EBS volumes being bound to an AZ. I heard that I should create an ASG per AZ, and make my StatefulSet have Pods across the AZs, but I don't know how to enforce that behavior. Most of my volumes are EFS and FSx so far, but I'd like to run a database in EKS, which would need EBS for speed, and I don't want to miss anything.
Hi Thanks for the brilliant content! Does the redis deployment automatically distribute across six note? If I have a taint on master nodes I guess that will be an issue? Cheers
Hello, thank you for the tutorial ! I've got some questions at 25:30 you show that PV are retained but the reclaim policy is set to Delete. How does it work ? Does it only occur when the StatefulSet get fully deleted ?
Great question. As per documentation, For reclaim policy "Delete", the volumes are only deleted when the PVC's are deleted. Current statefulset behaviour is PVC's are not deleted with pods or the statefulsets are deleted since they have a different lifespan.
Learning about PVs and noticed that all of the PVs deployed into our cluster have the reclaim policy = Delete, which had me concerned that our engineering team had possibly made a terrible oversight in the deployments. This comment is exactly what I wanted to understand. Thank you so much for this comment and the excellent statefulset tutorial!
Thank you so much for your video. I have a question about scaling down Cassandra cluster, as you know, Cassandra cluster will store all the data in different databases, let's assume we have 5 Cassandra databases with replica of 3, and then scaling down to 1 Cassandra database, how can we know if the left Cassandra database consists all the data of the 5 Cassandra databases. Or it is not possible to do scaling down Cassandra like Redis in k8s? Thank you very much
Hi, the yamls shown in the video are different from the git. For instance the service type for redis cluster used in the video is ClusterIP but in git its headless service. Also why service name is to be mentioned in statefulset definition?
I was looking over the Github repo for the redis stateful set. I have a question. If each replica is mounting to the same host path is there any chance for mounting collisions and data corruption?
hello Marcel !! could you please let me know how to troubleshoot based on the error at the end of ur video ( scale down to zero - and then making up the replicas , but still the cluster is down ) how to overcome this ?
It's an interesting issue right ? :) This is part of the reason I recommend SaaS solutions for storage. I myself am not a Redis expert, so I would need to dig into 1) Redis cluster best practises, 2) Redis cluster troubleshooting \ repair process. I am planning a Redis series in the future 💪🏽
Good question. I have not tried it. The statefulset spec.serviceName might have some mandatory requirements, but nothing stops you from creating another clusterIP service or LoadBalancer one and selecting the statefulset pods with a label selector, if you need to
Hello, I use microk8s and run $IPs = $(microk8s.kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ') I got =: command not found How can I solve this problem?
You are talking too fast sometimes, when I watch yours videos i need to pause coule of times and rewind 20 sek to catch everything I need. But sometimes there is a lack of details for example how this web app is connecting to redis in this example
Dude, your series on k8s is - by far - the celarest, most concise and illuminating set of vids out there. I sincerely commend you, sir.
Greatly explained . Currently working as DevOps intern and really was stuck with this statefulset and deployment. keep up good work
I am on the virtuous path of watching every single video of this channel. An practice it. So far it is going great and I have learned a ton.
Very helpful. Thanks for the videos with a good emphasis on the "Why" part.
you, sir, are answering all the exact questions which no one other answers. thank you.
Your vedios are really helpful and i love the the way you explain it with real time use cases. Would request you to cover helm charts from scratch to complex. Lets say how to create helm charts, how to convert our own applications to helm charts etc etc. I would love to have some series of vedios on this with the ongoing standard use cases which currently organisations are doing. Thanks! keep the great work going going. Happy learning!!
I think I said it in one of you first videos but again, you are the best DevOps educator on youtube. Clearly you've got experience that many others don't. Your vids aren't some robotic read-out of the official docs for views, so I can always find something new. Great style, great structure. You definitely deserve more recognition but it's getting there and certainly it will get there!
I will rate you 100% . The content was too notch and the explanation was just what I need now.
I wanted to deploy a postgres database on k8s, so I sort for resources on data persistent and I couldn't find any that is more explanatory.
Your sample that resides in github was top-notch.
Thanks to you, I learned a lot of "production / professional" ways to understand and implement those strategies. None of your videos get bored, I even watch some vid 5-6 times to strengthen my knowledge until I completely understand everything. Thank you for your supports.
You can explain very good and in this way many others can understand this technologies
I don't know what I would do without your channel. You have an amazing skill to explain complicated topics in such a wonderful, easy-to-understand way. Keep going!!
A video with Tekton would be great, if I can propose something.
Your video on the topic is excellent!! I would love to see more examples of statefulsets because your video has raised some critical issues about cluster failure which couldnt be automatically resolved by the kubernetes engine and may require manually fixing the cluster setup. Can you please show possible solutions on fixing cluster issues that cannot be automatically resolved by the k8s engine?
StatefulSets are an interesting beast. When I first ran into them, I didn't know what to make of it : I've always thought of k8s as stateless-oriented and StatefulSets felt like an anti-pattern. Once I looked into clustered / high availability databases and the on-going work in that space, it made a lot more sense (I'm a big fan of Stolon, if you're into distributed Postgres). K8s is really a great engine to distribute *stuff*, it doesn't have to be microservices.
You are so precise, even laser beam refracts. I really like your presentation. Thank you!
I loved this stateful sets explanation
Thank you for your awesome videos, one of the best series of videos on K8s on youtube. TYSM.
Man, your videos are awesome. Learnt a lot from it!
stateful explained in a very clear way. Thank you for the video.
You are a great instructor and cross Nana 👏
This is an awesome series, but I wanted to point out an issue with the command @22:28. The command adds an extra ":6379" at the end of the string which is recorded in the env var.
The output from:
kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 '
in my case was:
"10.244.0.6:6379 10.244.0.8:6379 10.244.0.10:6379 10.244.0.12:6379 10.244.0.14:6379 10.244.0.16:6379 :6379"
This causes the next cluster create command to fail. Took me a little bit to understand why the command was failing if anyone else runs into the issue.
Now that's called pure gold 💰 . Awesome stuff mate ! 👏
The way you explain concepts and then finish them with real world examples is what most engineers wish to see. Redis, RabbitMQ I have watched both and request you to deploy more such complex applications. Also, if you can tell us what could be the best setup for cluster monitoring and how to approach to solve real world cluster down scenarios, that would be great.
You are best keep sharing the concept and. Keep it simple as other devops video are not easy to understand on another channels
A very clear explanation, thanks!
Outstanding video mate. I'd say you are a welcome addition to our sunny shores but you chose VIC instead of WA :P
Just teasing Melbourne rocks, lived there for 4 years myself as a young lad.
Very Very helful vedio and was very easy to understand StatefulSet. Thanks you so much...
This fellow is no mere mortal
It’s very clear and concise.. really appreciate it..
very good in-depth explanation. just what was looking for. thanks
Peace Marcel, You are the man!
Love ur content! Keep on rockin'
This is a very very good tutorial. Thank you and subscribed!
just rewatched this...
thinking I'm picking up on the complexity of stateful sets and ... a stretched environment like multiple AZ's as present on AWS EKS... any chance you've done a video, where say 3 or more stateful sets are pined to nodes (AWS EKS Nodegroups).
this can then be used against the Redis deployment and PostgreSQL deployment ?
My Redis series does this already. A statefulset will pin a pod to the same node's volume claim. Unless that node disappeared, the pod will always stay on that node. If a rolling node upgrade occurs which my roll new VMs, you may lose data ( depending how the cloud provider does updates)
@@MarcelDempers understand the stateful pod will gravitate back to where the pvc is, but i want to pin/direct the pod before that, I have a EKS cluster spread across 3 x az's, and additionally the cluster have node groups for apps and nodegroups for db... i want to direct the stateful set to the database node groups, and additionally push each of the Redis masters onto a node in each of the az's.
Could you please make a video on stateless set
another good videos on Kubernetes.. can you please make video to capture logs of application in cabernets in ELK with cabana or any other tools to view app logs
Thanks for the kind words 💪🏽
We have started that journey already 🤓
Come follow along th-cam.com/play/PLHq1uqvAteVvfDxFW50Mdezk0xum-tyHT.html
you are awsom! thank you the DevOps guy))
super well explained! thank you!
Nice explanation! Keep up the good work!
I would like to add a correction, please correct me if I'm wrong. At 12:18 you said in deployment scaling the pods scale all at same time but I think with rolling updates we can choose % of scaling pods at a time.
Hello Marcel,
Can you make a video based on DevOps security aspects ?.. Like all things to be considered when we think about security in CICD? Like as a hiring manager what all u think is important and relevant.
Best of the best, you are the man, thanks a lot !
This is helpful .. thanks!
In your example you used ClusterIP type of service. Is headless service not one of requirements for statefulset? Or I can still use ClusterIP? Good video anyway, thanks.
Thanks for your support on kubernetes..
My request could you please make zuul api gateway for kubernetes cluster for routing
Wonderful - Well explained - does this apply to Kubernetes version 1.20 with containerd ?
Awesome explanation! Thank you.
You're the best man, this video is amazing as always
Thanks so much for the content!
love ur content please make a video on Daemonset
Hi At first would like to thank you for the wonderful videos...I am having request... Could you please make a detailed video on how MySQL High availability works and how to configure it on k8s. Also detailed video from. Basic to Advance on K8S Networking.. Thanks a lot in Advance..Stay Safe..!!
Very well laid out. Great job!
definitely helped explain stateful sets and pv's snd pvc's... but now to map to real world... 2 AZ's on AWS with Prometheus deployed, with a ALB in front ?
if the Prometheus servers were running on the same node I'd say lets use deployments, that way either node would have access to all the data, but then you going to have duplication where they scrape the same target.
lets make it simpler. one prometheus server, running active in AZ1.. but it needs to be protected to be able to fail over to AZ2...
snd as I discovered EFS not allowed.
Nice work!
it alway nice watch your videos ! best ever!
Does statefulsets already includes data & file replication?
MVP Channel! 10x Marcel 🙏
Hey, do you have a video describing how to make an EBS based StatefulSet in AWS? It seems that there's an issue reguarding EBS volumes being bound to an AZ. I heard that I should create an ASG per AZ, and make my StatefulSet have Pods across the AZs, but I don't know how to enforce that behavior. Most of my volumes are EFS and FSx so far, but I'd like to run a database in EKS, which would need EBS for speed, and I don't want to miss anything.
Hi Thanks for the brilliant content! Does the redis deployment automatically distribute across six note? If I have a taint on master nodes I guess that will be an issue? Cheers
Awesome. Thank you!
Informative video . Keep on Going with these type of videos.
Hello, thank you for the tutorial ! I've got some questions at 25:30 you show that PV are retained but the reclaim policy is set to Delete. How does it work ? Does it only occur when the StatefulSet get fully deleted ?
Great question. As per documentation, For reclaim policy "Delete", the volumes are only deleted when the PVC's are deleted. Current statefulset behaviour is PVC's are not deleted with pods or the statefulsets are deleted since they have a different lifespan.
@@MarcelDempers thank you
Learning about PVs and noticed that all of the PVs deployed into our cluster have the reclaim policy = Delete, which had me concerned that our engineering team had possibly made a terrible oversight in the deployments.
This comment is exactly what I wanted to understand. Thank you so much for this comment and the excellent statefulset tutorial!
Great explanation.
Thank you so much for your video. I have a question about scaling down Cassandra cluster, as you know, Cassandra cluster will store all the data in different databases, let's assume we have 5 Cassandra databases with replica of 3, and then scaling down to 1 Cassandra database, how can we know if the left Cassandra database consists all the data of the 5 Cassandra databases. Or it is not possible to do scaling down Cassandra like Redis in k8s? Thank you very much
Thanks a lot for your videos! It is extremely helpful!
holy crap! what a video... awesome!
Please let me know which software you are using for diagram
Good job mate! it easy to digest ;)
If I get the job I will send you a beer !
Great explanation!
You are beautifully pumped man. But please don't jump so quick from example code to yourself.
Nice videos. Thank you!!!
Best video ever...
Your Just Awsome!!! Very helpful.
Hi, the yamls shown in the video are different from the git. For instance the service type for redis cluster used in the video is ClusterIP but in git its headless service. Also why service name is to be mentioned in statefulset definition?
How does the redis cluster keep track of the new pod members when you scale up the statefulset ?
Checkout my Redis guide which includes a video on HA and Kubernetes
th-cam.com/play/PLHq1uqvAteVtlgFkmOlIqWro3XP26y_oW.html
That was master class :)
Great content. Thank you!
how do we make cluster of multiple nodes using docker for windows?
I was looking over the Github repo for the redis stateful set. I have a question. If each replica is mounting to the same host path is there any chance for mounting collisions and data corruption?
At least the subPath on the host needs to be unique. Each pod should have it's own data, yes
Checkout the Redis playlist on the channel for a more in-depth Redis setup
How do we fix that cluster outage then? 🤔
super thanks!!!
Ur arms are ripped dude
hello Marcel !! could you please let me know how to troubleshoot based on the error at the end of ur video ( scale down to zero - and then making up the replicas , but still the cluster is down ) how to overcome this ?
It's an interesting issue right ? :) This is part of the reason I recommend SaaS solutions for storage. I myself am not a Redis expert, so I would need to dig into 1) Redis cluster best practises, 2) Redis cluster troubleshooting \ repair process. I am planning a Redis series in the future 💪🏽
@@MarcelDempers Thank you for great video. Have you come up with any solution for this issue? Is Redis Sentinel answer for this issue?
Peace! ☮
Is it mandatory to use headless service with StatefulSet?
Good question. I have not tried it. The statefulset spec.serviceName might have some mandatory requirements, but nothing stops you from creating another clusterIP service or LoadBalancer one and selecting the statefulset pods with a label selector, if you need to
Is it ok to use pod ip's for clustering? I think If pods get rescheduled their pod ip addresses will change and break the redis cluster.
Correct, you dont want to use Pod IPs. You'll want to use K8s headless services with statefulsets. Take a look at my Redis playlist as an example.
first. can you please do a scheduler video?
THE BEST!
👏
Very helpful :)
Perfect
Sweet
Hello,
I use microk8s and run
$IPs = $(microk8s.kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
I got
=: command not found
How can I solve this problem?
I'm having the same problem
best
great channel - it would be better just to show the screen though - the montage is too hectic
Dude, I really like your efforts, I've learned alot. No offense but whenever I see you I think It's a murderer trying to teach k8s :D
You are talking too fast sometimes, when I watch yours videos i need to pause coule of times and rewind 20 sek to catch everything I need. But sometimes there is a lack of details for example how this web app is connecting to redis in this example
you are beautiful
Amazing video! thank you!
Great video, thanks!
Excellent video! Thank you!