I think I said it in one of you first videos but again, you are the best DevOps educator on youtube. Clearly you've got experience that many others don't. Your vids aren't some robotic read-out of the official docs for views, so I can always find something new. Great style, great structure. You definitely deserve more recognition but it's getting there and certainly it will get there!
Your vedios are really helpful and i love the the way you explain it with real time use cases. Would request you to cover helm charts from scratch to complex. Lets say how to create helm charts, how to convert our own applications to helm charts etc etc. I would love to have some series of vedios on this with the ongoing standard use cases which currently organisations are doing. Thanks! keep the great work going going. Happy learning!!
I will rate you 100% . The content was too notch and the explanation was just what I need now. I wanted to deploy a postgres database on k8s, so I sort for resources on data persistent and I couldn't find any that is more explanatory. Your sample that resides in github was top-notch.
Thanks to you, I learned a lot of "production / professional" ways to understand and implement those strategies. None of your videos get bored, I even watch some vid 5-6 times to strengthen my knowledge until I completely understand everything. Thank you for your supports.
I don't know what I would do without your channel. You have an amazing skill to explain complicated topics in such a wonderful, easy-to-understand way. Keep going!! A video with Tekton would be great, if I can propose something.
I would like to add a correction, please correct me if I'm wrong. At 12:18 you said in deployment scaling the pods scale all at same time but I think with rolling updates we can choose % of scaling pods at a time.
Your video on the topic is excellent!! I would love to see more examples of statefulsets because your video has raised some critical issues about cluster failure which couldnt be automatically resolved by the kubernetes engine and may require manually fixing the cluster setup. Can you please show possible solutions on fixing cluster issues that cannot be automatically resolved by the k8s engine?
just rewatched this... thinking I'm picking up on the complexity of stateful sets and ... a stretched environment like multiple AZ's as present on AWS EKS... any chance you've done a video, where say 3 or more stateful sets are pined to nodes (AWS EKS Nodegroups). this can then be used against the Redis deployment and PostgreSQL deployment ?
My Redis series does this already. A statefulset will pin a pod to the same node's volume claim. Unless that node disappeared, the pod will always stay on that node. If a rolling node upgrade occurs which my roll new VMs, you may lose data ( depending how the cloud provider does updates)
@@MarcelDempers understand the stateful pod will gravitate back to where the pvc is, but i want to pin/direct the pod before that, I have a EKS cluster spread across 3 x az's, and additionally the cluster have node groups for apps and nodegroups for db... i want to direct the stateful set to the database node groups, and additionally push each of the Redis masters onto a node in each of the az's.
Hello, thank you for the tutorial ! I've got some questions at 25:30 you show that PV are retained but the reclaim policy is set to Delete. How does it work ? Does it only occur when the StatefulSet get fully deleted ?
Great question. As per documentation, For reclaim policy "Delete", the volumes are only deleted when the PVC's are deleted. Current statefulset behaviour is PVC's are not deleted with pods or the statefulsets are deleted since they have a different lifespan.
Learning about PVs and noticed that all of the PVs deployed into our cluster have the reclaim policy = Delete, which had me concerned that our engineering team had possibly made a terrible oversight in the deployments. This comment is exactly what I wanted to understand. Thank you so much for this comment and the excellent statefulset tutorial!
This is an awesome series, but I wanted to point out an issue with the command @22:28. The command adds an extra ":6379" at the end of the string which is recorded in the env var. The output from: kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ' in my case was: "10.244.0.6:6379 10.244.0.8:6379 10.244.0.10:6379 10.244.0.12:6379 10.244.0.14:6379 10.244.0.16:6379 :6379" This causes the next cluster create command to fail. Took me a little bit to understand why the command was failing if anyone else runs into the issue.
The way you explain concepts and then finish them with real world examples is what most engineers wish to see. Redis, RabbitMQ I have watched both and request you to deploy more such complex applications. Also, if you can tell us what could be the best setup for cluster monitoring and how to approach to solve real world cluster down scenarios, that would be great.
StatefulSets are an interesting beast. When I first ran into them, I didn't know what to make of it : I've always thought of k8s as stateless-oriented and StatefulSets felt like an anti-pattern. Once I looked into clustered / high availability databases and the on-going work in that space, it made a lot more sense (I'm a big fan of Stolon, if you're into distributed Postgres). K8s is really a great engine to distribute *stuff*, it doesn't have to be microservices.
Outstanding video mate. I'd say you are a welcome addition to our sunny shores but you chose VIC instead of WA :P Just teasing Melbourne rocks, lived there for 4 years myself as a young lad.
In your example you used ClusterIP type of service. Is headless service not one of requirements for statefulset? Or I can still use ClusterIP? Good video anyway, thanks.
Hey, do you have a video describing how to make an EBS based StatefulSet in AWS? It seems that there's an issue reguarding EBS volumes being bound to an AZ. I heard that I should create an ASG per AZ, and make my StatefulSet have Pods across the AZs, but I don't know how to enforce that behavior. Most of my volumes are EFS and FSx so far, but I'd like to run a database in EKS, which would need EBS for speed, and I don't want to miss anything.
Thank you so much for your video. I have a question about scaling down Cassandra cluster, as you know, Cassandra cluster will store all the data in different databases, let's assume we have 5 Cassandra databases with replica of 3, and then scaling down to 1 Cassandra database, how can we know if the left Cassandra database consists all the data of the 5 Cassandra databases. Or it is not possible to do scaling down Cassandra like Redis in k8s? Thank you very much
I was looking over the Github repo for the redis stateful set. I have a question. If each replica is mounting to the same host path is there any chance for mounting collisions and data corruption?
Hi, the yamls shown in the video are different from the git. For instance the service type for redis cluster used in the video is ClusterIP but in git its headless service. Also why service name is to be mentioned in statefulset definition?
another good videos on Kubernetes.. can you please make video to capture logs of application in cabernets in ELK with cabana or any other tools to view app logs
hello Marcel !! could you please let me know how to troubleshoot based on the error at the end of ur video ( scale down to zero - and then making up the replicas , but still the cluster is down ) how to overcome this ?
It's an interesting issue right ? :) This is part of the reason I recommend SaaS solutions for storage. I myself am not a Redis expert, so I would need to dig into 1) Redis cluster best practises, 2) Redis cluster troubleshooting \ repair process. I am planning a Redis series in the future 💪🏽
Hi Thanks for the brilliant content! Does the redis deployment automatically distribute across six note? If I have a taint on master nodes I guess that will be an issue? Cheers
Hello, I use microk8s and run $IPs = $(microk8s.kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ') I got =: command not found How can I solve this problem?
Good question. I have not tried it. The statefulset spec.serviceName might have some mandatory requirements, but nothing stops you from creating another clusterIP service or LoadBalancer one and selecting the statefulset pods with a label selector, if you need to
Hello Marcel, Can you make a video based on DevOps security aspects ?.. Like all things to be considered when we think about security in CICD? Like as a hiring manager what all u think is important and relevant.
Hi At first would like to thank you for the wonderful videos...I am having request... Could you please make a detailed video on how MySQL High availability works and how to configure it on k8s. Also detailed video from. Basic to Advance on K8S Networking.. Thanks a lot in Advance..Stay Safe..!!
definitely helped explain stateful sets and pv's snd pvc's... but now to map to real world... 2 AZ's on AWS with Prometheus deployed, with a ALB in front ? if the Prometheus servers were running on the same node I'd say lets use deployments, that way either node would have access to all the data, but then you going to have duplication where they scrape the same target. lets make it simpler. one prometheus server, running active in AZ1.. but it needs to be protected to be able to fail over to AZ2... snd as I discovered EFS not allowed.
You are talking too fast sometimes, when I watch yours videos i need to pause coule of times and rewind 20 sek to catch everything I need. But sometimes there is a lack of details for example how this web app is connecting to redis in this example
Dude, your series on k8s is - by far - the celarest, most concise and illuminating set of vids out there. I sincerely commend you, sir.
Greatly explained . Currently working as DevOps intern and really was stuck with this statefulset and deployment. keep up good work
I am on the virtuous path of watching every single video of this channel. An practice it. So far it is going great and I have learned a ton.
I think I said it in one of you first videos but again, you are the best DevOps educator on youtube. Clearly you've got experience that many others don't. Your vids aren't some robotic read-out of the official docs for views, so I can always find something new. Great style, great structure. You definitely deserve more recognition but it's getting there and certainly it will get there!
you, sir, are answering all the exact questions which no one other answers. thank you.
Very helpful. Thanks for the videos with a good emphasis on the "Why" part.
Your vedios are really helpful and i love the the way you explain it with real time use cases. Would request you to cover helm charts from scratch to complex. Lets say how to create helm charts, how to convert our own applications to helm charts etc etc. I would love to have some series of vedios on this with the ongoing standard use cases which currently organisations are doing. Thanks! keep the great work going going. Happy learning!!
I will rate you 100% . The content was too notch and the explanation was just what I need now.
I wanted to deploy a postgres database on k8s, so I sort for resources on data persistent and I couldn't find any that is more explanatory.
Your sample that resides in github was top-notch.
Thanks to you, I learned a lot of "production / professional" ways to understand and implement those strategies. None of your videos get bored, I even watch some vid 5-6 times to strengthen my knowledge until I completely understand everything. Thank you for your supports.
I don't know what I would do without your channel. You have an amazing skill to explain complicated topics in such a wonderful, easy-to-understand way. Keep going!!
A video with Tekton would be great, if I can propose something.
Thank you for your awesome videos, one of the best series of videos on K8s on youtube. TYSM.
Now that's called pure gold 💰 . Awesome stuff mate ! 👏
You are so precise, even laser beam refracts. I really like your presentation. Thank you!
Man, your videos are awesome. Learnt a lot from it!
I would like to add a correction, please correct me if I'm wrong. At 12:18 you said in deployment scaling the pods scale all at same time but I think with rolling updates we can choose % of scaling pods at a time.
You can explain very good and in this way many others can understand this technologies
Your video on the topic is excellent!! I would love to see more examples of statefulsets because your video has raised some critical issues about cluster failure which couldnt be automatically resolved by the kubernetes engine and may require manually fixing the cluster setup. Can you please show possible solutions on fixing cluster issues that cannot be automatically resolved by the k8s engine?
stateful explained in a very clear way. Thank you for the video.
You are best keep sharing the concept and. Keep it simple as other devops video are not easy to understand on another channels
just rewatched this...
thinking I'm picking up on the complexity of stateful sets and ... a stretched environment like multiple AZ's as present on AWS EKS... any chance you've done a video, where say 3 or more stateful sets are pined to nodes (AWS EKS Nodegroups).
this can then be used against the Redis deployment and PostgreSQL deployment ?
My Redis series does this already. A statefulset will pin a pod to the same node's volume claim. Unless that node disappeared, the pod will always stay on that node. If a rolling node upgrade occurs which my roll new VMs, you may lose data ( depending how the cloud provider does updates)
@@MarcelDempers understand the stateful pod will gravitate back to where the pvc is, but i want to pin/direct the pod before that, I have a EKS cluster spread across 3 x az's, and additionally the cluster have node groups for apps and nodegroups for db... i want to direct the stateful set to the database node groups, and additionally push each of the Redis masters onto a node in each of the az's.
Hello, thank you for the tutorial ! I've got some questions at 25:30 you show that PV are retained but the reclaim policy is set to Delete. How does it work ? Does it only occur when the StatefulSet get fully deleted ?
Great question. As per documentation, For reclaim policy "Delete", the volumes are only deleted when the PVC's are deleted. Current statefulset behaviour is PVC's are not deleted with pods or the statefulsets are deleted since they have a different lifespan.
@@MarcelDempers thank you
Learning about PVs and noticed that all of the PVs deployed into our cluster have the reclaim policy = Delete, which had me concerned that our engineering team had possibly made a terrible oversight in the deployments.
This comment is exactly what I wanted to understand. Thank you so much for this comment and the excellent statefulset tutorial!
I loved this stateful sets explanation
You are a great instructor and cross Nana 👏
This is an awesome series, but I wanted to point out an issue with the command @22:28. The command adds an extra ":6379" at the end of the string which is recorded in the env var.
The output from:
kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 '
in my case was:
"10.244.0.6:6379 10.244.0.8:6379 10.244.0.10:6379 10.244.0.12:6379 10.244.0.14:6379 10.244.0.16:6379 :6379"
This causes the next cluster create command to fail. Took me a little bit to understand why the command was failing if anyone else runs into the issue.
Peace Marcel, You are the man!
It’s very clear and concise.. really appreciate it..
very good in-depth explanation. just what was looking for. thanks
MVP Channel! 10x Marcel 🙏
The way you explain concepts and then finish them with real world examples is what most engineers wish to see. Redis, RabbitMQ I have watched both and request you to deploy more such complex applications. Also, if you can tell us what could be the best setup for cluster monitoring and how to approach to solve real world cluster down scenarios, that would be great.
Love ur content! Keep on rockin'
A very clear explanation, thanks!
StatefulSets are an interesting beast. When I first ran into them, I didn't know what to make of it : I've always thought of k8s as stateless-oriented and StatefulSets felt like an anti-pattern. Once I looked into clustered / high availability databases and the on-going work in that space, it made a lot more sense (I'm a big fan of Stolon, if you're into distributed Postgres). K8s is really a great engine to distribute *stuff*, it doesn't have to be microservices.
Outstanding video mate. I'd say you are a welcome addition to our sunny shores but you chose VIC instead of WA :P
Just teasing Melbourne rocks, lived there for 4 years myself as a young lad.
Very Very helful vedio and was very easy to understand StatefulSet. Thanks you so much...
You're the best man, this video is amazing as always
This is a very very good tutorial. Thank you and subscribed!
you are awsom! thank you the DevOps guy))
In your example you used ClusterIP type of service. Is headless service not one of requirements for statefulset? Or I can still use ClusterIP? Good video anyway, thanks.
Does statefulsets already includes data & file replication?
Hey, do you have a video describing how to make an EBS based StatefulSet in AWS? It seems that there's an issue reguarding EBS volumes being bound to an AZ. I heard that I should create an ASG per AZ, and make my StatefulSet have Pods across the AZs, but I don't know how to enforce that behavior. Most of my volumes are EFS and FSx so far, but I'd like to run a database in EKS, which would need EBS for speed, and I don't want to miss anything.
holy crap! what a video... awesome!
Best of the best, you are the man, thanks a lot !
Nice explanation! Keep up the good work!
Wonderful - Well explained - does this apply to Kubernetes version 1.20 with containerd ?
Thank you so much for your video. I have a question about scaling down Cassandra cluster, as you know, Cassandra cluster will store all the data in different databases, let's assume we have 5 Cassandra databases with replica of 3, and then scaling down to 1 Cassandra database, how can we know if the left Cassandra database consists all the data of the 5 Cassandra databases. Or it is not possible to do scaling down Cassandra like Redis in k8s? Thank you very much
I was looking over the Github repo for the redis stateful set. I have a question. If each replica is mounting to the same host path is there any chance for mounting collisions and data corruption?
At least the subPath on the host needs to be unique. Each pod should have it's own data, yes
Checkout the Redis playlist on the channel for a more in-depth Redis setup
how do we make cluster of multiple nodes using docker for windows?
Hi, the yamls shown in the video are different from the git. For instance the service type for redis cluster used in the video is ClusterIP but in git its headless service. Also why service name is to be mentioned in statefulset definition?
How does the redis cluster keep track of the new pod members when you scale up the statefulset ?
Checkout my Redis guide which includes a video on HA and Kubernetes
th-cam.com/play/PLHq1uqvAteVtlgFkmOlIqWro3XP26y_oW.html
another good videos on Kubernetes.. can you please make video to capture logs of application in cabernets in ELK with cabana or any other tools to view app logs
Thanks for the kind words 💪🏽
We have started that journey already 🤓
Come follow along th-cam.com/play/PLHq1uqvAteVvfDxFW50Mdezk0xum-tyHT.html
super well explained! thank you!
This fellow is no mere mortal
hello Marcel !! could you please let me know how to troubleshoot based on the error at the end of ur video ( scale down to zero - and then making up the replicas , but still the cluster is down ) how to overcome this ?
It's an interesting issue right ? :) This is part of the reason I recommend SaaS solutions for storage. I myself am not a Redis expert, so I would need to dig into 1) Redis cluster best practises, 2) Redis cluster troubleshooting \ repair process. I am planning a Redis series in the future 💪🏽
@@MarcelDempers Thank you for great video. Have you come up with any solution for this issue? Is Redis Sentinel answer for this issue?
Hi Thanks for the brilliant content! Does the redis deployment automatically distribute across six note? If I have a taint on master nodes I guess that will be an issue? Cheers
it alway nice watch your videos ! best ever!
Is it ok to use pod ip's for clustering? I think If pods get rescheduled their pod ip addresses will change and break the redis cluster.
Correct, you dont want to use Pod IPs. You'll want to use K8s headless services with statefulsets. Take a look at my Redis playlist as an example.
Thanks so much for the content!
Please let me know which software you are using for diagram
Best video ever...
Very well laid out. Great job!
That was master class :)
Awesome explanation! Thank you.
Awesome. Thank you!
This is helpful .. thanks!
Informative video . Keep on Going with these type of videos.
Hello,
I use microk8s and run
$IPs = $(microk8s.kubectl -n example get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
I got
=: command not found
How can I solve this problem?
I'm having the same problem
Is it mandatory to use headless service with StatefulSet?
Good question. I have not tried it. The statefulset spec.serviceName might have some mandatory requirements, but nothing stops you from creating another clusterIP service or LoadBalancer one and selecting the statefulset pods with a label selector, if you need to
Hello Marcel,
Can you make a video based on DevOps security aspects ?.. Like all things to be considered when we think about security in CICD? Like as a hiring manager what all u think is important and relevant.
How do we fix that cluster outage then? 🤔
Nice work!
Thanks a lot for your videos! It is extremely helpful!
Hi At first would like to thank you for the wonderful videos...I am having request... Could you please make a detailed video on how MySQL High availability works and how to configure it on k8s. Also detailed video from. Basic to Advance on K8S Networking.. Thanks a lot in Advance..Stay Safe..!!
Thanks for your support on kubernetes..
My request could you please make zuul api gateway for kubernetes cluster for routing
love ur content please make a video on Daemonset
THE BEST!
Could you please make a video on stateless set
Great explanation.
definitely helped explain stateful sets and pv's snd pvc's... but now to map to real world... 2 AZ's on AWS with Prometheus deployed, with a ALB in front ?
if the Prometheus servers were running on the same node I'd say lets use deployments, that way either node would have access to all the data, but then you going to have duplication where they scrape the same target.
lets make it simpler. one prometheus server, running active in AZ1.. but it needs to be protected to be able to fail over to AZ2...
snd as I discovered EFS not allowed.
Good job mate! it easy to digest ;)
Great explanation!
Ur arms are ripped dude
Your Just Awsome!!! Very helpful.
Great content. Thank you!
super thanks!!!
Peace! ☮
Very helpful :)
If I get the job I will send you a beer !
first. can you please do a scheduler video?
Perfect
You are beautifully pumped man. But please don't jump so quick from example code to yourself.
Nice videos. Thank you!!!
👏
Sweet
best
Dude, I really like your efforts, I've learned alot. No offense but whenever I see you I think It's a murderer trying to teach k8s :D
great channel - it would be better just to show the screen though - the montage is too hectic
You are talking too fast sometimes, when I watch yours videos i need to pause coule of times and rewind 20 sek to catch everything I need. But sometimes there is a lack of details for example how this web app is connecting to redis in this example
you are beautiful
Amazing video! thank you!
Great video, thanks!
Excellent video! Thank you!