[ Kube 77 ] AWS EKS Kubernetes cluster provisioning using Terraform
ฝัง
- เผยแพร่เมื่อ 5 ก.ย. 2024
- In this video, lets see how to use Terraform to provision a Kubernetes cluster in AWS using ESK managed Kubernetes Service.
Learn Kubernetes Playlist:
• Learn Kubernetes
EKS Getting Started Documentation:
learn.hashicor...
Terraform Provider AWS Github Repo:
github.com/ter...
Install aws-iam-authenticator:
docs.aws.amazo...
Hope you enjoyed this video. Please share it with your friends and don't forget to subscribe to this channel. For any questions/issues/feedback, please leave me a comment and I will be happy to help. Thanks for watching.
If you wish to support me:
www.paypal.com...
#kubernetes #learnkubernetes #justmekubernetes #awseks #eksctl #justmeandopensource
![[ Kube 78.1 ] KOPS Part 1 - Creating a Kubernetes cluster in AWS](http://i.ytimg.com/vi/6NymWMtcByg/mqdefault.jpg)
![[ Kube 78.1 ] KOPS Part 1 - Creating a Kubernetes cluster in AWS](/img/tr.png)
![[ Kube 71 ] Creating Kubernetes cluster in AWS using Eksctl | AWS Elastic Kubernetes Service](/img/n.gif)
Thanks for this video! In my opinion, the best way to learn about AWS resources is through Terraform documentation
Yeah Terraform does a good job in documentation but still AWS documentation is extensive. Thanks for watching.
I was totally unaware of KUBERNETES with terraform..This video helped me in understanding everything that I need basically..
Hi Rajan, thanks for watching.
In this video, you are launching cluster in public subnet. Can you explain how do we launch in private subnet and also how about having a load balancer?
Thank you very much for the video, it was very useful.
Thanks for watching.
I don't know I was facing issue while seeing this video above 360 Pixel. Content was really awesome ! best channel to learn about K8s
Hi Omkar, thanks for watching. I have had complaints from few viewers that they can't play this video in anything higher than 360p. I am experiencing the same issue. Its just this one video in my entire channel having this problem. Not sure what happened. Apologies.
Venkat, vanakkam, i just sent some contribution from my paypal. Thanks much for such good effor and content..for K8s
Hi Muzibur, many thanks for your contribution. Much appreciated. And thanks for your interest in this channel. Cheers.
At 6:50.... Please take care Venket. Stay safe and Thanks for the video Venkat. Please try to include Pulumi and aws cdk. Compare all 3 (i may not know, anything else still out there)
Hi Srini, thanks for watching. This was recorded before we knew about the pandemic in UK :)
Hi,
Found it useful. Tried it today and it worked. Thanks for your time and sharing. Subscribed your channel too.
Thanks,
Chetan from Kandivali, Mumbai :)
Cool. Thanks for watching.
Hi Venkat, I have sorted lot of problems with the help of this video. Can you please do a video on how to setup ALB ingress controller and add path base routing to it
Hi Ruwan, thanks for watching. I can give it a try but my videos are primarily based on local bare metals.
500th like...thank you sosoo much....😀
Many thanks :)
your tutorial is awesome Venkat garu... really awesome. please do more video's and when i choose 1080p resolution video not getting rendered. may be try to re-upload the same
Hi Venkat, thanks for watching. Unfortunately I don't have the original recording to re-upload. All my recent videos should be okay. There are few that has this problem.
Good Job Venkat
Thanks and waiting terraform series
Hi Mouhamed, thanks for watching. I wish to start the Terraform series. Let me see if I manage to find time in between. Cheers.
Massive work Venkat.
Thanks for watching.
Fantastic Job Venkat. I've learned so much from your videos! Would you be able to do a series on kubernetes and Operators (e.g using/writing/setting up environments) ?
I would also be interested in operators
Thanks for your interest in my videos. Operators are one area where I haven't spent enough time exploring to be able to explain. I will see if I get some time to do that. Its difficult to set apart some time in these times.
@@LesterThom what are operators btw??
Great content! Subscribed.
Thanks for watching and subscribing.
I really wanted to see the video. But quality is not enough. I hope I would see it in a better quality in the future. Wish you the best!
Hi Hossein, sorry about that. Will make sure the quality is good in my future videos. Cheers.
I can't seem to play this video above 360p, anyone else?
Hi Carter, Yes you are right. I can't go any higher than 360p. Not sure what happened. All other videos are okay. I don't have the original video in my hard disk to re upload unfortunately. Think its a waste of time them.
@@justmeandopensource its was productive for me :)
Tkanks for sharing
Thanks for watching. Cheers.
Thank you this was an amazing video
Thanks for watching.
Informative. Can you please make a video on creating complete private eks cluster using terraform. Thank you
Great Job Venkat, Excellent, When you get a chance could you make a video creating in K8S an AWS ALB with kubectl , I am a little bit confused between differents open ports , eg, sg inbound rules , host port, target ports, It will be great to learn about that, Thanks a lot Sir
Hello Venkat, very nice presentation. Much appreciated. May I know which connection manager (mac compatible) you are using for you linux terminals?.
Hi Rajeev , thanks for watching. What do you mean by connection manager? Sorry I lost the context.
@@justmeandopensource I mean the multi window mputty like terminals for accessing your Linux box. Is that mobaxterm you use?
@@tharikida1000 I just use tmux for that.
@@justmeandopensource thanks for your prompt reply. Much appreciated
@@tharikida1000 no worries. Cheers.
Helped my assignment
Hi Sathwika, glad it helped you. Thanks for watching.
Hi Venkat, I hope you'll find some time to help me. After the command kubectl cluster-info - I get an error: error loading config file "Users/my user/.kube/config": yaml: line 3: mapping values are not allowed in this context. Do you have nay ideas, what can it be?
got the same error
Great Job Men!
Hi Johan, Thanks for watching.
Getting error like "mapping values are not allowed in this context " when we see kubectl cluster-info
I have created EKS through Terrarorm using the latest tag from GitHub.. Now what the point is, after creation I got the kubeconfig file and showing cluster in was and all is well ok... But the problem is I installed terraform on a Linux server from that server I have executed terraform... So how to login into the cluster... If I enter kubectl get nodes from the same server it's saying command not found
Hi Venkat, good video and appreciate your efforts on this. I have seen similar videos and articles demonstrating this but what I am trying to work out is, create this EKS using existing VPCs, IGs, Route tables, subnets & SGs rather than creating new AWS network components. As I believe that's how it will be real-time if you are creating a new EKS from scratch in the existing VPC network. I have been looking for this info and got some suggestions to look at datasource, but I didn't get it working or either I am using it incorrectly, as I am still learning both EKS & K8s, so struggling on these bits, so if you have got some inputs/suggestions on these requirements, that will be great. Maybe a link/video on this would be really helpful, if the video takes longer to create, quick suggestions as an email / here or link guiding to get this achieved would be highly appreciated. Many thx
Hi, thanks for watching. I will see if I can get sometime to explore that. But if you look at the eksctl help commands for creating cluster, there are options for all these.
@@justmeandopensource Thanks for checking and replying on this, sorry I should have mentioned building eks using terraform on existing vpc network resources and I believe using eksctl command will create new VPC, but will explore eksctl using existing vpc network options as well. I would be more interested in getting this done using terraform in existing VPC as that's how it will be deployed in real-time afaik. thx
@@sonuthestar I see. Still thats possible. Let me see.
@@justmeandopensource just checking if you got a chance to look at this requirement on buildin eks in a existing vpc & subnets, if you got it working, can you please code / gir repo on this or some reference to it would be really appreciated. thx
Hi Venkat,
Thanks for the Knowledge shared by you. its really very helpful.
Using the mentioned Terraform Code we can build cluster with one linux worker node.
Where i am looking for windows worker nodes, i had went through the documents and build the manually but finding difficulties in creating the same with terraform. Could you please help out with this.
HI Sai, thanks for watching. can you share your terraform code in pastebin.com. I will take a look later when I get some time. Cheers.
@@justmeandopensource Hi thanks for your response, I am using the terraform code that's given by AWS in their GitHub repository eks module. Can you check that once and I am facing issues during auth configmaps code execution. Thanks in advance
Nice one 👌😁
Hi, Thanks for watching.
Very useful tutorial Il liked, also I would like create an other cluster with 3masters, and 3 etcd can help me to do that??
this is a good tutorial but after the deployment of the Nginx pod or why there is no sample to expose the service to outside? is there another tutorial to follow?
do i need to setup kubernetes locally before doing this?
command : terraform output kubeconfig > ~/.kube/config is not working
Yeah you should have minikube and Kubectl locally
Hi Sathwika, thanks for answering. Cheers.
@@sathwikakatam4504 Why do we need minikube when we are creating EKS cluster?
Great video Venkat, also can we use tarraform to provision a k8s cluster without the EKS(managed) service, the way kops does?
Very well explained thank you, also interested in multiregion K8S cluster with terraform.
Could you please say, what the console with auto completion is this and how can i have the same on my mac?
Hi Alex, if you are asking about my terminal, i have done a video on the setup around Zsh which can be achieved in Mac as well.
th-cam.com/video/PUWnCbr9cN8/w-d-xo.html
Cheers.
Thank you for this video. How does one attach a RDS db to the worker nodes in this setup? What is the best practice for architecture and do you have any examples?
Hi Ron, thanks for watching. What do you mean by attaching RDS to worker nodes? RDS is accessed through its endpoint. Your applications deployed in the kubernetes cluster will have access to the RDS point if they need to use database. You just need to make sure the security groups on worker nodes and RDS allow this.
@@justmeandopensource No problem. Its wasy to watch with such great content :) Thanks for the reply!! yeah that is what I meant. To allow worker nodes to interact with RDS endpoint. So does the security group of RDS need to allow access for the nodes and the security group of the nodes need to allow access to RDS. Assuming that the RDS instance and the nodes live in the same VPC (which I think is the most efficient approach): Does the RDS instance need to live in a private subnet on the same VPC? Is that a good approach? In which case, does a private subnet need to be added to the example?
Yes RDS needs to be in a private subnet unless you want external public accesd directly to your RDS. It can be in the same vpc or different. If in different vpc, you need to make sure you have vpc peering and routing setup.
Awesome bro ,Can you please make a video for codepipeline deploy to ecs (EC2 ) for node projects with terraform
Hello Venkat, I Want to create EKS clusters and ALB . Will AWS charge me if i create resources for them for short time?
Hi, I am getting below error while applying in terraform, from windows10
Error: error waiting for EKS Node Groupe (terraform-eks-demo:demo) to create: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: %!s()
│
│ with aws_eks_node_group.demo,
│ on eks-worker-nodes.tf line 41, in resource "aws_eks_node_group" "demo":
│ 41: resource "aws_eks_node_group" "demo" {
Please upload a video on kubespray using terraform aws eks
Thanks for watching. Kubespray is based on Ansible for automation and terraform is different.
@@justmeandopensource thank you. Please upload kubespray using ansible aws eks
It's really great video, kindly help me how to add more masters and workers to existing eks cluster, thanks
Hey Venkat
I am unable to play this video after a minute it goes on buffer mode
Hi Manas, I am not sure what went wrong with this video. TH-cam seems to have problem on just this one video in my playlist.
I can't fix it but you can watch it if you change the video resolution to 360p.
Hey Venkat what if I want to add another kubernetes master node to the cluster can you please let me know what needs to be done?
Which image is being used in worker nodes in this EKS video..?
Amazon Linux 2 image is the default but you have an option to choose from a list.
registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group
Look for ami_type in the above link.
@@justmeandopensource How do i deploy a django app with Redis from Docker-compose file in EKS using Terraform..?
@@sanyamgrover1 Terraform is best suited for deploying infrastructures. There are other better ways to deploy apps/workloads onto k8s cluster.
Unable to connect to the server: dial tcp [::1]:8080: connectex: No connection could be made because the target machine actively refused it.
can you please help with this error
Good Work!
Thanks for watching.
@@justmeandopensource Which terraform version you are using I am getting following error
Warning: Version constraints inside provider configuration blocks are deprecated
│
│ on providers.tf line 7, in provider "aws":
│ 7: version = ">= 2.38.0"
│
│ Terraform 0.13 and earlier allowed provider version constraints inside the provider configuration block, but that is now deprecated and will be removed in a future
│ version of Terraform. To silence this warning, move the provider version constraint into the required_providers block.
╵
╷
│ Error: Error in function call
│
│ on vpc.tf line 12, in resource "aws_vpc" "demo":
│ 12: tags = map(
│ 13: "Name", "terraform-eks-demo-node",
│ 14: "kubernetes.io/cluster/${var.cluster-name}", "shared",
│ 15: )
│ ├────────────────
│ │ var.cluster-name is a string, known only after apply
│
│ Call to function "map" failed: the "map" function was deprecated in Terraform v0.12 and is no longer available; use tomap({ ... }) syntax to write a literal map.
@@GenerativeAI-Guru I used v0.12 I guess.
@@justmeandopensource Thanks it works
Do you have any reference for running terraform using git-cicd.yml and storing terraform state in s3 ?
how do I see my codes in the visual studio codes so i can paste in the CLI
Great video. Very well explained.
Hi, thanks for watching.
Hey Venkat, Thanks for this awesome explanation. Actually , I am trying to create a cluster on aws and using the latest tag to build the resources. My resources are getting build successfully and I could see some resources created on the console like vpc , SG, Iam etc. But I could not see any machines created as part of the worker node. And also while navigating to the AWS EKS service I could see the cluster created but cannot see any thing inside that . Seems like some permission is missing but not sure. Can you please help.
Also, As when I navigate to eks service and click on the new cluster then I am getting some info like " your current user or role does not have access to kubernetes objects on this EKS cluster"
Hi venkat, Its pretty good. I have followed your video. You have deployed nginx from local to cluster. How to see the whether nginx deployed or not? Any particular URL to show?
Hi, thanks for watching. If you just want to know whether nginx is deployed or not, you can do kubectl get all which will show whether the nginx pods are running fine. If you want to access the nginx application from your local machine, you will have to use loadbalancer or nodeport or port-forwarding.
@@justmeandopensource many thanks,.I want to access nginx application.Do you have any video for loadbalancing or nodeport or port forwarding.
@@lakshminarayanans3927 Just search in my Kubernetes playlist. I have over 100 videos on Kubernetes topic. I have covered the load balancing concept in various videos but nothing in particular.
@@justmeandopensource, i will look at this. Many thanks
Hi Venkat
Can u make some video on CloudFormation
Hi, thanks for watching. I have a separate series on AWS where I was planning to do few videos around CloudFormation. But I couldn't find time to record. If I were to upload videos, it will be in this series.
th-cam.com/play/PL34sAs7_26wMKAl2wcDXb7ko65V8KDBzG.html
Hello, I have one question, what was the reason to install Nginx server and after you installed why did you delete it??
Hi Ankush, thanks for watching. I deployed nginx application just to verify that the cluster is working fine.
@@justmeandopensource Thanks for your reply. Your videos are so helpful please make the whole tutorial on Terraform with some real-time labs.
@@ankushverma7957 Sure. Will do.
Hello Venkat, thanks for this awesome video, but I'm running into this error error: error loading config file "C:\Users\\.kube\config": yaml: line 4: mapping values are not allowed in this context" this happened after I copied the kubeconfig file to ./kube/config. I have tried to check online for a solution but I don't find anything. Any help would be highly appreciated.
please start videos on terraform
Hi Manish, thanks for watching. I have that in my mind. Let me see if I have enough time to work on that series. Cheers.
how can i get the source code
git clone github.com/hashicorp/learn-terraform-provision-eks-cluster
do you have a git repo link?
Hi Venkat, I was able to create the EKS cluster using:
assume_role {
role_arn = "arn:aws:iam::234257617375:role/role_admin_temp"
}
Which is an ugly workaround when you don't have the mfa permission set for the user.
I use aws-vault to manage my aws user credentials. I got unauthorized error after executing "kubectl cluster-info" -> (error: You must be logged in to the server (Unauthorized))
Do you know why I am getting this? I think that it is because I am assuming a role to create the cluster and my user does not have the access to the server.
Yeah I believe what you think is right although I haven't tried it myself to confirm.
You did not provide the code
github.com/hashicorp/terraform-provider-aws
Can I setup using t2.micro is possible
Hi Ashok, thanks for watching. I haven't tried using t2.micro with EKS. These instance type has only 512MB RAM which is not sufficient to run as worker nodes. You can give it a try and see what happens.
I have create all this things in t2.micro
@@ashok.9761 Cool. For learning thats fine. But t2.micro with its limited cpu and memory won't cater even to the lightest workload.
In this vpc
I have to create additional 2 private subnets
Plz tel me cidr block range using
Count 2
Cidr_block = "10.0.${count.index}.1/24
But getting error
@@ashok.9761 Networks are something you have to design. You will specify the network range while creating the VPC and the divide the network into subnets. You can use an online subnet calculator like www.subnet-calculator.com/ for this purpose. Thanks.
nice please provide sourcecode link
# TIL
Tks !!!!
once again i followed this video and trying to create. When i tried to hit "terraform apply" , it thrown below error. Whats the issue?
error waiting for EKS Node Group (terraform-eks-demo:demo) creation: Ec2SubnetInvalidConfiguration: One or more Amazon EC2 Subnets of [subnet-018fc2a0efe34c37c, subnet-0ce71f0d5f8bbf027] for node group demo does not automatically assign public IP addresses to instances launched into it. If you want your instances to be assigned a public IP address, then you need to enable auto-assign public IP address for the subnet. See IP addressing in VPC guide: docs.aws.amazon.com/vpc/latest/userguide/vpc-ip-addressing.html#subnet-public-ip. Resource IDs: [subnet-018fc2a0efe34c37c subnet-0ce71f0d5f8bbf027]
on eks-worker-nodes.tf line 41, in resource "aws_eks_node_group" "demo":
41: resource "aws_eks_node_group" "demo" {
I am having the same issue. any solution for this?
@Lakshminarayanan S - I found the solution. you need to add the key "map_public_ip_on_launch = true" to the vpc.tf file as below and save the file. then execute terraform apply and that solved the issue.
resource "aws_subnet" "demo" {
count = 2
availability_zone = data.aws_availability_zones.available.names[count.index]
cidr_block = "10.0.${count.index}.0/24"
vpc_id = aws_vpc.demo.id
map_public_ip_on_launch = true
@@dhanabalasingamjagannathan3135 , yes its working fine bro after added map_public_ip_on_launch = true.
The concepts are really awesome, while running this code its is asking some config files.so i got struck with this error
root@ip-172-31-26-108:~/terraform-provider-aws/examples/eks-getting-started# kubectl get nodes
W0521 11:02:26.700220 2738 loader.go:223] Config not found: /etc/kubernetes/admin.conf
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Thanks for watching. This video is about a year old and in Kubernetes space, things change quite frequently. I will have to check this video again to confirm whether something has changed or its just an issue in your setup.
@@justmeandopensource thank s for the response.ols update the status
Thank you so much for this video. It is very useful.
Thanks for watching Hari, Cheers.