How to set up AWS Kubernetes Jenkins pipeline? - Part 13
ฝัง
- เผยแพร่เมื่อ 11 ก.ค. 2021
- How to set up AWS Kubernetes Jenkins pipeline?
▬▬▬▬▬▬ * Instruction's Guides * ▬▬▬▬▬▬
1. AWS Kubernetes Jenkins Pipeline - jhooq.com/aws-kubernetes-jenk...
2. GitHub Repo - github.com/rahulwagh/k8s-jenk...
In this lab session, we are going to set up the AWS Kubernetes Jenkins pipeline. We will be using eksctl for creating an EKS cluster on AWS.
Setting up a pipeline on AWS requires you to have an active AWS account and you can not use the free tier of AWS because you can not use the EKS cluster on the free tier of AWS.
Here are steps which we will go through -
1. Setting up AWS EC2 Machine - First we need to set up EC2 machine where we will be installing the Jenkins along with Gradle, kubectl, eksctl, docker. We can not use t2.micro instance because we will need more CPU and memory to run Jenkins and Gradle.
2. Connect to EC2 machine - After starting the EC2 machine we will use the private key-pair file to connect to EC2 instance.
3. Install JDK - As we need to install Jenkins so we need to have JDK(adoptopenjdk.net/) pre-installed onto the EC2 machine. For this lab session I will be using adopt open JDK 11.
4. Install and Setup Jenkins - In this step we are going to install Jenkins and also going to the setup admin user
5. Assign administrative privileges to Jenkins user - After installing and setting up the Jenkins on EC2 machine we need to assign root privileges so that we can execute Docker and Gradle commands inside the Jenkins pipeline
6. Install Docker - The next step would be to install docker so that we can create a docker image and push the docker image to docker hub
7. Install and Configure AWS CLI - After installing the docker we need to install and configure AWS CLI so that we can use EKSCTL to set up and start the Kubernetes cluster.
8. Install kubectl - Also along with EKSCTL we need to install Kubectl so that we can view the deployments and services
9. Install eksctl - Now we need to install eksctl so that we can set up and start the Kubernetes cluster without using AWS UI
10. Add docker hub and GITHUB credentials - As we need to clone the github repo and push the docker image to docker hub we need to store credentials of github and docker inside Jenkins.
11. Jenkins script for stages - Once we are done all the above steps we are going to write the Jenkins pipeline scripts for all the stages
▬▬▬▬▬▬ ⭐️ 🕘Timestamps ⭐️ ▬▬▬▬▬▬
2:21 Setup an AWS EC2 Instance
7:17 Connect to EC2 Instance
10:05 Install JDK on AWS EC2 Instance
11:54 Install and Setup Jenkins
17:41 Update visudo and assign administrative privileges to Jenkins user
19:24 Install Docker
25:44 Install and Setup AWS CLI
29:32 Install and Setup Kubectl
31:01 Install and Setup eksctl
33:47 Create eks cluster using eksctl
48:25 Add Docker and GitHub Credentials into Jenkins
50:25 Add Jenkins stages
52:54 Build, deploy and test CI CD pipeline
To learn more on DevOps visit - jhooq.com
Disclaimer/Policy: All the content/instructions are solely mine. The source is completely open-source.
Video is copyrighted and can not be re-distributed on any platform.
#devops #aws #jenkins #cloud #kubernetes #eksctl #docker - วิทยาศาสตร์และเทคโนโลยี
I repent why i have not come across such a wonderful tutorial all these years. Brilliant Rahul. Great work and clear concise and pretty simple. Thanks a million Rahul
It will great to see logs for POD on the different instance in your AWS each time your hit the URL, amazing video!!
Thanks for the idea! Generally this session tends to get very long so its really hard to keep the track of every aspect. But i will make a not for the next time
Thanks a ton Rahul for the video, got clarified many doubts in this video GREAT EXPLANATION
Thank you so much brother amazing and detailed explanation. Just started to learn about devops and this was very helpful.
Most wonder video I ever seen on youtube on CICD with simple explanation. Kudos
Glad you liked it
Thank you so much @Rahul for the amazing content and for starting from scratch.
You are welcome
This is the best video I found in recent day on any devops topics
Thanks Rahul to make life easy :)
I hope it was easy to understand
wow...!!Crystal clear ,simple ,easy ..I was looking for this pipeline since long ,,,,first time ever seen this type of explanation about CI/CD pipeline .. you are real champion of DevOps
this video boosted my confidence
Glad to hear that
Very nice explanation by detailing each step.
You deserve a millions of followers.such a great tutorials. Appreciate your great work. Keep it up
Glad you like them!
Thank you Rahul sir..... :)
Very nice and simple steps in this video as you shown.
Keep watching there is more to come
I have become ur fan Boss!! u r teaching is out of this world!! Keep up ur good worK!!
Thank you so much 😀
nicely explained, Rahul. thanks, I was able to follow the steps :)
its very much impressive Rahul. you can make it very easy. Thank you
So nice of you, stay tuned for more similar sessions
Fantastic tutorial. I really enjoyed it and learned a lot. Thank you very much.
You're very welcome!
I was looking for this pipeline since long. Thanks a lot . It's the best tutorial.
Glad to know that
@@RahulWagh Just wanted to thank you again for this video. I cracked an interview today :)
Sorry for the late reply and but really glad to know you cleared the interview. Best of luck with your new job
Rahul Bhai amazing videos
Your my Guruji , thanks a lot .....
🙏🙏
Thankyou Rahul such a great step by step knowledge sharing ....
Thank a lot
Thanks and welcome
You are a life save Rahul. Awesome tutorial.
Happy to help
Thank you so much for sharing such wonderful video bro.. Please keep posting. It is very useful video for everybody.
So nice of you
Rahul, you are amazing, BEST of BEST. Thanks a lot
Thanks
Thanks a Bunch Rahul for sharing content
So nice of you
Great work Rahul.. Thank you..
Glad you liked it
Awesome videos whenever I see your videos first I will like then I will see because I know u will explain every steps with details waiting for openshift lecture for deploying application
Thanks and welcome
Best and clean explanation 👍
Glad you think so!
Bro... fantastic Explanation.. Got the clear picture !! :)
Glad to know
Brilliant tutorial.
Tremendous explanation 💥🤘❣️
Thanks a lot 😊
Amazing video !!! When do you plan to create a video more automation focused with with terraform, Ansible, docker, EKS, Jenkins, CI/CD all combined ?
Soon I will prepare something around that just not getting sufficient time
greate video and crestal clear explation ever👏
Glad it was helpful!
awesome but you forgot to explain how to connect Kubernetes cluster in Jenkins pipeline
very nice and helpful
Thanks and welcome
excellent video !!!
You welcome
Excellent session Boss!!
Glad you enjoyed it!
@@RahulWagh when do u have plan to make video on complete automation with Terraform, Ansible, Docker, K8s, Jenkins etc.
Let see when i get little more time in my schedule may be in the next month you will see
Nice explanation
Thanks for liking
Great tutorial!
Thank you! Cheers!
Which kube context are you using and how did you create that? Did you update aws-auth? any AWS role attached to the machine?
Hi Rahul. Trust you are doing great. Your video has been a good refrence for me.
Question: I was just thinking about the installation of the docker using jenkins user. Would it give us same result if we spin up a new EC2 instance for installing docker? Would our k8 cluster not be affected if jenkins server has issue? Anticipating your response. Thank you
Thanks you liked my videos.
Answering to you question and as per my experience I would keep isolation between kubernetes cluster and the EC2 instance where i have installed Docker,Jenkins.
There is no problem using docker using jenkins user or install docker and jenkins to docker user group but I would always install jenkins on separate server(ec2) and kubernetes cluster on separate(EKS cluster.)
If you keep the isoation the way i have explained in the video then there should not be any problem
Awesome sir
Hi Rahul,
Big thank you for taking time and explaining DevOps processes step by step. Appreciate your efforts. I have a one concern - I am not sure from where this "jenkins" user came in picture. Did it get created after installation of Jenkins on EC2 machine?
- धन्यवाद !
Glad you liked it. Jenkins user is created at the time of installation
When we install Jenkins by default Jenkins User has been created...
cd /etc/passwd
In last we can see the default Jenkins User...then set the password for the user with passwd command
@@lakshmanakumar1594 Thank you very much :)
Excellent!
Really this video is great !!
Glad to know that
Thanks Really Helpful
Glad it helped
Hii...Can you make video on cluster autos calling...it'll help us so much...bcs your way of explanation is Excellent tq
Will try surely on cluster auto scalling
Awesome you are the best
Cheer’s
instead of installing aws cli on 24:51, you could attach a role to the jenkins instances with ec2 assume role permissions, saves u the trouble of creating access keys
Great suggestion
It was a great video
Glad you liked it
Great thank you so much
You welcome
Rahul, am new to devops and your videos are awesome very easy to understand, am in the US and is there anyway I can get in contact with you for some guidance, thanks
Does `kubectl apply` inside pipeline work because you configured credentials with `aws configure` previously?
I have completed this pipeline today, great step by step guide. Do you have guide for Jenkins master slave architecture and deploy to EKS.
Not yet navnath but may be in future i can prepare around it
Dear Mr. Rahul, Hope you are doing well. Thanks a lot for your kind video. This is really helpful.
May I request you to please upload a new video on an end to end Microservice application deployment in AWS EKS .. like from pushing the code in GIT to EKS Deployment ... and the complete flow will be in a pipeline calling BuildSpec.yaml ( including Docker build to create Image) , Kubernetes Manifest file , services.yaml file etc..
I will be highly grateful if you can arrange something like this !! 🙂
Will try to put it onto my todo list. Hopefully i will come up something around that in upcoming months
@@RahulWagh Happy New Year Rahul !! Hope all is well . Kindly can you please look into this request ? I have some upcoming interviews so will be grateful if you kindly provide something on this . 🙂
Hi Rahul it's really amazing and you way to explain is very nice u made simple steps, will you pls share me playlist link for this whole task, as I am unable to find playlist ☺️☺️☺️☺️☺️
Here are the playlist -
1. Terraform - th-cam.com/play/PL7iMyoQPMtAOz187ezONf7pL8oGZRobYl.html
2. Kubernetes - th-cam.com/play/PL7iMyoQPMtAP4nNc7IqQncWm6DGlIee66.html
3. Helm chart - th-cam.com/play/PL7iMyoQPMtANm_35XWjkNzDCcsw9vy01b.html
Thank you so much for detailed tutorial. Have a doubt! How are we connecting to AWS k8 cluster without providing any AWS creds either in jenkins or jenkins file? Can you please clarify on this. Also, Please tell me the plugin names for docker and k8 that needs to be installed.
You need to install the AWS CLI onto your machine so that you can connect to AWS. Also after installing the AWS CLI you need to run the command aws configure where you need to enter access key and secret id
Thanks 👍 🙏
Welcome 👍
Thanks 👍👍👌
Welcome 😊
Great tutorial , but now i am more confuse , i have my jenkins application running on Ec2 instance , and i have multiple eks clusters in different aws regions , and i want to connect to my eks cluster , for that i am using Kubernetes plugins in jenkins ans providing, but getting error. connection details in cloud configuration section. also kubctl and eksctl is installed on my local machine., so accessing Eks cluster from my local machine , you seems to skip this step , or am I missing something here?
Well in past I tried using Jenkins kubernetes plugin but was not satisfied with it. It’s buggy and I think it’s not being maintained by the community
So. Smoothly explanation Can you please explain pipeline how to write. In your way
Hey paramesh thanks for the comment.
sorry i did not get your question about how to write?
@@RahulWagh He means how to write the JenkinsFile of our pipeline
did you created any RBAC role for jenkins deployment because i'm getting error while running deployment stage like Error from server (Forbidden): deployments.apps is forbidden and also
kubectl auth can-i create deployments --namespace default
no
no i did not create any RBAC for jenkins
Rahul, where did you provide Kube config details in Jenkins server? How is it able to connect to & authenticate with kibe-api server from Jenkins ?
I have installed the kubectl utility on my machine which will take of kube config details
@@RahulWagh Yes, having the Kubernetes config details on profile will work when kubectl are run from a terminal. But how Jenkins is able to detect those details. It must know some how to look at default Kubernetes config, there must be a plugin installed in your Jenkins setup which is doing that job.
In my case (unluckily windows env.), I have kubectl installed & in system path variable. From a command prompt, I'm able to query kibe-api server. But the same kubectl commands are not recognised by Jenkins. I think I miss a plugin here, but no idea which one.
@@RahulWagh Found the issue. Its because Jenkins is running as system account instead of its own/admin. Due to this, the relevant kubeconfig files are not used by Jenkins.
Great to know you found the issue
how do i change the final message in the rest end point?
can you tell us how you connected Kubernetes please let me know plugin used
You just need to install kubectl utility and it will take care of the connection
When I build in jenkins I got an error like kubectl command not found.. But in my ubuntu I'm able to get commands
I think either your jenkins user does not have permission to run kubectl command or you did not install the kubectl on jenkins virtual machine
Sir , how about testing within the CI/CD pipeline ?
Yes off course you could write unit tests or integration test which can be run and tested along with pipeline.
As the video was quite long so it was really hard to incorporate everything into one session.
I can see you are having trouble tagging the correct docker image name.
I would recommend follow these steps and use the correct docker image name along the docker repository name which you have created on docker.hub.com .
here is link on how to push correct docker image - jhooq.com/deploy-spring-boot-microservices-on-kubernetes/#step-6-create-docker-registry-at-httpshubdockercom
Also did you created repository with the name - kazishafin/jhooq-docker-demo on docker hub?
36:50 eks after execution of that command i got an error called...invalid apiversion version string is empty...how can I resolve the issue
It says this is 13, where is the beginning please. I want to start and follow from the beginning.
Gradle all dont use If I use maven then IMAGE CREATION and then check then image then go to ECR registry so no need to pull only push is OK
Where is the beginning
sir, pipeline was not able to use kubectl command . It says " rror from server (Forbidden): window.location.replace('/login?from=%2Fswagger-2.0.0.pb-v1%3Ftimeout%3D32s'); " and authentication required ! Help me out of this sir.
Probably something wrong with your certificates
bro same error....how did you solve it??
@@RahulWagh sir same error please help
Thank you for this great tutorial. It was so clear. However, my pipeline fails during Kubernetes deployment. I get the error below
+ kubectl apply -f k8s-spring-boot-deployment.yml
Error from server (Forbidden): window.location.replace('/login?from=%2Fswagger-2.0.0.pb-v1%3Ftimeout%3D32s');
Authentication required
.
Please can someone help me here?
Probably you are not able to pull the docker image from docker hub. Need to see the logs there could be many things behind it
@@RahulWagh Thanks so much for your reply. I was able to fix the issue. The problem was with my kubectl version. I just had to update my cluster version using the command aws eks update-kubeconfig --name cluster-name
It worked and everything built perfectly.
Thank you very much Rahul. This video is just perfect and well explained.
Most welcome!
Great tutorial!
Glad you liked it.