ฝัง
- เผยแพร่เมื่อ 29 ก.ย. 2024
- Quickly spin up an OpenShift 4 cluster to start building and running your container workloads.
This tutorial follows the Bare Metal UPI method of installation and uses ESXi to virtualise the 'bare metal' machines. More information and details instructions are available in the GitHub Repo.
GitHub Repo: github.com/rya...
---
Official OpenShift documentation: docs.openshift...
---
Music: Chill, Sakura Hz - • Chill - sakura Hz (No ...
Excellent video! So clear and well laid out. I like also how you give an overview of the whole process in the beginning, since this is almost impossible to get from the RedHat documentation when you start with it for the first time.
What’s ssh client you use in MacOS ? Thanks and excellent video
Hi Ryan,
OKD port group is the interface that facing to internet right ?
Hi Ryan, I want to install OKE, can you share a video on it
Hi Manoj, its the same process just use the OKD installer and Fedora CoreOS.
Just followed these instructions for a 4.6 install and it worked like a charm... only thing I did different is that I simply allowed the coreos image to boot completely and then I cut and pasted the following:
sudo coreos-installer install /dev/sda \
--insecure \
--image-url 192.168.22.1:8080/ocp4/rhcos \
--insecure-ignition \
--ignition-url 192.168.22.1:8080/ocp4/bootstrap.ign
(replacing bootstrap.ign with master.ign and worker.ign respectively)
I like this approach because it was much easier to cut and paste into the running vm than it is to type out the kernel options when booting each VM.
Thank you so much Ryan for doing this! Great How-To. Calm and very clear and structured explanations! Perfekt!
It is impossible to get it done by reading the official documentation, by watching this video, I believe that I can do it ! Thank you ! On question, It is the same way to installing the OKD?
Thanks Paul, yes the same method can be used.
Had done this with 4.2. But i used Ansible. Doing this by hand is pretty time consuming. Ocp-svc server usually is called Bastion server. :) Will do this in couple of days again but this time for provisioning will use Terraform and Vault for secrets. Hope there are not a lot of changes since 4.2. Great video! Also... if interested in S3 compatible alternative, there are great project called MinIO.
Hi ! Please, do you have a gitrepo or something like that where we could see a template of the provisioning with Terraform or Ansible ? Thanks
Hi Ryan,
many thanks for this excellent tutorial, i have done on 4.12 successfully.
Now I am able to login with kube:admin user only, not from other admin users. Can you or anyone can help on this?
How i can access to dashboard from my laptop?
please keep this up2date!
Hi Ray, this is an excellent video. We were using this guide video for a long time since 4.6.8 version. But with current 4.7.0 version, though all the machines turn into green but, the installation never completes. "authentication, machine-api, openshift-apiserver" cluster operators never gets up. Also we have installed, assisted version of this 4.7.0. It also stucks at that point. Do you have any experience with 4.7.0?
Great tutorial! I have a Question: I would like to change some network settings (IP, Domain). What files I need to change from your git repository?
Great question! I did the same and the dns wasn't right due to the db files having specific names. I changed them and tried redoing the steps but still couldn't resolve dns correctly.
Hi Bro ,I followed you tutorial to install openshift, but while fetching ignition file from server it encounters some connection exception .
I tried ping the host after not interrupting live boot then it shows like unable to reach host.Any idea on what is the issue?
Hi Ryan, I encountered an issue when installing the openshift cluster. After bootstrap+cp+workers are installed, all the vms are responding to "crictl ps" with "failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded". I am trying to install the 4.6.6 openshift cluster. Do you have any ideea regarding this issue?
i'm facing same issue on 4.6.8. How did you fix it?
@@purushdas1577 I didn't :)). I followed Ryan github to install openshift 4.5.6
@@nicu9735 - Also seeing this error. @Ryan Hay - Any thoughts?
: DEBUG OpenShift Installer 4.6.8
DEBUG Built from commit f5ba6239853f0904704c04d8b1c04c78172f1141
INFO Waiting up to 20m0s for the Kubernetes API at api.lab.ocp.lan:6443...
INFO API v1.19.0+7070803 up
INFO Waiting up to 30m0s for bootstrapping to complete...
W0427 15:10:21.649653 32330 reflector.go:326] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *v1.ConfigMap ended with: very short watch: k8s.io/client-go/tools/watch/informerwatcher.go:146: Unexpected watch close - watch lasted less than a second and no items received
E0427 15:10:22.651115 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:23.652676 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:24.654194 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:25.655750 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
Thanks man your video helped alot understanding pre install setup.
warning: utilize your .ign files with in 24 hours of creation else it will give certificate error.
Hi Ryan, I am unable to boot both worker node after reboot. Getting Get Result :internal server error but
bootstrap and all CP nodes boot properly after reboot. What is the issue??
Thank you so much Ryan, for doing this quality content very clear explanation and step by step. I'm planning my own lab with ESXi 6.7 and all VMS using Centos core OS, any suggestion on this?, I think using Centos on all VMware's is not a restriction forOpenShit :), am I right?.
Kind regards from Lima, Peru.
Thank you! Im not familiar with CentOS Core OS and can't find anything on it. The OpenShift documentation does however specify that the bootstrap and control plane machines must use Red Hat Enterprise Linux CoreOS (RHCOS) as the OS, the compute can use either RHCOS or RHEL.
Very Well done! I'm not an experet in this field. Certainly you need to know the basics of Linux, Routing and Protocolas like DHCP or TCP. But man...I by watgin the vide a have a pretty good sense how things need to executed in which order and why. thanks!
Hernandez Shirley Lewis Matthew Young Richard
Hi Ryan, an awesome video. One addition, you need to add a firewall rule to ocp-svc - firewall-cmd --zone=internal --add-port=53/tcp --permanent. I found that random Pods would fail DNS lookup. Further investigation showed that happened when the DNS response exceeded 512 bytes on UDP.
Hi Ryan,
This video is very helpful and easy understanding.
One question: what content we put in /share/registry (nfs share)
Loved the video, appreciate the effort. But just a warning to viewers. Don't spent all your time trying to set this up as a test environment and if you need to actually test something anytime soon. I tried this twice from front to back using the latest versions of RedHat with CentOS 8.3, triple checked every step, and I used all the time I had from Friday night to 2AM Monday. Its really close but non functional. Same errors on the GET calls to the api host. So I burned 3 days I didn't have to lose. Don't make my mistake! It looks so close to working but not quite there.
agreed! already spent 3+ days and still errors on GET calls will try a little bit more before I wrap up for a while I guess...
Hello, i am not sure what is wrong, but the farest I have gone is when bootstrap is ready but the controll plane nodes are going DOWN and UP.
I have restarted the procedure at least 6 times from scratch.
At one point I have realized the ocp-svc node has different timezone than the bootstrap and controll plane nodes hence the certificates generated by the installer on the svc node are different and bootstrap and controllers failing.
After fixing this, API is still not stable. Don't have a clue what is wrong.
Just while I have been typing the API bootstrap and one controller came UP, but after copule of minutes they usually go DOWN again. Never seen full green by all 3 cp nodes both API and config backend.
1) What is the process for booting the master/workstations from the ignition files?
Bootstrap,master,worker all should be powered on at the same time.
Bootstrap first then the master machine about an hour later, then the workers after another hour
What does the wait for bootstrap to complete mean
1) The bootstrap server has all the files from the bastion server independent of the status of the master/nodes
2) Or masters and workers are all configured and working
What indication can we look at to see what is happening on the bootstrap server. The are logs but what should we look for in the logs.
Thanks
I get internal server errors from the http server on the bootstrap node when a worker node tries to bootstrap.
Hi
Is it workable on my laptop (8 Core , 64GB RAM).
Regards
Zaheer
this video is so helpful to me! thanks
Glad it helped!
Hi Ryan, great video! very informative. Stuck on deploying the control-plane nodes however wondering if you could advise.... ignition[805] GET error: Get api-int.ocp.lan:22623/config/master I have validated DNS is working, however that file doesn't exist on the services node at all if I attempt to wget it etc...
Resolution was update from RHEL 8.2 --> 8.3. It seems there was a bug which was fixed with an update. bugzilla.redhat.com/show_bug.cgi?id=1708663
Hi @@matthewklos7184 , I have the same error...
ignition[925] GET error: Get api-int.lab.ocp.lan:22623/config/master: EOF
I'm using CentOS-8.2.2004-x86_64-dvd1.iso for ocp-svc
Shall I use rhel 8.3 for ocp-svc instead of CoreOs 8.2?.. will this solve the issue?. Thanks!
Im not sure about this issue. I would SSH into the bootstrap node and take a look around for any errors/containers not starting because it seems like the machine config server is not coming up for some reason.
@Matthew Klos
was the issue resolved, I hit the same error.
@@juanchicchon5017 Seeing this same issue now, i tried to send a CURL request to that endpoint and i keep getting this : OpenSSL SSL_connect: SSL_ERROR_SYSCALL. How did you manage to resolve this problem?
sometimes it works sometimes it doesnt, is this anything to do with the masking or DNS setup?
Thank you very much Ryan, It helped me a lot in setting up my first OCP4.6 .👍👍
This is amazing and supremely helpful! Especially the architecture definition and deployment flow in the beginning. Thank you so much.
Can you post some trouble shooting info on your github site
Thank you so much by the video. Can you help me? Is possible install OKD4 on a VPS Ubuntu 20.04 with 24gb for a homelab cluster all in one?
Hi Anderson, It depends if the VPS supports nested virtualisation for you to provision your cluster nodes. You would probably require at least double the RAM though.
Hi Ryan thanks for the comprehensive tutorial. I have a question what could be the architectural changes in case DNS being part of a separate machine ? does that machine also requires ocp network ? and do we need to update api, api-int, ocp-svc and * records to be pointing to external IP of machine having loadbalancer ?
Whats the difference between this and OKD?
Can this be installed any regions? Is it payable?
Hi Ryan Hay. Thank you very much for making the video available. I have one question. Your process is also supposed to work with the latest version of Red Hat CoreOS, at 4.72, right? I ask this because I am having some problems raising the initial setup
Forget. I put more power(CPU and memory) in my cluster and everything works fine
Hey Ryan, the HAProxy stats page is totally cool - but I just learned a valuable lesson. I have followed these instructions a couple of times now, with great success. Today, I started working with a slightly larger cluster - one with 3 workers instead of 2. When I looked at the haproxy stats page - one of my worker nodes was red. I freaked out and spent a bunch of time trying to figure it out. Turns out, Openshift "ingress" spins up 2 pods listening on http (80) and https (443) - so you are only going to get "green" on two of your worker nodes at any time.
TLDR; if you have more than 2 worker nodes, it's ok if HAProxy thinks some of them are down. Trust `oc get nodes` to tell you if the nodes are up and running!
Hi Eric, good point, I should have mentioned what the HAProxy check was doing. I recall someone else in the comments ran into a similar issue when they made the master nodes schedulable. The console pods were spun up on the master nodes instead of the workers and the HAProxy config wasn't pointing to the masters for http/https. I guess I could have left the HAProxy stats page out but I just found it convenient to visualise installation progress on a long running install.
Is this installation possible on Hyper V
Great Explanation. Could you please create more videos for OpenShift related? Not much videos for openShift in the TH-cam found... Ty
trying to ping github to get argo working
Hi Ryan Hay,
I want to install the latest version of OpenShift on VMWare but there is a condition that I would not have internet access instead of that I have to create a mirror Repository server where I will put all the images and software. So what required changes do I have to make on the Bootstrap virtual machine (ocp-bootstrap) machine and Services virtual machine (ocp-svc). If I follow your video for installation.
Hi! Is it possible to add the pull secret after install? I used {"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}} but now I need redhat operators, specifically Cluster Logging.
This installation guide has been extremely helpful. I am running into "failed fetching image headers from 192.168.22.1:8080/ocp4/rhcos". Any ideas or things I should be looking at? TIA!
Did you change the ownership and permissions of the web server directory? If so are you able to attach another host to the openshift network and try to wget the image? Does that work?
@@RyanHay Thank you for your reply. It was the internal network setup that caused the problem.
Hey Ryan, Great Video.... what spec NUC would you recommend to get this working?
was looking at a NUC10i7FNH, NUC 10 Core i7-10710U, 64GB DDR4 Mem, 128GB
SATA M.2 SSD, No OS
with NAS 4tb synology.
Thanks
Hi Ryan
I have separate servers for DNS,AD and DHCP so which part I have to skip for creating it. As there are 18 section for building UPI.
{"error":"server_error","error_description":"The authorization server encountered an unexpected condition that prevented it from fulfilling the request.","state":"ac7a094f"} why this error occured after every 5 to 10 mins
Please suggest me a solution please
I'm stuck on bring up the worker node. anybody know why? the error is worker node failed to fetch ignition file. worker.ign
Check file permissions in helper node make curl command
./openshift-install create manifests --dir ~/ocp-install/
? Platform [Use arrows to move, enter to select, type to filter, ? for more help]
> aws
azure
gcp
openstack
ovirt
vsphere
why my command appeared different the result
Hi Ryan, this is very helpful! Thank you! Are there plans for you to step through the creation of the Openshift UI for your CRs deployed by your custom Operators, using Spec Descriptors? If not, I'd love to see it! Subscribed! (See: Declarative Dynamic UI for your Operator)
I have a question. Is this tutorial works for OKD ? Because,, i dig other videos and tutorial.. i try my best to install this OKD on my Hyper V,, but it wont works.
Hello Ryan I was not able to find the CentOS 8 ISO image for linux online, instead I installed CentOS 8 Stream. Is there any difference? I am stuck at the Deploy Openshift step, the bootstrap command on git is not working. Is it because of the different vesion I installed. Any suggestion how to fix it?
What is the actual duration to fully configure and setup end to end? what if I had a 360 worker node config, how long should I expect to install that?
Don't know the time but haproxy stats will be there which will show you the status of the nodes
While booting coreos getting an error cant find /dev/sda....could anyone help on this please?
You should boot your vm for first time with CD DVD iso file after Boot enter our ignition details then stop the vm then change the boot orders like we have to follow it
Hello Ryan, I am trying to setup the lab by following this video. I am using VMware Vsphere to setup the openshift container cluster. I have a VLAn created for internal network but for external network, I don't know how to create an external VM network which you added as an additional network for the helper machine. I got struck here for past 1 week. I don't know how to take it further. Since I new to VMware and linux , learning it. Please guide me how to setup the external network and connect?
Ryan ,video very excellent..I am planning to install 4.11version , i believe installation instructions got changed. I request you to add new video specific to 4.11 version
Hi @ryan great tutorial! One question why did you use baremetal insteand of esx/vsphere provision?
Hey @czerogr, thanks! 2 reasons; firstly, I just wanted to explore a lot of the things the Installer Provisioned Infra (IPI) method was doing behind the scenes. Secondly, because of system resources... The host I was using only has 64GB of RAM and the vCenter Server required for the vSphere install I think at minimum takes up around 12GB of RAM which starts to take away from my already under spec'd master and worker nodes.
Hi Ryan,
Nice video. Bootstrap came up fine for me by fetching bootstrap ignition file and correct IP and hostname was also assigned automatically to bootstrap node. However, when I try to bring up control plane VMs, it fails with below error
time="2021-09-21T09:30:22-04:00" level=error msg="Bootstrap failed to complete: an error on the server (\"\") has prevented the request from succeeding"
time="2021-09-21T09:30:22-04:00" level=error msg="Failed waiting for Kubernetes API. This error usually happens when there is a problem on the bootstrap host that prevents creating a temporary control plane."
time="2021-09-21T09:30:22-04:00" level=fatal msg="Bootstrap failed to complete"
Could you help please. Where could be the issue?
Hi Ryan. When I power on bootstrap after writing the url in boot time after loading, I’m getting error “7 unrandom warning’s missed due to ratelimiting “ do you have any idea about this.
Thanks In advance.
Hello Ryan, I have a question , Is it necessary to use vmware vsphere? is it possible with workstation?
Or virtual box on windows?
quickly, LOL, uh no.....but so far that best instructions I have found!
Hi Ryan, i tried your mentioned method but two operators (Console and Authentication) shows not available, can you guide me on that i am stuck here.
Hello Ryan, Can you please make a video on openshift install in restricted network on vsphere using UPI Method
Hi, I need your help. I have doubt in the HAproxy section, I can not start the Haproxy it showing. please give me advice
Wow, this is very good. Thanks a lot Ryan! You won the internet with this video :)
Glad you liked it!
Hi Ryan, followed your video, but i am stuck waiting for all cluster operator to be available. Especially authentication, console, monitoring and openshift-apiserver. Any advice on this?
I followed the tutorial exactly until reached 18:20, kubectl: command not found . . .
Excellent video Ryan!, It helped me a lot to understand the whole installation process. I would like to know how to add advanced network configuration, for example to add bonding, I found related information on the RedHat documentation:
bond=bond0:em1,em2:mode=active-backup
ip=bond0:dhcp
Where em1 and em2 are the physical network devices. Those two lines should be added during PXE booting on the RHCOS machines? I found also that we could modify a live install ISO image, with customized network settings profile (using coreos-installer binary). Could you please help me to clarify?
Hey Ryan , can l use just 1 network , for example if l dedicate complete 1 esxi to ocp ??
wow this video is excellent resource , thanks for your hard work. keep working like this.
What is the best option to choose if I want to create private docker images registry, e.g. on the University server for the students? Portus/Harbor?
I am using same steps with uefi script and secure boot enabled its failing,,,can you help me
Hi Ryan, I am getting error while boostrap image from coreos os.. it says compressed file format is not support.
i tried to install 3 master, 7 worker cluster. Installed only 3 masters. Help pls.
Excellent step-by-step guide. This is the best one out there.
I've been stuck for two weeks though sorting out the x509 cert errors when the worker install pulls api-int.lab.ocp.lan:22623/config/worker. The CP's finished after a few tries, but the workers are stuck in a 1000 tries. Apparently COREOS 4.4 has a check that's not in 4.3. I tried moving back to 4.3 but same error. There's an expired cert somewhere but I'm not sure where or how to fix it. I tried extracting a PEM file from Firefox and pushing it to the HAProxy certs PEM, but same problem. Any suggestions appreciated. Thank you.
Did you approve any outstanding csr's? `oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs oc adm certificate approve`
@@RyanHay Hi Ryan. Yes I tried approving the csr but I couldn't even connect with oc. I got it up finally. I rebuilt using OKD installer and Fedora Coreos images and initially had same problem so that wasn't it. I rebuilt again but putting the SSH key in the yaml file in single quotes instead of double quotes and it worked. I'm pretty sure that was the problem bcz the error was referring to x509 certs.
Thank you for your reply and your exceptional channel!!
One more thought. I think more likely was that my yaml SSH key was not in sync with the SSH directory on my services vm. Beginner error.
They say you learn by figuring things out.
Okay enough of this. Now on to new challenges.
Actually this is brilliant. Thank you, good guy!
Great video, I have a question, does bootstrap have to have internet? I did the lab but bootstrap doesn't have internet, how did you get it to?
Failed to start Berkeley Internet Name Domain (DNS).
hi @ryan i have some issue here regarding fetching rhcos image from http but when i test the web page i can find it and install it on desktop and your response appreciated
can we install windows platform and linux platform on rhel openshift at same time?
Hello Ryan, do you provide any online training on openshift ? thanks
Can we do openshift on bare metal using azure?
Can it be like 3 control planes as VM and 2 worker nodes as baremetal ?
Hello Ryan, I'm trying to assign my worker/control plane/bootstrap nodes with ip addresses for the open-shift network "bottom portion of the diagram" however, when I edit the network configurations using nmtui command it does not retain the configs. I also tried restarting the network service using 'systemctl restart network' and 'systemctl restart networkmanager' after making these changes but I'm prompted for a password that I never created for these vms. this is preventing us from deploying open-shift on my worker/control plane/bootstrap nodes. any clue or insight would help!
What an amazing video Ryan. You ROCK !!. Thank you so much Sir. Really helpful and beneficial.
thanks for the demo Ryan!! question for you: why the svc machine has to be in the same network than your local network ?
Hi Ryan
what should i do for this error
ERROR Attempted to gather ClusterOperator status after wait failure: listing ClusterOperator objects: Get api.lab.ocp.lan:6443/apis/config.openshift.io/v1/clusteroperators: EOF
INFO Use the following commands to gather logs from the cluster
INFO openshift-install gather bootstrap --help
FATAL failed waiting for Kubernetes API: Get api.lab.ocp.lan:6443/version?timeout=32s: EOF
this error found when i run this command
./openshift-install --dir ~/ocp-install wait-for bootstrap-complete --log-level=debug
While the documentation of openshift is a complete mess, you explain the things like a walk in the park. Bravo!
Thank you very much! Great howto!
Really appreciate your work....thank you so much for your guidance
Thanks for sharing this fantastic tutorial! I hope to try it myself. I do not have vmware esxi so would virtualbox or Hyper-V be an acceptable substitute?
It's really useful,Is that possible to do it in 1st method using registry!
I was in need of this. Thanks @Ryan Hay :)
Hi, im a little bewildred regarding the following:
Power on the ocp-bootstrap host and ocp-cp-# hosts and select 'Tab' to enter boot configuration.
Select 'Tab' where do i do that? And is the image rhcos-live.x86_64.iso the correct one?
Hi Anders, its the tab key on your keyboard. The image is something like: rhcos-X.X.X-x86_64-metal.x86_64.raw.gz if you haven't renamed it.
great work. Thanks a LOT. Should have watched this a week before, may have saved time.
why we are using the bootstrap ?
Hi Ryan , thank you for your effort , just want to inform you that I have problem follwing your steps , as the redhat interface has been changed and I cannot find the redhat coreOS
Check documentation mirror images
I'm not sure that last remark about shutting down and start any time you like is accurate. I've experienced several times where a cluster is powered down for a couple weeks and then when you go to resume messing with it everything is badly broken because all the certificates have expired. Looks like they may only be good for about a month so if you shut it off just before whenever it renews until they expire it leaves you with a pumpkin.
Hey Matthew. As per the documentation As of OpenShift Container Platform 4.4.8, the cluster can automatically recover from expired control plane certificates. The exception is that you must manually approve the pending node-bootstrapper certificate signing requests (CSRs) to recover kubelet certificates. To do that in bulk you can run: "oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs oc adm certificate approve".
Useful information. Tks Ryan.