Install Hadoop Cluster on your Windows PC | HDP Sandbox installation step-by-step guide | Lecture 3
ฝัง
- เผยแพร่เมื่อ 20 ส.ค. 2024
- This lecture is all about setting up Hadoop cluster on your Windows PC where we have installed Hortonworks Data Platform which will allow to work with different Hadoop ecosystem tools such as
MapReduce
YARN
Hive
Apache Pig
Apache HBase and HBase components
Apache mahout
Sqoop
Apache Spark
Apache Flume
Ambari
Zookeeper
Apache Oozie
---------------------------------------------------------------------------------------------------------
Installation links:
Oracle VM Virtualbox: download.virtu...
HDP Sandbox link: archive.cloude...
HDP Sandbox installation guide: hortonworks.co...
-------------------------------------------------------------------------------------------------------------
Want to know more about Big Data? then checkout the full course dedicated to Big Data fundamentals: • Big Data Full Course
In the previous lecture we have seen Hadoop ecosystem and their different components in details and how they work together to crunch Big Data!
Here we have seen the basic introductory part for following Hadoop tools:
MapReduce, YARN, Hive, Apache Pig, Apache HBase and HBase components, Apache mahout, Sqoop, Apache Spark, Apache Flume, Ambari, Zookeeper and Apache Oozie.
Also check out similar informative videos in the field of cloud computing:
What is Big Data: • What is Big Data? | Bi...
How Cloud Computing changed the world: • How Cloud Computing ch...
What is Cloud? • What is Cloud Computing?
Top 10 facts about Cloud Computing that will blow your mind! • Top 10 facts about Clo...
Audience
This tutorial is made for professionals who are willing to learn the basics of Big Data Analytics using Hadoop Ecosystem and become a Hadoop Developer. Software Professionals, Analytics Professionals, and ETL developers are the key beneficiaries of this course.
Prerequisites
Before you start proceeding with this course, I am assuming that you have some basic knowledge to Core Java, database concepts, and any of the Linux operating system flavors.
---------------------------------------------------------------------------------------------------------------------------
Check out our full course topic wise playlist on some of the most popular technologies:
SQL Full Course Playlist-
• SQL Full Course
PYTHON Full Course Playlist-
• Python Full Course
Data Warehouse Playlist-
• Data Warehouse Full Co...
Unix Shell Scripting Full Course Playlist-
• Unix Shell Scripting F...
--------------------------------------------------------------------------------------------------------------------------
Don't forget to like and follow us on our social media accounts which are linked below.
Facebook-
/ ampcode
Instagram-
/ ampcode_tutorials
Twitter-
/ ampcodetutorial
Tumblr-
ampcode.tumblr.com
-------------------------------------------------------------------------------------------------------------------------
Channel Description-
AmpCode provides you e-learning platform with a mission of making education accessible to every student. AmpCode will provide you tutorials, full courses of some of the best technologies in the world today.By subscribing to this channel, you will never miss out on high quality videos on trending topics in the areas of Big Data & Hadoop, DevOps, Machine Learning, Artificial Intelligence, Angular, Data Science, Apache Spark, Python, Selenium, Tableau, AWS , Digital Marketing and many more.
#bigdata #datascience #technology #dataanalytics #datascientist #hadoop #hdfs #iot #hadoop #hdp
![ไปอยู่ไสมา - ช๊อปเปอร์ โตเกียวมิวสิค [ OFFICIAL MUSIC VIDEO ]](http://i.ytimg.com/vi/sWuVNbRTJbc/mqdefault.jpg)
You are awesome, & highly underrated, GBU bro
Thank you so much!
really Helpful
Hey hi@ampCode ,I am not able to find hdp sandbox download option on cloudera website.I tried other options too but not able to get it.Could you please share any other option to work on Hadoop.
same here.
@@AnanyaSingh13529 it is deleted now 😢😢😢
@@AnanyaSingh13529u got any other way to run it?
Localhost refuses to connect, how to resolve?
Sorry for late response. You can always restart your sandbox as these issues will be related to connection issues and will be restored in the next kickoff of your sandbox
Thank you
Thank you so much! 😇
When we launch localhost:1080 it's not showing HDFS..
Login also it takes admin for both username and password
Sorry for late response. Could you please confirm if your ambari is running fine? let know if you still face any issues. :)
As per instructions Oracle box and HDP sand box installed. But unable to get the Ambari Login. with localhost:1080. Help me in this issue.
Could you please try on localhost:8080 which is the port for Ambari login. please let me know if the issue still persists. :)
after selecting virtualbox and clicking let's go it is asking for signin and a no. of fields to be filled like business mail, company, job title, mob no etc..
Hello there!
Yes, they will ask you for your basic information before beginning the download. Once you are done, the download will start. It is completely free and no charges will incur so no need to worry😊
Hi, I have alloted 4gb ram to the hortonworks environment and started it, but the screen is stuck at Extracting and Loading the Hortonworks sandbox from a very long time. Any idea how to resolve this? i have 8 gb ram. will it work on my pc?
Hello, yes it will definitely work on you PC! 😃
During the first boot, it usually takes more time(maybe 2-3 hours also for some users) and its only one time activity. but once all the extraction and loading is done, it will only take around 20 mins to boot your sandbox afterwards. Please let me know if any😊
Hi Ashay, I am trying to learn the Big data concepts. HDP sandbox is no more available to download. what do you recommend ?
did you get any alternative??
Its not letting download the HDP sandbox.
Says that access is rectricetd.
Any other way ?
Sorry for late response. Have you tried downloading HDP from Cloudera website as mentioned? You just have to sign up and then you’ll able to download the file
Hi, I have alloted 10gb ram to the hortonworks environment and started it, but the screen is stuck at Extracting and Loading the Hortonworks sandbox from a very long time(hours). Any idea how to resolve this?
Hello Vedanth, I can totally understand your pain and I also faced the same thing. But let me assure you that this will only be the one time activity and it will only take around 20 mins after that to start all the associated services. :) Could you please let me know if your PC is at least having 16 GB ram and 8th Gen and above Intel Core i5/i7 processor or similar?
@@ampcode Hi Ampcode!! Yes I actually waited for sometime and it worked. Now I'm following your playlist!
@@vedanthvbaliga Thanks for confirming! Keep learning, keep growing! 😊
Hi, getting "FATAL: No bootable medium found! System halted". what to do?
Sorry for late response. Could you please check if all your services are running fine?
Hi, how do you decarese or slim the HDP 2.6.5? Because it grows in size when I practice ....
Yes I also faced the same issue😕 but as we kept practicing, our data files, logs generated by every services on the cluster will also gradually grow. But I found some suggestions on the cloudera community which you can refer. community.cloudera.com/t5/Support-Questions/HDP-2-5-Sandbox-Keeps-running-out-of-space/td-p/171335
Hope this’ll be helpful!😊
@@ampcode could removing/uninstalling services decrease resources like dispace requirements, etc.?
now impossible to download the hdp sandbox, they put it behind a paywall so its impossible to follow your tuto as it doesnt correspond anymore to the reality
any solutions now?
I am getting 3 alerts while starting Ambari - 2 in Hive and 1 in Ranger, can you please help me solve that
Hey Srushti,
I have also faced this issue multiple times. You can try shutting down the HDP Sandbox by doing ACPI Shutdown option and then restart it. It had always worked every-time for me. Also I’m giving u below link if the issue persists.
community.cloudera.com/t5/Support-Questions/Alerts-IN-Ambari-Sandbox-Pls-view-Attachment/td-p/298211
docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-operations/content/hive_alerts.html
Please do let me know in case of any further issues
i don't know how to dowload hdp sandbox on cloudera because i don't see anywhere.what else to use with hadoop.Can you suggest me?
@AmpCode I'm using laptop having 8 GB ram and 4th Gen and above Intel Core i5 processor so HDP showing mi error ....
How can solved it ....
or any other option for that one
Since you are having 8 GB ram, you can allocate 4 gb ram to the sandbox. But what is the error you are facing?
HI,
everythings good but when trying to get the local host 1080 I see error or could not be reached.
my virtual box -7.1
my hdp sandbox - 2.6.5
Please let me know how to connect
after launch the dashboard directly going to jenkin page
Sorry for late response. Could you please try connecting to Ambari once as mentioned in the video. Please let ne know if the issue persists😊
Sir, I do not have a powerful PC, only 8GB RAM Laptop and AMD Ryzen 3 3250U processor, Can I do the course along with you? If I just watch along, will it help, please sir reply.
HI @AmpCode i had confusion while downloading hadoop we can strait download hadoop or is it necessary to download hdp we can also do through frst install jdk then hadoop also have confusion that linux is important to download and will it affect pc
Hey Anushka,😊
Yes, you are absolutely correct!. We can install Hadoop by installing JDK then downloading and extracting Hadoop to a specified path and setting up the environment variables.
But, The HDP Sandbox makes it easy to get started with Apache Hadoop, Apache Spark, Apache Hive, Apache HBase, Druid and Data Analytics Studio which I have discussed in the subsequent lectures in this playlist.
Hope this’ll be helpful! 😊
@@ampcode do i need to install Linux and will it affect my pc or make it slow or not work properly as i am having 8gb ram 1tb hdd and 125gb ssd
@@_anushka_sharma777 yes you can definitely run HDP in you PC but make sure you are storing the image other than C drive(125GB SSD) as it should require around 25GB of free space and while running, make sure you don’t have any other apps/browser tabs running or else it will affect your system performance. You can allot 4-6 GB RAM to the HDP image while deployment process.
Hope this will help!
@@ampcode i will try this uh seriously a champ thanks for valuable reply
@@_anushka_sharma777 Thank you so much for your inputs. I really appreciate it.
Take care!😊
Hi, I have been using the sandbox for the past two months. However, for the last few days, when I start all the services, the Spark2 server is taking an unusually long time to initiate its service. It's literally taking hours to start. If you know how to fix this issue, could you please share the solution? I haven't been able to find any resolution.
access restricted showing in cloudera while downloading hortonworks HDP
what to do
Hello Abhishek,
Have you provided all your details including name, email. I guess they don’t allow us for further steps. I know it’s a bit time consuming but totally worth it. Pls let me know if you still face any issue😃
After installing virtual box and then importing the hdp sandbox (15gb) when i am starting it it shows critical error and whole pc got hung. I have, 8 gb ram). What could be the solution for this. please help
hello, i am unable to launch the sandbox, it errors out with EFAIL
Sorry for late response. Could you please let me know which error you are facing?
How can we set up a multinode cluster using hortonworks?
Hello there, here are the step-by-step official documentation on deploying multi node Hadoop cluster. Hope this will be helpful! :)
docs.cloudera.com/HDPDocuments/HDP2/HDP-2.3.0-Win/bk_HDP_Install_Win/content/ch_deploying.html
@@ampcode Thanks
Hi , after i tried to login with raj_ops and password , it is taking me back to reenter the credentials, any solution?
Sorry for late response. Could you please try maria_dev and give same as a password and let me know?
Hi Sir I am not able to download the hdp sandbox while downloading it shows you must be a hdp customer
Could you please sign in and provide some mandatory details. That way you’ll able to download it👍
@@ampcode I have already sign up but in my account section it shows a community user but for downloading it shows you must be a hdp customer
I filled the community form it shows their hdp sandbox product is removed
so can any other mthod to do so
HDP Sandbox downloading is too slow, how can I download it quickly?????
I can definitely understand your pain🤭 but since the sandbox is pretty large, only good and stable bandwidth connection will solve the problem😃
I downloaded virtual box...but while downloading hdp sandbox..I am facing the problem that access restricted is coming on screen...and HDP data platform is in legacy mode....can you please help me to solve this problem
Same Problem with me
Hi sir i encountered this error 502 Bad Gateway nginx/1.15.0 when I open it next time
When i starting sandbox its installing 48GB!!!!!!
Sorry for late response. Yes it's a very big file and also it'll eventually grow with the use but it will be the one time activity and you don't need to go through any hassles afterwards.
Is there any other way ❓️Access Restricted even after Account login
how to register with cloudera .it gives error of incorrect email?
Sir , I'm unable to login using both of the passwords .... The window of Ambari is not opening at all .. Please help me out .... :(
I have downloaded the same version (2.6.5) as shown in the video ..
After i tried to login with raj_ops and maria_dev , it is taking me back to re-enter the credentials, any solution sir?
Are you using maria_dev as username as well as the password?
Please let me know
@@ampcode yes sir
not able to download hdp from cloudera its showing acces denied and you are not hdp customer please help
Could you please login first and provide all the necessary details. Let me know if any further issues
@@ampcode have tried but same is coming
@@soumyadeep8107 sure. Let me have a look into it and get back to you
@@ampcode Any luck on this.. facing same issue
Any hope on this?
Who saw play grand theft auto icon😂
😂😂
the maria_dev and raj_ops password is not working..?????
Hello there,
Could you please check the admin account by giving ‘hadoop’ as a password.
This may work. Please let me know in case of any.😊
@@ampcode plz help i downloaded twice then also unable to login wrong password
@@thetechyguy4410 Hello there!
Have you tried checking the admin account. Password is hadoop all in lowercase. Pls let me know if issue still persists😊
@@ampcode brother can u plz plz help me with the issue can u send me mail id I can share the image
@@thetechyguy4410 Hey, sorry for late response. Yes, you can reach me out at ashaypatil11@gmail.com if issue is still there😊
Can you please make video for cloudera
Sure. Will definitely give it a try and keep you updated!😊
@@ampcode thanks bro but is there big difference between this and cloudera?like I know about installation but after installation there is difference between these two Hadoop?
@@iqbaldanishkhan3037 yeah bro there are some key differences like Cloudera has a commercial license whereas Hortonworks is open-source license and there are some differences in the Hadoop stack which they provide like HDP has Ambari web UI as compared to the Hue in Cloudera. Also Cloudera provides great database option like Impala which is way more optimised than Apache Hive. Please find the below for more info. 😊
www.educba.com/cloudera-vs-hortonworks/
@@ampcode thanks bro actually I try to find hands on video on TH-cam but didn't get too much data if you have any idea so please suggest thanks a lot bro👍
@@iqbaldanishkhan3037 Sure bro. I’ll keep you updated on this. Thank you!🙌
Bhai ap ka link se download kya lekin wo passward nehi le Rahi ye
Maria_dev or raj_ops nehi le rahiye
tried all the passwords unable to login plz help @AmpCode
Could you please let me know which HDP version you have deployed?
@@ampcode 2.65
@@ampcode i did same as seen in the video everything also getting one more issue server not yet listening on port 8080 after 50 second exited
@@thetechyguy4410 you need to wait for atleast 10 mins to kickoff the Ambari service in the background. Please do let me know if this error persists
@@ampcode but brother getting error server not yet listening on port 8080 after 50 second what to do for this error?