Please Subscribe🔔, like, 👍and share 📣the video to unlock the following bonus videos. Let me know which video topic you want me to cover here: forms.gle/UMp4GA3krcSMMWzy9 🎁 Bonus Videos: ▶ Hit 50,000 views to unlock a video about building an end-to-end machine-learning pipeline with PySpark. ▶ Hit 100,000 views to unlock another video video about end-to-end spark streaming.
Simple yet very informative and helpful content..please dont get discouraged by less views.. slowly more and more people will come and watch your videos
Wasn't expecting a 50 min video to be this informative! Covered all the major topics related to PySpark. Truly, a well structured video and quality content. Thanks!
I found this to be an excellent guided tour, but only because I've gone through a lot of tutorials on webpages, books, and videos. PySpark is so vast and diverse that, after seeing all these other tutorials, one needs a video like this one to knit them all together.
Have to question the AI voice. But honestly, with 99% of tech tutorials on TH-cam being presented by very, very thick Hindi accents, it's probably for the best.
This is the latest and most beginner friendly tutorial regarding apache spark. As an aspiring data engineering this is absolutely helpful! If you ever have free time, I would like to request creating a tutorial series regarding docker, dbt, and kafka, also if possible could you add more tutorials for dagster. As a student, out of all the tutorials in youtube I was able to follow you tutorial the best. Thank you for the airflow and pyspark playlist. I hope you still continue recording 😊
First, thank you for your video, it is really helpful for people who know nothing about PySpark like me. But I have searched for spark installation, and there are many tutorials that install very complicated above Hadoop Cluster. And it seems like you installed it so easily, I wonder if this is a version of installing Spark standalone, right?
It is standalone. To get started, it is enough. And for the real project, you wouldn't need to know how to set up the cluster yourself. You can use Spark in clouds like databricks instead.
I am unable to get the jupyter notebooks to work. It keeps complaining about py4j. I am able to run the commands in a terminal tho. Not sure what the setup is supposed to be. I wish this video started with how to install everything.
Thanks for the informative video. The pyspark environment is not activating on my windows 10 machine. Can you please help me what might be missing '.pyspark-env' is not recognized as an internal or external command, operable program or batch file.
Please Subscribe🔔, like, 👍and share 📣the video to unlock the following bonus videos. Let me know which video topic you want me to cover here: forms.gle/UMp4GA3krcSMMWzy9
🎁 Bonus Videos:
▶ Hit 50,000 views to unlock a video about building an end-to-end machine-learning pipeline with PySpark.
▶ Hit 100,000 views to unlock another video video about end-to-end spark streaming.
Hands down! The best Pyspark tutorial that I've come across.
Thanks!
Simple yet very informative and helpful content..please dont get discouraged by less views.. slowly more and more people will come and watch your videos
Thank you so much 🙂
Wasn't expecting a 50 min video to be this informative! Covered all the major topics related to PySpark. Truly, a well structured video and quality content. Thanks!
great explanation! thanks!
27:34 - "rdd.foreach(lambda x: print(x))" did not print for me - instead suggest: "for x in rdd.collect():
print(x)"
Thanks! This helped me set this up locally. I was able to run it on Collab and Kaggle but not locally.
Brilliant! Invaluable info. Thank you!
Thanks for this wonderful tutorial. Eagerly waiting for the Pyspark for advanced tutorial.
I found this to be an excellent guided tour, but only because I've gone through a lot of tutorials on webpages, books, and videos. PySpark is so vast and diverse that, after seeing all these other tutorials, one needs a video like this one to knit them all together.
Glad it helped! ☺️
Very detailed tutorial, hope to see more PySpark ML and Spark Streaming. Thank the author very much!
Thank you! 🤗
Thanks a lot for the clear and structured explanation, so helpful to grasp the basics of the Spark framework!
You are welcome 🤗
Have to question the AI voice. But honestly, with 99% of tech tutorials on TH-cam being presented by very, very thick Hindi accents, it's probably for the best.
Excellent work. Thanks so much for the content.
amazing tutorial, Thank you!
Glad it was helpful!
This was super helpful, thanks!! It answered all of my questions & provided me with everything I needed to know!
Thank you brother, this is the best Spark introduction.
Thank you!
This tutorial was pretty solid. Thanks.
This was good, thanks! The screen was super blurry on that first section in Jupyter. But after that it was fine.
This is the latest and most beginner friendly tutorial regarding apache spark. As an aspiring data engineering this is absolutely helpful! If you ever have free time, I would like to request creating a tutorial series regarding docker, dbt, and kafka, also if possible could you add more tutorials for dagster. As a student, out of all the tutorials in youtube I was able to follow you tutorial the best. Thank you for the airflow and pyspark playlist. I hope you still continue recording 😊
Thank you for your feedback. I will definitely consider that! 😍
Thank you for sharing it. Helped me a lot.
You are welcome 🤗
@@coder2j ♥
Really nice overall Tutorial! Thanks for creating it
Thank you!
this was super helpful! thank you for making this video!
You are welcome 🤗
Thank you a lot, that's super helpful
excellent video, please do hesitate to do more like these
13:49 give error for me "spark not defined"
First, thank you for your video, it is really helpful for people who know nothing about PySpark like me. But I have searched for spark installation, and there are many tutorials that install very complicated above Hadoop Cluster. And it seems like you installed it so easily, I wonder if this is a version of installing Spark standalone, right?
It is standalone. To get started, it is enough. And for the real project, you wouldn't need to know how to set up the cluster yourself. You can use Spark in clouds like databricks instead.
@@coder2j When we set SPARK_HOME variable, is pyspark library in your .spark-env could recognize that Spark and run above that?
Very great tutorial, I hope you can launch the tutorials of Spark ML and Streaming asap. Thanks
Thanks. Stay tuned!
Do i need to create a new env after closing or restarting a session ?
amazing as alaways
Thanks 🙏
I have the error of : Py4JJavaError: An error occurred while calling o42.showString. How can I solve this ?
I am unable to get the jupyter notebooks to work. It keeps complaining about py4j. I am able to run the commands in a terminal tho. Not sure what the setup is supposed to be. I wish this video started with how to install everything.
U need to set environment variable in the paths for java. Type it in Google it will give u steps
df.show() is throwing an error
Amazing video❤
Even though I've paid $20,000 to University still this video holds more value than that of learning in the class. Thank you @coder2j
Thanks for your comments! Glad it helped ☺️
Thanks for the informative video.
The pyspark environment is not activating on my windows 10 machine. Can you please help me what might be missing
'.pyspark-env' is not recognized as an internal or external command,
operable program or batch file.
Did you set the spark path correctly?
Thank you for you tutorial. It is amazing 👏
You are welcome 🤗
so good video
Thank you.
Thank you. :)
BEST
Hey, are you using any AI for voice modulation or content creation.?
nice content though
I am using the text to speech for the audio. Do you enjoy it or find it annoying?
@@coder2j content is very nice,tts sounds robotic
Thanks for your honest feedback. Would you enjoy it more if I use my own voice?
Absolutely! 💯
Thank you
Could you please share this PPT.
👍👍👍
windows version bugged or something. Can't run df.show() without weird error codes. Ah well. on to another tutorial I guess. Thanks for trying
Thank you.
we need apache nifi
Where is the pyspark ML? you promised :'(
9:51
Mason Terrace
Gonzalez Carol Gonzalez Melissa Miller Kevin
Deepfake, or not, keep responding to me using same word over and over again
Thank you for not to be a indian voice.
first thank you for your video. Just one thing you never say nothing about the file spark-defaults.conf and im having an error that i can fix it