For tech enthusiasts, one common issue is falling asleep while watching learning videos. However, after watching one such video, I found this gem. Thanks Man !
Hello Darshil is great to watch this content and learn some new concept each day. Can we get the same for Apache Hadoop , Apache Nifi the more reliabe tools which are ongoing used in Data engineering Projects Thank you!
great content and i have been looking at your content for years now, i still learning but i feel like something is missing, like a real life project , real time processing where one can use kafka,pyspark , orchastrated by airflow and the data goes in a medallion achitecture then transformed by dbt and then deployed in production using ci/cd ,and then monitor the dags using promethus and grafana ,something of that sort and then mayb deploy it on gcp or aws
Hi Darshil First of all , Great Content ! I am trying to replicate the steps as the video , and it works fine from the terminal. Messages sent from the producer is displayed at consumer end. But when I'm trying to run it with the producer/consumer scripts , the consumer does not receives the messages. Any idea why ? Both containers(kafka & zookeeper) are on the same bridge n/w on same subnet.
Hi bhaiya Needed a laptop in college and programming i started learning data science and ml but dont do gaming so i didnt take any GPU laptop I have 2 options First lenovo ideapad pro 5 with 14" ultra 9 H processor 32 gb ram 1 tb ssd and 84 wh battery Second is lenovo yoga slim 7i with ultra 7 processor 32 gb ram 1 tb ssd and 65 wh battery Both price same Which one should i go for?
I am facing issue while installing pydoop which is needed for me to write data from kafka topics to hadoop. Please can you help me put in installing pydoop.
if i started learning for now than there are very less data engineering internships(if one has data internship it is easy to get a job ,and that person also get hand on experience ) ,,,,so should i do data analyst first only for internship and after that i will study for data engineering for job because it is also a good way , and i need to do an internship because it is in our college criteria , they do not support much if you do not have done internship?? nice video on kafka
For data engineers scope is mostly limited to consumption side and rarely you write to topics as producers (for specific use cases) . Most of the complexity of setting up kafka and micro service should be handled by source systems. Consumer should only worry about consuming those events and setting up pipelines which is pretty easy.
Don’t forget to LIKEEEE
Never... Never Forget to 👍
What's the prior knowledge should have to learn Apache Kafka?
For tech enthusiasts, one common issue is falling asleep while watching learning videos. However, after watching one such video, I found this gem. Thanks Man !
I really love the way you explain. Especially you showed it in both Jupyter notebook and terminal.
No waste of time straight to the point. Loved it.
Thanks for the video. It was useful.
You're welcome
Great content to understand what Kafka is and it's components and how they work together
Very good content, looking for more videos...
More to come!
That is a serious unibrow. Bravo.
falling in love with your content quality 🔥
Top video! in such short time, you explained it clearly, Darshil!
Glad you liked it!
love this content, keep making it sir ❤
More to come!
Great stuff, could you also explain how to handle DLQ messages and consumer lag in sink connector
Will create some tutorial on this
Best one, Please explain indepth architecture in Kafka!!! Love to hear more!
Very soon!
@@DarshilParmar Hey Darshil, did you release this later?
Guys remember if you wanna learn kafka, go to documentation section and read it
All time best resource
great content sir keep up
Wow I was just looking for this !
Glad you found it
This Explanation was best , Can we get same for apache nifi?
Great content vro.. build a video series of the format. Killed it 🫡
Thank you
Great overview. Well done!
Nice information thanks ❤
Welcome
Hello Darshil is great to watch this content and learn some new concept each day.
Can we get the same for Apache Hadoop , Apache Nifi the more reliabe tools which are ongoing used in Data engineering Projects
Thank you!
Thank you ❤❤ It was really helpful 🎉.
You're welcome 😊
Go deeper into architecture and implementation of Kafka. Great video!
Thanks, will do!
Please consider this request from my side as well.
@@ManishSharma-fi2vr Sure
Yes, please go deeper
Great content man.
great content and i have been looking at your content for years now, i still learning but i feel like something is missing, like a real life project , real time processing where one can use kafka,pyspark , orchastrated by airflow and the data goes in a medallion achitecture then transformed by dbt and then deployed in production using ci/cd ,and then monitor the dags using promethus and grafana ,something of that sort and then mayb deploy it on gcp or aws
Can you please come up with Kafka end to end series for your people...
will be launching course on this soon
@@DarshilParmar i am thinking to buy, when it will come please let me know its very urgent !!!!!!
Hi Darshil
First of all , Great Content !
I am trying to replicate the steps as the video , and it works fine from the terminal. Messages sent from the producer is displayed at consumer end.
But when I'm trying to run it with the producer/consumer scripts , the consumer does not receives the messages.
Any idea why ? Both containers(kafka & zookeeper) are on the same bridge n/w on same subnet.
Hi, yours courses hace the translate to spanish?
when this course or combo will come on it ?, i just want to ask something regarding course buy, can we connect ?
Hi bhaiya
Needed a laptop in college and programming i started learning data science and ml but dont do gaming so i didnt take any GPU laptop
I have 2 options
First lenovo ideapad pro 5 with 14" ultra 9 H processor 32 gb ram 1 tb ssd and 84 wh battery
Second is lenovo yoga slim 7i with ultra 7 processor 32 gb ram 1 tb ssd and 65 wh battery
Both price same
Which one should i go for?
make a full courese on kafaka and snowflake integration
I already have on snowflake
@@DarshilParmar but kafka integration kind of
That will come soon
@@DarshilParmar I would very much appreciate one extensive tutorial on Kafka, thanks for the awesome lessons
I am facing issue while installing pydoop which is needed for me to write data from kafka topics to hadoop. Please can you help me put in installing pydoop.
if i started learning for now than there are very less data engineering internships(if one has data internship it is easy to get a job ,and that person also get hand on experience ) ,,,,so should i do data analyst first only for internship and after that i will study for data engineering for job because it is also a good way , and i need to do an internship because it is in our college criteria , they do not support much if you do not have done internship??
nice video on kafka
Start with any data field that you get, foundation skills are same for all
Hey let's connect and talk more about it
@@devparmar7420 ok
@@devparmar7420 ok
@@devparmar7420 share me your email are you also an student
3:32 how to create this map?
Epic 🥰
What about Google Cloud Pub/sub. Both Kafka and pub/sub are same
Yes it’s similar
We need a full tutorial on Kafka
on it!
For data engineers scope is mostly limited to consumption side and rarely you write to topics as producers (for specific use cases) . Most of the complexity of setting up kafka and micro service should be handled by source systems. Consumer should only worry about consuming those events and setting up pipelines which is pretty easy.
100%
What is the work of the zookeeper here?
Manage resources and work on coordination/synchronization
We are running multiple machines
After typing command docker-compose up i am getting eeror like incorrect username and password
you will have to login to docker using cli, just google about it
I have done bcom want to enter in it what should i do mtech vs mca
MTech is good option you’ll get in-depth understanding
I have some queries what is the way to contact you?