Hi Basant, Your videos helped immensely during the interview prep time and it helped to get a decent job as Senior Software Engineer... many thnaks to you !!
Very creative and helpful video anyone can understand. I went through many articles and videos, and I was very confused, and my thought was Kafka is very big and very vast. But by watching this video I am full cleared on Kafka concepts.
Hello @Java Techie, thank you for the amazing content , it is quite easy to understand. I have few queries, 1. What is the importance of replicationFactor? 2. How the kafka setup ensures delivery of message in case when consumer application is down? 3. Is it possible to create the topics from docker-compose.yml files?
Hi Basant Thank you for Kaka tutorial. Learned a lot from you in the fast. I have few queries 1. How can we create multiple topics programmatically? With @Bean we can create only topic at a time 2.. If topic already exist, it throws exception. How can we skip topic creation if topic already exist. 3. Does it creates new thread for each message send?
First and Two questions are quite interesting i will check and update you but regarding the 3rd one it won't execute by a separate thread message will concurrently be distributed to multiple partitions but to get those messages you need to implement multi threading in the consumer part . You can even use the Executor service in the producer itself
Hi This is not related to this video but in General I Want to know is that Mac Air laptop will supports our all Java tools? or we need Mac pro ?? I want to shifting from windows to Mac. I need AI ,ML tool Supper as Well. pls suggest
Have a quick question: if you are capturing the result into an object after sending, isn't that mean the thread will wait for result to come and it is still hampering the performance of the app ?
Also looks like the messages in kafka are getting encrypted by default, I dont see actual message in offset Explorer. Is ther a way to get plain message in kafka topic
I believe it's by default available in the community edition but not sure whether I add any plugin or not. Can you check the plugin name spring boot assistant
Hi Basant. I have a question. We have already used KafkaTemplate and it is creating the topic if topic is not already created. Why do we need to again use NewTopic bean and pass the topic name in KafkaTemplate instance. Pls explain. Thanks
Rajendra correct Kafka will create the topic but with default configuration like 1 partition and replication factor so if you want to customize it then you need to create topic by your own either using terminal or programmatically
Do you have offset Explorer license to use it for the educational purpose? It is not free to use for educational purpose as mentioned on its website. Just brining to your notice.
@@Javatechie This is what is mentioned on its website Offset Explorer is free for personal use only. Any non-personal use, including commercial, educational and non-profit work is not permitted without purchasing a license. Non-personal use is allowed for evaluation purposes for 30 days following the download of Offset Explorer, after which you must purchase a valid license or remove the software
How can we send events in to same partition rather than round robin. Because there is order in partition but no order among partition how we provide order by sending same partition.
Hi Basant, thanks for knowledge sharing.can you post video on replication factor other than one and how it will behave and in this case duplicate messages will be received by consumer or not
Hi sir WSL (10) ERROR: CreateProcessEntryCommon:505: execvpe /bin/bash failed 2 WSL (10) ERROR: CreateProcessEntryCommon:508: Create process not expected to return getting this while starting Zookeeper
Hello everyone, I am constantly gettinig the below given error: Required type: CompletableFuture Provided: ListenableFuture everytime I try declaring future variable
I was confused by the fact that not all messages were sent to the topic, when I'm trying to see the data only 100 messages are in presence. The reason is that you need to specify the number of messages you wanna see, hope this helps beginners not search for errors
@@Javatechie Everything is working well, as I said I couldn't see all the messages as there was a limit for messages that can be shown (in offset explorer)
@@vladstarichenko4937 it's a free tool buddy if I will send 1000 message still it will show max 100 message in app so better to check in each partition level message count
Hi Basant,
Your videos helped immensely during the interview prep time and it helped to get a decent job as Senior Software Engineer... many thnaks to you !!
Glad to hear this buddy 🥰. Thank you so much keep learning
send to Basant, at least some percent of your salary bro , he is doing it for free! )
hey buddy, I get you're joking, but I truly enjoyed the appreciation from you guys, which means more to me than money.
Java Techie, we miss you so much!!!
Very creative and helpful video anyone can understand.
I went through many articles and videos, and I was very confused, and my thought was Kafka is very big and very vast. But by watching this video I am full cleared on Kafka concepts.
very clear and concise explanation. Keep Rock!
Great series! Thanks for your very detailed and clear explanations... Keep it going!
Really helpful, detailed and amazing video. Keep up the good work brother.
Great tutorial on Kafka. Appreciate your efforts, Basant.
Thanks, mate!
Your tutorials are always awesome.
Hi Basant, thank you for sharing kafka series. I apricate your efforts
Thank you for your videos! They are super helpful
Amazing tutorial, thank you bro!
Simply Superb
Very nice explanation, Thank you sir.
Great.....God Bless you !!!!
Hello @Java Techie, thank you for the amazing content , it is quite easy to understand. I have few queries,
1. What is the importance of replicationFactor?
2. How the kafka setup ensures delivery of message in case when consumer application is down?
3. Is it possible to create the topics from docker-compose.yml files?
Great explanation!
Hi Basant
Thank you for Kaka tutorial. Learned a lot from you in the fast. I have few queries
1. How can we create multiple topics programmatically? With @Bean we can create only topic at a time
2.. If topic already exist, it throws exception. How can we skip topic creation if topic already exist.
3. Does it creates new thread for each message send?
First and Two questions are quite interesting i will check and update you but regarding the 3rd one it won't execute by a separate thread message will concurrently be distributed to multiple partitions but to get those messages you need to implement multi threading in the consumer part .
You can even use the Executor service in the producer itself
Great knowledge sharing. Hope we can see multithreading consumer tutorial
Yes i will cover that with consumer group don't worry
Please post a video for setting up the consumer for the same. Amazing video
Sure buddy that will be my next video.
Great video, thanks!
Excellent video!
Hi This is not related to this video but in General I Want to know is that Mac Air laptop will supports our all Java tools? or we need Mac pro ?? I want to shifting from windows to Mac. I need AI ,ML tool Supper as Well. pls suggest
No i am using a Mac desktop with custom congratulations 16 GB . You can use a Mac pro laptop with the latest configuration to play with ai
This is awesome
Have a quick question: if you are capturing the result into an object after sending, isn't that mean the thread will wait for result to come and it is still hampering the performance of the app ?
It will just capture metadata of your records nothing else so I don't think it will impact anything on performance
This is awesome !!!
How it will behave when multiple instance of application are launched?? It will create different topics or adhere to same topic??
Also looks like the messages in kafka are getting encrypted by default, I dont see actual message in offset Explorer. Is ther a way to get plain message in kafka topic
No in offset you need to change the data type to json or plain text in topic itself
Event Streaming with huge data reading and writing
Hi Basant, How do you get yml intellisense on intellij ?
I believe it's by default available in the community edition but not sure whether I add any plugin or not. Can you check the plugin name spring boot assistant
How will we handle an exception and implement retry on the publisher microservice if we face issue while publishing to kafka
Loved ur video, sir can u please make a video of inter service communication in spring Microserverices with real time example using kafka
Buddy i have done the same in the cqrs design pattern please check that
Hi Basant. I have a question. We have already used KafkaTemplate and it is creating the topic if topic is not already created. Why do we need to again use NewTopic bean and pass the topic name in KafkaTemplate instance. Pls explain. Thanks
Rajendra correct Kafka will create the topic but with default configuration like 1 partition and replication factor so if you want to customize it then you need to create topic by your own either using terminal or programmatically
@@Javatechie oh okay. Thank you 🙏🏻
Thanks for this nice video, I have configured offset exproler but for me instead of actual message it showing hascode of message
Okay just click on topic and change the type to string instead of byte
Hello Sir,, Here Topic 'java-techie-demo-3' will be created only If it is not exist already. Right?
Yes correct 👍
Do you have offset Explorer license to use it for the educational purpose? It is not free to use for educational purpose as mentioned on its website. Just brining to your notice.
It's open source and free
@@Javatechie This is what is mentioned on its website Offset Explorer is free for personal use only. Any non-personal use, including commercial, educational and non-profit work is not permitted without purchasing a license. Non-personal use is allowed for evaluation purposes for 30 days following the download of Offset Explorer, after which you must purchase a valid license or remove the software
Not sure then, there might be some changes but it's installed on my machine from past 2 years and software didn't force me for any licence
Sir if you don't mind please mention the link of microservice playlist taught by you...
Please go through it
Microservice: th-cam.com/play/PLVz2XdJiJQxxWhFkucZBoMxeYE6qTgEF8.html
How to listen the Kafka consume with some intervel time when producer send the data to the kafla server
can you please explain Kafka clustering topic using spring boot if possible
please share the link of the 1st video. Thanks
Hi Basant can you please tell me how can we integrate Kafka instead of active mq with spring boot
This is a Kafka integration example only right? What are you looking for?
How can we send events in to same partition rather than round robin. Because there is order in partition but no order among partition how we provide order by sending same partition.
Yes I will cover that
I owe you a drink.
Can you make video on spring boot jdbc template Mapsqlparametersource with stringutils and string builder using sql queries please
Okay i will do that
sir i have issue to download Offset Explorer for Ubuntu system
can provide .sh file
No it's a simple DMG file for me I just need to install it
How do you set idempotence= true for producer?
Hi Basant, thanks for knowledge sharing.can you post video on replication factor other than one and how it will behave and in this case duplicate messages will be received by consumer or not
I will do that buddy
Hi sir
WSL (10) ERROR: CreateProcessEntryCommon:505: execvpe /bin/bash failed 2
WSL (10) ERROR: CreateProcessEntryCommon:508: Create process not expected to return
getting this while starting Zookeeper
20:09 In my scenario my data stored only partition 2. No data stored in partition 0 & 1. Why ?? @javatechie
Try 2 or 3 times again it will store messages to different partition. Please check in video i mentioned why okay
@@Javatechie Thanks Sir. I got it.
👍 thanks
The next part is expected soon..maybe today 😅.. thanks
I will try to do it soon
Hi ..you have missed a crucial point as to how to distribute the messages among partitions using Kafka custom Partitioner
I will do that buddy along with read and write part
Hello everyone,
I am constantly gettinig the below given error:
Required type: CompletableFuture
Provided: ListenableFuture
everytime I try declaring future variable
The return type is incorrect just use your ide to get the correct return type brother
@@Javatechie hello brother, I tried but I'm not getting the option to use completable future in java 11. I am using Intellij
@sunnysingh7854 please share your code in GitHub.com
how to start Zookeeper and kafka on windows
I was confused by the fact that not all messages were sent to the topic, when I'm trying to see the data only 100 messages are in presence. The reason is that you need to specify the number of messages you wanna see, hope this helps beginners not search for errors
No it should send all the messages. Check the total message count in each partition
@@Javatechie Everything is working well, as I said I couldn't see all the messages as there was a limit for messages that can be shown (in offset explorer)
@@vladstarichenko4937 it's a free tool buddy if I will send 1000 message still it will show max 100 message in app so better to check in each partition level message count
Please teach something complex like Kafka streams as not many good videos are there
I will do that concept step by step
🙏👍
Great series! Thanks for your very detailed and clear explanations... Keep it going!