Apache Kafka® Producer Example using SpringBoot 3.x | Java Techie
ฝัง
- เผยแพร่เมื่อ 29 ก.ย. 2024
- In this tutorial, we will be creating a simple Kafka Producer using Spring Boot 3.0 and will understand below key points
👉 Default topic create by Spring Boot
👉 How to handle the results asynchronously in kafka producer
👉 How to create topic programatically
👉 How messages will be distributed to multiple partition
👉 Offset Explorer to visualise kafka events
#JavaTechie #SpringBoot #Kafka
Spring boot microservice Premium course lunched with 70% off 🚀 🚀
Hurry-up & Register today itself!
COURSE LINK : javatechie5246...
PROMO CODE : JAVATECHIE50
GitHub:
github.com/Jav...
Blogs:
/ javatechie
Facebook:
/ javatechie
Join this channel to get access to perks:
www.youtube.co...
guys if you like this video please do subscribe now and press the bell icon to not miss any update from Java Techie
Disclaimer/Policy:
--------------------------------
Note : All uploaded content in this channel is mine and its not copied from any community ,
you are free to use source code from above mentioned GitHub account
Hi Basant,
Your videos helped immensely during the interview prep time and it helped to get a decent job as Senior Software Engineer... many thnaks to you !!
Glad to hear this buddy 🥰. Thank you so much keep learning
send to Basant, at least some percent of your salary bro , he is doing it for free! )
hey buddy, I get you're joking, but I truly enjoyed the appreciation from you guys, which means more to me than money.
Really helpful, detailed and amazing video. Keep up the good work brother.
Great tutorial on Kafka. Appreciate your efforts, Basant.
Amazing tutorial, thank you bro!
Great.....God Bless you !!!!
Event Streaming with huge data reading and writing
Also looks like the messages in kafka are getting encrypted by default, I dont see actual message in offset Explorer. Is ther a way to get plain message in kafka topic
No in offset you need to change the data type to json or plain text in topic itself
How it will behave when multiple instance of application are launched?? It will create different topics or adhere to same topic??
I owe you a drink.
how to start Zookeeper and kafka on windows
How to listen the Kafka consume with some intervel time when producer send the data to the kafla server
How do you set idempotence= true for producer?
Hello everyone,
I am constantly gettinig the below given error:
Required type: CompletableFuture
Provided: ListenableFuture
everytime I try declaring future variable
The return type is incorrect just use your ide to get the correct return type brother
@@Javatechie hello brother, I tried but I'm not getting the option to use completable future in java 11. I am using Intellij
@sunnysingh7854 please share your code in GitHub.com
Hi sir
WSL (10) ERROR: CreateProcessEntryCommon:505: execvpe /bin/bash failed 2
WSL (10) ERROR: CreateProcessEntryCommon:508: Create process not expected to return
getting this while starting Zookeeper
Simply Superb
Great series! Thanks for your very detailed and clear explanations... Keep it going!
very clear and concise explanation. Keep Rock!
How will we handle an exception and implement retry on the publisher microservice if we face issue while publishing to kafka
Do you have offset Explorer license to use it for the educational purpose? It is not free to use for educational purpose as mentioned on its website. Just brining to your notice.
It's open source and free
@@Javatechie This is what is mentioned on its website Offset Explorer is free for personal use only. Any non-personal use, including commercial, educational and non-profit work is not permitted without purchasing a license. Non-personal use is allowed for evaluation purposes for 30 days following the download of Offset Explorer, after which you must purchase a valid license or remove the software
Not sure then, there might be some changes but it's installed on my machine from past 2 years and software didn't force me for any licence
Have a quick question: if you are capturing the result into an object after sending, isn't that mean the thread will wait for result to come and it is still hampering the performance of the app ?
It will just capture metadata of your records nothing else so I don't think it will impact anything on performance
Thanks for this nice video, I have configured offset exproler but for me instead of actual message it showing hascode of message
Okay just click on topic and change the type to string instead of byte
Hi Basant, thank you for sharing kafka series. I apricate your efforts
Hi This is not related to this video but in General I Want to know is that Mac Air laptop will supports our all Java tools? or we need Mac pro ?? I want to shifting from windows to Mac. I need AI ,ML tool Supper as Well. pls suggest
No i am using a Mac desktop with custom congratulations 16 GB . You can use a Mac pro laptop with the latest configuration to play with ai
Very creative and helpful video anyone can understand.
I went through many articles and videos, and I was very confused, and my thought was Kafka is very big and very vast. But by watching this video I am full cleared on Kafka concepts.
Hi Basant can you please tell me how can we integrate Kafka instead of active mq with spring boot
This is a Kafka integration example only right? What are you looking for?
Hello Sir,, Here Topic 'java-techie-demo-3' will be created only If it is not exist already. Right?
Yes correct 👍
sir i have issue to download Offset Explorer for Ubuntu system
can provide .sh file
No it's a simple DMG file for me I just need to install it
Can you make video on spring boot jdbc template Mapsqlparametersource with stringutils and string builder using sql queries please
Okay i will do that
Java Techie, we miss you so much!!!
Hi ..you have missed a crucial point as to how to distribute the messages among partitions using Kafka custom Partitioner
I will do that buddy along with read and write part
How can we send events in to same partition rather than round robin. Because there is order in partition but no order among partition how we provide order by sending same partition.
Yes I will cover that
Hi Basant
Thank you for Kaka tutorial. Learned a lot from you in the fast. I have few queries
1. How can we create multiple topics programmatically? With @Bean we can create only topic at a time
2.. If topic already exist, it throws exception. How can we skip topic creation if topic already exist.
3. Does it creates new thread for each message send?
First and Two questions are quite interesting i will check and update you but regarding the 3rd one it won't execute by a separate thread message will concurrently be distributed to multiple partitions but to get those messages you need to implement multi threading in the consumer part .
You can even use the Executor service in the producer itself
Sir if you don't mind please mention the link of microservice playlist taught by you...
Please go through it
Microservice: th-cam.com/play/PLVz2XdJiJQxxWhFkucZBoMxeYE6qTgEF8.html
20:09 In my scenario my data stored only partition 2. No data stored in partition 0 & 1. Why ?? @javatechie
Try 2 or 3 times again it will store messages to different partition. Please check in video i mentioned why okay
@@Javatechie Thanks Sir. I got it.
Thank you for your videos! They are super helpful
Loved ur video, sir can u please make a video of inter service communication in spring Microserverices with real time example using kafka
Buddy i have done the same in the cqrs design pattern please check that
Hi Basant. I have a question. We have already used KafkaTemplate and it is creating the topic if topic is not already created. Why do we need to again use NewTopic bean and pass the topic name in KafkaTemplate instance. Pls explain. Thanks
Rajendra correct Kafka will create the topic but with default configuration like 1 partition and replication factor so if you want to customize it then you need to create topic by your own either using terminal or programmatically
@@Javatechie oh okay. Thank you 🙏🏻
Hello @Java Techie, thank you for the amazing content , it is quite easy to understand. I have few queries,
1. What is the importance of replicationFactor?
2. How the kafka setup ensures delivery of message in case when consumer application is down?
3. Is it possible to create the topics from docker-compose.yml files?
Please teach something complex like Kafka streams as not many good videos are there
I will do that concept step by step
Very nice explanation, Thank you sir.
Hi Basant, How do you get yml intellisense on intellij ?
I believe it's by default available in the community edition but not sure whether I add any plugin or not. Can you check the plugin name spring boot assistant
Great explanation!
Please post a video for setting up the consumer for the same. Amazing video
Sure buddy that will be my next video.
can you please explain Kafka clustering topic using spring boot if possible
Great knowledge sharing. Hope we can see multithreading consumer tutorial
Yes i will cover that with consumer group don't worry
I was confused by the fact that not all messages were sent to the topic, when I'm trying to see the data only 100 messages are in presence. The reason is that you need to specify the number of messages you wanna see, hope this helps beginners not search for errors
No it should send all the messages. Check the total message count in each partition
@@Javatechie Everything is working well, as I said I couldn't see all the messages as there was a limit for messages that can be shown (in offset explorer)
@@vladstarichenko4937 it's a free tool buddy if I will send 1000 message still it will show max 100 message in app so better to check in each partition level message count
Great video, thanks!
Great series! Thanks for your very detailed and clear explanations... Keep it going!
This is awesome
Hi Basant, thanks for knowledge sharing.can you post video on replication factor other than one and how it will behave and in this case duplicate messages will be received by consumer or not
I will do that buddy
🙏👍
👍 thanks
The next part is expected soon..maybe today 😅.. thanks
I will try to do it soon
please share the link of the 1st video. Thanks
Excellent video!
This is awesome !!!