Azure Event Hubs for Apache Kafka | Azure Friday
ฝัง
- เผยแพร่เมื่อ 20 ก.ค. 2024
- Shubha Vijayasarathy joins Scott Hanselman to discuss Azure Event Hubs, which makes data ingestion simple, secure, and scalable. As a distributed streaming platform, Event Hubs enables you to stream your data from any source-storing and processing millions of events per second- so you can build dynamic data pipelines and respond to business challenges in real time.
With Azure Event Hubs for Apache Kafka, we're bringing together two powerful distributed streaming platforms, so you can access the breadth of Kafka ecosystem applications without having to manage servers or networks. Event Hubs for Kafka provides a Kafka endpoint so that any Kafka client running Kafka 1.0 or newer protocols can publish/subscribe events to/from Event Hubs with a simple configuration change.
[06:45] Demo Start
Azure Event Hubs docs
aka.ms/azfr/495/01
Azure Event Hubs samples (GitHub)
aka.ms/azfr/495/02
Azure Event Hubs overview
aka.ms/azfr/495/03
Azure Event Hubs pricing
aka.ms/azfr/495/04
Announcing the general availability of Azure Event Hubs for Apache Kafka® (blog post)
aka.ms/azfr/495/05
Create a free account (Azure)
aka.ms/azfr/495/free - วิทยาศาสตร์และเทคโนโลยี
A very succinct comparison of these 2 easily-confused distributed streaming/messaging systems. Your example with AzureEventHub using a "Namespace" to provide a "Kafka Abstracion Layer" to Java-coded Producers & Consumers was PRECISELY the explanation I was seeking for explaining AzureEventHubs to a customer! You just made my day!
What’s the persistence properties of Eventhub, can I also store like 60 days and replay form there? How would a consumer manage its offsets?
The big question is it as good as kafka as far as performance? i am not sure folks who use kafka today will switch and give away all the producer and consumer configuration fine tuning options and replace it with a magic box in the cloud.
Such an amazing discussion! love it! Thank you for sharing!
Well explained. Thanks folks.
Why Microsoft is not positioning eventhub for service to service communication? Why its always depicted as event streaming?
Good one and nice explanation on kafka data streaming platforms
Do we absolutely need Event Hub to consume this Kafka data? Or can we connect to Kafka using the Azure Data Factory or Databricks?
@4:32 that sarcastic smile from Scott, says it all. Just scales smile (with cost :))
That's incredible! Although funny to see java in demo when we have C# :)
Thanks for this video!
Fantastic
Thank you.
Can we read data from blob storage using Kafka?
Hi, I am trying same implementation in my Maven Spring Boot + Kafka application for Producer after sending data it is hitting Azure Matrix but mentioned error I am getting ,
Error:
2019-07-04 12:27:31.729 INFO 92607 --- [nio-8050-exec-2] c.g.k.service.KafkaPublisherService : sending message='t'
2019-07-04 12:28:31.730 ERROR 92607 --- [nio-8050-exec-2] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='null' and payload='Hi' to topic testhub:
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
And for consumer while clean build I am getting below mentioned error:
2019-07-04 12:48:26.837 WARN 94752 --- [ main] o.s.w.c.s.GenericWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
Please let me know if there is any solution for this.
@@snehanjanaray8795 The EventHubs team doesn't monitor this video for issues. Open a github issue or file a support ticket with Microsoft!
2:41 did she mean when the CONSUMERS r ready ?
Hdinsights or event hub for Kafka?
Hi Bill Gates