Apache Kafka in 1 hour for C# Developers - Guilherme Ferreira - NDC London 2023

Are you curious about Apache Kafka? But you never had the time to dig into it?
No problem. In this session, we will dig into it. In under 60 minutes, we will learn basic concepts and write our first Kafka application using C# and KafkaFlow, an Open Source framework for .NET.
Even if you know nothing about it, you can expect to go home with the basics to start your Kafka journey.
Check out our new channel:
NDC Clips:
@ndcclips
Check out more of our featured speakers and talks at
ndcconferences.com/
ndclondon.com/

[@Qrzychu92]

Great talk, my mind finally clicked what kafka is, thank you!

[@gui.ferreira]

WOW! So happy with it! Glad it helped.

[@gustavo-santos-dev]

Great talk!

[@gui.ferreira]

Thanks Gustavo!

[@iamkaransethi]

Great talk, following along was so easy. Loved the bookmark example.

[@gui.ferreira]

That's really valuable feedback! Thank you, Karan 🙏

[@ismailm123]

Excellent, will definately be looking to use Kafkaflow.

[@gui.ferreira]

Brilliant! Let me know if you need any help.

[@rahullore8428]

Great talk! However I always get confused between Topic and Partition. My use case is, every hospital has multiple IOT devices. Every IOT device sends two sets of data points Vitals and Waveform and there are multiple devices in one hospital and there are multiple hospital. Should I creating 2 unique topic per IOT device ?

[@gui.ferreira]

It's hard to have a strict rule to find topics and partitions. As an example, you can have a topic "vitals" and use one partition per hospital. Or, having a topic per hospital. So, topics are mostly about domain scope, while partitions are mostly about throughput.

[@ronenelahav1]

Thanks for all the information, It was very helpful..
I have multiple scenarios where I don't know if Kafka can solve...
1. how do you update services (that use internal cache on startup) to get updates when running in case you have multiple instances? (using groupId - you will need to manage those groupIds).
2. If I will use a topic to send messages out to my clients ( I want to make sure that my client will get them.. and he isn't offline) how can you ensure that (is there a peek and ack for bulk messages by ClientId or something- that will save the order of messages for each client?

[@gui.ferreira]

Hi! Thanks 🙏
Regarding your questions:
1. Each service would need to be a consumer with a distinct group ID (like a GUID-generated one). However, I think it's worth considering using a shared cache like Redis.
2. You can have a client-specific topic. Then, you can use Retry Policies to ensure that they are delivered. KafkaFlow has an extension for that. Search for KafkaFlow.Retry.

[@cecilia_mackie]

So cool! Shall we connect?

[@timur2887]

Great, thanks! Do you have these projects on Github?

[@gui.ferreira]

Thanks 🙏 No. They are in fact production code that powers my Patreon. However, most code regarding the Kafka connection is disclosed during the session.

[@user-pm7kt8tm1s]

Where is github link?

[@cecilia_mackie]

That’s superb! Let’s connect?

[@nickchapsas]

I'll have to pop out some lovesongs then

[@gui.ferreira]

Finally, some serious competition!

[@we_run_the_house]

Gorgeous! Let’s connect?

[@atari1040]

Konrad Siuda is now famous 🤣

[@rafaelferreiraalves2258]

Legal ver um brasileiro voando.

[@carlinhos10002]

Eu acho que ele é português

[@dbgmitre]

Ele diz que é português em 0:55

[@gui.ferreira]

@@dbgmitre Confirmo 😉

[@gui.ferreira]

​@@carlinhos10002 os meus pais e o meu cartão de cidadão concordam 😉

[@gui.ferreira]

Olá Sep! Não sou brasileiro, mas partilhamos o idioma. 🇵🇹 🇧🇷

[@sdstorm]

Extreme red shift makes this unwatchable for me.

[@gui.ferreira]

😞 Sad to hear that. Is it because of the annotations?

[@cecilia_mackie]

Nice! Let’s be friends?