Apache Kafka load testing with JMeter

แชร์
ฝัง
  • เผยแพร่เมื่อ 23 ธ.ค. 2024

ความคิดเห็น • 38

  • @gurbina
    @gurbina 2 ปีที่แล้ว +3

    Thank you, it works like a charm!

  • @fedoratestxyz1720
    @fedoratestxyz1720 ปีที่แล้ว

    this video was really helpful to me in my project

  • @sreenivasansetu7017
    @sreenivasansetu7017 2 ปีที่แล้ว +2

    Hi zz.. Can the above framework be used for Kakfa with AVRO Schemas ?

    • @zztalk
      @zztalk  2 ปีที่แล้ว +2

      Hi, I believe so. AVRO schemas is created in JSON, correct? If that's the case, you just need to construct the json on JSR223 sampler.

  • @amithrajk9841
    @amithrajk9841 2 ปีที่แล้ว +1

    Thank you for the video❤️

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      Thank you for watching.

  • @TheNeoamerica
    @TheNeoamerica ปีที่แล้ว

    hi good video, do you have any pointer on how to send a json value instead of a string message .. thanks.

  • @veerakanchustambham9118
    @veerakanchustambham9118 ปีที่แล้ว

    Hi ZZ, could you please let me know how to send keystore which is in file along with request using jsr223 sampler

  • @archimedecornely5341
    @archimedecornely5341 ปีที่แล้ว

    can we use this on an existing consumer, for example: a consumer made in spring boot?

  • @shaktyee
    @shaktyee ปีที่แล้ว

    Hi
    Script is not working for kafkaProps.put("value.serializer","io.confluent.kafka.serializers.KafkaAvroSerializer");
    Is any specific imports required to support this ?

  • @saeedyasrebi6271
    @saeedyasrebi6271 ปีที่แล้ว

    Hi zz talk, thank you for this sharing. I have trouble with the line "records = consumer.poll(Duration.ofMillis(100));" and i get NullPointerExceptionError.Can you help me to solve this issue?

  • @kahmed2222
    @kahmed2222 ปีที่แล้ว

    Hi zz talk, How would we handle a JSON payload?

  • @adwaitdamle8721
    @adwaitdamle8721 ปีที่แล้ว

    Thanks for this !!

  • @999NightCrawler
    @999NightCrawler 2 ปีที่แล้ว +1

    Good video. Can you tell me if the script can easily be amended to send messages of different sizes (random) for each iteration for each user thread? Currently all threads send the same message size

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      I believe so.. In that case, you would like to program a little bit, use JSR223 processor to produce dynamic load.

  • @rtaaskype
    @rtaaskype ปีที่แล้ว

    Hi i have few questions:
    1. I just want to send 14messages /sec to a remote kafka topic. This system requires sslhandshake. Security.protocol:sasl_ssl, sasl.mechanism:plain, sasl.jaas.config is also required with username,password.
    How can i send these in the test?
    2. If im just using publisher to publish messages, then i hope i need the consumer part from the .jmx file in project?
    Thanks.

  • @JosiasValentimdeFigueredo
    @JosiasValentimdeFigueredo ปีที่แล้ว

    Hi ZZ, first of all thnx for sharing with us, well i´m already saw some videos about some Kafka plugins for JMeter but i see that for all the configuration is for a local server, is that possible to test for a remote server? these plugins don´t have any config to this situation, any suggestions?

    • @zztalk
      @zztalk  ปีที่แล้ว

      I didn’t use the plugin in. All were done by JSR223 samplers if you watched my video. And I don’t see why it cannot test for a remote server. You can simply change the Kafka broker ip and port.

  • @veerendragottiveeti8539
    @veerendragottiveeti8539 2 ปีที่แล้ว

    Hi, I used your implementation but the consumer portion never pops up in the view results tree it only shows the steps up to the producer. Can you please help me?

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      Hmmmm, did you see any error?

  • @nareshbabu4773
    @nareshbabu4773 2 ปีที่แล้ว

    Hi zz talk, can we consider record timestamp - event creation time for calculating elapsed time/ processed time. record.timeStamp().

  • @sonaliverma5515
    @sonaliverma5515 2 ปีที่แล้ว

    I am adding Json in create message and my project requirement is to send around 1500Json data. How Can I do that? When adding 2MB data Jmeter is getting hanged
    Please suggest

    • @zztalk
      @zztalk  2 ปีที่แล้ว +1

      How did you construct your json data? with JSR223 sampler?

    • @sonaliverma5515
      @sonaliverma5515 2 ปีที่แล้ว

      @@zztalk Yeah

    • @sonaliverma5515
      @sonaliverma5515 2 ปีที่แล้ว

      I am passing data in create message like this syntax
      props.put(“message”, “””Json1,Json2,Json3…..”””);
      Upto 100 Json it is fine, above that Jmeter gets hanged.
      P.S I couldn’t find out how can I add directly Json txt file in JSR

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      @@sonaliverma5515 you can construct whatever json message via JSR223 processor, which is more efficient when you’re dealing with huge amount of data. Looks like your JMeter running into performance issue because of this level of load you’re pushing. You might wanna tune your JMeter infra by distributing the load cross a JMeter cluster, or try constructing the JSON via processor to see if it helps.

    • @zztalk
      @zztalk  2 ปีที่แล้ว +1

      @@sonaliverma5515 also you might wanna keep an eye on your Kafka side to see if there is any perf bottleneck you need to tune Kafka settings.

  • @sundarr6756
    @sundarr6756 2 ปีที่แล้ว

    What throughput you able to achieve?

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      Good question. It depends on your Kafka infra. Actually I didn't test the max throughput. It was not my use case. I just demo'd how to test Kafka here with JMeter using a single node Kafka.. (My use case was to show some stats on how much redis is faster than Kafka.. This video only covers Kafka part, btw. :))

  • @amithrajk9841
    @amithrajk9841 2 ปีที่แล้ว

    Can i send the message as JSON?

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      Yes.

    • @amithrajk9841
      @amithrajk9841 2 ปีที่แล้ว

      @@zztalk should I change the value serilizer to "org.apche.kafka.common.serialization.JSONserializer"??
      Also I'm wokring on kafka for the first time. How can I monitor if the message is being received at kafka end?

    • @zztalk
      @zztalk  2 ปีที่แล้ว

      @@amithrajk9841 You can verify from Kafka cli.. Maybe take a look this video first: th-cam.com/video/7yrwTLnPewQ/w-d-xo.html