Spark Interview Question | Scenario Based Questions | { Regexp_replace } | Using PySpark

แชร์
ฝัง
  • เผยแพร่เมื่อ 24 ธ.ค. 2024

ความคิดเห็น • 45

  • @sumantaghosh9299
    @sumantaghosh9299 3 ปีที่แล้ว +1

    Nice explanation..nd gd questions too

  • @gayathrilakshmi6087
    @gayathrilakshmi6087 3 ปีที่แล้ว +4

    Can you please attach data set and solution set ..so that we can practice...thanks for all the excellent videos

  • @SurendraKapkoti
    @SurendraKapkoti 2 ปีที่แล้ว

    Great explanation.. Keep it 👆

  • @jk_gameplay9195
    @jk_gameplay9195 ปีที่แล้ว

    Awesome bro

  • @bunnyvlogs7647
    @bunnyvlogs7647 3 ปีที่แล้ว

    Great brother...God bless u

  • @pavanp7242
    @pavanp7242 3 ปีที่แล้ว +1

    Good work bro

  • @deepakkini3835
    @deepakkini3835 3 ปีที่แล้ว +5

    This was thoroughly explained, nice scenario .
    Could you please do some videos on weekly cohort analysis using window functions in spark?

    • @AzarudeenShahul
      @AzarudeenShahul  3 ปีที่แล้ว

      Sure, we will plan.. thanks for your support

    • @sensibleandhrite
      @sensibleandhrite 3 ปีที่แล้ว

      @@AzarudeenShahul can you explain expressions used in regex replace? i didnt understand whats $0.

  • @sravankumar1767
    @sravankumar1767 2 ปีที่แล้ว

    superb explanation, do more videos bro........

  • @anuragvinit
    @anuragvinit 3 ปีที่แล้ว

    Thanks for this . This has been asked to be during Mindtree interview

    • @AzarudeenShahul
      @AzarudeenShahul  3 ปีที่แล้ว

      Cool.. hope you were able to answer the question and crack interview...

  • @cpkabil
    @cpkabil 3 ปีที่แล้ว

    Very helpful.. thank you!

  • @murrthuzaalibaig1205
    @murrthuzaalibaig1205 3 ปีที่แล้ว +2

    can u expalin about regex in detail and how did u get the expression

  • @sangramrajpujari3829
    @sangramrajpujari3829 3 ปีที่แล้ว

    good video to improve our logic.

  • @arnabbangal766
    @arnabbangal766 2 ปีที่แล้ว +1

    Sir, can you explain the regex expression more clearly or provide any youtube link where regex is explained nicely ? Thanks. Your videos are very helpful.

  • @sudarshanthota4444
    @sudarshanthota4444 3 ปีที่แล้ว

    Thank you very much for your video's

    • @AzarudeenShahul
      @AzarudeenShahul  3 ปีที่แล้ว

      Thanks for you support:)

    • @bikersview9926
      @bikersview9926 2 ปีที่แล้ว

      @@AzarudeenShahul please txt file and code snippets

  • @maheshk1678
    @maheshk1678 3 ปีที่แล้ว

    Thanks for the nice video

  • @maheshk1678
    @maheshk1678 3 ปีที่แล้ว +1

    Could you explain the same with kafka message streaming

  • @umakanthtagore6003
    @umakanthtagore6003 2 ปีที่แล้ว

    Thanks for this information, can you please help me that
    Is this aproch works large data as well ?
    Thanks in advance !!

    • @AzarudeenShahul
      @AzarudeenShahul  2 ปีที่แล้ว

      Yes, this approach can be scale out to large dataset. Let me know if you face any problem

  • @arifulahsan8803
    @arifulahsan8803 ปีที่แล้ว

    Hi do you teach spark course?

  • @ihba02_official
    @ihba02_official 3 ปีที่แล้ว

    Thank you so much bro

  • @aN0nyMas
    @aN0nyMas 3 ปีที่แล้ว

    In this the last record in my test data had just four columns. hence I got a schema error. is there a way to specify to handle this by ignoring the malformed data?

  • @dipakchavan4659
    @dipakchavan4659 2 ปีที่แล้ว

    Hey Azharuddin, Superb 👍🏻. Can u plz provide dataset.

  • @riyazalimohammad633
    @riyazalimohammad633 2 ปีที่แล้ว

    Hello Azar! Amazing Video. Is there a way we could replace the 5th pipe occurrence rather than adding "-". I want to replace the pipe with "-".

  • @shiyamprasath3105
    @shiyamprasath3105 2 ปีที่แล้ว +1

    hii bro....i have seen this 5 th occasion change in Scala but code is too difficult compared with pyspark... please share easy step of code for scala bro

  • @ankitapriya6671
    @ankitapriya6671 2 ปีที่แล้ว

    Can you share the post where you have provided answer for this

  • @sumitrastogi1
    @sumitrastogi1 3 ปีที่แล้ว

    Can you please share how to deploy in production environment for pyspark job.your videos very helpful

  • @jittendrakumar3908
    @jittendrakumar3908 3 ปีที่แล้ว

    Also please upload for reading the occurance of an string in an word.

  • @jittendrakumar3908
    @jittendrakumar3908 3 ปีที่แล้ว

    Please upload the video for ingesting the data from sap server. This is very important as we need to ingest the data from different source via pyspark.

  • @purnimabharti2306
    @purnimabharti2306 2 ปีที่แล้ว

    I didn't understand at some places you convert rdd to df and then df to rdd...why is it so?

    • @khushbusalunkhe677
      @khushbusalunkhe677 2 ปีที่แล้ว

      Some transformations are not allowed in dataframe but they are availble in RDD so to perform those operations it was converted into RDD and then back to DF

  • @VinodR-vx8uh
    @VinodR-vx8uh 11 หลายเดือนก่อน

    Please someone explain that regexp pattern(.*?\\){5} and why $0 in $0-

  • @duskbbd
    @duskbbd 2 ปีที่แล้ว

    Why you kept $0 before delimiter -

    • @riyazalimohammad633
      @riyazalimohammad633 2 ปีที่แล้ว

      The $0 in awk syntax means to return the output, so when azar uses "$0-" in the function, it will preserve the output of the regex and add "-" to it.

  • @Azardeen-sb1wr
    @Azardeen-sb1wr ปีที่แล้ว

    Mohammed,Azar,BE-4year
    Prakesh,Kummar,Btech-3year
    Ram,Kumar,Mtech,3year
    jhon,smith,BE,2year # any one can share the pyspark code to delimit the "-"

  • @shyammtv.v.s.p4262
    @shyammtv.v.s.p4262 2 ปีที่แล้ว

    123#Australia,india,Pakistan
    456#England,France
    789#canada,USA
    output is
    123#Australia
    789#canada
    456#England
    456#France
    123#india
    123#Pakistan
    how to solve this using pyspark or scala