Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Michael Armbrust

แชร์
ฝัง
  • เผยแพร่เมื่อ 14 ธ.ค. 2024

ความคิดเห็น • 20

  • @FernandoRacca
    @FernandoRacca 7 ปีที่แล้ว +8

    Scala + Kafka + Spark = SuperDataPipeline

  • @7Powers
    @7Powers 7 ปีที่แล้ว +3

    Superb! I must say the best presentation for a long time..

  • @sitientibus
    @sitientibus 7 ปีที่แล้ว

    the best youtube spark video I found so far!

  • @markevogt
    @markevogt 6 ปีที่แล้ว

    Found this presentation both informative and engaging - GLAD you recorded it.
    Like others, there's SO much here that I've already stop/rewound/restarted portions numerous times, until it took me an hour to get through a 28 min presentation ;-)
    I too noticed a few "verbal typos", but it was clear you UNDERSTOOD the terms, so it was easy to follow the slides while listening ;-) ..
    ONE QUESTION:
    I've reviewed it repeatedly, but I'm still NOT sure about one thing:
    Around the 8:05 mark you show a slide with "Stringly-typed methods will 'downcast' to generic "Row" objects.
    - the term "Stringly-typed" is a term, a SLANG term hinting at developers who type MOST of their variables as String, but IS this what you really MEANT?
    - the term "Strongly-typed" is indeed a NON-slang term, and actually (to me) makes as much or MORE sense in the sentence.
    QUESTION: DID you really mean STRINGLY-typed, or STRONGLY-typed, and
    - IF STRINGLY-typed, why? and why NOT STRONGLY-typed?
    Hope you (or anyone else) still reply after 18 months :-)
    KEEP UP THE GREAT POSTS !

  • @donluc007
    @donluc007 8 ปีที่แล้ว +2

    Great add-ons and crisp and clear presentation!

    • @louisalfonso9493
      @louisalfonso9493 3 ปีที่แล้ว

      I know Im randomly asking but does anyone know a trick to get back into an Instagram account??
      I stupidly lost the account password. I love any tips you can offer me.

    • @drewxzavier9220
      @drewxzavier9220 3 ปีที่แล้ว

      @Louis Alfonso Instablaster ;)

    • @louisalfonso9493
      @louisalfonso9493 3 ปีที่แล้ว

      @Drew Xzavier thanks for your reply. I got to the site on google and Im trying it out atm.
      Seems to take a while so I will reply here later with my results.

    • @louisalfonso9493
      @louisalfonso9493 3 ปีที่แล้ว

      @Drew Xzavier it did the trick and I now got access to my account again. Im so happy:D
      Thank you so much you saved my ass!

    • @drewxzavier9220
      @drewxzavier9220 3 ปีที่แล้ว

      @Louis Alfonso You are welcome =)

  • @djibb.7876
    @djibb.7876 7 ปีที่แล้ว

    Great talk!!!
    I set up a spark-cluster with 2 workers. I save a Dataframe using partitionBy ("column x") as a parquet format to some path(same path) on each worker. The matter is that i am able to save it but if i want to read it back i am getting these errors: - Could not read footer for file file´status ...... - unable to specify Schema ... Any Suggestions?
    REPLY

  • @lilhard
    @lilhard 8 ปีที่แล้ว +3

    Man! this is great.

  • @jeffreyqiu5128
    @jeffreyqiu5128 7 ปีที่แล้ว

    Very lean, good stuff

  • @harihs85
    @harihs85 7 ปีที่แล้ว +1

    Can someone tell how to update a column in dataframe pls

    • @TheUsaview
      @TheUsaview 6 ปีที่แล้ว

      You can just overwrite it with df.withColumn("col_name", lit("message_content"))

    • @saikrishnapujari
      @saikrishnapujari 5 ปีที่แล้ว

      Use dataframe.withColumn(newColumnName, any objective of type Column)

  • @bool29
    @bool29 7 ปีที่แล้ว

    Good, to the point presentation besides few wrong spellings and mispronunciations. It is "Cartesian" and "Kyro" instead of "Cartesian" and "Crayo" respectively.

  • @jeevand5060
    @jeevand5060 7 ปีที่แล้ว

    this man is awesome

  • @MolotovWithLux
    @MolotovWithLux 6 ปีที่แล้ว

    #ApacheSpark

  • @louisburke8927
    @louisburke8927 7 ปีที่แล้ว

    Kyro not Kryo