87. Databricks | Pyspark | Real Time Project: ETL Pipeline Integrating ADF, ASQL, ADLS, Key Vault

แชร์
ฝัง
  • เผยแพร่เมื่อ 27 ธ.ค. 2024

ความคิดเห็น • 34

  • @satijena5790
    @satijena5790 ปีที่แล้ว +1

    Nicely explained the simple real life scenario. It gives lot of learnings e.g. how to call a Databricks notebook from a ADf pipeline (parametrised), integration with AKV, Using connections to ADLS as well as SQL DB with complete clarity. Since, i did see the previous videos of this channel, i could understnad it end to end. Very eager to do hands-on once i get an dev enironment. Great job. Many thanks.

  • @kranthikiran8318
    @kranthikiran8318 2 หลายเดือนก่อน +1

    This guy needs more recognition

  • @Elkhamasi
    @Elkhamasi 5 หลายเดือนก่อน +1

    You are a great teacher.

  • @sravankumar1767
    @sravankumar1767 2 ปีที่แล้ว +1

    Superb explanation Raja 👌 👏 👍

  • @rambevara5702
    @rambevara5702 ปีที่แล้ว +1

    Brother I have been waiting for this video..I am devops engineer need to explore and prepare adf adb SQL integration..this video helped me lot..my request please put devops engineers in your mind and do simple video with python or SQL rather pyspark as it is not familiar for us..many thanks brother

    • @rajasdataengineering7585
      @rajasdataengineering7585  ปีที่แล้ว

      Thanks Ram for your comment. Glad it helps devops engineers too.
      Sure will make simple videos

    • @rambevara5702
      @rambevara5702 ปีที่แล้ว

      @@rajasdataengineering7585 thanks brother

    • @rambevara5702
      @rambevara5702 ปีที่แล้ว

      @@rajasdataengineering7585 where we can find that example notebook you have used for this video..

  • @sivag98
    @sivag98 2 ปีที่แล้ว +2

    Hi Raja.. Your videos are awesome...Thank you so much...
    could you please make Databricks delta table real time projects videos and incremental load full load pipeline related videos... Thanks advance

    • @rajasdataengineering7585
      @rajasdataengineering7585  2 ปีที่แล้ว

      Sure Anand, will make delta lake real time videos

    • @ravilikhar007
      @ravilikhar007 ปีที่แล้ว

      @@rajasdataengineering7585 Hi Raja, Please make video on delta live real time project. Thanks. Keep doing good work.

  • @navalsaini3229
    @navalsaini3229 2 ปีที่แล้ว +1

    Nice explanation kindly make an video on stream processing via spark using stream processing as well

    • @rajasdataengineering7585
      @rajasdataengineering7585  2 ปีที่แล้ว

      Sure Naval. I have already created one video on streaming
      th-cam.com/video/zx5H82fmUPU/w-d-xo.html
      Will also make more videos on advanced streaming exercises

  • @neerajsharma-zt8jo
    @neerajsharma-zt8jo ปีที่แล้ว +1

    How have you configured SQL server table details in sink for load to log table before mapping the columns.. Can you share more details there.. We are unable to get insert option there..

    • @rajasdataengineering7585
      @rajasdataengineering7585  ปีที่แล้ว

      We are just writing the dataframe output into SQL table using jdbc connection

  • @abhimanyukakde849
    @abhimanyukakde849 ปีที่แล้ว +1

    Hi, could you please make a video on aws ETL Pipeline in real time use case

  • @ManukaDeepika
    @ManukaDeepika 6 หลายเดือนก่อน +1

    databricks using pyspark do more vedios

  • @shankardonthireddy9422
    @shankardonthireddy9422 2 ปีที่แล้ว +1

    Hi, could you please make a videos on Delta live tables if possible?

  • @mission_possible
    @mission_possible 2 ปีที่แล้ว +1

    Some of the videos are showing as unavailable can you please look into it?

    • @rajasdataengineering7585
      @rajasdataengineering7585  2 ปีที่แล้ว

      Hi Noor, sure I can look into it. Could you please share one of the video topic for which you face such issue

    • @mission_possible
      @mission_possible 2 ปีที่แล้ว +1

      @@rajasdataengineering7585 when I open data bricks playlist there it is showing as 4 unavailable videos

    • @rajasdataengineering7585
      @rajasdataengineering7585  2 ปีที่แล้ว

      Ok, I will check

    • @mission_possible
      @mission_possible 2 ปีที่แล้ว +1

      @@rajasdataengineering7585 Thank you

  • @Mehtre108
    @Mehtre108 ปีที่แล้ว

    Could you pls upload one real time project with experienced people sir

  • @suresh.suthar.24
    @suresh.suthar.24 ปีที่แล้ว

    can we run the community edition notebook using ADF?

  • @ManukaDeepika
    @ManukaDeepika 6 หลายเดือนก่อน

    Can u show how to install databricks and datalakes demo everytime we can learn easily one more real project show me