Use Asserts for Data Quality and Data Validation in ADF

แชร์
ฝัง
  • เผยแพร่เมื่อ 22 ต.ค. 2024

ความคิดเห็น • 12

  • @andyparkerson
    @andyparkerson 2 ปีที่แล้ว +2

    I love the directness of these videos. No flashy title cards, or "Hey TH-cam! It's ya boy, Mark!" It reminds us that good documentation can be as easy as walking through a few examples on video. It doesn't have to be fancy or take a long time to produce.
    I'm going to create asserts in my dataflows from now on. Thanks for the video!

  • @emiliogarzacantu4296
    @emiliogarzacantu4296 ปีที่แล้ว

    Awesome video, thanks for the explanation. wish all videos were like this

  • @cburys
    @cburys ปีที่แล้ว +1

    solid video! question, what's the point of the description? I don't see where it's ever useful. It must have a purpose though.

    • @MSDataFactory
      @MSDataFactory  10 หลายเดือนก่อน

      what description are you referring to?

    • @cburys
      @cburys 10 หลายเดือนก่อน

      @@MSDataFactory The Assert Description

    • @gielnijkamp8857
      @gielnijkamp8857 9 หลายเดือนก่อน +1

      @@cburys I had the same question. Where do we see the assert description back? I was hoping in a report of the error log?

  • @DataForgeAcademy
    @DataForgeAcademy 2 ปีที่แล้ว

    Great feature and explanation

  • @tybutton318
    @tybutton318 2 ปีที่แล้ว

    Love this demo and have adapted asserts where we can. Checking for lengths, data types ir null values seems to work well. But as soon as we introduce an assert checking a field for a set of values slows us down dramatically and 4 or 5 will cripple the data flow. Is there a better way to implement something along the lines of contains('a', 'b', 'c', item = myfield). Using 'or' seems to have similar issues for us. We've scaled the processors up for the dataflow and still find that some of our validation has to be implemented directly in a conditional split because the logic seems to overwhelm the assert transformation.

  • @PicaPauDiablo1
    @PicaPauDiablo1 2 ปีที่แล้ว +1

    Excellent

  • @nissarahmad8545
    @nissarahmad8545 ปีที่แล้ว

    What will be your approach of system testing in azure data factory.

  • @paulhernandezgermany
    @paulhernandezgermany 2 ปีที่แล้ว

    Really cool! 😎

  • @RameshP-ds4xt
    @RameshP-ds4xt 2 ปีที่แล้ว +1

    👍