Apache Airflow Architecture 101
ฝัง
- เผยแพร่เมื่อ 21 ก.ค. 2024
- Apache Airflow is a popular open-source Linux and Python based workflow engine. Before you jump in and start using it, there are some things you need to know. In this video, I explain the Airflow architecture: the good, the bad, and the ugly, so you can decide where and when to use it.
Join my Patreon Community and Watch this Video without Ads!
www.patreon.com/bePatron?u=63...
Slides
github.com/bcafferky/shared/b...
Video: Don't Use Apache Airflow
• Don't Use Apache Airflow
Follow me on Twitter
@BryanCafferky - วิทยาศาสตร์และเทคโนโลยี
The best intro regarding Airflow I found. Thank you...
I really appreciate the parsimony segment. Very helpful, thanks.
Nice explanation
Hi, great and helpful video, is there a reliability strategy between the scheduler and the dag directory? It seems that's the signal point of the fail point. Thanks!!!
Not sure but I think the Airflow UI is where failures are expected to surfaced. But good question.
How do you feel about “Prefect”? They’re trying to replace Airflow.
A video on Prefect from you would be great too!
Do you need a WorkFlow engine?
@@BryanCafferky yes
Awesome!
Hi guys! I am looking for Apache Aiflow study group. I have some intermediate knowledge on Kafka-KSQL, Spring-Batch, DataEngineering concepts etc. I really want to improove my skills.
would airflow be an alternative for ssis/ssdt?
Hi Nathan,
See my other vide to answer that.
Video: Don't Use Apache Airflow
th-cam.com/video/YQ056EKzCyw/w-d-xo.html