Thanks for this. My question - the official apache airflow helm chart and the docker compose file splits airflow into multiple pieces using the same container image. Which instance of this image cares about the dag and it's related files, given every instance has the dags folder?
In 18:24 You are presenting DAGs Dashoard. Please noticed DAG called airflow2_good_example has schedule_imterval = '@daily', but Last run and Next run are showing the same date (2021-11-29, 00:00:00). Do You know when we can expect improvements? Its so confusing for end users.
Hey, Awesome webinar! Thank you! I do have one question about the best practices of structuring a DAG. So, It is better to put the task implementations (python operators) into a separate file. If my tasks require a lot of imports, is it better to import inside every task (method) or is it fine to import it all on the top level of the tasks file?
I honestly usually just add them all to the top of DAG file as Taskflow tasks, instead of pulling in from files, but if you're using the same python functions in multiple DAG's, the import method is probably best for you!
@Astronomer Please share a direct link to the CLI library you mention (for proper files strcuture) th-cam.com/video/zVzBVpbgw1A/w-d-xo.htmlsi=HiJa9Afi-53yLZOG&t=873
Could you please make video on best practices for kubernetesexecuter on K8s and how we can run parallel pods?
Thanks for this. My question - the official apache airflow helm chart and the docker compose file splits airflow into multiple pieces using the same container image. Which instance of this image cares about the dag and it's related files, given every instance has the dags folder?
Are presented examples available in any code repo?
Thank you for this video! I was wondering how you would see all the variables available or add new variables?
Check out the Airflow registry, it shows all the variables for each operator! registry.astronomer.io/
Where can we get the presentation slides?
In 18:24 You are presenting DAGs Dashoard. Please noticed DAG called airflow2_good_example has schedule_imterval = '@daily', but Last run and Next run are showing the same date (2021-11-29, 00:00:00). Do You know when we can expect improvements? Its so confusing for end users.
Hey, Awesome webinar! Thank you!
I do have one question about the best practices of structuring a DAG.
So, It is better to put the task implementations (python operators) into a separate file.
If my tasks require a lot of imports, is it better to import inside every task (method) or is it fine to import it all on the top level of the tasks file?
I honestly usually just add them all to the top of DAG file as Taskflow tasks, instead of pulling in from files, but if you're using the same python functions in multiple DAG's, the import method is probably best for you!
@Astronomer
Please share a direct link to the CLI library you mention (for proper files strcuture)
th-cam.com/video/zVzBVpbgw1A/w-d-xo.htmlsi=HiJa9Afi-53yLZOG&t=873
You can find documentation on the Astro CLI, including download instructions, here: docs.astronomer.io/astro/cli/overview
Why can't americans say words? eye-dempotency?
How do you say it? I and everyone else I work with (in London) say it like that