Why every one using yaml everywhere? with no code completion, difficult to test/validate, every thing needs to be in a single huge file because we can't use function abstraction ? This is fine for simple "transform"-> "train" -> "test" pipeline, but become very hard for complexe ones. I prefer the Airflow way of defining pipelines with Python code.
managing airflow infra in house is a task in itself. flexibility comes at a cost. and btw yaml is what kubernetes thrives on and most of infra-as-code tools :)
Why every one using yaml everywhere? with no code completion, difficult to test/validate, every thing needs to be in a single huge file because we can't use function abstraction ? This is fine for simple "transform"-> "train" -> "test" pipeline, but become very hard for complexe ones. I prefer the Airflow way of defining pipelines with Python code.
managing airflow infra in house is a task in itself. flexibility comes at a cost. and btw yaml is what kubernetes thrives on and most of infra-as-code tools :)
How do we move the artifacts to prodiution
Love this. Thanks for the great session. 👍
Notebook & Slides Link
This is great!
this was a very good session
wow awesome
Cool