I've been doing something similar to this for quite some time and I thought I was being lazy 😁 As pointed out here, consumers often get a lot of their questions solved just by looking at the loaded data and then you get a trimmed down, more concise and focused set of transformations for a clean final product.
I felt that same way with the user analytics process I'd been running. I created it as an ELT process, and it worked great. It only really broke down when I tried to integrate it with a corporate ETL initiative, which vastly increased the complexity and slowed down development of new features in the data model (developing transformation was planned to be re-done by a totally separate team, which would have come out of my team's budget, and even that took over half a year just to finish requirements gathering). It essentially killed the agility that the project initially had, and made it harder to deliver value to stakeholders. I think ETL still has a place, like you pointed out, when you have a "concise and focused set of transformation for a clean final product". But up to that point, ELT is much more effective at exploring datasets and developing the right set of transformations.
Hello - thanks for your question. With our upcoming SaaS based Data Transformations as well as our existing Data Warehouse Automation product called Qlik Compose, Qlik leverages the power and scale of common Cloud Targets such as Snowflake, Synapse and GBQ by “pushing down” data transformation execution. This is ELT. Thanks for your question!
got very good quick overview.
I've been doing something similar to this for quite some time and I thought I was being lazy 😁
As pointed out here, consumers often get a lot of their questions solved just by looking at the loaded data and then you get a trimmed down, more concise and focused set of transformations for a clean final product.
I felt that same way with the user analytics process I'd been running. I created it as an ELT process, and it worked great. It only really broke down when I tried to integrate it with a corporate ETL initiative, which vastly increased the complexity and slowed down development of new features in the data model (developing transformation was planned to be re-done by a totally separate team, which would have come out of my team's budget, and even that took over half a year just to finish requirements gathering). It essentially killed the agility that the project initially had, and made it harder to deliver value to stakeholders.
I think ETL still has a place, like you pointed out, when you have a "concise and focused set of transformation for a clean final product". But up to that point, ELT is much more effective at exploring datasets and developing the right set of transformations.
So Qlik is ETL or ELT?
Hello - thanks for your question.
With our upcoming SaaS based Data Transformations as well as our existing Data Warehouse Automation product called Qlik Compose, Qlik leverages the power and scale of common Cloud Targets such as Snowflake, Synapse and GBQ by “pushing down” data transformation execution. This is ELT.
Thanks for your question!