As a "manager", I would start off by having a top notch data engineer who knows how to work with complex data and do complex joins, they should also know about data warehousing best practice techniques. My second employee would be a data analyst with superb customer service skills that understands what the customer wants.... she/he should have superb analytical/visualization/story telling techniques, more analytical than technical.
Great video. Quick question for anyone: if I have my data warehouse set up, do I connect my reporting tool directly to query the DWH? The reason I'm asking is because of cost. If the reporting dashboards are constantly pinging the data warehouse, doesn't this greatly increase the compute costs? Storage seems relatively cheap for our purposes, but if the data warehouse needs to stay on all the time because dashboards are querying data from it, then that could be too expensive. But I don't know if I'm thinking of this the wrong way.
Generally speaking its probably not always great to run complex queries straight against your DW due to cost and speed reasons. Some people pre-calculate metrics and store them in tables to ensure you avoid increasing costs. Other people might pre-calculate metrics and create datamarts on less expensive databases. Etc. Unless you need the data live because the dashboards are used for operational purposes, you can probably use a few different methods to avoid live running of complex queries.
Thanks Seattle Data Guy for creating so many useful videos. Will you be able to share some idea for the career growth of Data engineer vs Analytics Engineer? (Career path, impact, salary, job levels, etc) Since I’ve found that there’re more and more Analytics Engineer position coming out. Just want to see if it’s a good option for me (current Data engineer) to change to Analytics Engineer position in the long run? I want to train myself more advanced data engineering skills (like Kafka, spark, flink, etc) so that I can add more value to myself and that would make me hard to be replaced and get paid more than just using modern data tool like dbt (Analytics Engineer) Am I right? Thanks a lot!
I think analytics engineers have always exists. I often reference the fact that there usually is a split between DEs, those that sit more on the top layer and those that are more ont eh bottom. I have seen people who are smart make just as much or a lot more doing analytics engineering as people who do complex kafka and spark work. I would say its worth learning spark and kafka though.
Great video! One of the challenges my company faces is we are working with HR data so we cannot expose it to the DW due to GDPR, sensitive information, etc. Do you have any recommendations? We can encrypt it but PowerBI does not read from an encrypted source.
Shoot sorry about that. I deleted that video and I don't have the question currently. feel free to reask it. I will be re-releasing that video in the future.
Hi Ben, I'll have my bachelor in accounting and economics. Can I ask you how accounting could help me to be a data analyst?Because it seems as if everyone involved with data analysis came from CS, DS, IT background. Moreover I wanted to add that my bachelor includes 5 courses on programming (excel, excel vba, Python, R and powerBi, they give me 3 credits each) and another exam called business intelligence which gives me 6 credits. I found interesting the part about data engineering even though in my case I'd be better being a data analyst role (descriptive, prescriptive and diagnostic analysis) Anyway I guess that accounting may count as domain knowledge hence great for financial data analyst or sales data analyst or supply data analyst with specific industry domain (even auditing or accounting per se).
Data analysts generally have a few different sides. They have the technical side like Excel, SQL and python but they also have a business side. The business side needs to understand how businesses operate, which numbers, metrics, etc are important and how to tell a story with the numbers they are seeing. Accounting can help from this perspective.
@@SeattleDataGuy well I was just starting out with my cloud data engineering journey few months ago and the algorithm gods made me come across your channel, been following it ever since. Keep up the good work! Regards from India!!❤️❤️
As a "manager", I would start off by having a top notch data engineer who knows how to work with complex data and do complex joins, they should also know about data warehousing best practice techniques. My second employee would be a data analyst with superb customer service skills that understands what the customer wants.... she/he should have superb analytical/visualization/story telling techniques, more analytical than technical.
Simple, straight forward and provides a good foundation for a data analytics team
Glad you found it helpful!
Great video. Quick question for anyone: if I have my data warehouse set up, do I connect my reporting tool directly to query the DWH? The reason I'm asking is because of cost. If the reporting dashboards are constantly pinging the data warehouse, doesn't this greatly increase the compute costs? Storage seems relatively cheap for our purposes, but if the data warehouse needs to stay on all the time because dashboards are querying data from it, then that could be too expensive. But I don't know if I'm thinking of this the wrong way.
Generally speaking its probably not always great to run complex queries straight against your DW due to cost and speed reasons.
Some people pre-calculate metrics and store them in tables to ensure you avoid increasing costs.
Other people might pre-calculate metrics and create datamarts on less expensive databases.
Etc.
Unless you need the data live because the dashboards are used for operational purposes, you can probably use a few different methods to avoid live running of complex queries.
@@SeattleDataGuy thank you for the quick response. I appreciate it.
Which data warehouse are you using?
Great question @baw5xc; thanks for the concise and practical answer @Seattle Data Guy
I'm a freelance data Analyst, competent with the use of Excel, SQL and Power Bi,....i will be willing to join a team of data analysts
Congrats!
Great content as always, keep It comin
Glad you enjoyed it!!
Thanks for the vid, very helpful
Glad it was helpful!
Interesting stuff. Thank you.
Glad it was helpful!
Thanks Seattle Data Guy for creating so many useful videos.
Will you be able to share some idea for the career growth of Data engineer vs Analytics Engineer? (Career path, impact, salary, job levels, etc)
Since I’ve found that there’re more and more Analytics Engineer position coming out. Just want to see if it’s a good option for me (current Data engineer) to change to Analytics Engineer position in the long run? I want to train myself more advanced data engineering skills (like Kafka, spark, flink, etc) so that I can add more value to myself and that would make me hard to be replaced and get paid more than just using modern data tool like dbt (Analytics Engineer) Am I right?
Thanks a lot!
I think analytics engineers have always exists. I often reference the fact that there usually is a split between DEs, those that sit more on the top layer and those that are more ont eh bottom. I have seen people who are smart make just as much or a lot more doing analytics engineering as people who do complex kafka and spark work. I would say its worth learning spark and kafka though.
Great info!
Glad you liked it!
How to set up the Data Analytics vertical in a product based company??
Great video! One of the challenges my company faces is we are working with HR data so we cannot expose it to the DW due to GDPR, sensitive information, etc. Do you have any recommendations? We can encrypt it but PowerBI does not read from an encrypted source.
Hey Ben
I had a really good question for you on the yesterday video but its not visible to me :D
I think it has been deleted
Shoot sorry about that. I deleted that video and I don't have the question currently. feel free to reask it. I will be re-releasing that video in the future.
Hi Ben, I'll have my bachelor in accounting and economics. Can I ask you how accounting could help me to be a data analyst?Because it seems as if everyone involved with data analysis came from CS, DS, IT background. Moreover I wanted to add that my bachelor includes 5 courses on programming (excel, excel vba, Python, R and powerBi, they give me 3 credits each) and another exam called business intelligence which gives me 6 credits.
I found interesting the part about data engineering even though in my case I'd be better being a data analyst role (descriptive, prescriptive and diagnostic analysis)
Anyway I guess that accounting may count as domain knowledge hence great for financial data analyst or sales data analyst or supply data analyst with specific industry domain (even auditing or accounting per se).
Data analysts generally have a few different sides. They have the technical side like Excel, SQL and python but they also have a business side. The business side needs to understand how businesses operate, which numbers, metrics, etc are important and how to tell a story with the numbers they are seeing. Accounting can help from this perspective.
Yo.. where can we see the deleted video from yesterday..😂
Wow, never expected anyone would notice. There were a few edits that didn't get added that made the video wonky. I will put it up at a later date.
@@SeattleDataGuy well I was just starting out with my cloud data engineering journey few months ago and the algorithm gods made me come across your channel, been following it ever since.
Keep up the good work! Regards from India!!❤️❤️