If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
The explanation is interesting, I immediately subscribed. I want to try something, which is transforming SQL to NoSQL, do you think that is interesting and still within the realm of data engineering? Is it possible to use a hybrid of SQL and NoSQL data? And is it possible to implement the CDC approach in transforming SQL to NoSQL?
Is it necessary to implement CDC when transferring data from source systems to the data lake? Alternatively, does CDC play a role specifically when working with the data warehouse? Or does its implementation depend solely on the business requirements?
It's generally just good practice to read from a replica database when pulling data from analytics because yes, ther is always a risk that whether you're pulling from tables or the WAL that the operational system may be impacted. Also here is a great video talking about the WAL very in depth th-cam.com/video/wI4hKwl1Cn4/w-d-xo.html
Really good info and I am always glad to see MySQL and Postgres being used in a company/work use as the bigger budget names seem to always take the light. Now to the important part....whats the cats name @6:55
Hi! Thanks for your videos,they are unimaginably helpful! Can you please make a video on how to transition from junior data engineer to junior machine learning engineer(jr.MLE) or just MLE? Thanks!!
If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
Lovely cameo at the end there
she has a way of always showing up!
Thanks for this informative video Ben. I heard about CDC quite often before but couldn't understand it enough until I watched your video. Thank you!
Glad this helped!!
I have learned so much from you. Thank you
You are so welcome
Great explanation Ben!
Glad you liked it!
Great video! Both tools you mentioned were new to me, so I'll keep them in mind the next time I need to build a CDC pipeline. Thank you very much! ❤
Glad you found the video helpful! Let me know what you think about the tools!
Thanks for sharing!
glad you liked it!
The explanation is interesting, I immediately subscribed.
I want to try something, which is transforming SQL to NoSQL, do you think that is interesting and still within the realm of data engineering? Is it possible to use a hybrid of SQL and NoSQL data? And is it possible to implement the CDC approach in transforming SQL to NoSQL?
Is it necessary to implement CDC when transferring data from source systems to the data lake? Alternatively, does CDC play a role specifically when working with the data warehouse? Or does its implementation depend solely on the business requirements?
It's not required and very much depends on the businesses requirements.
Does implementing a WAL impact performance of operational source systems?
It's generally just good practice to read from a replica database when pulling data from analytics because yes, ther is always a risk that whether you're pulling from tables or the WAL that the operational system may be impacted. Also here is a great video talking about the WAL very in depth th-cam.com/video/wI4hKwl1Cn4/w-d-xo.html
Really good info and I am always glad to see MySQL and Postgres being used in a company/work use as the bigger budget names seem to always take the light. Now to the important part....whats the cats name @6:55
Mia 😅
Hi! Thanks for your videos,they are unimaginably helpful! Can you please make a video on how to transition from junior data engineer to junior machine learning engineer(jr.MLE) or just MLE? Thanks!!
Great video, the music was a bit distracting.
Thanks for letting me know!