What is Data Integration ?
ฝัง
- เผยแพร่เมื่อ 3 ก.ค. 2024
- In this live stream, I will introduce you to the concept of Data Integration. We will also have an overview of the 3 different types of Data Integration, with examples of each. Make sure you add in a reminder to join me at the scheduled time, and get any questions clarified. Thanks !
nicely explained. Thanks!
Thank you Mukesh. I am glad you found the content useful. Please do Like, Share & Subscribe!
It seems that all other channels assume prior knowledge in this topic, because they sound like they're speaking in another language, even in educational videos. Thank you for making it easy to understand for the layman.
Comments like this Make my Day :) Thank You !
Very knowledgeable lecture
Thank you very much for your encouraging comments. Please do Like, Share & Subscribe!
You really great explanation bro .....I searched a lot I gained good knowledge to see your video
Thank you for your comments Somu. Please do subscribe!
This is an awesome tutorial, thank you!!
You are most Welcome. Thank you for your comments. Please do subscribe!
Great Video! Worth every second! Could you please explain how system integration and data integration are different.
First, Apologies for the delay in responding.
Thank you for your comments ! Glad you liked the video.
Regarding the difference between System Integration & Data Integration:
System Integration is a broad term that describes multiple separate Systems working together to deliver a service or feature. Data Integration is very much a part of it. Data Integration specifically deals with how systems work with each other through data exchange. For ex- A booking website like Trivago.com could be said to be integrated with a airline website of Lufthansa.com. In this case, the integration between these 2 systems is achieved by exchanging data, hence the term Data Integration. So, "Data Integration" is a sub-topic of "System Integration" that specifically deals with the guidelines, pattern, best practices & architecture of how data exchange is used to integrate systems.
Hope that answers your question !
Do you have a full live project that i can join or any prerecorded on Data Integration
Very good explanation. Wanted to get your perspective on this scenario. Let's say we have an already established EDW, and due to M&A we want to integrate the new company's Sales DB data, with the one we already have in our EDW? I'm leaning towards it being in the B2B type category. What are your thoughts w.r.t. data-integration process/steps to actually make this merger happen with the end result being that we have all the sales data from the new company into ours at the end of the day.
Hello Deepak. Thanks for such an interesting question. Given the example of a Merger, if you want to integrate the new company's Sales DB data, then I would not consider it as B2B. Here's why.
As a CEO, whether you manage 1 retail company or 2 retail companies, do not matter much to your EDW.
You will still have only "1" EDW, for both your companies (assuming both companies are in the same business).
From an EDW perspective, the only change that has happened is, that instead of sourcing data from a single Sales table, you are now sourcing data from 2 Sales tables. So you will update your "ETL" setup accordingly to manage the additional data source. That's it. So, as you can see, the B2B type still isn't applicable here. It is only a change to the "ETL" setup, as "ETL" is what is used to setup your EDW.
To clarify, "B2B" is a type of data integration to be used when businesses/applications want to exchange data in real-time or near real-time. Most often these use cases are critical (for ex- flight booking, hotel booking, bank wire transfers etc.). They often deal with small amount of data being exchanged with very low latency. Do notice that I use the word "Exchange", since data flows both ways. For ex- if you book a hotel through a travel website, then the travel website sends a packet of information to the Hotel's servers. The Hotel's reservation system will then respond with an acknowledgement and a confirmation of the reservation (or a decline message). Here, only transaction is processed in rapid succession by both systems, where they talk to each other. This is a B2B. Whereas in ETL, it is mostly a one-way traffic. The ETL tool pulls information from all the different data sources in bulk, but is not expected to send any data back to the source system, simply because it is not required in a EDW. Hence the paradigm of ETL and B2B are totally different.
To your last question, about how to merge the data from 2 sales tables, the answer can be quite exhaustive based on the specific scenario. So, instead let me give you some high level pointers -
- Ensure the granularity (Level of Detail) of both Sales table is same. MOST IMPORTANT.
- Highlight any additional dimensions that are available in the newer Sales table but not in the older one
- Detect any cases of Late Arriving Dimensions
- Detect if any Sales Data can be updated over time in the Source. This may indicate any kind of off-hours reconciliation
- Take care of conflicting Degenerate Dimensions across the 2 Sales Fact table. for ex- SalesOrderNbr. VERY IMPORTANT
- Have exactly 1 column in your Fact table, that can be used to distinguish between the 2 different sources.
That's all I could think of now :) Hope this helps !
really great explained nicely
Thank you Parvez !
Excellent.
Very nicely explained Bindu...!
Thanks Vinit. I am glad you found it interesting ! Stay subscribed.
@@DataAcademyIN off late I have watched almost all videos you've prepared. The
concepts are explained at ease.
Looking forward to more such industry wide videos.
-Vinit
Many thanks for the encouraging words@@vinzy147. I am a bit busy at work, so I am finding it hard to prepare new videos. I will pickup the pace in a few weeks time. Stay tuned :)
🎩
Thank you !! Please do subscribe