Great way to handle the incremental data insert and update the existing.👍 Just 2-3 days back, I've come accross this scenario where I've used SQL queries (with merge statement) to insert new and update the existing records. I had 40-50 tables and in each table atleast 30-35 columns. So to manage this I've used SQL queries but your solution is great too. 👍 Nice video keep it up. Keep posting such a great informative videos ❤️
Sir thanks a lot for the video. Lets say a record is deleted from the source table so how do we handle that using lookup we will be either inserting or updating based on EMPID.
Yes you are absolutely right 👍. I have explained about the techniques of incremental loads. This is a technique by which we can also do the incremental load. I have recorded other videos where I explained what you are suggesting. Thanks for your comment 🙏
Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.
Thanks for your tutorial. Very good. I have a question, in the step which you use 'OLE DB Command', can that be done also by using 'Execute SQL Task''? Thanks :)
Sir I have a requirement where I need to get data from multiple tables and views and the total records will be around 50000 per day. The target system uses REST so db data needs to be converted to JSON so how to do this. scenario - 1 If i want to send data in batch of 5k till it reaches 50k records scenario-2. Pull 50k records and then convert to json and send it as is(i think its not possible but advise on this approach)
Great way to handle the incremental data insert and update the existing.👍
Just 2-3 days back, I've come accross this scenario where I've used SQL queries (with merge statement) to insert new and update the existing records.
I had 40-50 tables and in each table atleast 30-35 columns. So to manage this I've used SQL queries but your solution is great too. 👍
Nice video keep it up. Keep posting such a great informative videos ❤️
Thanks for your lovely appreciation ❤️😘
Keep watching and sharing your thoughts.
Thanks
Great video on incremental load.. Good to see other techniques as well
Thanks.. uploaded other techniques as well..watch SSIS playlist..
Very good video. Many thanks.
Thanks for your appreciation
Please share to others
Thank you soo much Bhai you saved my day... Less time with more information!! 😊👍
Thanks for your appreciation,
Please share to others
Sir thanks a lot for the video. Lets say a record is deleted from the source table so how do we handle that using lookup we will be either inserting or updating based on EMPID.
I have already recorded video on this and uploaded.
Please check below link
th-cam.com/video/MY1KV4rZoTI/w-d-xo.html
Use a checksum aggregation between each table and compare the checksum. It is quicker. Lookups on SCD are slow if you have n100k or millions of rows.
Yes you are absolutely right 👍.
I have explained about the techniques of incremental loads. This is a technique by which we can also do the incremental load. I have recorded other videos where I explained what you are suggesting.
Thanks for your comment 🙏
@@ssunitech6890 Hi sir, In which video did you explain about checksum technique?
I know this was a while ago, but do you think you could further explain how to do this?
Very good vide Sirji
Thanks 🙏
Thanks sir for this great video. All doubt gets cleared. Sir, Could you please upload 1 more video on sequence container with example like this?
Thanks Kaushal,
Sure will record a video on sequence container and upload soon .
Stay tune .keep watching and please share to others..
Thanks
Hey,
Check out the video on sequence container.
th-cam.com/video/BeJ3y3t4Zxc/w-d-xo.html
Thanks
Great video
Thanks
Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.
Sure, you can add one table which will always have last execution time. Use this table in your query
Thanks for your tutorial. Very good. I have a question, in the step which you use 'OLE DB Command', can that be done also by using 'Execute SQL Task''? Thanks :)
Oledb is available inside dataflow. But Execute SQL task will only available in control flow. So how can you do directly there
@@ssunitech6890 thanks for your reply
Your Welcome
simple but super!
Thanks Jahangir..
Share to your friends..
Thanks- SS Unitech
Hi Sir, Incremental load is better or CDC is better , when to use both of them , are they same or different?
Hello Vishwajeet,
Both are the incremental load techniques.
Thanks.
@@ssunitech6890 thank you, I wanted to ask lookup should be used or cdc should be used , what's the difference?
keep going bro. It is nice video to understan
Thanks brother.
Can you please share the videos to others.?
Thanks
@@ssunitech6890 my pleasure
Sir I have a requirement where I need to get data from multiple tables and views and the total records will be around 50000 per day.
The target system uses REST so db data needs to be converted to JSON so how to do this.
scenario - 1 If i want to send data in batch of 5k till it reaches 50k records
scenario-2. Pull 50k records and then convert to json and send it as is(i think its not possible but advise on this approach)
why do you pulling all 12 records from source every time?
Because source has 12 records.
Sir, Can you please upload one video on SSIS configuration?
Specially XML and SQL Configuration file
I wanted to upload video on SSIS configurations..
I will take this request first.
Thanks for your comment..
If you are using SQL Server 2012 or above. Use the project deployment model and do your deployment and configuration through SSISDB.
Yes but I guess he wants to know how to deploy package in file deployment.
Thanks
I get only o in place off false. How we change
Check video again
Same also I AM getting 0 in place of false
Same table insert update why not possible ?
Stagging table ? Use ?
Yes that you can do.
I am explaining the concept
Adabaly ...
Understand question
Adbaly
tks
Thanks 🙏
If there are duplicate ID's. Like if I am having a fact table which is having duplicate records