But what about modified rows. We have transactions that can be modified after some period. How can I deal with such rows. And we have simple ERP that don't store any "modification date" in DB.
Millik to load incrementally it will need one of three things. 1 - Primary key (where you can do a Where Exists to do a diff) - primarily for added / deleted rows only 2- Last Updated or Modified Date (To check for dates sequentially after date QVD or QVW/QVF was last loaded) - added, deleted, modified 3- Use a Hashed Value for every row, and then do a where exists. This could be done on the server side as a column, If these are not present and you have to parse the different tables before getting the delta, this may lead to performance issues. It will still however, be faster than multiple users hitting your ERP every time a click-request is formed since the Qlik Sense / View Server leverage only a Singular Query. - added, deleted, modified Hope this helps!
Hello Millik - thanks for your question. Alexander has provided some great suggestions in the comments below. As stated in the video - there are a number of resources available on the Qlik Communty - these are also referenced in the description here on TH-cam. Take a look at: community.qlik.com/groups/qlikview-india/blog/2017/02/20/incremental-loading-all-scenarios - this may provide additional answers. My plan is to start creating additional videos on this topic - but you can also search TH-cam for Qlik Incremental Load (loading) - you may also find additional examples. As mentioned this is a Best Practice and can be implemented a number of ways. Please continue the discussion with me and others in the Qlik Community as we are better equipped to handle these types of questions, including screenshots and attachments. Thanks again - hope to hear from you.
But what about modified rows. We have transactions that can be modified after some period. How can I deal with such rows. And we have simple ERP that don't store any "modification date" in DB.
Millik to load incrementally it will need one of three things.
1 - Primary key (where you can do a Where Exists to do a diff) - primarily for added / deleted rows only
2- Last Updated or Modified Date (To check for dates sequentially after date QVD or QVW/QVF was last loaded) - added, deleted, modified
3- Use a Hashed Value for every row, and then do a where exists. This could be done on the server side as a column, If these are not present and you have to parse the different tables before getting the delta, this may lead to performance issues. It will still however, be faster than multiple users hitting your ERP every time a click-request is formed since the Qlik Sense / View Server leverage only a Singular Query. - added, deleted, modified
Hope this helps!
Hello Millik - thanks for your question. Alexander has provided some great suggestions in the comments below. As stated in the video - there are a number of resources available on the Qlik Communty - these are also referenced in the description here on TH-cam. Take a look at: community.qlik.com/groups/qlikview-india/blog/2017/02/20/incremental-loading-all-scenarios - this may provide additional answers. My plan is to start creating additional videos on this topic - but you can also search TH-cam for Qlik Incremental Load (loading) - you may also find additional examples. As mentioned this is a Best Practice and can be implemented a number of ways. Please continue the discussion with me and others in the Qlik Community as we are better equipped to handle these types of questions, including screenshots and attachments. Thanks again - hope to hear from you.
Good