Please watch other videos for incremental load testing in "ETL Testing" playlist: There 2 seperate video one for RDBMS and other for Flat file as source system
Good evening sir, Can you do a dedicated video showing actual work of an etl tester like how to write test cases for this video of scd type 2 and how to load and run the job in Informatica and validation part of insert logic and update logic in any DB. Taking very small example of two tables The practical work which you greatly explain would be helpful
Hello sir Thanks for sharing this much knowledge free of cost. i have a question, for validating SCD2 transformation- how can i do data validation in source and target, and we have only read permission in source and target system... Also could you elaborate on.. How staging area is actually present in database. Thanks Kiran
It depends on project and management decision however you should have all access on database/source systems as part of testing and you should have a dedicated QA environment so you are free to manipulate the data in QA environment. I have seen some projects do not encourage a separate QA environment where QA and dev environments are shared , in that case you can ask DBA to do run the insert/updates
Hello @ETL QA Labs, actually I didn't get that delete one row in target and run WF? after doing this what will be the impact on SCD2 and what we are actually validating?
at which stage "scd type 2" is applied in your project (Source to Landing OR Landing to Staging OR STAGING to DW.?) is question asked by interviewer. could you please answer..??
Scd 2 - slowly changing dimension type 2 where full history is preserved which means it's available for history tracking for the business, as you know any other stage in etl process is only for the purpose of etl processing and only non volatile space is Target so the answer should be it's applied for Target table in your case it should be staging to Target.
Ideally QA should have a dedicated test environment which should have all the required access for QA to perform however in some situations and projects, a seperate environment is not feasible to provide so QA has to share with development team and hence access may not be available for QA so we should agree with such arrangement on the part of actions that dev team need to perform for QA to test as part of test planning and should be captured in the Test strategy document for smooth facilitation of testing life cycle and avoid any conflicts with development team
While performing this testing :- when we perform insert , update , delete and run the workflow to check the result but what will happen on actual table target and source that will get disturb .. Or we have to do this on view table
Hello Sir, Thanks for this video. Really appreciated Can you please tell me what are the expected technical interview question that ask in mangerial round for 3+ years exp in etl?
if we have more than 100 clm than how can we perform minus query? write all clm names in select query is not a goop option. is there any alternate way?
Yes , Ideally you would perform all the testing in QA environment which is owned by QA team and you normally would be using a generic( common account for QA team ) account to peform all these insert , updates etc . In certain case if you don't have dedicated QA environment , you can request DBA to run all these DML statement before you run the workflows. Hope that clarifies
@@etlqalabs5048 In Real-time Testers run the workflow ? Developers create mappings n create workflows n runs those workflows then data is loaded into target with the business logics applied... Developer environment - Etl process completed n data is loaded into target. QA environment - replica of data what is loaded into target is given to QA environment for testing. So when we do update delete or insert that won't effect the actual target data. This is what I understood. Please correct me if I'm wrong n Lemme know the actual process. Thank you.
I heard that tester is only have access to select command. How can we insert, delete n update. ? I saw ur response for same question but didn't understand clearly like qa environment something. If u don't mind could you give some more clarity please. Thank you.
As a ETL TESTER you need read/write access on database in QA environment however UAT users testing in UAT environment usually have read only access on db. Just to summaries - There are various environment for specific purpose related to each phase of SDLC - 1) Development environment for developing ETL job and performing Unit Testing 2) QA environment for testing by QA team 3) UAT environment for testing by Users ) end users before production release. 4) Production environment - final environment where actual business runs Hope this clarifies some doubts
How do we validate the hundreds of rows of scd2 transformation by writing querry...? I mean i want to write a querry after loading data in source table such that it should validate the fromdate ,todate, flag status all in querry..?? It wil be very helpful plz guide on this
Hi Sir, I'm working in some company, I got project, they said I need to work on production support(like monitoring the jobs and scheduling the ODI loads)... I'm not sure whether I will get knowledge on informaticaPC. Could you suggest what are the difficulties we will get
You are really doing very great job... Thanks a lot sir very simple explanation. Want practical video on same topic.
Very helpful video to understand SCD type 1 and type 2
Thanks for helping the community with your experience :)
Thanks for your good words !
Superb … thanx sir
It covers :- Incremental load testing
Please watch other videos for incremental load testing in "ETL Testing" playlist: There 2 seperate video one for RDBMS and other for Flat file as source system
Wonderful Explanation Sir !
Thankyou sir !! Great help
Good video
Good evening sir,
Can you do a dedicated video showing actual work of an etl tester like how to write test cases for this video of scd type 2 and how to load and run the job in Informatica and validation part of insert logic and update logic in any DB. Taking very small example of two tables
The practical work which you greatly explain would be helpful
Ok- Will try to cover in future. Thanks for your question.
@@etlqalabs5048 YES SIR,,,,DO PRACTICALLY!!...THANKS ALOT SIR....YOUR VIDEOS ARE HELPING ALOT? FOR FRESHERS
Thank you sir... if possible please cover more ETL interview questions in upcoming videos..
Hello sir
Thanks for sharing this much knowledge free of cost.
i have a question, for validating SCD2 transformation- how can i do data validation in source and target, and we have only read permission in source and target system... Also could you elaborate on.. How staging area is actually present in database.
Thanks
Kiran
Very good information you’re sharing sir thank you.👍In real time do we testers have permission to insert or update or delete data?
It depends on project and management decision however you should have all access on database/source systems as part of testing and you should have a dedicated QA environment so you are free to manipulate the data in QA environment. I have seen some projects do not encourage a separate QA environment where QA and dev environments are shared , in that case you can ask DBA to do run the insert/updates
In real time how we are going to apply transformation logic based on STM document, pease help us with all types of transformation logic
Yes sir in real time project how to apply transformation logic based on the mapping document ,please take all types of transformation logic
Hello @ETL QA Labs, actually I didn't get that delete one row in target and run WF? after doing this what will be the impact on SCD2 and what we are actually validating?
THANK YOU HETU
Hi sir
Please make a dedicated vedio on transformation logic..
Very useful
Glad you think so!
at which stage "scd type 2" is applied in your project (Source to Landing OR Landing to Staging OR STAGING to DW.?) is question asked by interviewer. could you please answer..??
Scd 2 - slowly changing dimension type 2 where full history is preserved which means it's available for history tracking for the business, as you know any other stage in etl process is only for the purpose of etl processing and only non volatile space is Target so the answer should be it's applied for Target table in your case it should be staging to Target.
Query for null value checks ? And steps are same for scd2 ? Is there any additional steps for scd2 validation?
Please make video on how to apply transformation logic and please take all kind of transformation logic
Will plan to do a video on this soon. Hope you are looking this video to be specific from development perspective?
Hello sir, please do video on how to apply transformation logic and use all types of transformation logic, it will be usefull for us
Hi what can we do if no access to insert db records
Ideally QA should have a dedicated test environment which should have all the required access for QA to perform however in some situations and projects, a seperate environment is not feasible to provide so QA has to share with development team and hence access may not be available for QA so we should agree with such arrangement on the part of actions that dev team need to perform for QA to test as part of test planning and should be captured in the Test strategy document for smooth facilitation of testing life cycle and avoid any conflicts with development team
While performing this testing
:- when we perform insert , update , delete and run the workflow to check the result
but what will happen on actual table target and source that will get disturb ..
Or we have to do this on view table
The data in QA environment is only for testing purpose so it won't impact us anyways
@@etlqalabs5048 ohhk got it thanx 👍
Could you please make video on actually how to apply transformation logic ,please explain with all types of transformation logic
Hello Sir,
Thanks for this video. Really appreciated
Can you please tell me what are the expected technical interview question that ask in mangerial round for 3+ years exp in etl?
Yes sir make a video on this
Ok- will try to do a video in future.
if we have more than 100 clm than how can we perform minus query? write all clm names in select query is not a goop option. is there any alternate way?
wrt Update logic ETL Tester does having the chance to update the Target table?
Yes , Ideally you would perform all the testing in QA environment which is owned by QA team and you normally would be using a generic( common account for QA team ) account to peform all these insert , updates etc . In certain case if you don't have dedicated QA environment , you can request DBA to run all these DML statement before you run the workflows. Hope that clarifies
@@etlqalabs5048 Thank you so much for your valuable response.
@@etlqalabs5048 In Real-time Testers run the workflow ?
Developers create mappings n create workflows n runs those workflows then data is loaded into target with the business logics applied...
Developer environment - Etl process completed n data is loaded into target.
QA environment - replica of data what is loaded into target is given to QA environment for testing. So when we do update delete or insert that won't effect the actual target data. This is what I understood.
Please correct me if I'm wrong n Lemme know the actual process.
Thank you.
I heard that tester is only have access to select command. How can we insert, delete n update. ? I saw ur response for same question but didn't understand clearly like qa environment something. If u don't mind could you give some more clarity please. Thank you.
As a ETL TESTER you need read/write access on database in QA environment however UAT users testing in UAT environment usually have read only access on db.
Just to summaries - There are various environment for specific purpose related to each phase of SDLC -
1) Development environment for developing ETL job and performing Unit Testing
2) QA environment for testing by QA team
3) UAT environment for testing by Users ) end users before production release.
4) Production environment - final environment where actual business runs
Hope this clarifies some doubts
Thank you sir
How do we validate the hundreds of rows of scd2 transformation by writing querry...?
I mean i want to write a querry after loading data in source table such that it should validate the fromdate ,todate, flag status all in querry..??
It wil be very helpful plz guide on this
Sir can you please make video on transformation logic using mapping document.
Will plan to do a video on this soon. Hope you are looking this video to be specific from development perspective?
Can you give real time projects or company examples which use type 1 and type 2
Hi Sir,
I'm working in some company, I got project, they said I need to work on production support(like monitoring the jobs and scheduling the ODI loads)...
I'm not sure whether I will get knowledge on informaticaPC.
Could you suggest what are the difficulties we will get
Hello Sir, please explain effort estimation in ETL project
How to check surrogate key validation in SCD 2
Good afternoon sir, I was into project where I need to work on testing of SCD TYPE 3 table, could you please help me on it or make a video
Thank you