Interviewer (Chandrali) definitely have deep understanding of topics. If we can get some interview videos for senior post where she is giving the interview for scenario based question that will be more helpful.
Data governance includes access permission for a user or a group , we can restrict user to read any specific location or we call it data masking ... Second thing unity catalog also store metadata
If repartition happens at driver, do u think entire gbs of data travell to driver from all executors and then driver divide and give it to all executors... Then your understanding about spark is wrong.....
Interviewer (Chandrali) definitely have deep understanding of topics. If we can get some interview videos for senior post where she is giving the interview for scenario based question that will be more helpful.
Please continue python and Sql series
Data governance includes access permission for a user or a group , we can restrict user to read any specific location or we call it data masking ... Second thing unity catalog also store metadata
If repartition happens at driver, do u think entire gbs of data travell to driver from all executors and then driver divide and give it to all executors...
Then your understanding about spark is wrong.....
Insightful! thank you chandrali and abhirup
Her question on rank and dense rank was not complete. Rank and dense rank but based on what was not told to the candidate.
Data lake vs delta lake
Unity catalog
Data profiling
Data governance