(Re-upload) What is Shuffle | How to minimize shuffle in Spark | Spark Interview Questions

Hi Friends,
In this video, I have explained about Shuffle and given the ways to minimize the shuffle.
github.com/sra...
Please subscribe to my channel for more interesting learnings.

[@sravanalakshmipisupati6533]

Hi Friends, Reuploaded the video with volume enhancements. Please check and let me know if this is fine. Thanks.

[@rahulpandit9082]

Nice content, Thanks a lot Sarvana..

[@sravanalakshmipisupati6533]

Thank you 😊 👍

[@udaypratapsingh2245]

Thanks for this wonderful video! If you explain Lil more about clusterbykey point .

[@rahulpandit9082]

Read about Bucketing in Hive..

[@sravanalakshmipisupati6533]

@Uday Pratap Singh - Please checkout - th-cam.com/video/pZMTv0090rI/w-d-xo.html for cluster by key in Hive.

[@satishmajji481]

Can you explain how to pick the correct/optimal number of partitions dynamically while processing data?

[@sravanalakshmipisupati6533]

Spark has its own mechanism to have dynamic partitioning. If you want to overwrite then you can analyse the data and then give appropriate partition number.

[@satishmajji481]

@@sravanalakshmipisupati6533 How to analyse and decide the appropriate partition number? Can you explain please?