Spark Interview Question | Handle Data Skewness in Apache Spark | LearntoSpark

[@ramesh-bigdatalearner3223]

Salting technique we can apply to avoid the skewness when we know data skew before we running the job.But in real time if job is failed due to skew how to handle it? and how to avoid such skew failures in production?

[@vijeandran]

Thanks Azarudeen, How can we do salting technique in Pyspark for data skewness?

[@srinuch9531]

Thanks for your efforts in making videos on spark. It would be helpful if you show sample code for these options for better understanding

[@AzarudeenShahul]

Great Suggestion !!! I will try my best to bring out some sample code

[@monangiavinash5653]

please explain us the salting technique in detail

[@AzarudeenShahul]

I will try for a video with Demo on salting alone. Provide all your support :)

[@gurumoorthysivakolunthu9878]

Hi Bro... This is Ultimate... Very well explained...
I am a begginer... So I couldn't understand...
1.. What is single dominant partition id mean...? Why partitions with single dominant partition id cannot be repartitioned...
2. Also, what is multiple dominant partition id...
3. How to change in the source itself... Partitions are made by spark - based on partitions size, right... How can we specify one or more columns for partitioning... What is the command for that...
Please help me in these doubts...
Thank you, bro...

[@krishnarupeshnagubandi596]

Thanks for your explanation. Can you please provide this with example. I mean code and data.

[@AnandSharma-yb3cv]

please show it through codes

[@SpiritOfIndiaaa]

How come repatriation would work ??

[@ashwinc9867]

Please explain solution with a real timecode scenario

[@sravankumar1767]

nice explanation........

[@awanishkumar6308]

Hi Shahul i have sent the screenshots as per your description

[@AzarudeenShahul]

Will check and reply you Awanish

[@minalmoon4605]

Expected more detailed explanation 🙄

[@swagatikaskavita-ekkoshish9707]

Can you pls add English subtitles as well to all your videos