HDFS - Namenode Recovery (role of editlogs, fsimage and secondary namenode)

[@itversity]

For any technical discussions or doubts, please use our forum - discuss.itversity.com
For practicing on state of the art big data cluster, please sign up on - labs.itversity.com
Lab is under free preview until 12/31/2016 and after that subscription
charges are 14.99$ per 31 days, 34.99$ per 93 days and 54.99$ per 185 days

[@namanahg3748]

Hi sir.. when namenode is crashed and unable to make service up!! In this scenario hdfs will be in read only mode.. how we can bring latest FSImage from SNN to namenode? Please do needfull

[@dhruvaj9845]

Thanks for your details explanation. I have one query here. When name node crashed or formatted manually , Latest FSIImage is created by merging latest Edit logs. As you said , FSI image does not store the block location, How FSI Image details are mapped with block locations when data nodes are started registering block with Name node??

[@ajajtgkg]

FSImage has block names.
DataNodes have blocks.
Each datanode will send the block report to NameNode as when sending heartbeat as explained in last video.So this is how namenode will map block names with block location.

[@mnerkar99]

Usually FSI image does not have block location but when NN crashes and sn takes over after leaving the safe mode all datanode sends blockreport to NN within that span of time so it recovers the physical location of the block size on the cluster

[@beinghuman2187]

Will the secondary NN has latest FSImage? I though standy will have ...

[@praveenchandna4607]

In the above video we have discussed the restore using the fsImage and edit log. In this section we have discussed the Secondary name node . Are you talking about this as per Hadoop 1 not Hadoop 2. As per my understanding in Hadoop2(Yarn) we have active name node.
Can you please help.

[@itversity]

Whether it is Hadoop 1 or 2, in the typical configuration you need to have secondary namenode

[@satycse06]

Hi Sir,
As you said block location is not store in FSimage it store in RAM.So if RAM will corrupt or some hardware error come then how it will restore the block location? will it restore applying chekpoint?

[@itversity]

Please use discuss.itversity.com as I am not continuously monitoring TH-cam comments

[@Vamsh]

Hi sir,when namenode fails,where will be the checkpoint created?is it in secondary name node?

[@karthiknedunchezhiyan7935]

yes it will be created on the secondary name node and it send back new fsimage to namenode once the checkpoint process is competed..