Great tutorial! I'm new to working with SD cards and need to create a bootable image of one to send to my team and was struggling with read errors. It seems like there's more reddit threads than real docs on how to do this. You're a lifesaver thank you.
I just want to write an OS image to an sd card, but etcher keeps threatening me that this will erase my system drive. And the card isn't in /sda or /sdb anything. This is getting complicated, and I don't want to risk my system!
thanks for the info. you presented the information with the patience of a great teacher, it shows that you are well informed in the subject matter and wanting to share your knowledge. a huge thumbs up
NO! Use this form instead: sudo dd if=/dev/mmcblk0 of=sdcard.img bs=512 conv=noerror,sync status=progress. Not all SD cards are the same and one brand 8GB SD card may actually be smaller than another brand 8GB card. If you use a large block size and subsequently put the image on a new SD card you can get a not enough space error. Also, the large block size can result in extra random bytes from RAM added to the end of the image which are not part of the application you are backing up. A subsequent imaging of a new card may result in the application not running because the app doesn't like what's in the extra bytes - I've had this happen and fixed it with the smaller block size. Also do use the "status=progress" option. Large SD cards take a while to back up to an image and you don't want to have the system appear to be hanging - might cause you to kill the process needlessly. The smaller 512 block size will take longer to image than the 4M block size but it's worth the wait to get a good image. Also, probably best to test the image is good by writing it to another SD and trying the new SD card in the device.
Dear , Another best route is to use gparted app to resize the sdcard so that you get to compress data and then use standard sd card copier to copy your Raspi image.....This hardly requires much effort.....
There is one important step in restoring the .img file to another SD card which this video overlooks. The host did not need to do this because he did not take the SD card out or put in a new one. So His SD card is already unmounted. If you are doing this for real, you will need to unmount the dd card you just inserted in the computer using the umount cmd as before. He should have mentioned this and he should add this info to his notes.
I can't seem to post where to find PiSafe. It is on that opensource developers website that starts with git and ends with hub. Then RichardMidnight then pi-safe. - peace
If you are super careful selecting your devices you should be able to use the dd command to copy directly from the SD card you are copying from directly to the blank SD card.
Exactly what I am trying to do, but I do not have any idea how the command would be...for the purpose fo the video I guess this youtuber should have shown first this process, which is straight forward to get the solution, shouldn't he?
QUESTION: Restoration to a new card --- size must be the same or larger than the original. Only older clone programs, metadata from the original used to be copied to the image as well. When that image was restored to new media and that media was physically larger than the original, the metadata would tell the computer OS that the new media had been transformed to (logically) the same size as the original media, rendering any larger data space unusable. Q: does the dd command do the same here? or will the full data space still be available after the "restoration" function is complete... THX
I prefer to squash it: mksquashfs empty_folder Your_img.img -p 'Your_img.img f 444 root root dd if=/dev/Your_dev bs=4M' requirements: directory named "empty_folder" peresent in current working directory. It is not very speedy but does serious crunching, cpu dependant.
Very clear and helpful! However, I'm experiencing a problem with not knowing the device identifier of the SD Card reader (running Ubuntu under Win10). 'df -h' shows the filesystem as 'D:' when mounted, and not /dev/, which isn't recognized by 'dd if=D:' anyone have a clue how to solve?
Hello Jay, Could you please make a video on the best way to move a /home partition from /dev/sda to a newly added ssd, /dev/sdb. I need to do this because my root partition is running out of free space. Thanks. Keep up the good work.
Thanks for this tutorial, very useful! Just a question, I noticed your dd command is much smaller when you pipe to gzip. Is this ok? and if yes, why use the extra option of block size followed by sync when you use it by itself?
You could use the other options before the pipe ( | ), but not the " ; sync " part. The blocksize option (bs=) tells the dd command to read and write with a specific amount of blocks. Useful when writing to a block device. But if you pipe it to gzip you don't write to a block device so you can view it more as a "stream". The sync command is of influence to syncing the blockdevices, which is again not applicable for a pipe to gzip. Cheers! Happy dd'ing :-D
Great video! I have an SD card that has some damage (as you noted) - will the dd command skip them and write nothing in their place? Or write as best it could? I'd really like to get it to write exactly what it read to minimize repair.
Great video tutorial man!!! Just what the doctor ordered :) I noted some differences however when running from the command line on my Mac OSX Catalina. I wanted to make an iso/img file of my RPI3B+ SD card so I used the 'sudo dd if=/dev/disk3 of=/Volumes/'Data Drive'/rpi3b_sd.img bs=4; sync' and it is in fact still running as I write this comment. (OR I HOPE IT IS...). It gave me fits when I enter the 'bs=4M' it said invalid value so I then tried 'bs=4MB' and that crapped out, and finally I used 'bs=4'. Have you done this on Mac OSX v10.14 or later??? After all, at the root from the command line so to speak, it is a unix/linux based OS. ALSO... 'lsblk' DID NOT work from my Mac's command line so I used 'diskutil list' to try to figure it out... Look forward to your reply with any hints on how to better do these steps on Mac OSX Catalina... Thanks again, Tim
So if I have a 128gb image and I restore to a 256gb SD card, is there a way after the fact to increase the partition size to the remaining free disk space on the 256gb sd?
Thanks for your video, but let me give some notes about: 1. By unknown reason your last videos hang Apple TV and send it to reboot while playing your video including this one. Probably the program you encode your video is creating not fully compatible video files with old Apple devices. The videos from others play well. Your old ones playing well also. Very strange. 2. To backup SD cards is best way via ‘bsdtar’ utility which save permissions of files and even links. Sure, in this case partitions must be restored manually, but this way allows backup from bigger to smaller SD card. Thanks.
How long is this supposed to take? I am on a Linux Mint computer and it has been all night long and it is still in process. One change I did have to make was from "bs=4M" which kept coming up as invalid to "bs=4" and that seemed to work. But now we are going on 8 hours in and still the process is not yet complete. It was a 64gb micro sd card put into a slot SD card reader. It was nearly full. Any input here is appreciated.
UPDATE: It completed successfully after 12 full hours. All good! :) Btw, I was able to view progress by searching the file name I made on my computer and refreshing to show the file size growing until it reached the full 64 gb.
Hi that is a fantastic explanation!! Great job, many thanks. Would be nice if someone out there cold do that in German Language (I am from Austria :-) )Many thanks!!
You probably know that compressing a stream, such as the output of dd, is not as efficient as compressing a complete file. The difference in size of the compressed file is more noticeable when the compressible portions of the stream are not contiguous. But compressing a stream is much faster than creating a file and then compressing that. Thanks for the video, though, it's a neat way of duplicating a filesystem.
Be careful about SD sizes! All are not the same between brands! For example, an 8Gb SanDisk is most likely NOT the same exact size as an 8Gb Samsung. This could result in a dd copy FAIL depending on the direction of the transfer! Happened often for me. ☠️
Great tutorial! I'm new to working with SD cards and need to create a bootable image of one to send to my team and was struggling with read errors. It seems like there's more reddit threads than real docs on how to do this. You're a lifesaver thank you.
This is the best video on drive backups I have ever seen. Thanks a ton for this.
Finally I found someone who doesn't click and talk too fast. An you know what your doing and explain everything ty
I just want to write an OS image to an sd card, but etcher keeps threatening me that this will erase my system drive. And the card isn't in /sda or /sdb anything. This is getting complicated, and I don't want to risk my system!
thanks for the info. you presented the information with the patience of a great teacher, it shows that you are well informed in the subject matter and wanting to share your knowledge. a huge thumbs up
Brilliant video, clear and concise with a few good extra linux tips along the way. I have just subscribed to your channel.
NO! Use this form instead: sudo dd if=/dev/mmcblk0 of=sdcard.img bs=512 conv=noerror,sync status=progress. Not all SD cards are the same and one brand 8GB SD card may actually be smaller than another brand 8GB card. If you use a large block size and subsequently put the image on a new SD card you can get a not enough space error. Also, the large block size can result in extra random bytes from RAM added to the end of the image which are not part of the application you are backing up. A subsequent imaging of a new card may result in the application not running because the app doesn't like what's in the extra bytes - I've had this happen and fixed it with the smaller block size. Also do use the "status=progress" option. Large SD cards take a while to back up to an image and you don't want to have the system appear to be hanging - might cause you to kill the process needlessly. The smaller 512 block size will take longer to image than the 4M block size but it's worth the wait to get a good image. Also, probably best to test the image is good by writing it to another SD and trying the new SD card in the device.
why conv=noerror...?
@@Drjacques So dd doesn't stop on an error. You want dd to dump the whole SD content.
You can use PiSafe to easily create a compressed image of your SD card or USB stick. Then restore it to a different device any time you like.
Dear , Another best route is to use gparted app to resize the sdcard so that you get to compress data and then use standard sd card copier to copy your Raspi image.....This hardly requires much effort.....
There is one important step in restoring the .img file to another SD card which this video overlooks. The host did not need to do this because he did not take the SD card out or put in a new one. So His SD card is already unmounted. If you are doing this for real, you will need to unmount the dd card you just inserted in the computer using the umount cmd as before. He should have mentioned this and he should add this info to his notes.
Thank you. This is just what I need to back up my gps memory card.
I can't seem to post where to find PiSafe. It is on that opensource developers website that starts with git and ends with hub. Then RichardMidnight then pi-safe. - peace
If you are super careful selecting your devices you should be able to use the dd command to copy directly from the SD card you are copying from directly to the blank SD card.
Exactly what I am trying to do, but I do not have any idea how the command would be...for the purpose fo the video I guess this youtuber should have shown first this process, which is straight forward to get the solution, shouldn't he?
You did a great job explaining this. Thank you!
Thanks for this great video. Could you please make one when using a "usb flash drive"? Thanks
Thanks so much! It was so helpful and precise
Great Video. It's full of information and it's nice when you show some another possibility to do the same thing. Thanks a lot.
Great tutorial, great teacher. Thanks you!
This was very well explained. Thanks heaps for your time.
Excellent tutorial. And it's complete and very accurate. Great job LearnLinuxTV. :)
Great tutorial! Thank you!
QUESTION: Restoration to a new card --- size must be the same or larger than the original.
Only older clone programs, metadata from the original used to be copied to the image as well.
When that image was restored to new media and that media was physically larger than the original, the metadata would tell the computer OS that the new media had been transformed to (logically) the same size as the original media, rendering any larger data space unusable.
Q: does the dd command do the same here? or will the full data space still be available after the "restoration" function is complete...
THX
The partition will be as large as the image was. The remaining space can be used when you enlarge the partition or if you create a second one.
my dude! thanks for sharing this ! I really appreciate it !
Thanks. Very useful tutorial
Great job Sir! Thank you very much.
Very helpful. Thank you.
Great stuff! Thank you.
SUPER HELPFUL SIRE. THANX A LOT.
Very Cool! ? How to make automation scripts with Racket and SBCL for Linux?
Stellar content!
Tried this process and after restore SD card won't boot my R Pi :( any clues why?
great video!
14:50 can you help me which video that you show how to restore the sd card?
in my case, i just want to only read file in sdcard.img and save them
Linux is such a black art. It is like Windows 20 years ago. Love the way the OS ignores telling you it is actually doing what you asked.
I prefer to squash it:
mksquashfs empty_folder Your_img.img -p 'Your_img.img f 444 root root dd if=/dev/Your_dev bs=4M'
requirements: directory named "empty_folder" peresent in current working directory.
It is not very speedy but does serious crunching, cpu dependant.
Very clear and helpful! However, I'm experiencing a problem with not knowing the device identifier of the SD Card reader (running Ubuntu under Win10). 'df -h' shows the filesystem as 'D:' when mounted, and not /dev/, which isn't recognized by 'dd if=D:' anyone have a clue how to solve?
Hello Jay,
Could you please make a video on the best way to move a /home partition from /dev/sda to a newly added ssd, /dev/sdb. I need to do this because my root partition is running out of free space. Thanks. Keep up the good work.
Good stuff!
Excellent tutorial.
so does this help me back up my current OS of my raspberry pi4
Thanks Very Cool!
thanks.. it worked perfectly.
Thanks for this tutorial, very useful! Just a question, I noticed your dd command is much smaller when you pipe to gzip. Is this ok? and if yes, why use the extra option of block size followed by sync when you use it by itself?
You could use the other options before the pipe ( | ), but not the " ; sync " part. The blocksize option (bs=) tells the dd command to read and write with a specific amount of blocks. Useful when writing to a block device. But if you pipe it to gzip you don't write to a block device so you can view it more as a "stream". The sync command is of influence to syncing the blockdevices, which is again not applicable for a pipe to gzip.
Cheers! Happy dd'ing :-D
Thank you for your tips...at the same time i m to learn inglish.
Great video! I have an SD card that has some damage (as you noted) - will the dd command skip them and write nothing in their place? Or write as best it could? I'd really like to get it to write exactly what it read to minimize repair.
Thank you, Jay. GUI methods are always less stable to use than the CLI original tool.
Great video tutorial man!!! Just what the doctor ordered :) I noted some differences however when running from the command line on my Mac OSX Catalina. I wanted to make an iso/img file of my RPI3B+ SD card so I used the 'sudo dd if=/dev/disk3 of=/Volumes/'Data Drive'/rpi3b_sd.img bs=4; sync' and it is in fact still running as I write this comment. (OR I HOPE IT IS...). It gave me fits when I enter the 'bs=4M' it said invalid value so I then tried 'bs=4MB' and that crapped out, and finally I used 'bs=4'. Have you done this on Mac OSX v10.14 or later??? After all, at the root from the command line so to speak, it is a unix/linux based OS. ALSO... 'lsblk' DID NOT work from my Mac's command line so I used 'diskutil list' to try to figure it out... Look forward to your reply with any hints on how to better do these steps on Mac OSX Catalina... Thanks again, Tim
So if I have a 128gb image and I restore to a 256gb SD card, is there a way after the fact to increase the partition size to the remaining free disk space on the 256gb sd?
awesome !!
Thanks for your video, but let me give some notes about:
1. By unknown reason your last videos hang Apple TV and send it to reboot while playing your video including this one. Probably the program you encode your video is creating not fully compatible video files with old Apple devices. The videos from others play well. Your old ones playing well also. Very strange.
2. To backup SD cards is best way via ‘bsdtar’ utility which save permissions of files and even links. Sure, in this case partitions must be restored manually, but this way allows backup from bigger to smaller SD card.
Thanks.
what is the purpose of setting block size? is it necessary if it is the same for reading and writing?
I've never found it to be necessary, it's become muscle-memory to do this.
Super tip! this makes it easy to make an image of my linux system drive also! Thanks man! :-D
@jay please do us a solid on android 9+ rooting
How long is this supposed to take? I am on a Linux Mint computer and it has been all night long and it is still in process. One change I did have to make was from "bs=4M" which kept coming up as invalid to "bs=4" and that seemed to work. But now we are going on 8 hours in and still the process is not yet complete. It was a 64gb micro sd card put into a slot SD card reader. It was nearly full. Any input here is appreciated.
UPDATE: It completed successfully after 12 full hours. All good! :) Btw, I was able to view progress by searching the file name I made on my computer and refreshing to show the file size growing until it reached the full 64 gb.
Is there a way to move (backup and restore) a windows to Linux virtual box ?
Yes, you can do that with Clonezilla, but you'll need to run Sysprep on the Windows installation first.
Thanks a lot! This saved me!!!!
I like your shirt, I watched the movie it was awesome ! :D
This is huge
очень актуально для меня. спасибо
Hi that is a fantastic explanation!! Great job, many thanks. Would be nice if someone out there cold do that in German Language (I am from Austria :-) )Many thanks!!
You probably know that compressing a stream, such as the output of dd, is not as efficient as compressing a complete file. The difference in size of the compressed file is more noticeable when the compressible portions of the stream are not contiguous. But compressing a stream is much faster than creating a file and then compressing that.
Thanks for the video, though, it's a neat way of duplicating a filesystem.
Be careful about SD sizes! All are not the same between brands! For example, an 8Gb SanDisk is most likely NOT the same exact size as an 8Gb Samsung. This could result in a dd copy FAIL depending on the direction of the transfer! Happened often for me. ☠️
Dude. There are many progs/utils that do this for people...
He already explained why he's using the terminal for this.
The problem with linux is someone has to make a video/ make a guide to show you how to do everything on linux lol.
"super secret password"
ha,your password is 1234
Very nicely explained, thanks!