Thursday, 8 January 2015

Making your own SD Card clone army

This term I'm going to a few weeks of Raspberry Pi activities with my Yr 5-6 CodeClub. They all really enjoyed the physical computing aspects of the DIY Gamer project from last term and asked to do more hardware-related stuff.

Thanks to various donations and competition wins, and by raiding the family collection, I now have enough Pis for 10 children. I tested the school Computing Suite infrastructure with and end-of term Pi Minecraft edition session (with the younger CodeClub) before the holidays so I'm reasonable confident that I have everything I need.

I'd like the Pis to all run the latest Raspbian UI, but that will mean updating all the SD cards I burned in December. I could just lets the children do the update during the session, but to be honest, watching apt-get upgrade running is not that much fun or all that informative (I'll get them to perform some updates, but I don't want them to have to wait for a whole load of packages to install).

When I burned the cards before Xmas, I just used a single USB reader/writer, doing all 10 over the course of a few days using dd. Invariably it took longer than necessary because I'd leave one copying job running and then forget about it. It seemed like the whole process was ripe for automation, so I purchased a couple more cheap readers/writers from eBay and dug out my USB hub.


Because I have some full size SD cards and some micro SD cards (with adapters to allow use in model Bs), I have two reader/writers for each size of card.

The excellent enhanced version of dd -  dcfldd - allows you to create multiple output copies of whatever input you give it, and also has some extra options to monitor progress.  You can install it using

apt-get install dcfldd

 A few fairly un-scientifc tests suggested that performance was not significantly degraded by writing multiple copies: a 4 GB image file took  20 minutes when writing 4 copies (2 SD and 2 micro SD).  That means I can create a full set of 10 in under an hour!

It's very simple to run from the command line:

dcfldd if=ImageFileToBeCopied.img of=/dev/sda of=/dev/sdb of=/devsdc of=/dev/sdd sizeprobe=if

The 'sizeprobe' argument is optional but tells dcfldd to display a helpful percentage progress value which is quite helpful in seeing how far the copying has got and how much time remains.

Finally, because I'm toying with the idea of using my UnicornHat to display progress, I also wrote a simple Python wrapper to log events to a file.
import subprocess 
tool = subprocess.Popen(['dcfldd', 'if=CodeclubJan15.img', 'of=/dev/sdd','of=/dev/sdc', 'of=/dev/sdb', 'of=/dev/sda', 'statusinterval=4092', 'sizeprobe=if'], stderr=subprocess.PIPE) 
tee = subprocess.Popen(['tee', 'log_file'], stdin=tool.stderr) 
tool.stderr.close() 
tee.communicate()

6 comments:

  1. Sounds like you could use http://pi-ltsp.net

    ReplyDelete
  2. Thank-you, thank-you, thank-you. Was dreading the cloning of 6 8GB SD cards in preparation for Cotswold Raspberry Jam's Minecraft tutorial session (Sat 31 Jan, Cheltenham, free tickets still available folks!). Took forever last time. Genuinely didn't realise that the ever-excellent dcfldd utility could take more than one OF output file parameter. I've already got a powered USB hub, so off I go to Poundland to get half a dozen USB SD adaptors.

    ReplyDelete
  3. I wrote my own batch file for this. Bash shell isn't too powerful, but since you know python, I imagine you can do better than my own bash script hack. Do you really need to install another program?

    ReplyDelete
  4. @harry You can easily do it with dd... But you will have to read the file to write N times the number SD cards. dcfldd allows one input, multiple outputs! Therefore, you read once for the multiple SD cards.

    ReplyDelete
  5. What's the best way to do this to a smaller SD card? I want to clone Raspbian and my app to multiple SD cards, but I developed on a 64Gb card in a Pi4 and don't need such a large card for the clones. I'd like to go with 16Gb SD cards as all I need is the OS and my app. Do I need to resize the image first, or is there a way to copy to a smaller card in one go?

    ReplyDelete