Quickly create an img file from a Raspbian SD card on OSX

Introduction

Bottom line (given that your SD card is /dev/disk2):

$ ddCount=$(sudo fdisk /dev/disk2 | grep Linux | awk '{printf "%i", ($11+$13)*512/1024/1024 + 1}')
$ diskutil unmountdisk /dev/disk2

Unmount of all volumes on disk2 was successful

$sudo dd if=/dev/rdisk2 bs=1m count=$ddCount | gzip > myimage.img.gz

Featured image

Long story:

Sometimes you need to make a backup of an SD card with your favorite installation on it. This recipe will show you how to do this in the most efficient way on Apple, Mac OSX. This is of course not a good replacement of making notes and automating your installations. But for the times when you really need to make an image copy of your SD card, it is helpful.

Most recipes online will simply tell you to use “dd” to make an image of the whole card. This will create an image that is the size of your SD card, which is rarely necessary. A base installation of Raspbian Lite clocks in at around 1 GB, and the base partition created when you first install Raspbian on an SD card is “only” 1.7GB. If you can keep the root (/) partition at this size until you make your backup, you save both time and space on your harddrive. Most people use larger SD cards than 1.7GB nowadays, often 32GB - 128GB. I personally try and get as small SD cards as I can, and the current sweet spot for price/performance is found around 16GB. That will change.

The first thing that normally happens when you start up your Raspberry Pi, is that rasbpian will resize your root partition from 1.7 GB to the size of your SD card. In the good old days you had to do this yourself though the use of the “raspi-config” command. In the later releases, the resize is done at the first boot.

So, the basic recipe for optimizing

  • Create your initial installation (install Raspbian on an SD card)

  • Make sure that you do not expand the root filesystem at first boot (edit /boot/cmdline.txt)

  • Do your stuff/set up your environment

  • Make a backup of the SD card

  • Get the size of the partitions

  • Use dd to get only the good parts

Initial installation

  • Download the image
  • Unzip it
  • dd the image to your SD card

Be cautious. Make sure you use the correct “/dev/rdiskX” file, as you WILL destoy the destination, replacing it with your image. You figure out the correct disk name by using “diskutil list”. When you “dd” the image, use the “/dev/rdiskX” device, NOT the “/dev/diskX” device. The latter will be much slower.

$ diskutil list
...
/dev/disk2 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *16.1 GB    disk2
   1:             Windows_FAT_32 boot                    43.7 MB    disk2s1
   2:                      Linux                         1.7 GB     disk2s2

$ sudo dd if=2017-09-07-raspbian-stretch-lite.img of=/dev/rdisk2 bs=1m

Root filesystem expansion

The root filesystem expansion will happen at first boot. It is initialized through a script called by _init=/usr/lib/raspi-config/init_ressize.sh in the /boot/cmdline.txt. After it has resized the filesystem, it removes the init= clause and reboots. You should remove this from the cmdline.txt file before booting the first time, in order to keep the root filesystem as small as possible.

$cat /Volumes/boot/cmdline.txt 

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=a8790229-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh


#--- edit the file and remove init=...

$cat /Volumes/boot/cmdline.txt 

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=a8790229-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet

Whenever you would like to expand the filesystem, you can do so by running “sudo raspi-config” on the command line of your raspberry pi, and resize it.

Setup your environment

This is where you do your thing. Install some software, play around. Go wild!

Backup SD card

This is where the magic happens. Or at least the essence of this blog entry. You will need to figure out the last block of the linux partition (your root partition), and only create an image until that point.

  • Figure out the SD card partition layout
  • Calculate the number of MB you should read from the SD card
  • Create the image, using “dd” and a block size of 1MB

You get this from the partition table, which you display by using the “fdisk” command.

$ sudo fdisk /dev/disk2
Disk: /dev/disk2 geometry: 1955/255/63 [31422464 sectors]
Signature: 0xAA55

         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
 1: 0C    0 130   3 -    5 210  42 [      8192 -      85405] Win95 FAT32L
 2: 83    5 220  24209 202  59 [     94208 -    3276162] Linux files*
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
 4: 00    0   0   0 -    0   0   0 [         0 -          0] unused      

Here you can see that the linux partition starts at block 94208 and goes on for another 3276162 blocks.

Use the “hdiutil imageinfo” command to get the block size:

$ sudo hdiutil imageinfo /dev/disk2 | grep block-size
 block-size: 512

You now have all info needed to calculate the end of your SD card partitions, numMB = round((94208 + 3276162) * 512 / 1024 / 1024) + 1. The +1 is for when the number of blocks does not and at an even MB boundary. I throw in one more MB for good measure. You could in principle skip part of the calculation, if your dd-command would use the disk block size (bs=512) instead of “bs=1m”, but the copy would take forever to complete. In my example, I will read 1646 MB from the card, which is a few kB too much, but you can skip the calculation and use 1700, 1800, or 2000 MB as well without over shooting too much.

$ diskutil unmountdisk /dev/disk2 
$ #--- jessie: sudo dd if=/dev/rdisk2 bs=1m count=1646 | gzip > myimage.img.gz
#--- stretch
$ sudo dd if=/dev/rdisk2 bs=1m count=1800 | gzip > myimage.img.gz

To write this image back to your SD card:

$ diskutil unmountdisk /dev/disk2
$ gzcat myimage.img.gz | sudo dd of=/dev/rdisk2 bs=1m

Edits:

  • Mats Karlsson noted that one can use “xz -e9v” instead of gzip, to produce a file that is ca 30%
  • malu@kmg-mcp001.local:/temp/raspberry/raspiMake $ls -latrh my* -rw-r–r–  1 malu  wheel   347M Sep 19 11:48 myimage.img.gz -rw-r–r–  1 malu  wheel   225M Sep 19 11:58 myimage.img.xz

Raspbian Stretch (September 2017) is just a bit larger than Jessie, add a few MB toyour dd command (I use 1800)