SD CArd and MMC card raw disk images

Introduction

This page contains raw disk images (compressed using bzip2) of SD cards and MMC cards. These images are for use by software developers, particularly those working on embedded systems, to help ensure compatibility. If you are developing an MP3/Vorbis player, a digital camera, a PDA, an answering machine, a data logger, or some other device that needs to use SD cards to store data, it is helpful to have examples of the various formats.

There are many permutations possible in file formats on the cards. Is the data stored on a SD or MMC card. Is the filesystem FAT16 or FAT32. Does the card hold a raw filesystem or does it have a partition table first. If there is a partition table, how many partitions are there and which partition has the bulk of the cards capacity.. Some manufacturers are rumored to store application software on a separate partition, for example. Cards can have different cluster sizes. It is even possible that cards can have sector sizes other than 512 bytes. Some cards might have the unused areas initialized to 0xFF instead of 0x00. Cards smaller than 32MB may store the sizes in different locations in the boot record.

These raw images can also be used to produce a table listing the characteristics of each card.

The images

SD Cards

• impact_128MB_SD128-604_onefile.image.bz2 - This image has one file ("termcap", 738418 bytes) stored on an otherwise virgin "Impact" brand 128MB SD128-604 SD Card. UPC 6-50590-11849-6. Measures 2.15mm thick. Impact appears to be an off-brand pseudonym or spinoff of Lexar.
• impact_128MB_SD128-604_revirginized.image.bz2 - same as previous card but manually restored to its virgin state based on a hexdump I had in my scrollback buffer. 250 bytes.
• rawimages/sandisk_SDSDB-256-A10_256MB_virgin.image.bz2 - Sandisk SDSB-256-A10 256MB SD Card
• sandisk_SDMRJ-128-A10M_128MB_RS-MMC_virgin.image.bz2 - Sandisk SDMRJ-128-A10M 128MB RS-MMC card. UPC 19659-02125 $50 at Radio Shack (only $16 on froogle). This is a Reduced Size (half length) MMC card that comes with an extender.
• lexar_SD128-267_128MB_virgin.image.bz2 - Lexar SD128-267 128MB SD Card (P/N 9379-128, Rev A, UPC 6-50590-10626-4)

bzip2 is the new standard for compressing individual files. The raw images need to be compressed to save disk space and bandwidth. Uncompressed files are likely to be 128MB to 2GB in size. Compressed files of virgin cards should be only a few hundred bytes. The compression using gzip is not as efficient as you might expect because it doesn't compress arbitrarily long runs of zeros into a single repeat instruction, so bzip2 generates a file 250 bytes long where gzip generates one 120K long on the same image. For these unusually redundant files, the old "compress" program actually did better than gzip producing a file about 20K long. The enhanced compression comes at a price: bzip2 is slower than gzip.

Note that bzip2 is not the same as pkzip, infozip, winzip, etc. It replaces gzip which used the same compression algorithm as zip but the former is a file compressor and the others are file archivers. They used the same compreession algorithm but different containers. bzip2 has neither the same compression algorithm or container format.

bzip2 is availible for Linux, Windows, Mac OSX, *BSD, and many other operating systems.

Howto use images

Compressed images can be uncompressed using bunzip2 which is part of the bzip2 package.

Images can be used by having your software open the raw image file instead of a raw device on unix. For embedded software, the software can be compiled on the desktop for testing purposes with the sector_read() and sector_write() functions modified to use fread() and fwrite().

To modify the disk images to contain actual files to manipulate, the disk images can be loopback mounted on Linux and similar OSes. Copying the images to a card can be a problem due to size differences but the right image could be used to restore a card to its factory state. If your software is not too picky, you can probably copy from smaller cards to larger cards for compatibility testing. Just be sure to save an image of the card first so you can restore it.

Not having images availible for testing can be expensive for manufacturers of embedded devices and the memory card manufacturers. Incompatabilities can cause.

Making Images

Images of virgin cards as shipped by the manufacturer are preferred. Images of cards that have been used, even if all files are erased, may be too large and may contain confidential informatin. Images of cards which have been reformated may be useful but do not reflect the state as shipped from the manufacturer. If reformatted, a full format must be done, not a quick format. Images should reflect all sectors on the card, not just those on the default partition. If you reformat the card to a different file system type (FAT16 vs FAT32) you may also need to change the partition type using "fdisk". Virgin card images should have "virgin" as the last card image.

A memory card reader is needed. The Stratitec USB2CR8 ("USB 2.0 10-IN-1 Card Reader") is availible for about $20 from Sam's Club and works under Linux, Windows, and macs. It says it needs linux 2.4.25+ but it seems to work under 2.4.18, at least for SD cards (I did get a read error on a smart media card).

Creating the disk image for a 128MB flash card takes about 3 minutes and 30 seconds using a USB 2.0 card reader. Larger cards will take proportionally longer. A 1GB card will take about half an hour.

If you do not have bzip2 but do have a way to read the raw image to a disk file, you can compress using zip archive format and send me the files and I can convert them.

Sending me the files: you can email the files as an attachment to whitis@freelabs.com. Be sure to include the text "[SD Card]" in the subject line so your message doesn't get lost in the sea of spam. You can also post the files on an HTTP or FTP server and send me the URL. Include the UPC codes and model numbers if availible. Include URLS to the specification sheet, data sheet, and purchase pages where availible. If you have the information from the card specific registers (OCR, CID, CSD) send that as well.

Linux, FreeBSD, OpenBSD, NetBSD, and other compatible operating systems

The device name used may vary depending on what other devices are present on your system. Try "/dev/sda", "/dev/sdb", "/dev/sdc", "/dev/sdd", etc. Check to make sure you got the right drive using "hexdump -C" or "od -a" . If you don't see "FAT16" or "FAT32" in the first screenfull or two of data, you probably have the wrong device. Unplug the card and try again, if you still get a dump of data, you have the wrong drive.

mkdir card_images
cd card_images
hexdump -C /dev/sdd | less       # check 
bzip2 manufacturer_size_model_virgin.image.bz2

Root priveledges are required to read/write the raw disk sectors unless you have changed permissions on the raw block device.

Windows

Creating a raw image of a disk in a standard format would be a trivial operation if Windows was a decent OS. It isn't. One of the problems with windows is that there isn't an easy way to read raw disk data. Expensive programs like Norton Ghost can be used to do this but they apparently do not produce standard (or even documented) file formats.

Note that under windows, you must have administrator priveleges to read the raw disk sectors.

Cygwin over Windows

Cygwin is a free set of tools that provide a Unix like environment on Windows systems. Installing cygwin is far easier if you have a fast net connection accessable from the target machine and a 2GB of free disk. It is possible to strip down the installation if you are just going to use it for this application; minimum packages required are "cygwin", "bzip2", "dd", maybe "bash", and "od" and any dependencies.

The device name used may vary depending on what other devices are present on your system. Try "/dev/sda", "/dev/sdb", "/dev/sdc", "/dev/sdd", etc. Check to make sure you got the right drive using "hexdump -C" or "od -a" . If you don't see "FAT16" or "FAT32" in the first screenfull or two of data, you probably have the wrong device. Unplug the card and try again, if you still get a dump of data, you have the wrong drive. For some reason, bzip2 and gzip won't read from the device directly on cygwin so you need to use dd.

mkdir card_images
cd card_images
od -a /dev/sda | less       # check 
dd if=/dev/sda | bzip2 >manufacturer_size_virgin.image.bz2

Bootable utility disk

A variety of free utility disks are availible that provide a standalone floppy or cdrom that runs a miniature version of unix. Some of these are intended to replace norton ghost but use standard file formats. Search freshmeat for "disk image" yields packages such as Partition Image, Webclone, g4u, Ghost for Linux. Others, such as knoppix, are intended as linux demo/rescue CDs. If your hard drive is formatted using the proprietary undocumented NTFS, then it is not feasible to write to your hard drive from these disks so you will have to store the files on a floppy drive or on a ramdisk and then copy to an MMC card (not necessarily the one being imaged) from the ramdisk.

# Ramdisk example
# substitute appriate drive  device for /dev/hdd below
mkdir /mnt/ramdisk
mkfs /dev/ram0        # if already in use, choose another ramdisk
mount /dev/ram0 /mnt/ramdisk
cd /mnt/ramdisk
bzip2 manufacturer_size_virgin.image.bz2  
# swap cards
mount -t msdos /dev/sda /mnt/ramdisk
cp manufacturer_size_virgin.image.bz2 /mnt/ramdisk/
umount /mnt/ramdisk

Roadkil's Sector Editor (almost works)

Roadkil's Sector Editor is a windows GUI program that can save a raw copy of a disk to an uncompressed file. Requires more user interaction than the command line based solutions. Note, however, that this only saves the contents of the FAT partition; the partition table (MBR) is lost; add "incomplete" to the file name. Download bzip2 to compress the file.

• Start Sector Edit
• Select the appropriate drive when prompted. On my system it is drive G: but your mileage may vary.
• Make sure you are still displaying sector 0 if you were browsing
• Select File -> Export Sectors
• Provide a filename, the program will add ".bin" at the end. For example: manufacturer_size_model_virgin_incomplete.bin
• When asked for the number of sectors, enter the number displayed on the bottom line of the main window (ie. "Current 000000000 of 000248221")
• Open a DOS window (on newer versions of windows this is something like Start -> All Programs -> Accessories -> Command prompt)
• change to the directory where the file was saved and compress the file. For example,

  
           cd "C:\Documents and Settings\Owner\My Documents"
           bzip2 manufacturer_size_model_virgin_incomplete.bin
        

VAIOsoft Recovery Manager (almost works)

This program looks like it can back up the whole disk, including MBR, into a file that is almost standard format if you use the Backup function with "Exact Duplicate, copies both used and unused space" except that it has 51 bytes of crap at the beginning. Leave the FMB extension on the filename so I will know it is a broken file. Does not support industry standard compression; download Bzip2 to compress file before sending. It has one additional bug: it forgets to save the last sector on the disk. Fortunately, that is likely to be all zeros in our application (but must be present to prevent seek errors when data is added). I can recover the raw data using:

   bunzip2 junk.image.bz2

Programs that don't work

One would expect to be able to use rawread/rawwrite to create disk images that could be compressed but that doesn't seem to work. The version found at http://www.pamarsystems.com/raw.html only works on floppy drives. The version at http://www.drivesnapshot.de/freedos/rawread.htm is only good for benchmarking. A program named MBRWork looks like it would almost work but it can only save up to 64 sectors. A program named directdisk is similar to roadkil's sector editor but it can only save one sector at a time and also can't read the MBR on USB drives. A program called "Sector Inspector" doesn't allow saving data.

Send me the card

If you send me the card (and a pre-franked return mailer if you want it back), I can read the image from the card.

Who should contribute and why

Card manufacturers

I recommend that manufactuerers make raw images of all SD/MMC cards you produce. While creating raw images of your cards carries a small initial cost (in the hundreeds of dollars) you are likely to save a lot of money in the long run by cutting technical support calls. A technician should be able to create raw images of every card you currently manufacture in a day or two. If you have a linux box availible (most likely in your engineering labs or a local web server) and a compatible card reader, it will only take the time a tech needs to collect one of each card plus one minute of time per card (the tech can do other work while the copy is being done). If you only have Windows computers availible, it will take a little longer to install cygwin. The cards can then be returned to the production line and sold. Or, if tech support already has one of each card, they can take them to the production line to reinitialize them to their factory default state, make a raw image, and then they don't have to go to the production line again to restore the cards in the future. If you want to save labor costs, you can donate or loan the cards to me. You can then put a link to this web page in your FAQ file or other suitable web pages. You may also want to put the images on your web site. Manufacturers who provide cards or images will receive acknowledgement here. If sending me any reduced sized cards, please include an adapter to full size.

Embedded developers

Developers of embeded devices are likely to purchase a number of cards for testing purposes. By cooperating, in the spirit of open source, developers can reduce the costs of development.

Online retailers specializing in flash cards

Increase goodwill and sales. If you provide links to your sales pages for each card, I will include those with the card images. Your company is in a good position to provide this data since you have cards from many vendors in stock and merely need to be able to open and reseal packages. Even if the package shows it has been open, you should be able to ship to customers afterwards with a sticker that says something like "package opened for compatibility testing". In the long run, the availability of this information may reduce your tech support costs as fewer devices will be manufactured with compatibility problems. Also, the files themselves and tables generated from them can be used by your tech support staff. Also, the information obtainable from the files can be used to increase the quality of your product pages making it more likely that your customers will find the information they need on your site to obtain a card that meets the compatability requirements of their devices. The files can also be used to restore product returns to their original state before shipping to a customer.

End users

End users who have a computer with Linux, *BSD, or Cygwin installed and a brand new flash card can produce the files in a few minutes. Once you have done so, the image provides you with a way to restore your card to its factory default state. If you found this page while searching for pre-purchase info, drop a bookmark and come back here when you have purchased your card but before you put any data on it.

Restoring the card

For linux, *BSD, and cygwin users, restoring your card to its factory state is a simple as:

   bunzip2 /dev/sda

Card Registers

I would also like to have copies of the CARD registers:

• OCR - Operation conditions register (32 bits)
• CID - Card identification register
• CSD - Card Specific Data register
• RCA - Relative card address (dynamic, not needed, not avail in SPI mode)
• DSR - Driver Stage register (not needed, not avail in SPI mode)

Other card formats

I am most interested in SD and MMC card formats as I think these are most appropriate for designing into embedded systems. I will also accept images or cards in other formats. I have a reader which is capable of reading Compact Flash Type I, Compact Flash Type II, Smart Media, Secure Digital (SD Card), Microdive,, XD Compatible MMC card, Memory Stick, Memory Stick Pro. Reduced size SD cards or Memory stick Duos will require an adapter and reduce size MMC cards will require an extender.