Open Source Society

Better Clones and Hard Drives

The 100GB hard drive seemed more than enough when I first bought my laptop, but as time and music collections go on, I’ve begun to run out of space. The solution, of course, is to purchase a bigger hard drive. But do I want to go through the headache of completely reinstalling my system, remembering all the settings and patches I’ve applied, remembering all my passwords, and moving all 70GB of music and personal files to the new drive? This could literally take days, and I don’t have that kind of time. So what is the better solution? Clone the hard drive.

Some of you may already be familiar with commercial cloning programs like Norton Ghost or Acronis True Image. These programs are okay, but they lack a few features I think are essential to quickly and safely duplicating a drive. For starters, it’s usually an educated guessing game with either of these programs as to which drive should be the source and which should be the destination. Guess wrong, and you end up sending all of your data into the ether forever. If you’re motivated enough, you can yank both drives and track down their device IDs, but even then you don’t always have the ability to view the device IDs in the previously mentioned applications. If you happen to be cloning one drive to another one of identical size and manufacturer, which does happen in cases of system migration or failure recovery, you are taking a shot in the dark as to which drive is which.

Enter dd, a very small, unassuming application which comes as part of the core utilities on any Unix or Linux system. Dd’s primary role in life is copy files bit for bit from one place to another, which sounds useless until you recall that Linux can treat a partition or even a whole storage device (like a hard drive for instance?) as a pseudo-file. This means we can take an input file (the original hard drive), and copy it bit for bit to the output file (the new hard drive). Since this is a command line tool, we have all the other command line tools at our disposal to verify that the devices we’re working with are the correct ones.

To start with, let me point out that you do not want to clone a drive this way while booted from the source drive. This will cause an unending list of failures and eventually destroy the source drive. We need to boot into a virtual environment, like the kind provided by a LiveCD, and leave both drives unmounted during the cloning process. Since this is the case however, it means we can clone a drive no matter what operating system is installed on it. Dd copies bit for bit, and doesn’t care about the actual structure or format of the data it’s copying, so you can boot into a liveCD like SystemRescueCD and clone Windows, MacOS, or Linux drives all the same way without any problem.

For those of you who aren’t already Linux gurus, here is a bit of background on how Linux treats devices, particularly hard drives. There is a folder on most Linux systems called “/dev” which holds a number of files. There are entries here for everything from the text terminal you read and type with, modems, network cards, sound cards, video cards, hard drive controllers, CD or DVD drives, and of course, hard drives. The naming conventions on different systems may differ, but generally the first IDE drive will be called /dev/hda, the second will be called /dev/hdb, and so on. Be careful, because an IDE CDROM drive is treated the same way, so if you have a hard drive and a CDROM drive, you will probably have active devices for both /dev/hda and /dev/hdb, but you’ll need to determine which one is the hard drive, which we’ll discuss in a moment. If you have SATA or SCSI hard drives or CD drives, they will be named something like /dev/sda or /dev/sdb and so on. This makes it extremely easy to clone from an IDE drive to a SATA or SCSI drive and vice versa.

Let’s say however, for the sake of argument, that you are cloning between two drives which are identical in technology and size; two 100GB SATA drives for instance. How do we tell which is which before we start copying? Simple, we’ve got all the power of the Linux command line at our disposal. So first off we’ll use ’stat’ to see which devices are actually active:

stat /dev/sda

If there is in fact a device there, you should see something similar to the following:

File: `/dev/sda’
Size: 0 Blocks: 0 IO Block: 4096 block special file
Device: fh/15d Inode: 3980 Links: 1 Device type: 8,0
Access: (0640/brw-r—–) Uid: ( 0/ root) Gid: ( 6/ disk)
Access: 2008-01-21 08:59:42.269741059 -0500
Modify: 2008-01-21 08:59:21.304389417 -0500
Change: 2008-01-21 08:59:25.304617435 -0500

If there is nothing at /dev/sda, you’ll see something like this:

stat: cannot stat `/dev/sda’: No such file or directory

Now if you have two SATA or SCSI drives attached, you should be able to stat both /dev/sda and /dev/sdb successfully. Be careful though, because if you have a SATA or SCSI CDROM drive, it will show a successful stat also. However, we can tell which drive is which by mounting them and checking the contents.

mkdir /mnt/test

mount /dev/sda1 /mnt/test

cd /mnt/test

ls

This series of commands creates a directory called /mnt/test, “mounts” or attaches the device /dev/sda1 (the 1 signifies the first partition on device sda) to the /mnt/test directory, changes to that directory and attempts to list any files found there. If it doesn’t find any files, the ls command simply won’t list anything. If it does find files, check them to see if they are from the source hard drive, or a CD-ROM. If they are from the source drive, we’re good to go. Unmount the drive to prepare it for cloning.

umount /dev/sda

If ls didn’t find any files, chances are this is the blank drive you want to clone to. Go through the procedure with /dev/sdb and so on until you find the files from your source drive, then make sure all the drives are unmounted before proceeding to the next step.

*WARNING! YOU CAN DESTROY ALL OF YOUR DATA IF THE NEXT STEP IS NOT DONE CAREFULLY! DO NOT SUE OR EMAIL ME IF YOU WIPE YOUR DRIVE BY DOING THE NEXT FEW STEPS INCORRECTLY. PROCEED AT YOUR OWN RISK!*

Once you have determined the device names of the source and destination drive, it’s time to do the actual cloning. Dd takes a number of arguments which you can read about in the Unix manual page, but normally the only two you’ll need are if (input file) and of (output file). In our case the input “file” is the device /dev/sda, and the output “file” is the device /dev/sdb. In this case, the command to clone /dev/sda to /dev/sdb is:

dd if=/dev/sda of=/dev/sdb

You will get no indication of what is going on for a while. How long this takes depends on the size of the source drive, the speed of the drives, and the speed of your computer. Even on very fast machines however, cloning an 80 to 100GB hard drive will generally take an hour or two, so go grab some lunch and read the rest of the great articles here on OSS. Once the process has finished, you’ll get a response similar to this:

80GB copied

You’re all set! Remove the original drive and boot into the new drive to test things out. If you have any problems, post them here so that other users can avoid them beforehand and see the solutions.

DD provides a free and effective alternative to commercial programs that cost a lot but don’t deliver much. With the full power of the Linux command line at your disposal, you have far more options for performing a safer, more reliable copy. Happy cloning!