Growing a RAID5 array (with LVM) online with MDADM

Growing a RAID5 array (with LVM) online with MDADM

Trawling through the web I've found a lot of information about growing RAID5 arrays, and some sparse information about growing EXT4 partitions. The only problem is they did not appear in the same spots, making this procedure a bit of a guessing game.

I recently decided to upgrade my homebrew NAS. In its original form, it had 2x500GB disks and 2x1TB disks. In RAID5, its capacity is limited to 1.5TB. By replacing the 2x500GB with 2x1TB (totally 4x1TB disks), total capacity expands to a whopping 3TB!

But that's enough talk, let's get down and dirty.

General assumptions:

  • the disks to be replaced are /dev/sdb and /dev/sdc (array goes from sdb to sde);
  • the md array is /dev/md0;
  • the LVM is setup as /dev/lvm-raid/lvm0;
  • the mount point for the array is /media;
  • you are running all this as root; and
  • you have three separate backups in case stuff goes wrong.

First of all, we'll need to replace each 500GB disk in the array individually, and let MDADM rebuild the array after each of the new disks are introduced. To prepare MDADM for the rebuild, set the first disk to be replaced as faulty and remove it. (Don't worry, it's not gonna damage it in any way! It's just the way MDADM works.)

mdadm /dev/md0 --faulty /dev/sdb --remove /dev/sdb

If you don't have a case with hot-swap HDD trays, turn off your computer, remove the old HDD and add the new one, then turn it back on. The computer should detect it as /dev/sdb as well.

Add the new drive to the array:

mdadm /dev/md0 --add /dev/sdb

Wait until MDADM is finished with the rebuild. You can watch with:

mdadm --detail /dev/md0

Once it's done, do the same for the other drive. Do not proceed before ensuring the array has all the new drives active (and the rebuild is finished).

Time to grow the array size to its maximum logical limits (from 1.5TB to 3TB) in MDADM:

mdadm --grow /dev/md0 --size=max

Expand the PV:

pvresize /dev/md0

Extend the LV to the maximum available size of the VG:

lvresize -l+100%FREE /dev/lvm-raid/lvm0

Then proceed with the filesystem resize operation:

resize2fs -p /dev/lvm-raid-lvm0

If all went well, you'll have a lot more space in your mount point (remember, mine is /media) to play with, and your old data is still there! Doing an fsck is a good idea, but that means bringing it offline. And there's plenty of info out on the web explaining how to do that.

One cool thing to notice is that I replaced the first disk and rebuilt the array while watching an HD movie on the HTPC. No skips, jumps or anything. Pretty cool stuff!

If things turn out different for you, let me know in the comments.

PS 1: My original setup didn't use LVM, and resize2fs wouldn't work on it. To grow the EXT4 partition, I'd have to turn the computer off and use Gparted. A hassle. That's when I decided to setup one logical volume (using LVM) on top of the RAID5 array. For instructions on how to setup LVM on RAID, visit this page and check the comments from the author on 19-April-2007 at 16:26.

PS 2: After seeing my backup restore write speed never go beyond 10MB/s, I decided to do some tuning to the RAID array. Turns out there is a nice script here. It didn't fully work for me, so I took the liberty to improve it and you'll find it here. But take caution, as it needs to be configured for your own array properties before it is called, or all hell breaks loose!



Professional site reliability engineer, amateur chef, practicing traveller.

View Comments
Next Post

Bye bye Delicious. Hello Evernote!

Previous Post

That damn xenserver-linuxfixup-disk.iso!

Success! Your membership now is active.