This blogs lists the steps around creating RAID5 volume on Linux using local disks.

We will start by installing software RAID on Linux system

# yum install mdadm
Loaded plugins: ulninfo
Resolving Dependencies
--> Running transaction check
---> Package mdadm.x86_64 0:4.1-1.0.1.el7 will be installed
--> Processing Dependency: libreport-filesystem for package: mdadm-4.1-1.0.1.el7.x86_64
--> Running transaction check
---> Package libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                  Arch       Version               Repository      Size
================================================================================
Installing:
 mdadm                    x86_64     4.1-1.0.1.el7         ol7_latest     435 k
Installing for dependencies:
 libreport-filesystem     x86_64     2.1.11-43.0.1.el7     ol7_latest      40 k

Transaction Summary
================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 475 k
Installed size: 1.0 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): libreport-filesystem-2.1.11-43.0.1.el7.x86_64.rpm   |  40 kB   00:00
(2/2): mdadm-4.1-1.0.1.el7.x86_64.rpm                      | 435 kB   00:00
--------------------------------------------------------------------------------
Total                                              1.8 MB/s | 475 kB  00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libreport-filesystem-2.1.11-43.0.1.el7.x86_64                1/2
  Installing : mdadm-4.1-1.0.1.el7.x86_64                                   2/2
  Verifying  : mdadm-4.1-1.0.1.el7.x86_64                                   1/2
  Verifying  : libreport-filesystem-2.1.11-43.0.1.el7.x86_64                2/2

Installed:
  mdadm.x86_64 0:4.1-1.0.1.el7

Dependency Installed:
  libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7

Complete!

Now let’s identify disks which we will use for creating RAID5 volume. These disks can be local or storage LUNs. You can list the disks using running below command. In this example we will use 5 local disks – /dev/xvd(b-f)

# fdisk -l | grep /dev/
Disk /dev/xvda: 53.7 GB, 53687091200 bytes, 104857600 sectors
/dev/xvda1   *        2048     4098047     2048000   83  Linux
/dev/xvda2         4098048    86018047    40960000   83  Linux
/dev/xvda3        86018048   104857599     9419776   82  Linux swap / Solaris
Disk /dev/xvdd: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvde: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdb: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Disk /dev/xvdf: 53.7 GB, 53687091200 bytes, 104857600 sectors

Before you begin make sure that selected drives do not have existing RAID configured.

# mdadm -E /dev/xvd[b-f]
mdadm: No md superblock detected on /dev/xvdb.
mdadm: No md superblock detected on /dev/xvdc.
mdadm: No md superblock detected on /dev/xvdd.
mdadm: No md superblock detected on /dev/xvde.
mdadm: No md superblock detected on /dev/xvdf.

Next step is to create partition on selected disks using fdisk. Follow below steps for each selected disks – /dev/xvd(b-f) in this example

# fdisk /dev/xvdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x6845b853.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-104857599, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599):
Using default value 104857599
Partition 1 of type Linux and of size 50 GiB is set

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): p

Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x6845b853

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdc1            2048   104857599    52427776   fd  Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Once you’ve successfully created the partitions for all disks, you can verify the changes using below command

# mdadm -E /dev/xvd[b-f]
/dev/xvdb:
   MBR Magic : aa55
Partition[0] :    104855552 sectors at         2048 (type fd)
/dev/xvdc:
   MBR Magic : aa55
Partition[0] :    104855552 sectors at         2048 (type fd)
/dev/xvdd:
   MBR Magic : aa55
Partition[0] :    104855552 sectors at         2048 (type fd)
/dev/xvde:
   MBR Magic : aa55
Partition[0] :    104855552 sectors at         2048 (type fd)
/dev/xvdf:
   MBR Magic : aa55
Partition[0] :    104855552 sectors at         2048 (type fd)

Now create RAID device (/dev/md1 in below example) using all the newly created disk partitions

# mdadm --create /dev/md1 --level=5 --raid-devices=5 /dev/xvdb1 /dev/xvdc1 /dev/xvdd1 /dev/xvde1 /dev/xvdf1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.

Note that it takes some time to complete the RAID creation. You can monitor the progress using below command. Below command will show you current % completed.

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : active raid5 xvdf1[5] xvde1[3] xvdd1[2] xvdc1[1] xvdb1[0]
      209575936 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_]
      [=>...................]  recovery =  7.2% (3808256/52393984) finish=13.1min speed=61363K/sec

unused devices: <none>

But you prefer to watch screen till 100% completion then run below command. Result of this command will be persistent on the screen and will show you real time progress.

# watch -n1 cat /proc/mdstat
Every 1.0s: cat /proc/mdstat                            Mon Dec  2 09:41:04 2019

Personalities : [raid6] [raid5] [raid4]
md1 : active raid5 xvdf1[5] xvde1[3] xvdd1[2] xvdc1[1] xvdb1[0]
      209575936 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_]
      [==>..................]  recovery = 13.9% (7335304/52393984) finish=12.1mi
n speed=61622K/sec

unused devices: <none>

Once RAID creation is 100% you can run below command to see the summary of the RAID device configuration. RAID device is /dev/md1 in below example with 5 devices – /dev/xvd(b-f)1

# mdadm --detail /dev/md1
/dev/md1:
           Version : 1.2
     Creation Time : Mon Dec  2 09:39:05 2019
        Raid Level : raid5
        Array Size : 209575936 (199.87 GiB 214.61 GB)
     Used Dev Size : 52393984 (49.97 GiB 53.65 GB)
      Raid Devices : 5
     Total Devices : 5
       Persistence : Superblock is persistent

       Update Time : Mon Dec  2 09:53:20 2019
             State : clean
    Active Devices : 5
   Working Devices : 5
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : ip-10-10-10-207:1  (local to host ip-10-10-10-207)
              UUID : 07f28a7b:3c70d798:1deecf9a:292ddd2c
            Events : 18

    Number   Major   Minor   RaidDevice State
       0     202       17        0      active sync   /dev/xvdb1
       1     202       33        1      active sync   /dev/xvdc1
       2     202       49        2      active sync   /dev/xvdd1
       3     202       65        3      active sync   /dev/xvde1
       5     202       81        4      active sync   /dev/xvdf1

At this point we have RAID5 device created (/dev/md1). Now we need to create file system. In below example we are creating EXT4 file system

# mkfs.ext4 /dev/md1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=512 blocks
13099008 inodes, 52393984 blocks
2619699 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2199912448
1599 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Once the file system is created on RAID5 device, create the mount point and mount the RAID5 device on the same.

# mkdir /mnt/ebs_r5
# mount /dev/md1 /mnt/ebs_r5
# df -h
Filesystem        Size  Used Avail Use% Mounted on
devtmpfs           32G     0   32G   0% /dev
tmpfs              32G     0   32G   0% /dev/shm
tmpfs              32G   17M   32G   1% /run
tmpfs              32G     0   32G   0% /sys/fs/cgroup
/dev/xvda2         39G  2.9G   34G   8% /
/dev/xvda1        1.9G  234M  1.6G  13% /boot
tmpfs             6.3G     0  6.3G   0% /run/user/1000
/dev/md1          197G   61M  187G   1% /mnt/ebs_r5

Also make sure you’ve added entry for new mount point in the /etc/fstab file. So that mount point will be persistent across system reboots.

# vi etc/fstab
/dev/md1                /mnt/ebs_r5              ext4    defaults        0 0

Lastly most important step is to save the RAID configuration. Run below command to make sure RAID configuration is saved.

# mdadm --detail --scan --verbose >> /etc/mdadm.conf

At this point you’ve successfully created RAID5 device on Linux.