This blogs lists the steps around creating RAID5 volume on Linux using local disks.
We will start by installing software RAID on Linux system
# yum install mdadm Loaded plugins: ulninfo Resolving Dependencies --> Running transaction check ---> Package mdadm.x86_64 0:4.1-1.0.1.el7 will be installed --> Processing Dependency: libreport-filesystem for package: mdadm-4.1-1.0.1.el7.x86_64 --> Running transaction check ---> Package libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: mdadm x86_64 4.1-1.0.1.el7 ol7_latest 435 k Installing for dependencies: libreport-filesystem x86_64 2.1.11-43.0.1.el7 ol7_latest 40 k Transaction Summary ================================================================================ Install 1 Package (+1 Dependent package) Total download size: 475 k Installed size: 1.0 M Is this ok [y/d/N]: y Downloading packages: (1/2): libreport-filesystem-2.1.11-43.0.1.el7.x86_64.rpm | 40 kB 00:00 (2/2): mdadm-4.1-1.0.1.el7.x86_64.rpm | 435 kB 00:00 -------------------------------------------------------------------------------- Total 1.8 MB/s | 475 kB 00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libreport-filesystem-2.1.11-43.0.1.el7.x86_64 1/2 Installing : mdadm-4.1-1.0.1.el7.x86_64 2/2 Verifying : mdadm-4.1-1.0.1.el7.x86_64 1/2 Verifying : libreport-filesystem-2.1.11-43.0.1.el7.x86_64 2/2 Installed: mdadm.x86_64 0:4.1-1.0.1.el7 Dependency Installed: libreport-filesystem.x86_64 0:2.1.11-43.0.1.el7 Complete!
Now let’s identify disks which we will use for creating RAID5 volume. These disks can be local or storage LUNs. You can list the disks using running below command. In this example we will use 5 local disks – /dev/xvd(b-f)
# fdisk -l | grep /dev/ Disk /dev/xvda: 53.7 GB, 53687091200 bytes, 104857600 sectors /dev/xvda1 * 2048 4098047 2048000 83 Linux /dev/xvda2 4098048 86018047 40960000 83 Linux /dev/xvda3 86018048 104857599 9419776 82 Linux swap / Solaris Disk /dev/xvdd: 53.7 GB, 53687091200 bytes, 104857600 sectors Disk /dev/xvde: 53.7 GB, 53687091200 bytes, 104857600 sectors Disk /dev/xvdb: 53.7 GB, 53687091200 bytes, 104857600 sectors Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors Disk /dev/xvdf: 53.7 GB, 53687091200 bytes, 104857600 sectors
Before you begin make sure that selected drives do not have existing RAID configured.
# mdadm -E /dev/xvd[b-f] mdadm: No md superblock detected on /dev/xvdb. mdadm: No md superblock detected on /dev/xvdc. mdadm: No md superblock detected on /dev/xvdd. mdadm: No md superblock detected on /dev/xvde. mdadm: No md superblock detected on /dev/xvdf.
Next step is to create partition on selected disks using fdisk
. Follow below steps for each selected disks – /dev/xvd(b-f)
in this example
# fdisk /dev/xvdc Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0x6845b853. Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-104857599, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): Using default value 104857599 Partition 1 of type Linux and of size 50 GiB is set Command (m for help): t Selected partition 1 Hex code (type L to list all codes): fd Changed type of partition 'Linux' to 'Linux raid autodetect' Command (m for help): p Disk /dev/xvdc: 53.7 GB, 53687091200 bytes, 104857600 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x6845b853 Device Boot Start End Blocks Id System /dev/xvdc1 2048 104857599 52427776 fd Linux raid autodetect Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
Once you’ve successfully created the partitions for all disks, you can verify the changes using below command
# mdadm -E /dev/xvd[b-f] /dev/xvdb: MBR Magic : aa55 Partition[0] : 104855552 sectors at 2048 (type fd) /dev/xvdc: MBR Magic : aa55 Partition[0] : 104855552 sectors at 2048 (type fd) /dev/xvdd: MBR Magic : aa55 Partition[0] : 104855552 sectors at 2048 (type fd) /dev/xvde: MBR Magic : aa55 Partition[0] : 104855552 sectors at 2048 (type fd) /dev/xvdf: MBR Magic : aa55 Partition[0] : 104855552 sectors at 2048 (type fd)
Now create RAID device (/dev/md1
in below example) using all the newly created disk partitions
# mdadm --create /dev/md1 --level=5 --raid-devices=5 /dev/xvdb1 /dev/xvdc1 /dev/xvdd1 /dev/xvde1 /dev/xvdf1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started.
Note that it takes some time to complete the RAID creation. You can monitor the progress using below command. Below command will show you current % completed.
# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md1 : active raid5 xvdf1[5] xvde1[3] xvdd1[2] xvdc1[1] xvdb1[0] 209575936 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_] [=>...................] recovery = 7.2% (3808256/52393984) finish=13.1min speed=61363K/sec unused devices: <none>
But you prefer to watch screen till 100% completion then run below command. Result of this command will be persistent on the screen and will show you real time progress.
# watch -n1 cat /proc/mdstat Every 1.0s: cat /proc/mdstat Mon Dec 2 09:41:04 2019 Personalities : [raid6] [raid5] [raid4] md1 : active raid5 xvdf1[5] xvde1[3] xvdd1[2] xvdc1[1] xvdb1[0] 209575936 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_] [==>..................] recovery = 13.9% (7335304/52393984) finish=12.1mi n speed=61622K/sec unused devices: <none>
Once RAID creation is 100% you can run below command to see the summary of the RAID device configuration. RAID device is /dev/md1
in below example with 5 devices – /dev/xvd(b-f)1
# mdadm --detail /dev/md1 /dev/md1: Version : 1.2 Creation Time : Mon Dec 2 09:39:05 2019 Raid Level : raid5 Array Size : 209575936 (199.87 GiB 214.61 GB) Used Dev Size : 52393984 (49.97 GiB 53.65 GB) Raid Devices : 5 Total Devices : 5 Persistence : Superblock is persistent Update Time : Mon Dec 2 09:53:20 2019 State : clean Active Devices : 5 Working Devices : 5 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K Consistency Policy : resync Name : ip-10-10-10-207:1 (local to host ip-10-10-10-207) UUID : 07f28a7b:3c70d798:1deecf9a:292ddd2c Events : 18 Number Major Minor RaidDevice State 0 202 17 0 active sync /dev/xvdb1 1 202 33 1 active sync /dev/xvdc1 2 202 49 2 active sync /dev/xvdd1 3 202 65 3 active sync /dev/xvde1 5 202 81 4 active sync /dev/xvdf1
At this point we have RAID5 device created (/dev/md1
). Now we need to create file system. In below example we are creating EXT4 file system
# mkfs.ext4 /dev/md1 mke2fs 1.42.9 (28-Dec-2013) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=128 blocks, Stripe width=512 blocks 13099008 inodes, 52393984 blocks 2619699 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2199912448 1599 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done
Once the file system is created on RAID5 device, create the mount point and mount the RAID5 device on the same.
# mkdir /mnt/ebs_r5 # mount /dev/md1 /mnt/ebs_r5 # df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 32G 0 32G 0% /dev tmpfs 32G 0 32G 0% /dev/shm tmpfs 32G 17M 32G 1% /run tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/xvda2 39G 2.9G 34G 8% / /dev/xvda1 1.9G 234M 1.6G 13% /boot tmpfs 6.3G 0 6.3G 0% /run/user/1000 /dev/md1 197G 61M 187G 1% /mnt/ebs_r5
Also make sure you’ve added entry for new mount point in the /etc/fstab
file. So that mount point will be persistent across system reboots.
# vi etc/fstab /dev/md1 /mnt/ebs_r5 ext4 defaults 0 0
Lastly most important step is to save the RAID configuration. Run below command to make sure RAID configuration is saved.
# mdadm --detail --scan --verbose >> /etc/mdadm.conf
At this point you’ve successfully created RAID5 device on Linux.
Categories: Back To Basics
Reblogged this on Nguoidentubinhduong.
LikeLike