Have you been looking for open-source storage management tools that are easy to use and provide a graphical representation of your storage. Alas, there are no comprehensive tools but there are graphical tools that you can pair with command-line wizardry, particularly LVM.
One of the seemingly most popular GUI tools for LVM has the very unassuming name, “system-config-lvm.” Redhat has written a number of “system-config” tools for performing various tasks. Typically they use Glade and Python to create these tools which is precisely how they created system-config-lvm. The code is released under GPL v2 and can be easily built (assuming you have all of the prerequisite packages installed).
For this article system-config-lvm was tested on a CentOS 5.3 system using a 2.6.30 kernel and e2fsprogs was upgraded to 1.41.9. The tests were run on the following system:
GigaByte MAA78GM-US2H motherboard
An AMD Phenom II X4 920 CPU
8GB of memory
Linux 2.6.30 kernel
The OS and boot drive are on an IBM DTLA-307020 (20GB drive at Ultra ATA/100)
/home is on a Seagate ST1360827AS
There are two drives for this article. They are Seagate ST3500641AS-RK with 16 MB cache each. These are /dev/sdb and /dev/sdc.
For the purposes of this article /dev/sdb and /dev/sdc will have be partitioned into two partitions each – /dev/sdb1, /dev/sdb2, /dev/sdc1, and /dev/sdc2 for use within LVM.
For illustrating how to use LVM with system-config-lvm we’ll use /dev/sdb1 and /dev/sdc1 as the initial two devices in LVM. We’ll walk through the steps of creating PV’s, a Volume Group (VG), and ultimately a Logical Volume (LV). Using this LV we’ll create a file system and make it available on the server. Then /dev/sdb1 will be added to the Volume Group (VG) and the Logical Volume (LV) and it’s associated file system will be extended to use this additional storage space.
When system-config-lvm is started we see all of the hard drives in the system as shown in Figure 2.
Figure 2 – Initial Screen for system-config-lvm
You can see down the left hand side that the four drives in this system are listed.
You can drill down on each drive by clicking on the down arrows next to the drive on the left hand side. Figures 3 below shows the screen when we drill down on /dev/sdb.
Figure 3 – Drill Down on /dev/sdb1
The first thing to notice is that on the main screen in the center, it says that /dev/sdb1 is an “Uninitialized Disk Entity”. This means it is not yet ready for LVM (recall that you have to create a “Physical Volume” from a disk partition). On the right hand side you will notice that it has information about the partition including size, type, mount point, file system, etc.
The next step is to “Initialize” the partition to create a Physical Volume (PV). Notice at the bottom the button labeled, “Initialize Entry”. You just click on this button to create a Physical Volume from the disk partition (you will need to do it for /dev/sdb1 and /dev/sdc1). Figures 4 and 5 below show the screen after /dev/sdb1 and /dev/sdc1 have been initialized.
Figure 4 – Screen After Initializing /dev/sdb1
Figure 5 – Screen After Initializing /dev/sdc1
Both screens are shown so that you can see that you have to perform the initialization on both disk partitions.
Again, examine the right hand side of the tool to get more details on the newly created PV’s. It lists information such as the Physical Volume Name, Physical Volume Size, Space User, Space Free, and so on. Also notice that in the middle of the screen it lists the PV’s as “Unallocated Physical Volume”. So the next step is to create a Volume Group (VG) and add the newly initialized PV’s to it.
If you look at the bottom of either Figure 4 or 5 you see a button labeled “Create new Volume Group”. Just click on this button and it brings up a dialog box, as in Figure 6, where you can input your choices for the Volume Group
Figure 6 – Creating volume Group – Input Dialog Box
The dialog box is pretty straight forward and for the purposes of this article I will just name the VG and use the defaults for the other options.
After the VG is created, the tool returns and provides information about the VG as in Figure 7.
Figure 7 – After Creating Volume Group
On return notice that the left hand side of the screen now has a category listed as “Volume Groups” since we now have one created. The screen shows both a “logical view” and a “physical view” of the Volume Group. The logical view just shows the VG itself and what it is being used for. The Physical view shows what Physical Volumes (PV’s) are used in the VG. In this case, the logical view shows that the VG as “Unused Space” which makes perfect sense since the VG has not been used by any Logical Volumes (yet). The Physical View shows that the VG currently only consists of a single PV, /dev/sdb1.
On the far right hand side of the screen it gives all kinds of information about the volume group including the name, format, volume group size, available space, Number of physical volumes (1), and the number of Logical Volumes (LV’s) that are using the VG.
The next step is to add /dev/sdc1 to the Volume Group. To do this you navigate on the left hand side of the tool to the “Unallocated Volumes” and click on sdc1. When you do there will be a button at the bottom of the screen with the label “Add to existing Volume Group”. Click on this button and the tool will bring up the screen in Figure 8.
Figure 8 – Adding the Physical Volume /dev/sdc1 to the Volume Group
You just click on the volume group you want, in this case vg0, and click “Add”. After the PV is added to the VG, the tool returns to the main screen as was shown in Figure 7. However, now the physical view shows two PV’s as in Figure 9 below.
Figure 9 – Volume Group After Adding /dev/sdc1 to the Volume Group
Notice that the logical view has not changed (we haven’t done anything with the VG yet) while the physical view now shows the PV’s that are part of the VG. On the right hand side of the screen are the details of the VG. Notice that it lists 2 Physical Volumes and the size of the VG has doubled.
Now that the initial VG has been created, we can move to the next step, the creation of a Logical Volume (LV). To do this, you go to the “Logical view” of the Volume Group of interest. So in this case, on the left hand side you go to “Volume Groups” and then look for “vg0″ and then drill down to “vg0 Logical View“. When you do, the screen should look something like Figure 10 below
Figure 10 – Vg0 Logical View – Getting Ready to Create a Logical Volume
You can see that the VG has not been used since the entire VG is listed as “unused”. Detail information about the VG is listed on the far right hand side of the screen. At the bottom of the screen is a button labeled, “Create New Logical Volume”. Clicking on this button allows you to create a new LV within this VG.
When you click on the button it brings up a window allowing you to specify the details of the LV as show in Figure 11.
Figure 11 – Specifying the Details of the Logical Volume
The window has options for defining the LV properties, the size of the LV, and the ability to create a file system on the LV. This last option combines the steps of creating the Logical Volume and creating a file system into a single logical step.
For this article I chose the inputs for the Logical Volume as shown in Figure 12 below.
Figure 12 – Definitions of the Logical Volume
In this case I have chosen to label the LV as “lv0″ and make it linear (that makes it easier to extend in the future). Plus I chose to decide on the size of the LV in Gigabytes (easier than “extents”) and I’m using the entire VG for this particular LV. I have also chose not to create a file system at this time. The reason is that I like to separate these two steps to make sure I have my LV the way I want it.
The result of creating the LV is shown in Figure 13.
Figure 13 – Result of Creating the Logical Volume
Notice that the Logical Volume, lv0, spans both PV’s.
At this point we need to create a file system on the LV. On the far left hand side of the screen in Figure 13, we navigate to the vg0 logical view and select lv0 (the first Logical Volume). At the bottom of the screen should be a button with the label “Edit Properties”. If you click on it, then a dialog box will come up that is the same as Figure 12. Except in this case I’m going to have create a file system and mount it (Note: the mount point /mnt/data has to exist for this operation to succeed). The inputs I chose for this example are shown in Figure 14.
Figure 14 – Input into Creating the File System
At the bottom of the dialog box are the inputs into the file system creation. For the version of Centos I used (5.3) only two file systems were available: ext2, and ext3. So I chose ext3 for the purposes of this article. I also chose to mount the resulting file system at /mnt/data which is a mount point I created. If the mount point does not exist, the tool will not be able to mount the file system.
After clicking “OK” the tool creates the file system with a quasi-progress bar shown in Figure 15.
Figure 15 – File System Create – Progress
After the file system is created and mounted, the screen returns as in Figure 16.
Figure 16 – File System Create Results
On the right hand side it lists the file system (ext3) and the mount point. The one thing that system-config-lvm does not do is modify /etc/fstab. You will have to modify this file by hand (just use the output from “mount” as a guide).
To be sure that the file system is mounted, we can just use the command “df” as shown below.
The file system using LVM is ready to be used as we see fit.
Growing our Storage
One of the advantages of LVM is that we have great control over our storage. One of the typical tasks we might face is to extend or grow the storage space that is mounted as /mnt/data. The system-config-lvm tool allows us to do this.
The first step is to ready another drive partition that we want to add to the existing volume group. Basically we are going to create another Physical Volume (PV). Fortunately we have a couple of partitions that can be used so let’s use /dev/sdb2. During the creation of the first PV we went the left hand side title, “Uninitialized Entities”. We do the same thing again navigating to /dev/sdb2. After we initialize /dev/sdb2 the screen comes back and let’s us know we have an unused PV as show in Figure 17.
Figure 17 – Results of Initializing /dev/sdb2
Once the PV is created we can click on the button at the bottom of the screen labeled “Add to existing Volume Group” to add the new PV to our existing VG, vg0 as shown in Figure 18.
Figure 18 – Adding the Physical Volume /dev/sdb2 to vg0
Just click “Add” to add the PV to the existing VG, vg0. The result is shown in Figure 19.
Figure 19 – Volume Group After /dev/sdb2 is Added
Notice that the Physical View, in red at the bottom of the screen, shows the added PV, /dev/sdb2. The Logical view in blue shows that some space has been added but is unused. Out next step is to extend the Logical Volume (LV) to include this unused space.
To extend the LV, you need to navigate on the left hand side to the logical view of lv0, our Logical Volume. At that point you click on the button at the bottom labeled “Edit Properties”. When you do a dialog box pops up that allows you to extend the LV as shown in Figure 20.
Figure 20 – Extending the LV
You just grab the slider in the middle of the dialog box and drag it to the right to use the entire LV.
When you click “OK” on the button you will get a pop up message box that says something similar to the following.
Logical Volume /dev/vg0/vg0 is currently mounted on /mnt/data. In order to complete
request, it has to be unmounted. Are you sure you want it unmounted?
This dialog box is warning you that you cannot extend the LV while it is mounted. Be very sure there are no users who are using that file system or that any application running on the system is using that file system when you click “OK”.
The system-config-lvm tool will also extend the file system for you when it extends the LV. So it will resize the ext3 file system in this case. Once you click “OK” it will begin it’s extending/resizing operations. Figure 21 below shows the screen during the file system re-sizing process.
Figure 21 – Extending the LV – Resizing the File System
The process can take a little while depending upon how much data is in the file system, the size of the file system, and how much you are adding to the file system. Figure 22 below is a screen shot from from gkrellm that I run on my system.
Figure 22 – Gkrellm Output During Re-sizing
Notice that one of the CPU cores (CPU3) is somewhat busy, but also the disk “sdb” is very busy (this consists of /dev/sdb1 and /dev/sdb2).
Once the resizing is complete, the tool returns to show you the physical and logical view of the storage as shown in Figure 23.
Figure 23 – Physical and Logical View After Resizing
You can see that there is no unused space in the LV and the information on the far right hand size tells you that the LV has increased in size. You can drill down into the logical view of the LV by using the left hand navigation by clicking on the down arrow next to the label “vg0 Logical View”. Figure 24 shows this Logical View.
Figure 24 – Logical View After Resizing
You can see on the far right hand side that the file system has indeed grown. An easy way to check it is to use the “df” command again.
I have been searching for good open-source storage management tools for some time. Something that allows me to easily control and manipulate my storage devices including networks (if applicable). In addition, I want a way to monitor the status of my storage from the status of the hard drives (SMART data), to file system status, to system logs on the storage servers, and to network information such as dropped packets and retransmitted packets. So far, my search is yielded very little.
However, rather than cry over spilled milk, or rather, the lack of milk, I decided to see what tools are available now that move things in the general direction of storage management. One such tool that is extraordinarily important is LVM (Logical Volume Management).
Virtually every Linux distribution comes with LVM configured and ready to go. The concepts in LVM are fairly easy to memorize if you remember that LVM abstracts the storage devices. While this may seem counter-intuitive, “why put more layers between the storage and the file systems?”, it gives you more flexibility and more capability. It allows you to combine storage devices into virtual devices that can be used by file systems. You can extend or grow storage devices and then grow file systems very easily. While not discussed in this article but in a previous one you can also use LVM to create snapshots of the file system (think backups and disaster recovery).
The fundamental steps in LVM are fairly simple.
Disk Partition -> Physical Volumes (PV) -> Volume Group (VG) -> Logical Volume (LV) -> File System
You start with drive partitions, then create Physical Volumes (PV’s), then use PV’s to create Volume Groups (VG’s) which are like virtual drives, and then create Logical Volumes (LV’s) from the VG’s (LV’s are like virtual partitions). Finally you can use the LV’s as though they are storage devices for creating file systems.
LVM in Linux comes with command line tools that are fairly easy to learn and use but it is fairly easy to get confused about what real partitions or PV’s correspond to which VG’s or LV’s. That’s where a GUI can be very helpful. The LVM GUI tool that I used in this article is system-config-lvm from Redhat.
It presents a nice, easy to use interface for LVM. You have to know the basic steps of using LVM to use the tool but it does make things painless and graphical (i.e. easy to interpret). The tool can accommodate a number of PV’s, VG’s, and LV’s so it’s very flexible. It doesn’t necessarily understand Linux’ software RAID (md), at least not yet, so you have to create PV’s from these devices by hand. Nonetheless, the tool proved remarkably easy to use.
System-config-lvm has the essence of what I’m searching for in storage management – easy to use tools that can give me a graphical presentation of the state of my storage. In this case it gives me a very good representation of the state of my storage that uses LVM (most likely a big hunk of my storage).