Your prime directive as a system administrator is to keep those systems whipped into shape and running without interfering with user’s daily operations. Sometimes the systems put you in a difficult position when their disks overflow with files. The simple solution is to remove files. But, what if you can’t do that? You have to add disk to your ailing system. But, how and where do you add that space? And, how do you keep this from happening again?
This article leads you through the discovery and remediation of disk space problems. Start your favorite virtual machine and follow along.
The current scenario includes a Debian 5 virtual machine (VM) in VirtualBox 3.2 that currently uses two virtual disks: /dev/hda1 (5 GB) for / and /dev/hdb1 for /var (2 GB). The original 5GB wasn’t enough and the addition of the 2 GB /var disk came later to relieve those issues.
As you can see from the df, the original disk is full again.
This VM is a web server, therefore, it made sense to expand /var to alleviate any future problems with disk space for that service. Can you think of any other directories that would benefit from more space on this system? Let’s explore how a system administrator can fix this problem.
The most likely candidates for expansion are /usr, /home and /opt. Current disk usage in each of these directories is as follows:
~$ du -sk /usr
$ du -sk /home
$ du -sk /opt
Clearly, /usr is the space criminal in this scenario and the best candidate for expansion into its own virtual disk. To create a new disk for /usr, would also return 3GB of space to the / filesystem that includes /home and /opt for this system.
The ten steps involved in this expansion are:
Create a new 5 GB virtual disk.
Associate the new disk with the Debian VM.
Boot the VM.
Initialize the new disk.
Format the disk.
Temporarily mount the disk.
Copy /usr to the mount point.
Make the mount permanent with an entry in /etc/fstab.
Remove the contents of /usr.
Mount the new disk on /usr.
Step One: Create a 5 GB Virtual Disk
# VBoxManage createhd --filename Debian3.vdi --size 5000 --variant Fixed
Oracle VM VirtualBox Command Line Management Interface Version 3.2.4
(C) 2005-2010 Oracle Corporation
All rights reserved.
Disk image created. UUID: f88dc8a9-dbf5-4ac4-845a-c73bc0de1f53
Step Two: Associate the new disk with the Debian VM (Debian5-1).
Login to the VM and become root or use sudo for these privileged command steps.
Step Four: Initialize the New Disk. In this step, you’ll use the sfdisk command to list the new disk device. Then, you’ll step through the simple initialization process.
Note: The following steps only display relevant output to decrease the length and complexity of the screens.
# sfdisk -l
Disk /dev/hdc: 10158 cylinders, 16 heads, 63 sectors/track
sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/hdc: unrecognized partition table type
No partitions found
From the sfdiskoutput, you see that the new disk’s device name is /dev/hdc. Since we’re dedicating the entire disk to /usr, initialization is greatly simplified. The following screen is an interactive command line session using fdisk.
# fdisk /dev/hdc
Command (m for help):n
p primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-10158, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-10158, default 10158): 10158
Command (m for help): p
Disk /dev/hdc: 5242 MB, 5242880000 bytes
16 heads, 63 sectors/track, 10158 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Disk identifier: 0x761001bb
Device Boot Start End Blocks Id System
/dev/hdc1 1 10158 5119600+ 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
# sfdisk -l
Disk /dev/hdc: 10158 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/hdc1 0+ 10157 10158- 5119600+ 83 Linux
/dev/hdc2 0 - 0 0 0 Empty
/dev/hdc3 0 - 0 0 0 Empty
/dev/hdc4 0 - 0 0 0 Empty
Step Five: Format the new Disk (Create the filesystem).
# mkfs.ext3 /dev/hdc1
mke2fs 1.41.3 (12-Oct-2008)
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
320000 inodes, 1279900 blocks
63995 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1312817152
40 block groups
32768 blocks per group, 32768 fragments per group
8000 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 35 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
The new disk is ready to mount and use.
Step Six: Temporarily Mount the New Disk.
# mount /dev/hdc1 /mnt
/dev/hdc1 on /mnt type ext3 (rw)
Step Seven: Copy /usr to the Mount Point.
# cp -Rp /usr/* /mnt
Step Eight: Make the Mount Permanent with an Entry in /etc/fstab. Edit /etc/fstab and create the following entry.
/dev/hdc1 /usr ext3 errors=remount-ro 0 1
Step Nine: Remove the contents of /usr.
# rm -rf /usr/*
Step Ten: Mount the New Disk on /usr.
# umount /mnt
# mount /dev/hdc1 /usr
/dev/hdc1 on /usr type ext3 (rw)
The true test of your handiwork is to reboot the system and confirm that everything went as planned. If something goes wrong, your system will panic. The most likely culprits, should you have any problems, are that the /etc/fstab isn’t correctly setup or that the copy and mount didn’t work correctly. Follow the ten steps exactly as shown and you should have no problems.
If your system panics and doesn’t boot, boot the system using a rescue CD, mount the disks to see what went wrong, work through the steps again and take corrective action.
A quick look at the entire system now shows that the filesystems are happy and have plenty of useable space.
Taking what you’ve learned here, you can create a new disk for /home and one for /opt, if necessary. Systems run out of space. Filesystems fill up. And, it’s your job to fix it before it’s critical.
In the old days (1990s), systems could survive on two disks (not including any mirroring or RAID): A system disk and a “home” disk. The system disk held the standard filesystems, which were partitions or slices created by the system administrator to limit users from killing the system by filling /tmp or /home. The “home” disk, mounted on /home or /h kept user’s home directories isolated from the rest of the system. For a system administrator, managing user’s misuse of space is just another annoyance. These practices helped resolve that annoyance.
Some “old timers” still practice this kind of user isolation today. The rest of us use quotas (A topic of a future article).
Next week, you’ll pay a visit to Father Time and learn to lean on NTP to keep you in sync with the rest of the world.
Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62