Using OCFS2, A Clustering File System

Get a grip on the next-gen Oracle Cluster File System, an alternative to GFS.

Traditionally in Linux, if you wanted to access a file system from more than one machine you would mount it on one machine, and then export it to the others via NFS, CIFS or something similar. This is a stable, mature and well documented procedure, but it does have its downsides. Performance is not great, there can be locking issues, there are potential security implications and worst of all you have a single point of failure.

If your NFS server goes down, all clients immediately lose access. Luckily, there is another way to accomplish this task. A clustered file system is a file system which is simultaneously mounted on multiple servers. Implementing a clustered file system used to be a complicated and expensive proposition. It typically involved a high priced SAN, Fiber Channel HBA’s and switches and a proprietary file system.

Now you have iSCSI and your choice of Open Source clustered file systems available. The most popular is probably GFS, which Red Hat Open Sourced after acquiring Sistina. GFS now ships with both Fedora and CentOS, in addition to being a supported option on top of Red Hat Enterprise Linux. It is included in the mainline kernel as of 2.6.19.

Another option is OCFS2, which is the next generation of the Oracle Cluster File System. It is an extent based, POSIX compliant file system. Unlike the previous release (OCFS), OCFS2 is a general-purpose file system. As of this writing, you should be aware that OCFS2 does not support mmap. While support is planned for the near future, if you need a clustering file system that supports mmap now, you will want to look into GFS.

The OCFS2 user space tools are available for a variety of distributions, either directly from Oracle or from the distribution manufacturer. OCFS2 is licensed under the GPL. Check here for the main project page. To utilize OCFS2 you’ll need to install the ocfs2-tools package and the appropriate kernel modules (ocfs2-`uname -r`).

The ocfs2console package is not required but is recommended for ease-of-use. With these installed we’re ready to setup an OCFS2 partition. Setting up your shared storage is beyond the scope of this article and should be completed before you proceed. The examples used in this article are based on a RHEL 5.1 installation.

Some commands or file locations might differ slightly on other distributions. Note that if the current RHEL kernel does not have a corresponding OCFS2 kernel module RPM available directly from Oracle, you can easily rebuild one from the official ocfs2 tarball. As with GFS, OCFS2 is now in the mainline kernel, so if you are using a newer distribution you should check to see if you already have OCFS2 support before looking for the appropriate kernel module package.

First, you’ll need to edit /etc/sysconfig/o2cb and ensure it contains this line:


Next, create /etc/ocfs2/cluster.conf on each machine. The following is for a three node cluster. For multi-homed machines, the node parameter should match the host name, regardless of which IP you use.

        ip_port = 7777
        ip_address =
        number = 0
        name = host1.domain.com
        cluster = ocfs2
        ip_port = 7777
        ip_address =
        number = 1
        name = host2.domain.com
        cluster = ocfs2
        ip_port = 7777
        ip_address =
        number = 2
        name = host3.domain.com
        cluster = ocfs2
        node_count = 3
        name = ocfs2

You can now run /etc/init.d/o2cb start to start the cluster services. If everything worked, running /etc/init.d/o2cb status should look as follows:

Module "configfs": Loaded
Filesystem "configfs": Mounted
Module "ocfs2_nodemanager": Loaded
Module "ocfs2_dlm": Loaded
Module "ocfs2_dlmfs": Loaded
Filesystem "ocfs2_dlmfs": Mounted
Checking O2CB cluster ocfs2: Online
  Heartbeat dead threshold: 31
  Network idle timeout: 30000
  Network keepalive delay: 2000
  Network reconnect delay: 2000
Checking O2CB heartbeat: Active

You are now ready to format your OCFS2 partition.

# mkfs.ocfs2 -b 4k -C 32K -L "label" -N 4 /dev/sdxX

Now mount your partition on each machine.

# mount -L "label" /dir

With everything working you should add the partition to your fstab and ensure both o2cb and ocfs2 are set to start at boot.

/sbin/chkconfig o2cb on
/sbin/chkconfig ocfs2 on

This article has covered all the basics, and should be enough to get you a working OCFS2 setup. If you plan to run OCFS2 in production you should read the FAQ, which includes more advanced topics such as resizing, quorum and fencing, limitations and rolling upgrades.

Comments on "Using OCFS2, A Clustering File System"

I am sure this paragraph has touched each of the internet visitors, its really really fastidious post
on strengthening new internet site.

Here is my website LucienXCanty

Please take a look at the web sites we comply with, such as this one particular, because it represents our picks in the web.

Every once in a while we select blogs that we read. Listed beneath would be the latest sites that we choose.

We prefer to honor several other web sites on the internet, even when they aren?t linked to us, by linking to them. Underneath are some webpages worth checking out.

Always a large fan of linking to bloggers that I like but do not get a great deal of link adore from.

Usually posts some pretty intriguing stuff like this. If you are new to this site.

Check beneath, are some totally unrelated websites to ours, on the other hand, they may be most trustworthy sources that we use.

Just beneath, are several completely not connected websites to ours, however, they’re certainly really worth going over.

Wonderful story, reckoned we could combine several unrelated information, nonetheless actually really worth taking a search, whoa did a single master about Mid East has got much more problerms also.

Very couple of web sites that come about to become detailed beneath, from our point of view are undoubtedly nicely worth checking out.

We prefer to honor a lot of other online sites on the internet, even if they aren?t linked to us, by linking to them. Underneath are some webpages really worth checking out.

Here are a few of the web-sites we advocate for our visitors.

Leave a Reply