Storage Convergence: Fibre Channel over Ethernet

The ever-increasing demand for storage, network and compute resources for instant access of data and computational requirements lead to the continued growth of the variety of networks found in the data center. HPC is no exception to this trend.

Data centers can be found in multiple markets spanning multiple industries including, financial, government and education, manufacturing, automotive, health care, bio-technology, and high-tech. These data centers host multiple compute, storage, communication and infrastructural equipment for digital storage/retrieval, communications and management of information. The ever-increasing demand for storage, network and compute resources for instant access of data and computational requirements thereof, has in turn lead to the continued growth of the variety of networks found in the data center. HPC is no exception to this trend. Indeed, HPC often pushes the envelope for storage technologies.

The resulting network proliferation has increased IT costs through the need for multiple adapters that connect the servers found at the edge of the network through separate cables to the various networks. As networks proliferate demand is satisfied through the deployment of more servers and adapters. The growth in the number of servers and adapters directly lead to increased power and cooling demands placed on the data center. One of the largest consumers of data center costs is power and cooling, which for many data centers over the life of the equipment, can exceed the capital equipment expenditures.

The three most common types of networks found in the data center are:

  • Storage Area Networks (SAN)

  • Local Area Networks (LAN)

  • Inter-processor Communication (IPC) Networks

As shown in Figure One, a single server may have some or all of these three networks. v

Figure One: Common Server Interconnects
Figure One: Common Server Interconnects

Each network has evolved to solve a specific requirement delivering specific characteristics for its unique traffic type. The SAN network uses Fibre Channel technology to provide a guaranteed in-order, lossless transport for data to be sent to and from storage devices. The LAN network provides the traditional TCP/IP based Ethernet network for best effort data communications. The IPC network is typically used for High Performance Computing (HPC) clustered environments, where multiple servers communicate with each other using low latency messaging.

One of the biggest challenges for the IT administrator is how to reduce the data center costs while maintaining the Service Level Agreements (SLA) and performance that is expected of the network. Many technologies that have attempted to solve this problem have either been relegated to niche markets or have been found inadequate to solve the problem and thus have been discarded.

A promising solution is emerging to solve this problem. The various standards bodies — IEEE, ANSI and IETF — are collectively defining in the realms of their respective domains pieces of this solution that is expected to gain industry-wide adoption. A protocol and frame format called Fibre Channel over Ethernet (FCoE) is being defined by the ANSI T11. FCoE would need to be transported over an improved or enhanced Ethernet transport referred to as Converged Enhanced Ethernet (CEE). The Data Center Bridging (DCB) task force in the IEEE is working on a number of specifications for this protocol.

With this new protocol, comes a new class of adapter called a Converged Network Adapter (CNA). The basic premise of the CNA is to allow multiple networks to connect to a server over the same physical cable using a single adapter. A single physical connection from the CNA that can be used to transport SAN, LAN and IPC traffic can provide a lower Total Cost of Ownership through a single network infrastructure from the server edge to the access layer using common hardware and software management platforms. To achieve a converged I/O medium a widely adopted transport protocol is required.

Fibre Channel over Ethernet Introduction

Fibre Channel over Ethernet (FCoE) in essence is a mechanism to transport Fibre Channel over a DCB-enabled Ethernet infrastructure. A DCB-enabled Ethernet network differs from Ethernet networks widely deployed today for transporting TCP/IP. It avoids the complications of prior technologies which were designed to collapse infrastructures and be the ubiquitous network technology in that it does not require “fork-lifting” the network.

To transport an FCoE frame, a complete Fibre Channel frame is built in the same way as if it was being built for a traditional Fibre Channel network. FCoE layer encapsulates a fully-formed Fibre Channel frame and inserts a defined frame header before it is sent on the CEE link. Figure Two and Figure Three illustrate the FCoE Protocol Stack and the FCoE Frame Format respectively.

Figure Two: FCoE Protocol Stack. FC-2, FC-3, FC-4 are various Fibre Channel services.
Figure Two: FCoE Protocol Stack. FC-2, FC-3, FC-4 are various Fibre Channel services.

FCoE encodes the Start of Frame (SOF) and includes it in the FCoE header prepended to the standard Fibre Channel frame. It then encodes End of Frame (EOF) at the end of the Fibre Channel frame before the frame is handed over to the CEE layer. A header is added and a Cyclic Redundancy Check (CRC) is completed before transmitting the unmodified Fibre Channel frame as data. The type of packet, FCoE or TCP/IP is defined in the type field.

Figure Three: FCoE Frame Format
Figure Three: FCoE Frame Format

As with any new protocol, an FCoE switch is required to expose the CEE and Fibre Channel ports, making forwarding decisions based on the Fibre Channel headers inside the CEE frame. It is similar to how a basic Fibre Channel switch would switch Fibre Channel frames. The only difference is that an FCoE switch supports communication between any ports on the switch, Fibre Channel or CEE. In addition an FCoE switch is also capable of receiving and processing basic TCP/IP traffic on the CEE ports since it is connected to a host—capable of acting as a standard IEEE 802.1Q bridge between all CEE ports in this Layer 2 mode. This operation is very similar to how a multi-layer Internet Protocol (IP) switch performs basic Ethernet switching at Layer 2 (L2) as well as IP routing at Layer 3 between its ports. An FCoE switch treats Fibre Channel traffic as a Layer 3 network protocol, and no LAN traffic is forwarded to Fibre Channel ports in the L2 mode.

An FCoE switch can be connected to other FCoE switches via the CEE ports, and it also can be connected to any existing Fibre Channel switch via the Fibre Channel ports.

Comments on "Storage Convergence: Fibre Channel over Ethernet"

I just want to say I am just beginner to blogging and certainly enjoyed this blog site. Almost certainly I’m going to bookmark your website . You actually come with awesome well written articles. Cheers for revealing your website.

I do agree with all the ideas you’ve presented to your post.
They are very convincing and will definitely work.

Nonetheless, the posts are very brief for novices.

May just you please prolong them a little from next
time? Thanks for the post.
creditos rapidos con asnef

I do not even know how I ended up here, but I thought this
post was good. I do not know who you are but definitely
you’re going to a famous blogger if you aren’t already
;) Cheers!
prestamos con asnef

mulberry purse Do you have any video of that? I ad like to find out more details.

Pretty! This has been an extremely wonderful article. Thank you for supplying this info.

Thanks for the blog article.Much thanks again. Want more.

Rock ‘n Roll pioneer Dale Hawkins dead in Little Rock. Obviously, to earn all 48 stars, you’ll need to play every mission on veteran or run it as fast as possible.
Some have suggested that mobile phones are already as great a threat to privacy as RFID.

Also visit my web page … Instrumental Music

Heya i am for the first time here. I found this board and I find It truly
useful & it helped me out a lot. I hope to give something back and aid others
like you helped me.

website; Brook,

Hi, i think that i noticed you visited my web site
thus i got here to return the desire?.I am attempting to find issues to
improve my web site!I guess its adequate to use a few of your ideas!!

Asking questions are genuinely fastidious thing if you are
not understanding something completely, except this post gives fastidious understanding even.

This information is worth everyone’s attention. When can I find out more?

site – Carol

Hello there! This post could not be written any better! Reading
this post reminds me of my old room mate! He
always kept talking about this. I will forward this write-up to him.
Fairly certain he will have a good read. Thank you
for sharing!
webpage (Benjamin)

Hi! Do you use Twitter? I’d like to follow you
if that would be okay. I’m absolutely enjoying your blog and look forward to new updates.

What a information of un-ambiguity and preserveness of precious knowledge concerning
unexpected emotions.

Hey there! I’ve been reading your site for
some time now and finally got the bravery to go ahead
and give you a shout out from Kingwood Tx! Just wanted to say keep up the
great job!
web page (Nichol)

??????? ????? ?????-????? ???? ??? – ????? ???? ???? – ????? ????? ???? – ????? iran.sc-????? ???? ????-????? ??????

Im obliged for the blog article.Really thank you! Really Great.

I really liked your article post.Much thanks again. Keep writing.

Thanks again for the blog post. Want more.

A big thank you for your post.Thanks Again. Will read on…

One of our guests not too long ago advised the following website.

Here are a few of the web pages we advocate for our visitors.

Here are a number of the web sites we advise for our visitors.

The details mentioned within the write-up are several of the most effective out there.

Leave a Reply