Firm and Steady Wins the Race (Fibre Channel Vs iSCSI)

Fibre Channel (FC) Vs Internet Small Computer Systems Interface (iSCSI) has been one of the most disputed topics in storage area networks (SAN) for at least a decade.  It kind of reminds me of Tortoise and Hare story which we use to hear as kids. Most people still have misconception that iSCSI is low cost, less performance and easy to deploy because it works on the same kind of ethernet network that servers and clients are already running on. In this blog I am going to explain about FC and ISCSI and how each has developed over the years.

Figure 1: Block Protocol Summary

Figure 1: Block Protocol Summary

Please take a clear look at the Block Protocol layout, in Figure 1, which I am going to refer throughout my blog as a basis of my discussion.

Fibre Channel (FC)

Fibre Channel is a set of related physical layer networking standards. It was developed to transport data at the high speed with low overhead.

There are two basic types of data communication;

  1. Between processors – Channels
  2. Between processors and peripherals – Networks.

A channel provides a direct or switched point-to-point connection between the communicating devices. A channel is typically hardware-intensive and transports data at the high speed with low overhead.

In contrast, a Network is an aggregation of distributed nodes with its own protocol that supports interaction among these nodes. A network has relatively high overhead since it is software-intensive, and consequently slower than a channel. Networks can handle a more extensive range of tasks than channels as they operate in an environment of unanticipated connections, while channels operate amongst only a few devices with predefined addresses.

Fibre Channel attempts to combine the best of these two methods of communication into a new I/O interface that meets the needs of channel users and also network users.

Like Ethernet, its main competitor, Fibre Channel can utilize copper wiring. However, copper limits Fibre Channel to a maximum recommended reach of 30 meters, whereas with more expensive fiber optic cables, it reaches up to 10 kilometers.

The technology was specifically named Fibre Channel rather than Fiber Channel to distinguish it as supporting both fiber and copper cabling.

Fibre Channel does not follow the OSI model layering and is split into five layers:

  • FC-4 – Protocol-mapping layer, in which upper level protocols such as SCSI, IP or FICON, are encapsulated into Information Units (IUs) for delivery to FC-2. Current FC-4s include FCP-4, FC-SB-5, and FC-NVMe.
  • FC-3 – Common services layer, a thin layer that could eventually implement functions like encryption or RAID redundancy algorithms; multiport connections.
  • FC-2 – Signaling Protocol, defined by the Fibre Channel Framing and Signaling standard, consists of the low level Fibre Channel protocols & port to port connections.
  • FC-1 – Transmission Protocol, which implements line coding of signals;
  • FC-0 – Physical layer, includes cabling, connectors etc.
Layers of Fiber Channel Protocol

Figure 2: Layers of Fiber Channel Protocol (Reference used from BytePile.com)

When it was first introduced, Fibre-channel enabled the campus wide consolidation of high throughput storage. In fact network attached storage using ethernet had existed for many years before the introduction of Fibre-channel, but its low throughput (based on the 10 Mbps rates at the time) made it unsuitable for many applications compared to directly attached storage which was about 20x faster.

Following are the advantages that made Fibre Channel popular in the past:

  • Price Performance Leadership – Fibre Channel delivers cost-effective solutions for storage and networks.
  • Solutions Leadership – Fibre Channel provides versatile connectivity with scalable performance.
  • Reliability – Fibre Channel, a most reliable form of communications, sustains an enterprise with assured information delivery. It is a lossless network. It also came with reduced amount of internal cabling within the servers and storage systems, because Fibre-channel was a serial interface, compared to SCSI which was a parallel interface before (now it has both options of serial and parallel interface) and because of which it had a very high cable core and connector count.
  • Multiple Topologies – Dedicated point-to-point, shared loops, and scaled switched topologies meet application requirements.
  • Multiple Protocols – Fibre Channel delivers data. SCSI, TCP/IP, video, or raw data can all take advantage of high- performance, reliable Fibre Channel technology.
  • Scalable – From single point-to-point gigabit links to integrated enterprises with hundreds of servers, Fibre Channel delivers unmatched performance.
  • Fault Tolerance – Fibre-channel included fault tolerant mechanisms for rerouting around failed cable loops – although it was many years before these were properly supported by transparent failover software.
  • High Efficiency – Real price performance is directly correlated to the efficiency of the technology. Fibre Channel has very little transmission overhead. Most important, the Fibre Channel protocol, is specifically designed for highly efficient operation using hardware.

As decades passed by the original advantages of Fibre-channel have been blurred as other technologies have adopted similar attributes. We are used to the concept of networking data at high speed over large distances using many different types of technologies. At the building or campus network level, IP technology in the form of iSCSI offers identical performance these days for most common tasks like backup, replication, etc. Also the term SAN, which originally referred just to a Fibre-channel connected network has become defocused by common usage. So we get IP-SAN which usually refers to iSCSI, and which uses Ethernet (not Fibre-channel) as the underlying transport in a local area network.

Over the years Fibre Channel protocol has also meta-morphed into many new protocols to provide FC over Ethernet (FCoE), Fibre Channel over ethernet using IP (FCIP) and internet Fibre Channel (iFCP) because of various needs of the IT industry but each has its own pros and cons which I would discuss in the coming blogs. This blog I would want to stick to just FC and iSCSI.

Internet Small Computer Systems Interface (iSCSI)

iSCSI, which stands for Internet Small Computer System Interface, an Internet Protocol (IP)-based storage networking standard for linking data storage facilities. It provides networked block-level shared access to storage devices by carrying SCSI commands over a TCP/IP network. iSCSI is used to facilitate data transfers over intranets and to manage storage over long distances. It can be used to transmit data over local-area networks (LANs), wide-area networks (WANs) or the Internet.

IBM & Cisco developed iSCSI as a proof of concept in 1998, and presented the first draft of the iSCSI standard to the Internet Engineering Task Force (IETF) in 2000. The protocol was ratified in 2003.

How iSCSI Works ?

Before I explain how iSCSI works you need to understand how SCSI works as it is part of iSCSI.

The Small Computer Systems Interface (SCSI) is a popular family of protocols for communicating with I/O devices, especially storage devices.

There are two types of devices in SCSI protocol;

  • The SCSI Initiators (Mostly OS) to start the communications

The initiators are devices that request commands be executed.

  • The Targets (file servers or storage) to respond.

Targets are devices that carry out the commands. The endpoint, within the target, that executes the command is referred to as a “logical unit” (LU). A target is a collection of logical units, in general of the same type, and are directly addressable.

The structure used to communicate a command from an application client to a device server is referred to as a Command Descriptor Block (CDB). An SCSI command or a linked set of commands is referred to as a “task.” Only one command in a task can be outstanding at any given time.

SCSI command execution results in an optional;

  • Data phase and

In the data phase, data travels either from the initiator to the target, as in a WRITE command, or from the target to the initiator, as in a READ command.

  • Status phase

In the status phase, the target returns the final status of the operation. The status response terminates an SCSI command or task.

The basic function of the SCSI driver is to build SCSI Command Descriptor Blocks (CDB) from requests issued by the application, and forwards them to the iSCSI layer. The SCSI driver also receives CDBs from the iSCSI layer and forwarding the data to the application layer.

iSCSI_Works

Figure 3: iSCSI Network connectivity

iSCSI provides initiators and targets with unique names as well as a discovery method  (described below). The iSCSI protocol establishes communication sessions between initiators and targets, and provides methods for them to authenticate one another. An iSCSI session may contain one or more TCP connections and provides recovery in the event connections fail. Following is the way data is transmitted over iSCSI;

  • SCSI CDBs are passed from the SCSI generic layer to the iSCSI transport layer.
  • The iSCSI transport layer encapsulates the SCSI CDB into an iSCSI Protocol Data Unit (PDU) and forwards it to the Transmission Control Protocol (TCP) layer.
  • On a read, the iSCSI transport layer extracts the CDB from the iSCSI PDU, received from the TCP layer, and
  • Forwards the CDB to the SCSI generic layer.
  • iSCSI provides the SCSI generic command layer with a reliable transport

The following diagram illustrates the layering of the various SCSI command sets and data over different transport and physical layers.

Layering of SCSI command sets and data over different transport and physical layers

Figure 4: Layering of SCSI command sets and data over different transport and physical layers (Reference used from diskdrive.com)

iSCSI Naming & Addressing

In iSCSI network, each component (initiator or target) has its unique name. Let’s have a look at naming types. iSCSI provides three name formats;

1. iSCSI Qualified Name (IQN)

Briefly, the fields are:

  • Literal iqn (iSCSI Qualified Name)
  • Date (yyyy-mm) that the naming authority took ownership of the domain
  • Reversed domain name of the authority (e.g. org.alpinelinux, com.example, to.yp.cr)
  • Optional “:” prefixing a storage target name specified by the naming authority.
IQN_Naming_Convention

Figure 5: IQN Naming Format

2. Extended Unique Identifier (EUI)Format: eui. {EUI-64 bit address} (e.g. eui.02004567A425678D)

3. T11 Network Address Authority (NAA)

Format: naa. {NAA 64 or 128 bit identifier} (e.g. naa.52004567BA64678D)

IQN format addresses occur most commonly. They are qualified by a date (yyyy-mm) because domain names can expire or be acquired by another entity.

NAA name formats were added to iSCSI, to provide compatibility with naming conventions used in Fibre Channel and Serial Attached SCSI (SAS) storage technologies.

Usually, an iSCSI participant can be defined by three or four fields:

  1. Hostname or IP Address (e.g., “iscsi.example.com”)
  2. Port Number (e.g., 3260)
  3. iSCSI Name (e.g., the IQN “iqn.2003-01.com.ibm:00.fcd0ab21.shark128”)
  4. An optionalCHAP Secret (e.g., “secretsarefun”)

iSCSI Discovery

An iSCSI initiator can discover an iSCSI target in the following different ways:

  • By configuring the target’s address on the initiator.
  • By configuring a default target address on the initiator and the initiator connects to the target and requests a list of iSCSI Names, via a separate SendTargets command.
  • By issuing Service Location Protocol (SLP) multicast requests, to which the targets may respond.
  • By querying a storage name server (iSNS) for a list of targets that it can access.

iSNS: Internet Storage Name Server , is centralized server that has iSCSI configurations of initiators and targets.

SLP : Service Location Protocol, is not widely implemented, but anyway it helps computers to find iSCSI services across network.

iSCSI Security

iSCSI supports two separate security mechanisms:

  1. In-band authentication between initiator and target at the iSCSI connection level.(like CHAP authentication), occurs during login into storage
  2. Packet protection by IPsec at the IP level (all packets are secured).

 iSCSI will login to storage for the first time (here we implement in-band authentication if needed ),after successful login it will start exchanging  packets ( here we can implement IPSec if needed).

By now you might have understood that iSCSI protocol enables universal access to storage devices and storage-area networks (SANs) over standard Ethernet-based TCP/IP networks.

IP Ethernet network infrastructures provide major advantages for interconnection of servers to block-oriented storage devices. Following are the advantages that you would get using iSCSI;

  • Easy installation and maintenance of iSCSI SANs – Skills developed in the design and management of IP local-area network (LAN) networks can be applied to native IP SANs. Trained and experienced IP networking staffs are available to install and operate these networks.
  • Low cost – iSCSI uses the existing network infrastructure so there is no need to buy expensive equipment. Economies achieved from using a standard IP infrastructure, products, and service across the organization
  • Excellent performance– it is a very good alternative to the more expensive Fibre Channel technology. Gigabit Ethernet switches and routers have advanced capabilities including: ultra-low error rates, flow control, link aggregation, and full duplex operation. – Transfer data at optimal data rates over LAN, WAN, and metropolitan-area networks (MANs).
  • No distance limitation– using IP networking solves the problem with data replication to remote sites.
  • Interoperability and flexibility– iSCSI uses standard Ethernet switches so there is no need to install special cabling and switches required with Fibre Channel, it can also run at different Ethernet speed (users can choose between Gigabit and 10GbE or higher)
  • Compatibility– It is compatible with many commonly used standards, respected and recognized by the Internet Engineering Task Force (IETF). iSCSI is compatible with existing Ethernet and IP WAN infrastructures. iSCSI will coexist with other IP protocols on a network infrastructure.
  • Multipathing– iSCSI supports multi-pathing to improve network resiliency.
  • Security– iSCSI offers security features such as Challenge Handshake Authentication Protocol (CHAP) and Internet Protocol Security (IPsec).

After reading this blog you might have understood that neither FC nor iSCSI is a lower performance or lower cost protocol at this point and each comes with its own advantages and disadvantages and each is contesting by mutating itself into different forms to meet the market demand and price. So it’s actually up to you to choose what best fits for which purpose and deploy accordingly.

References: BytePile.com, Wikipedia, http://www.tomshardware.com, www,diskdrive.com

Advertisements