Converged Data Center: FCoE, iSCSI, and the Future of Storage Networking

The Journey to Convergence
Stuart Miniman, Office of the CTO

The iSCSI Story

• Transport SCSI over standard Ethernet
• Reliability through TCP
• SCSI has limited distance, iSCSI extended the distance

Non-Ethernet Convergence Options

• Infiniband
  • Used broadly for High Performance Computing (HPC) environment
  • Low cost and ultra-low latency geared for server to server cluster
  • Separate use from general network (Ethernet) or storage (FC or Ethernet)
• PCIe
  • Extension of the server bus to an I/O aggregation box
  • Not a standard, small players
  • Still using Etherhet and FC network and storage from an aggregation box

Maturation of 10Gb Ethernet

• Allows replacement of n x 1Gb with must smaller number of 10Gb adapters
• Single network allows for easier mobility for virtualization/cloud deployments
• Simplifies server, network and storage infrastructure

Standards

• 40Gb and 100Gb Ethernet (IEEE) standards will be completed in June 2010
• 16Gb FC (T11) standard is targeted for completion at the end of 2010
• 32Gb is in the works
• Server Adoption of FC ~3+ years, of Ethernet ~5+ years

Protocols and Standards
Fibre Channel over Ethernet

• Developed by T11, International Committee for Information Technology Standards (INCITS) T11 Fibre channel Interfaces Technical Committee
• FC-BB-5 standard ratified in June 2009

Converged Enhanced Ethernet

• Developed by IEEE Data Center Bridging (DCB) Task Group
• Commonly referred to as Lossless Ethernet
• IEEE standards targeting ratification mid-2010

iSCSI and FCoE Framing

• iSCSI is SCSI functionality transported using TCP/IP for delivery and routing
• FCoE is FC frames encapsulated in Layer 2 Ethernet frames over Lossless Ethernet

FCoE Frame Formats

• Ethernet frames give a 1:1 encapsulation of FC frames, no segmenting FC frames across multiple Ethernet frames

FC-BB-6

• Next step
• Not required for multi-hope FCoE or other current deployments
• Likely to support point-to-point configuration which allows two FCoE devices to communicate without going through an FCF (or switch)
• Today initiator cannot talk directly to a target without a switch in between. FC-BB-6 is investigating this.

Lossless Ethernet

• IEEE 802.1 Data Center Bridging (DCB) is the standards task group
• Converged Enhanced Ethernet (CEE) is the industry consensus term
• Link level enhancements (Priority Flow Control, Enhanced Transmission Selection, Data Center Bridging Exchange Protocol) are shipping in products today
  • PAUSE and Priority Flow Control
    • Classic 802.3x PAUSE is rarely implemented since it stops all traffic
    • New PAUSE known as PFC that can halt traffic according to priority tag while allowing traffic at other priority levels to continue. This creates lossless virtual lanes.
  • Enhanced Transmission Selection
    • Maintain low latency treatment of certain traffic classes
  • Data Center Bridging Exchange Protocol
    • Auto-negotiation for devices as they determine the link parameters
• The CEE cloud or DCB-enabled LAN is only for the portion of your network that requires lossless Ethernet

Beyond Link Level

• End-to-end
• Congestion notification
  • IEEE 802.1Qau ratified
  • Allows a switch to notify attached ports to slow down transmission due to heavy traffic
• Layer 2 multipathing
  • IETF TRILL – Transparent Interconnection of Lots of Links
  • Used with STP to provide more efficient bridging and bandwidth aggregation
  • Focuses on bridging capability that will increase bandwidth by allowing and aggregating multiple network paths

Solution Evolution
iSCSI

• iSCSI was >15% of revenue ($1.8B in 2009) and >20% capacity in SAN market in 2009
• 10Gb iSCSI solutions are available
• iSCSI natively routable (IP)
• iSCSI solutions are much smaller scale than FC

FCoE

• FCoE with direct attach of server to Converged Network Switch at top of rack or end of row
• Tightly controlled in 2009
• First solutions are with FcoE aware Ethernet switch (FIP snooping)
• Blade > FCoE switch > FC switch > Storage

Rack Area Network (RAN)

• Cisco UCS
• Vblock
• VCE (virtual computing environment)

Timeline

• End of year, you will see FCoE native in CX and DMX