Storage Ruminations

Converged Enhanced Ethernet

• Lossless Ethernet must be FoE aware
• FIP snooping

Encapsulation

• Maximum size frame is 2240 bytes
• Mini-jumbo frame support is required
• Not an issue for direct connect environments today
• 802.1q tag carried CoS information

Data Center Bridging Capability eXchange Protocol (DCBX)

• Extension of the link layer discovery protocol (LLDP)
• Allows for exchange of priority map values for both FCoE and the FCoE initialization protocol (FIP)
• Enables lossless behavior

FIP – FCoE Initialization Protocol

• Bridges the gap between the expectations of FC and the reality of FCoE
• Allows initiators and targets to know who to login with
• One-to-many relationship is built in
• LKA (link keep alive) and FIP CVL (clear virtual links) allows for logout from fabric should the logical link be lost
• Implicit security (man-in-the-middle is difficult) provided that FIP snooping and dynamic ACLs are implemented
• However, FCoE with FIP is more complex and more open for mis-configuration issues
• Allows enode to:
  • Perform VLAN and FCF discovery
  • Ensure that the layer 2 network is capable of supporting mini-jumbo frames
  • Fabric login
  • LKA protocol

• FIP VLAN request
  • Multi-cast
  • Allow the enode (CNA) to discover which VLANs FCoE services are being provided
  • All FIP requests and responses use a pre-defined set of TLV (type, length, value) data structures.

• FIP VLAN notification
  • Unicast
  • Both FCFs response

• FIP Solitication
  • Multicast
  • Allows the enode to discover which FCFs (fibre channel forwarders) are available for login
  • 802.1q tag and max FCoE size field tag

• FIP Advertisement
  • Unicast
  • Note the priority, name id (fabric WWN), and max FCoE size
  • Max FCoE size is a field padded to the proper size (2158 bytes in length)
  • Dynamic ACL updated via FIP snooping

• FIP FLOGI
  • Unicast
  • The FCF with the lower priority is sent the FLOGI

Priority Flow Control (PFC)

• Necessary since FC requires lossless environment
• Without PFC, normal periodic SAN congesntion will cause frames to drop
• Uses 8 transmissions lines with independent transmit queues and receive buffers
• Maximum distance is today is 50 meters. Not a hard limit. It’s a function of the high water mark of the buffer and ?? (missed that detail)

Use cases

• Already have investment in FC or are building a new DC
• Already have a LAN that supports 10 GE to core
• FCFs today are anywhere between 2.5:1 – 5:1 oversubscribed (assumes FCoE switch is fully pop’ed with CNAs, will be solved when FCoE functionality is integrated into director class switches)

Physical Connectivity

• Twinax (copper) and short range optical (fibre)
  • Twinax is much cheaper then fibre solutions (1/10 the cost)
  • Twinax uses less power