Home

Fibre Channel Part 1 – FCP and WWPN

https://learn.flackbox.com/courses/81445/lectures/5204536

Terminology

Aliases: Easy to identify WWPNs and WWNNs

• Alias EXCHANGE-SERVER = WWPN 01:23:45:67:89:ab:cd:ef
• Configurable on both storage systems and Switches
• Much easier to identify during troubleshooting and configuration

Client: The host accessing storage

FC HBA: Fibre Channel Host Bus Adaptor

• See HBA

FCP: Fibre Channel Protocol

• SAN specific. Original SAN protocol but still very popular
• Dedicated hardware, adapters, switches on all layers of the OSI model
• Used to send SCSi commands over the Fibre Channel network
• Very stable and reliable. Lossless, unlike TCP and UDP
• Bandwidths of 1, 2, 4, 8, 16, 32, 128 Gbps
• Uses WWN World Wide Names for addressing

HBA: Host Bus Adaptor

• Controller board for network ports. Often plug into the MoBo’s PCIe slots
• Possible to have Multi-port HBAs
• Generally used for Fibre Channel Ports(FC HBA)

LUN: Logical Unit Number

• Represents a disk that will be presented to the host.
• Can be thought of as a Storage device. Might be a disk partition, whole disk, or combination of disks.
• Used with SAN

NAS: Network Attached Storage

• File level. Typical shared storage between multiple usesrs
• Cannot be used to boot a device

SAN: Storage Area Network

• Block level. Appears as a disk mounted directly to the device.
• Can be used to boot a device

SCSI: Small Computer System Interface

Target: The storage system

WWN: World Wide Names

• 8 byte address of 16 hex characters – 01:23:45:67:89:ab:cd:ef

WWNN: World Wide Node Name

• Assigned to a node in the storage network
• A WWNN can identify multiple network interfaces of a single network node
  • You might have a multi-port HBA

WWPN: World Wide Port Name

• Like a MAC address. You can only have 1 WWPN per port (Physical Interface)
  • Globally unique
  • Burned into each port by manufacturer
• Multi-port HBAs will have different WWPNs on each port
• Primarily concerned with WWPNs vs WWNNs when configuring Fibre Channel networks.

Fibre Channel Part 2 – Security

https://learn.flackbox.com/courses/81445/lectures/5204538

Zoning

• Zoning is configured on the switches to control which hosts are allowed to communicate with each other
• Initiators (servers) will be allowed to communicate with targets (storage systems) but initiators will not be allowed to communicate with each other of Fibre Channel
  • Initiators talk to each other over TCP, not FCP
• Note that in the diagram, both hosts connect to the storage over the same WWPN (NETAPP-CTRL1)
• Zoning is generally configured on 1 switch then configured to auto-replicated to others
• Zoning prevents unauthorized hosts from reaching the storage system, but does not prevent a host from accessing the wrong LUN!  See LUN Masking

Zone Set: A group of Zone configurations

• See Zone diagram

LUN Masking

• Used to ensure the correct host is connecting to the correct LUN.
  • In the Zone diagram, both hosts can reach all LUNs on the storage.
  • This can cause corrupt data and is a security concern.
• Configured on the Storage system to lock a LUN down to the host or hosts authorized to access it.
• To secure storage, you need to configure Zoning on the Switches and LUN Masking on the Storage systems

Fibre Channel Part 3 – Fabric Login

https://learn.flackbox.com/courses/81445/lectures/5204537

Switch Domain IDs

• Each switch in the Fibre Channel network is assigned a unique Domain ID
• One switch is automatically assigned as the Principle Switch.
  • The Principle switch assigns the Domain IDs to the other switches
• Each switch learns about the other switches in the network and how to route to them based on their Domain ID.

FLOGI Fabric Login

• When a server or storage’s HBA powers on, it sends a FLOGI (Fabric LOGIn) request to its locally attached Fibre channel switch.
  • FLOGI Request includes the HBA Port’s WWPN
• The switch will assign it a 24 bit FCID Fibre Channel ID address
• The FCID assigned to hosts is made up of the switch’s Domain ID and the switch port the host is plugged into
• FCIDs are similar to IP addresses (but not exactly). It’s used byfibre channel switches to route traffic between servers and their storage
• Switches maintain a table of FCID to WWPN address mappings and which port the host is located on.

FCNS Fibre Channel Name Service

• Fibre channel switches share the FLOGI database information with each other using the FCNS Fibre Channel Name Service
• Each switch in the network learns where each WWPN is and how to route traffic.

PLOGI Port Login

• After the FLOGI Fabric Login process is complete, the initiator will send the PLOGI Port Login.
• Based on the zoning configuration on the switch, the host will learn its available target WWPNs

PRLI Process Login

• Finally, the initiator host will send a PRLI Proccess Login to its target storage.
• The storage system will grant access to the host based on its configured LUN masking.

Conclusion

As long as LUN masking is setup on the Storage and Zoning is configured on the switches, all of this happens automatically!

Fibre Channel Part 4 – Redundancy and Multipathing

https://learn.flackbox.com/courses/81445/lectures/5204539

Redundant SAN Fabrics

• Storage is mission critical. Access should provide no single point of failure.
• Redundant Fibre Channel networks should be put in place, known as Fabric A and Fabric B (or SAN A and SAN B)
• Each server and storage system host will be connected to both fabrics with redundant HBA Ports.
• Fibre Channel switches distribute shared information to each other such as Domain IDs, the FCNS database, and zoning.
• If an error in Fabric A was able to propagate to Fabric B, this would bring down both fabrics and drop the servers connection to their storage.
• For this reason switches in different sides of the fabric are NOT cross connected to each other. Both sides of the fabric are kept physically separate
• End hosts are connected to both fabrics, but the switches are not.

True Redundant Configuration

This would include redundant Storage Controllers as well as redundant servers!

TPG Target Portal Groups

• All ports on the storage system which initiators can access their storage through are added to a Target Portal Group
• Example: Ports CTRL1-A,CTRL1-B,CTRL2-A,CTRL2-B are added to a Target Portal Group
• Used for Security and/or Performance

ALUA – Asymmetric Logical Unit Assignment

• Defines Optimized paths for connection to the LUN.
• There are likely 2 or of these as long as
  • The number of “hops” remains the same.
  • You terminate at the controller that owns the desired LUN
• These can easily go through both Fabric A or Fabric B since ‘hop counts’ would be the same.

Multipathing

• During the Process Login, initiators will detect ports available to connect to their storage in the TPG (Target Portal Group) and ALUA will notify which are the preferred paths.
• Multipathing software on the initiator will choose which path or paths to take to the storage.
  • Will typically take the optimized path
• All popular operating systems have multipathing software which supports active/active or active/standby paths.
  • Active/Active should auto load balance.
• The client will automatically fail over to an alternate path if the one it is using fails.

Conclusion

• Client connectivity to SAN storage is fundamentally different to how Ethernet networking works
  • In Ethernet networking, all of the routing and switching decisions are handled by network infrastructure devices
  • In SAN storage, multipathing intelligence is enabled on the client end hosts.
• In Fibre Channel, the initiator will automatically detect the available paths to its storage through the FLOGI, PLOGI and PRLI process
• Multipathing software on the initiator will then choose which path or paths to use.

FCoE Overview

https://learn.flackbox.com/courses/81445/lectures/5208174

FCoE – Fibre Channel over Ethernet

• Viewed as “The Next Big Thing” in storage networking when it first came out.
  • Has been falling out of favor over the past few years.
  • More likely to find ISCI when deploying SAN over an Ethernet network.
• All MFGs support it.
• FCoE became possible with 10Gbps Ethernet. Enough bandwidth to support both data and storage traffic on the same physical network infrastructure.
  • 1G was a bit too slow.
• FCoE uses the FCP (Fibre Channel Protocol), but encapsulates it in an Ethernet header
• QOS is used to guarantee the required bandwidth to the storage traffic
• It retains the reliability and performance of Fibre Channel

Traditional FC Network

• There are
  • 4 ports
    • 2 Ethernet NICs
    • 2 FC HBAs
  • 4 Cables
  • 4 Switches

FCoE Network

• Shared Infrastructure using 10Gbps.
• Does require a CNA – Converged Network Adaptor
  • Same as regular Ethernet card, but supports FCoE
• Cut infrastructure in half.
• Still works the same as FCP, but is encapsulated.
  • Still have WWPNs, uses FLOGI, PLOGI, PRLI Process.
  • Still uses MAC addresses for TCP/IP

How this works

• Virtualize the Physical Interface into two virtual interfaces
  • Virtual NIC with a MAC addr. for Ethernet data
  • Virtual HBA with a WWPN for storage traffic.
  • Storage and data traffice are split into two different VLANS

Data VLan

Storage VLAN

All together now, 1, 2, 3.

Lossless FCoE

• FCP is lossless.  It ensures that no frames are lost in transit between the initiantor and target
• Ethernet is NOT lossless.  TCP recognizes when a packet does not reach its destination and resends.
  • This is expected and is normal SOP.
• FCoE uses FCP, which assumes a lossless network, so we need a way to ensure the storage packets are not lost while traversing the Ethernet network.
• PFC Priority Flow Control FCoE extension for Ethernet is used to ensure lossless delivery.
• PFC works on a hop by hop basis
• Each NIC and switch in the path between the initiator and target must be FCoE capable.
• FCoE capable NICs are known as CNA Converged Network Adaptors

FCoE Lab Demo

https://learn.flackbox.com/courses/81445/lectures/5204540

iSCSI Overview

https://learn.flackbox.com/courses/81445/lectures/5204542

SAN Protocol that runs over Ethernet Networks

• iSCI Internet Small Computer System Interface protocol
• Runs over Ethernet and was originally viewed as less expensive alternative to Fibre Channel
• Higher packet overhead than FC, lower reliability and performance
• Mature and popular SAN technology.
• Can share Ethernet or can have its own dedicated network infrastructure
• TOE (TCP Offload Engine) cards are specialist adapters which can be used to offload the storage TCP/IP processing from a server’s CPU.
  • These are sometimes called iSCSI HBAs.
  • Can still be used for normal Ethernet networks

iSCSI Addressing

• Fibre Channel using World Wide Names to identify initiators and targets
• iSCSI uses IQN iSCSI Qualified Names
  • or less common EUI Extended Unique Identifier)
• The IQN can be up to 255 characters
  • iqn.yyyy-mm.naming-authority:uniqueName
  • Ex: iqn.1991-05.com.microsoft:testHost
• The IQN is assigned to the host as a whole, similar to the WWNN in FC
• iSCSI runs over Ethernet, so individual ports are addressed by IP
• iSCSI does not support FLOGI/PLOGI/PLRI Process, so administrators must explicity point the initiator at its target by specifying one of the IP addresses in the Target Portal Group
  • It will then discover the target’s IQN and other ports in the TPG
  • Example: if the target has 4 ips, once it connects to one, it will learn the remaining 3.
• Multipathing software on the initiator can then choose which path or paths to take
• Although it runs on Ethernet, iSCSI is still a SAN protocol with multipathing intelligence on the initiator.

iSCSI Security

• LUN Masking is configured the same way, except it uses the IQN vs. the WWPN to identify the client.
• Zoning is not supported
• Password based authentication is typically configured on the initiator and darget to guard against spoofing attacks
• End to end IPSec encryption can be enabled to enhance security.

San Protocol Stack Comparison