Nexus 5500 to Nexus 7000 Multi-Hop FCoE Configuration Example

Configure

In order to begin this process, you must verify that:

• The Nexus 7000 has the FCoE feature enabled.
• The module F1 is licensed for FCoE.
• The system Quality of Service (QoS) is set correctly.
• The storage Virtual Device Contexts (VDC) are created.
• The dedicated and shared interfaces are configured.

In order to configure the Nexus 7000 Switch for Multi-Hop FCoE, complete these steps.

1. Enter these commands:

   default_vdc(config)# license fcoe module 1
   default_vdc(config)#system qos
   default_vdc(config-sys-qos)# service-policy type
   network-qos default-nq-7e-policy
   default_vdc(config)# install feature-set fcoe
   default_vdc(config)# vdc fcoe type storage
   default_vdc(config-vdc)# allocate fcoe-vlan-range [VLAN#] from vdcs [VDC Name]
   default_vdc(config-vdc)# allocate interface 
   ethernet1/31-32 <-- For FCoE VE port
   connectivity, you NEED dedicated interfaces;
   they cannot be shared!
   default_vdc(config)# switchto vdc fcoe

2. Configure the Ethernet and Virtual Fibre Channel (VFC) interface in the storage VDC. In order to configure the features needed for FCoE in the storage VDC (called FCoE in this example), enter:

   fcoe(config)# feature lacp
   fcoe(config)# feature-set fcoe
   fcoe(config)# feature lldp

3. Create the virtual storage area network (VSAN) and map the FCoE VLAN to the VSAN. This example uses VLAN 2 and VSAN 2.

   fcoe(config)# vlan [VLAN#]
   fcoe(config-vlan)# exit
   fcoe(config)# vsan database 
   fcoe(config-vsan-db)# vsan [VSAN#]
   fcoe(config-vsan-db)# exit
   fcoe(config)# vlan [VLAN#]
   fcoe(config-vlan)# fcoe vsan [VSAN#]
   fcoe(config-vlan)# exit

4. In order to configure the Ethernet interface as trunk and enable the FCoE VLAN on the trunk, enter:

   fcoe(config)# interface eth1/31-32
   fcoe(config-if)# channel-group 2 force mode active
   fcoe(config-if)# int port-channel 2
   fcoe(config-if)# switchport mode trunk
   fcoe(config-if)# switchport trunk allowed vlan 2 
   fcoe(config-if)# no shut

   Note: On Nexus 7000, by default the source-destination-oxid load balancing mechanism is used for FCoE traffic.

5. In order to create the VFC interface and bind it to the Ethernet interface, and enable the VSAN on the trunked interface, enter:

   fcoe(config)# interface vfc-port-channel 2 <--  There is an implicit bind; 
   it will be automatically mapped to the port-channel created in the previous set.
   fcoe(config-if)# switchport mode E 
   fcoe(config-if)# switchport trunk allowed vsan 2
   fcoe(config-if)# no shut

In order to configure the Nexus 5500 switch for Multi-Hop FCoE, complete these steps.

1. The first step on the Nexus 5000 switch is to enable the FCoE switching operation, and then ensure that LACP is enabled. The feature fcoe command is required in order to enable both Fibre Channel (FC) and FCoE on the Nexus 5500 switch.

   AwesomeN5k(config)# feature lacp
   AwesomeN5k(config)# feature fcoe

2. Enable QoS for FCoE operation on the Nexus 5500. Four lines of QoS statements map the baseline system QoS policies for FCoE. Without these commands, the virtual FC interface does not function when activated.

   system qos
   service-policy type qos input fcoe-default-in-policy
   service-policy type queuing input fcoe-default-in-policy
   service-policy type queuing output fcoe-default-out-policy
   service-policy type network-qos fcoe-default-nq-policy
   end

3. Create the VSAN and map the FCoE VLAN to the VSAN. This example uses VLAN 2 and VSAN 2.

   AwesomeN5k(config)# vlan [VLAN#]
   AwesomeN5k(config-vlan)# exit
   AwesomeN5k(config)# vsan database 
   AwesomeN5k(config-vsan-db)# vsan [VSAN#]
   AwesomeN5k(config-vsan-db)# exit
   AwesomeN5k(config)# vlan [VLAN#]
   AwesomeN5k(config-vlan)# fcoe vsan [VSAN#]
   AwesomeN5k(config-vlan)# exit

4. In order to configure the Ethernet interface as a trunk and to enable the FCoE VLAN on the trunk, enter:

   AwesomeN5k(config)# interface eth1/47-48
   AwesomeN5k(config-if)# channel-group 2 mode active
   AwesomeN5k(config-if)# int channel-group 2
   AwesomeN5k(config-if)# switchport mode trunk
   AwesomeN5k(config-if)# switchport trunk allowed vlan 2
   AwesomeN5k(config-if)# no shut

   Note: On Nexus 5000 the default load balancing mechanism on the LACP port-channel for FCoE traffic is “source-destination”. In this default state, all the FCoE traffic takes the same link in the port-channel when the Nexus 5000 forwards frames over FCoE VE ports. In order to enable all links to be used in the port-channel for FCoE traffic, enter the port-channel load-balance ethernet source-dest-port command to configure Nexus 5000 “port-channel load balancing” to “source-dest-port”. With this configuration “source-destination-oxid” load balancing is used for FCoE traffic.

5. In order to create the VFC interface and bind it to the Ethernet interface, and enable the VSAN on the trunked interface, enter:

   AwesomeN5k(config)# interface vfc 2
   AwesomeN5k(config-if)# bind interface port-channel 2
   AwesomeN5k(config-if)# switchport mode E
   AwesomeN5k(config-if)# switchport trunk allowed vsan 20

The VFCs are now online!