Modernize your SAN with Cisco MDS 64G Fibre Channel – Cisco Blogs

Today’s Storage Area Networks (SAN) need to evolve to 64G Fibre Channel to support the increasing application workloads and performance requirements. 

So the question is “should you upgrade your Cisco MDS 9700 Series to 64G Fibre Channel?”  To help answer this question, let’s review the unique and innovative capabilities of Cisco’s MDS 64G Fibre Channel:

Key Cisco MDS 64G Fibre Channel (FC) Innovation

Double the data rate

The new Cisco MDS 64G FC module supports an I/O data rate of 6400MB/s that provides double the I/O throughput per FC port compared to the MDS 32G FC module at 3200MB/s. As the All-Flash Array storage vendors rapidly consolidate towards 32G FC connectivity to support FC-NVMe workloads, it is imperative that the FC inter-switch links (ISLs) also step up their speeds in proportion (to 64G) to maintain the same levels of oversubscription that the fabrics were designed to support.

Investment protection

The Cisco MDS 64G Module is compatible with the existing MDS 9700 chassis with Supervisor-4 (SUP4) and Fabric-3 (FAB3) modules. The SUP4 hosts the 64G capable Arbiter ASIC and FAB3 hosts the 64G capable Crossbar ASIC. With the addition of 64G support, the MDS 9700 switch offers unprecedented investment protection over the three Cisco MDS FC 16G/32G/64G module generations we have offered over the last eight years.

To upgrade to Cisco MDS 64G is as simple as finding an empty slot on the Cisco MDS 9700 switch (with SUP4 and FAB3) and inserting the Cisco MDS 64G module and upgrading NX-OS software to 9.2(1). The upgrade is non-disruptive with no impact to data traffic.

Each of the 48 ports of the Cisco MDS 64G Module can operate at 8G/16G/32G/64G speed with any Cisco certified 8G/16G/32G optics. The Cisco MDS 64G optics are scheduled to be certified and available for purchase soon.

Denser and Energy efficient ASIC

The earlier Cisco MDS 32G Module had 3 separate switching ASICs of 16 ports each. The 64G Module takes advantage of latest advancements in chip fabrication and uses 2 dies of 24 FC ports each packaged into a single dual-die ASIC of 48 FC ports. This results in two key benefits:

• Fewer components and interconnections on the Module leading to simple design and increased hardware reliability as shown in the diagram below
• Better power efficiency. The typical power consumption of the 64G Module is roughly the same as the previous generation 32G Module. This means for the same power consumed, the 64G Module can switch twice the amount of traffic as the 32G Module.

More ASIC resources

The Cisco MDS 64G ASIC provides more capacity and better performance:

• The MDS 64G module doubles the number of B2B credits (buffers) to 24K as compared to the Cisco MDS 32G module. By default, Cisco MDS 64G Module E-ports are allocated 1000 buffers and F-ports are allocated 100 buffers. The additional buffers can help absorb short bursts of I/O traffic resulting in improved I/O performance. It also means lower probability of congestion spreading in the fabric.

The Cisco MDS 64G long-distance E-ports are allocated 3500 buffers by default which can be extended all the way till 16000 buffers. This allows for the farthest native FC extension on the Cisco MDS switch available today.

• The MDS 64G module provides 128K Ingress ACL entries for 24 ports. This is significantly more ACL entries per port than the Cisco MDS 32G Module. As a result, larger zones (more members) can be supported for hardware zoning. This is particularly important when the MDS 9700 switch is acting as an NPIV core switch for multiple downstream NPV switches. It may be noted that zoning ACLs of NPV devices are programmed on the NPIV F-port of the MDS 9700 and requires large number of ACL resources.
• The MDS 64G module provides 8 QoS levels. This is double the number of QoS levels that the Cisco MDS 32G Module provides. The additional QoS levels allows differentiated handling of varying levels of SAN congestions. The I/O traffic patterns can be classified into 8 different ways and segregated into dedicated QoS levels with independent B2B crediting for each QoS level.

More Secure Encryption

The Cisco MDS 32G ASIC supports 128-bit AES encryption and decryption. The Cisco MDS 64G ASIC supports for up to 8 FC TrustSec ports with 256-bit AES encryption/decryption at 64G rates. The 256-bit AES adds an extra layer of security.

FC TrustSec based encryption is important when interconnecting SAN fabrics across data centres using long distance FC links. The Cisco MDS 64G Module now allows you to extend your SAN the farthest with the most secure encryption technology.

Inline Analytics for a Billion IOPS

The SAN Analytics capabilities of the Cisco MDS 64G ASIC is a huge architectural uplift from the Cisco MDS 32G ASIC. The Cisco MDS 64G ASIC can analyze every SCSI or NVMe I/O being switched on all the 48 ports simultaneously at line rate and in real time without involving the NPU.

One Cisco MDS 64G FC port can be switching a theoretical maximum of about 28 million IOPS (512B I/O Size) and a 48 port Module can be switching well over a Billion IOPS. The Cisco MDS 64G ASIC can investigate each one of the Billion I/Os per second and extract about 70 I/O metrics on a per flow basis.

The Cisco MDS 64G Module also comes with a dedicated 1GbE telemetry port so that flow metrics can be optionally streamed directly out of the Module. This is designed to support high scale or high frequency streaming use cases.

There are also 8 new metrics with the Cisco MDS 64G Module compared to Cisco MDS 32G Module that deliver deeper insights into I/O behaviors.

Enhanced congestion detection

“TxWait” is a critical per-port counter supported on the Cisco MDS 32G ASIC. It helps quantify the severity of a slow drain condition in the transmit direction of a switch port. In addition to “TxWait”, the Cisco MDS 64G ASIC adds supports for a “RxWait” counter that can quantify slow drain conditions in the receive direction of a switch port. The “TxWait” on an F_Port in combination with “RxWait” on ISL ports can together help detect some unique congestion spreading scenarios in the SAN.

In addition, the credit monitoring counters like “credit not available”, “credit loss” etc., are now computed in the Cisco MDS 64G ASIC itself improving the precision of these metrics.

Many more new features, capabilities, and enhancements

The Cisco MDS 64G ASIC has many more exciting new capabilities like programmable lookups on NVMe command frames, VMID Analytics, Drop frame analysis, Inline Analytics capabilities just to name a few.

Conclusion

Cisco’s 64G module provides a range of innovations to help SAN customers adopt and evolve their SAN. With Cisco MDS 64G, SAN teams do not have to choose between deploying a 64G capable infrastructure or leveraging their existing storage investment. Cisco is staying true to its long-standing promise to customers to increase value of the existing SAN infrastructure verses forcing a chassis upgrade and replacement of optics.

We have provided information about the many key benefits of the new Cisco MDS 64G Module that establish the value of upgrading your Cisco MDS 9700 Series to 64G Fibre Channel (FC). Whether you are looking to improve the performance of your current storage workloads, adopt FC-NVMe, increase security, improve power efficiency, or enhance congestion detection, the Cisco MDS 9700 switches with 64G FC are ready to deliver these benefits and help you increase the value of your existing SAN.

To learn more, visit 64G Fibre Channel Done Right

Unlock the full value of your Storage Area Networking solution

Cisco MDS 9000 series switches simplify implementation, operations, and administration with its ability to non-disruptively upgrade any SAN fabric. Cisco MDS SAN switches help maximize the value of any data center investment for any organization, across any vertical.

Resources:

64G Fibre Channel Done Right – Blog

Key Innovations to Modernize Your Storage Network – Video on Demand

Optimize, Accelerate, and Simplify SANs Non-disruptively – ESG white paper

Cisco SAN Insights Discovery (SID) Tool – Meet your new storage fabric sidekick

Prevent SAN Congestion with Dynamic Ingress Rate Limiting – Video on Demand

Share: