In this short video we want to show how to identify Inter-switch Link (or ISL) contention and show how adding additional ISL’s can spread out the load and reduce the utilization on your SAN fabric. We will show how we identified a problem with these links and then we will compare the utilization and buffer credit shortages/sec after an additional ISL was added.
Today I’m going to demonstrate how you can leverage IntelliMagic Vision to identify fabric bottlenecks. This real world case study shows how adding additional ISL’s can spread out the load and reduce the utilization on your fabric.
The SAN Fabric is the collection of switches, and sometimes routers, that connect hosts to storage systems over a fibre channel network. Currently, 8 Gbit/sec is the standard host interface speed, although we are seeing storage connections and switch connections running 16 Gbit/sec. In some Storage Area Network environments, or SAN for short, there are switches that connect to other switches. These are called Inter-switch Links, or ISLs for short, and the port type for ISLs is ‘E’. You can think of the ISL’s as the connectors that link major freeways. When these connectors do not have enough lanes to keep traffic flowing smoothly it can cause congestion and result in buffer credit shortages which can lead to performance delays. In this use case we will show how we identified a problem with these links, and then we will compare the utilization and buffer credit shortages/sec after an additional ISL was added.
This is a port utilization chart by switch. The utilization data is aggregated at the switch level so the average utilizations are pretty low. The ratings for the busiest port on each switch are rolled up to the switch level, so that if you look in the legend you will see ratings for several of the switches in bold and with ratings ranging between .40 and .76. In this next chart we will drill down on SWITCH-000000015 to inspect the individual ports. Anything over .3 indicates that the warning threshold has been exceed at least 30% of the time or the exception threshold has been exceeded 10% of the time.
In this chart you see the switch port average utilization by port type. The E port shows as the group of ports that has the highest rating with a rating of .46. We will drill further down to the individual E ports by selecting the E port line in the chart.
There are four ports in the legend that have a rating of .46. During the peak period around 8:30 the utilization averages 90% for these four ports. This is very busy and is most likely contributing to buffer credit shortages at this time. Now let’s look at the buffer credit shortages at this time.
During the period from 8:00 – 9:00 PM there were a significant number of buffer credit shortages. At this customer site we recommended adding an additional ISL to spread the load over another port. Let’s take a look at what happened when we added the additional ISL.
The Zero Buffer to Buffer credits (Errors/sec) decreased by 27.64% or 6,040.70 errors per second. Let’s look at the utilization.
The port utilization improved by 19.29% with an absolute change of 7.02%. While it did not resolve all the issues, adding the additional ISL improved the performance and reduced the number of buffer credit errors/sec during the peak periods.
In this brief video we demonstrated that IntelliMagic Vision for Fabric was able to identify high ISL port utilization that once addressed, resulted in a significant reduction in the overall utilization and buffer credit shortages on the ISL ports affected.
For more information on how IntelliMagic Vision can reduce risk and improve performance in your SAN environment please contact us at firstname.lastname@example.org
Medavie Blue Cross Proactively Ensures Availability of HPE 3PAR and Brocade Fabric
Medavie Blue Cross uses IntelliMagic Vision to help maintain a reliable infrastructure in their two data centers comprising over 1 PB of HPE 3PAR Storage and Brocade Fibre Channel SAN switches.
Finding Buffer Credit Issues
Buffer credit issues are typically caused by discrepancies in the I/O workload capabilities of an initiator or target. This video demonstrates how to identify these issues.
Availability Intelligence for End-to-End SAN Performance & Capacity
Transform unpredictable outages into predictable ones by creating Availability Intelligence from the storage performance and configuration data.
Sign Up for a Free Trial
Experience firsthand the deepest visibility into the performance, capacity, and configuration of your VMware, fabric and storage systems. Whether you’re in the early stages of product research, evaluating competitive solutions, or trying to solve a problem, we’re happy to help you get the information you need to move forward with your IT initiatives.