Finding Buffer Credit Issues

 

Buffer credit issues are typically caused by discrepancies in the I/O workload capabilities of an initiator or target. Usually the targets are the storage systems, and they are extremely fast, and often times the initiators are hosts that may be fast but are oversubscribed or have saturation or contention issues. This video demonstrates how to identify these issues.

Find the Root Cause of Buffer-to-Buffer Credit Shortages

A significant amount of buffer credits would be more than 20,000 or 30,000 buffer credit shortages per second. Oftentimes when we see this on ISL’s, the ISL’s are on the same switch, and oftentimes the shortages are caused by the same issue if they’re on the same switch.

An important thing to look for when we see buffer credit shortages is SAN Port Transmit patterns: transmit on the fabric and on the VMware systems, to see where the issues are. We usually don’t see the F ports or the initiator or target ports showing the buffer credit issues, but we do usually see some pattern that matches the buffer credit shortages on one of the F ports.

Additionally, we look at the VMware read throughput for a host that matches the buffer credit issue pattern exactly. And looking at the VMware guest, it runs on an ESX host, where we can see a significant imbalance in the port traffic. So, there’s an issue here where we have a host that is reading data off of the storage and fabric, but it’s not able to read it as fast as it’s being sent, and so there are some delays with the buffer credit shortages. We also see an imbalance in terms of the host activity on its two paths.

Resolve Buffer Credit Issues

There are a couple ways to resolve this. Ideally, we’d like to have traffic evenly going across the two different ports, and that can be done with Active-Active configurations, or by having multiple data stores that are hosting the active data and assigning those data stores to different physical paths. By balancing things better in the environment, you should be able to resolve the buffer credit shortages.

If that doesn’t resolve the issue, you need to look at the different speeds and feeds on the wire, and it may be a situation in which you have to upgrade the host port to a higher speed.

IntelliMagic Vision has reports that easily detect buffer credit issues. If you would like to discuss buffer credit shortages or issues in your environment with an IntelliMagic expert, feel free to reach out to reach out to us at info@intellimagic.com or contact us here.

Related

Blog

5 Things Every Storage Professional Should Be Checking - Prevent Fires Before They Start

This blog identifies five key areas that every storage professional should regularly check to identify and eliminate upcoming issues and, in turn, save time in the long run.

Read more
Blog

Common SAN Anomaly Signatures and How to Identify Them

The focus on this blog is to show some examples of the types of common zoning issues and how to identify their signatures proactively so you can remove risk in your SAN infrastructure.

Read more
Customer Experience

Medavie Blue Cross Proactively Ensures Availability of HPE 3PAR and Brocade Fabric

Medavie Blue Cross uses IntelliMagic Vision to help maintain a reliable infrastructure in their two data centers comprising over 1 PB of HPE 3PAR Storage and Brocade Fibre Channel SAN switches.

Read more

Go to Resources

Request a Trial or Demo

Whether you’re in the early stages of product research, evaluating competitive solutions, or trying to solve a problem, we’re happy to help you get the information you need to move forward with your IT initiatives.

Discover End-to-End SAN Performance, Capacity, and Configuration Monitoring

IntelliMagic Vision for SAN

End-to-End SAN Performance, Capacity, and Configuration Monitoring

Learn More

IBM Spectrum Virtualize, SVC

Performance Management for IBM Spectrum Virtualize, SVC and Storwize

Learn More

HPE 3PAR

Performance Management for HPE 3PAR StoreServ Storage

Learn More

NetApp

Performance Management for NetApp FAS

Learn More

VMware

Breakthrough integration of VMware performance and configuration data

Learn More

Dell EMC VMAX

Performance Management for Dell EMC VMAX

Learn More

Show all

Flexible Deployment and Monitoring

On Premise

Install the software on premise and use it offline

In the Cloud

The Cloud based deployment can be accessed from everywhere in the world and is easy to share with colleagues,

Add On Services

Get help with the deployment and the management of the software