Lee LaFrese - 13 October 2014

I live in Tucson, AZ. The joke here is that most of the year the weather is so stable you can publish the forecast on a billboard – sunny, clear skies. Sounds nice, doesn’t it?  But every July we have monsoon season and the weather turns “interesting”. I suppose that you could still put the forecast on a billboard – sunny with a chance of wild and crazy storms! These storms are unpredictable and sometimes quite violent. Unfortunately, weather science is not to the point where it can clear up the uncertainty. Thankfully, in the world of storage performance we can do better.

Typical IT shops have various real-time monitors designed to raise alerts when something goes wrong. In theory this sounds like a good arrangement. If you get an alert, you can take action and fix things quickly, right? But in reality this won’t always be as effective as it would seem. On the one hand alerts may point to symptoms after you have already felt the impact. Do you want to hear that it is raining when you are already soaked to the bone? It would be much preferable to have advance warning before the problem manifests itself. On the other hand, sometimes you get more alerts than you know what to do with. It may be unclear whether these alerts indicate a real problem or whether they are just a bunch of false alarms. This is the classic “alert storm” and it can sometimes be as disruptive as a real problem. You don’t want your team to scramble because of a bunch of false positives. However, you can’t just discount alerts, because if there really is a problem you ignore them at your own peril.

Once I was talking performance with a storage manager who casually mentioned that they had over 60,000 alerts in their monitoring system. Needless to say having 60,000 alerts is the same as having 6,000 alerts or no alerts at all. Ideally a monitoring system will give you a manageable number of alerts that are actionable. Otherwise why do performance monitoring at all?

This is where a solution like IntelliMagic Vision can help. Vision is designed to be proactive and give you advance warning when issues are starting to pop up, well before they impact users. Vision comes preset with thresholds based on the capabilities of your specific storage infrastructure. In many cases these thresholds are dynamic and adjust from interval to interval based on the workload attributes. These factors all contribute to mitigating false alarms and alert storms. Thresholds may be adjusted to include service level objectives if desired but often this is unnecessary.

Traditional performance monitors report on key performance indicators (KPIs) and may alert you when there is a statistical deviation from the norm.   However short term changes in KPIs may or may not tell you anything useful. Alerts in Vision are based on knowledge of the hardware capabilities and are presented as key risk indicators (KRIs) that reflect sustained over-utilization of internal components. Thus KRIs are inherently more meaningful than KPIs.

With IntelliMagic Vision you are able to avoid alert storms. The deep insight and analysis tools that IntelliMagic Vision provides helps you quickly determine root cause if a storm should develop. Vision can also help you decide if the alert storm is truly a problem or just a red herring.

The bottom line here is that real-time is too late. Alert storms turn into fire drills that just suck up resources. If you supplement your real-time monitoring with the right tools, you can find shelter from the storm.

Related

Brochure

The z/OS Performance and Capacity Skills Gap

IntelliMagic Vision collects performance and configuration information on the VMware, fabric and storage systems to provide a complete and end-to-end picture.

Download
Webinar

Understanding & Dealing with z14 Traffic Patterns

The z14 is designed for massive, parallel processing. So why do delays still occur? This webinar will explore common sources of application delays and discuss practical solutions to reduce these delays.

Watch Webinar
Blog

Best Practices for Managing your SAN Performance (Part 3: Planning)

Within infrastructure capacity management it is important that we consider growth to help us understand future costs for budgeting purposes.

Read more

Go to Resources

This article's author

Lee LaFrese
Senior Storage Performance Consultant
Read Lee's bio

Share this blog

Subscribe to our Newsletter

Subscribe to our newsletter and receive monthly updates about the latest industry news and high quality content, like webinars, blogs, white papers, and more.