This doesn’t happen often, but let’s assume you have been assigned to find some cost savings, and you have tools to employ. Finding ‘gold’ in a productive mine field is fun! Your experience with other systems, in similar and other industries, has given you some pretty good ideas on what to look for.

Frequently, CICS ‘tuning’ exercises focus on unnecessary TCB switches. Without getting into what these are and why these might be happening. What makes a good candidate for improvement opportunities?

Figure 1: Pareto Chart-CICS transaction MSU consumption

This report has a clear winner for biggest consumer, but after #1, the results tail off quickly. There’s always the possibility it might be ‘fools gold’, but you can’t ignore the size of the 1^st item compared to the others.

There’s sometimes a debate on whether TCB switch time or TCB switches are the key item of focus for tuning efforts. Since both metrics are directly related to the same SMF field, it’s not surprising that there is a strong correlation between them, as seen below. It’s also important to recognize that the big hill in the middle of the report is the day-time peak for this transaction rates. Analysts should always be mindful of the distinction between correlation and causation. It may not be the what, but the how that’s important for TCB switches.

Figure 2: Transaction Rate compared with TCB Switch count and TCB switch time

If TCB Switch time is an important correlation of CPU consumption, then the next Pareto can help us with some early verification of the quality of the nuggets we are looking for. The top CPU consumer, call it Claims, in the first Pareto chart may only be 4^th or 5^th down the line in TCB Switch Wait Time Pareto chart, so that keeps us on track for a more in depth view of what / how TCB Switch Wait is occurring.

Figure 3: Pareto Chart-CICS transaction TCB Switch Time

The CICS and application experts typically apply their knowledge and experience to find several options for reducing CPU time. I’ll dig deeper into that in the paragraph below, but the next report shows some remarkable improvements!

Figure 4: Before-and-after results show a dramatic drop in CP Processor Time and TCB Switch Wait Time

After making a change, TCB switches were greatly reduced – saving up to 2000 CPU seconds at peak for essentially the same transaction workload.

CICS Transaction Details and CPU drivers

Sometimes a fresh set of experienced eyes can spot things that should be done and are easily done. With hundreds of CICS SMF metrics, what are some that we haven’t covered yet that may drive CPU and/or TCB switches?

Some that were involved in these savings include: VSAM Strings, and Buffers and just plain I/O. As this IBM reference suggests, some of this is configuration related and may have been relatively easy to implement. I/O is often overlooked as a driver of CPU, but it does drive CPU, and avoiding I/O with application changes and configuration changes should always be on the list of savings ideas. Trading memory for MSUs $ave$.

Below are a few reports highlighting the changes made over time. Note the critical dates around Dec. 2022, and May 9, 2023. The first report shows a change in mid-December where the number of CICS Active Strings begins showing up and progressively growing worse until early May.

Figure 5: CICS File Control Statistic – Active Strings (current active updates against a file)

The applications changes included CICS changes to ensure performance, but there aren’t always easy ways to ensure performance is protected and costs are controlled. This report below shows that buffer changes went in with the application changes.

Figure 6: Defined file control data buffers by region

Big Changes in May

As with many efforts, there’s often more than one cause, and more than one action required to remedy. In this situation, a few more actions may be required, but some action was taken to reduce VSAM browse requests. This large drop coincides with the drop in CPU and TCB switch time.

Figure 7: CICS VSAM File Browse Requests

There are more interesting details, and with so many CICS metrics and additional insights, it’s hard to keep things concise. The intent here is to keep it brief and highlight a few of the interesting points. The upshot for the system is a lower processor demand. In addition, a few key transactions are much more responsive. Specifically, the response time dropped from a peak of 700 ms, down to below 10 ms, while the CPU time went from nearly 40 ms, down to under 6 ms per transaction.

Figure 8: Claims transaction profile

Sharing the Success

The tendency of experts can be to share so many details on what was done that we lose track of the important message. How the changes reduced the TCB switch time, which in turn saved a ton of CPU, up to 2,000 seconds. This part of the story was intended to share some of the details and provide some incentive to proactively look for changes in workloads before problems arise. I look forward to sharing a bit more on effectively sharing the success in an upcoming blog. We would love to help you find the ‘nuggets’ quicker and would welcome an opportunity to share the kinds of insights that will help you deliver more for less!

Most of us can identify with situations where we are deluged with data, and we face challenges wading through it all and pulling out the value. A scenario that comes to mind is the proliferation of TV shows now available. As if the shows on the hundreds of networks available on cable TV weren’t enough, the challenge is compounded by the plethora of programming now available through streaming services.

Current cable TV “GUIDE” functionality (where you scroll page by page through a list of hundreds of shows) certainly doesn’t cut it. The streaming services attempt to provide some help through lists of shows divided into categories like drama or comedy, but unless you know specifically what you are looking for in advance, their search capabilities still leave a lot to be desired.

Analyzing CICS Performance

Mainframe performance analysts can consider themselves fortunate to have access to more extensive metrics than on any other platform, but often find themselves facing a similar uphill battle trying to sift through gigabytes if not terabytes of SMF data produced daily at their sites. This task of deriving value from massive quantities of data can be particularly challenging for analysts charged with effectively analyzing CICS performance from the SMF 110 CICS Monitoring Facility data.

It is clear this cannot be accomplished by analysts sifting through tabular data presented in countless static reports. To successfully achieve this, there are conceptually two ways you need to put the power of the computer to work for you.

One is for the computer to be automatically checking key metrics associated with the thousands of elements in your infrastructure to identify hidden risks to availability.

A second is to exploit a powerful graphical user interface to dynamically interact with the data, leveraging context-sensitive drilldowns enabling you to determine based on the current chart what the next logical analytical step should be.

In this blog I am going to focus on that second aspect, the analytical value provided as a highly flexible user interface makes data easily accessible.

Analyzing Time Measurements for CICS Transactions

Of the nearly 400 fields in the SMF 110-1 records, approximately 100 represent various time measurements (e.g., CPU, response, wait), so that is clearly an important focus for analysis. As a starting point, one might choose top transactions by total CPU consumed (typically called “CPU intensity”) or by transaction volume. The latter appears in the screenshot below generated using IntelliMagic Vision.

From here, we can drill down to examine the response time of a specific transaction over the indicated time interval, divided into the commonly-defined categories for CICS response times.

The ability to select other metrics for potential correlation can also facilitate analysis. In the following example, the report was customized to add “transaction rate” (the blue-green line) on the right axis (any variable could be selected).

Further analysis into the “Total I/O Wait Time” category may be warranted and can be achieved with a single click. In this example, “Wait for Control at end of MRO” constitutes almost all the wait time.

CICS also reports on the components of CPU in great detail, as reflected in the legend of this chart.

“Before and after” analysis (e.g., of the impact of an application release) on CPU per transaction can quickly be determined by converting this into an area chart and selecting any interval in the database for comparison (requiring only a few clicks).

It may be advantageous to begin analysis from the perspective of response times for high volume transactions (as in this example).

Analysis for a selected transaction could continue through drilldowns into response times by CICS regions and/or by z/OS systems (pictured below) to determine if response times varied significantly across the sysplex (in this case response times were generally similar).

Analysis could also begin from the viewpoint of CICS regions, with similar drilldowns including by transaction, response time categories and components, CPU time, z/OS system, etc. (not shown here due to space limitations).

Making CICS Data Manageable

In addition to the time-based metrics previously considered, there are twenty other categories of metrics provided in the SMF 110-1 records, as seen in the column on the right.

Easy visibility to explore this data and context-sensitive drilldowns for detailed analysis make these metrics readily accessible and positions the analyst to quickly derive value from them.

Whether you’re searching for the right TV show or the response time of certain CICS transactions, you need an easy and effective way to filter out the noise and rapidly find what you’re looking for.

From high level overviews to detailed, intuitive drilldowns, IntelliMagic Vision takes the previously unmanageable quantity of CICS data and makes it manageable. Rather than sifting through page after page or report after report trying to identify the significant data points, your analysis can proceed in a direct and expeditious manner to find key insights and actionable information.

This is one of the key reasons why over the course of my decades-long career spent primarily as a performance analyst I found IntelliMagic Vision to be a game-changing tool enabling me to perform analysis at the highest level.

See more on IntelliMagic Vision’s CICS reporting here.