Shifting from keeping the lights on to turning on new lights

This article was co-authored by Chris Boyd, a Senior Associate at Metis Strategy.

Today’s CIOs lead innovation efforts aimed at increasing revenue, accessing new markets, and growing product lines. However, according to Foundry’s 2022 State of the CIO Survey, 76% of CIOs say it’s challenging to find the right balance between business innovation and operational excellence. If CIOs can’t master operational excellence – “keeping the lights on” – they lose credibility with their peers and run the risk that the spotty Wi-Fi in the executive conference room will overshadow new innovations.

To strike the right balance, many technology leaders have adopted a Service Quality Index (SQI) to consistently measure and communicate the quality of “keep the lights on” work so their peers can quickly confirm the basics are covered and endorse innovation initiatives without pause.

 A CIO Service Quality Index (SQI) defines the key operational capabilities a CIO is responsible for delivering, the relative weight of the capability through the eyes of the customers, and the corresponding metrics that determine quality. In practice, it works like an equity index such as the S&P 500, summarizing the holistic performance of the CIO’s operational capabilities (e.g., the underlying stocks) with a single number, providing a basis for macro trend analysis (e.g., period-over-period performance), and allowing stakeholders to double-click into specific capabilities (e.g., sectors) for root cause analysis.

The customer-driven origins of SQI

FedEx built a SQI to better understand the level of quality being delivered to customers. They began by assessing their customer experience, which revealed 71 critical data points that strongly influenced the quality of service delivered, such as lost or damaged packages, missed pickups, overcharges, delivery delays, complaints, or unanswered customer requests.  Each of the 71 data points were weighted to reflect its relative importance to the customer based on the assessment.

The beauty of FedEx’s SQI is in its simplicity. In the dashboard, the company summarized the performance of all 71 data points with a single grade on a 0-100 scale. Once the SQI was published, FedEx executives could monitor the dashboard to identify trends and understand how well the company was serving its customers. Leaders also used the SQI to prioritize their work and increase focus on initiatives that made the biggest impact on customer experience. With a common set of data points to rally around, teams across the organization spun up projects aimed at “moving the needle” on one or many of the dimensions on the dashboard. FedEx’s SQI also helped ensure priorities were clearly communicated across the organization, and today it plays a major role in further improving customer experience and enabling the company’s growth.

Building a CIO Service Quality Index

Chances are you are among the majority of CIOs struggling to balance innovation and operational excellence. Or perhaps you are looking for a mechanism to show how aging infrastructure affects service quality to justify a new investment. In either case, consider constructing a SQI using the following five-step process as a smart first step.

1. Identify and weigh the dimensions of service quality
   • Just like FedEx examined its customer experience, start by assessing how various personas interact with IT. Identify the key operational capabilities your organization provides that visibly impact your constituents. A multi-billion industrials client, for example, chose to focus on employee help desk, network performance, data quality, platform availability, and issue resolution, since poor performance in any of these areas create negative customer sentiment and headwinds for the transformation agenda.
2. Weigh dimensions of service quality, identify metrics, and get buy-in
   • Determine the relative weight for each dimension of service quality based on the impact to the personas you identified. Work with leaders in your organization to understand what metrics you are tracking today, determine which ones will be useful to measure quality dimensions, and where new metrics may be needed. For example, you may determine that 30% of your overall SQI will be driven by the Issue Resolution quality metrics. For that dimension, you can use total ticket volume, average time to resolution, and business impact of critical issues to represent 20%, 20% and 60% of the quality dimension, respectively. Share the first draft of dimensions and metrics with your constituents. Make sure your weighting scheme appropriately reflects the views of the stakeholders that will be responsible for endorsing and using the dashboard for decision analysis. 
3. Build the MVP SQI and the “prospectus”
   • Build your MVP on a spreadsheet knowing there will be frequent changes in the early innings. If you want your index to have staying power, provide transparency into the mechanics. Consider creating a SQI “prospectus” that, like an equity index prospectus, allows stakeholders (investors) to understand the quality dimensions (underlying stocks in the index), how quality metrics are calculated, sourced and weighted (index performance), who is measuring and calculating quality (fund managers), and how frequently performance will be updated (monthly, quarterly, etc.).
4. Publish, iterate, integrate, and automate
   • Share the MVP and prospectus and provide ample opportunities for feedback and Q&A. Explain the purpose of the SQI and how it can be used as a prioritization mechanism as your organization thinks about capacity planning or where AI/ML might be used to drive quality improvements. Be open to feedback on quality measures and calculation and update the index using an iterative approach. Once you have achieved alignment, identify opportunities to integrate the SQI into other communication channels. Steering committees, existing operational reports, and department meetings are good starting points. One client decided to display the IT Service Quality Index, which was updated monthly, on large monitors that adorned the walls of the IT department to keep the team focused on quality. You can explore automation solutions for updating the index once you reach a steady state and changes become less frequent.

Bringing it together

Building a Service Quality Index will not make operational issues vanish. However, it will give you a tool that simplifies performance measurement and allows you to be surgical about remediation plans. Whether you are in the early or late innings of mastering operational excellence, the SQI can give you some breathing room and begin shifting the conversation from keeping the lights on to turning on new lights.