What is a KPI anyway?

It seems that almost any indicator is referred to as a KPI, so what are they really? As the name states, a KPI is an indicator of key performance, not just of performance. Find out which indicators to follow, and which to ignore. Recently, there has been an upsurge in the level of interest in management by objective, or in establishing performance measures to track progress towards asset maintenance goals. One of the terms that are regularly misused and poorly defined is that of the Key Performance Measure, or KPI. It seems that almost any indicator is referred to as a KPI, so what are they really? As the name states, a KPI is an indicator of key performance, not just of performance. In order to try to better explain this point we will look at an example of performance measurement as applied to a fleet of haulage trucks within a mine site. However, the same principles can be applied to fixed equipment within a process plant, manufacturing facility, or even to a distributed asset company such as utilities or rail. Within a mine site there are three or four different operations, exploration drilling and blasting, mining the ore, processing it or filtering it, and then loading it for sale or transport. While all of these are important processes the management of the fleet of haulage units or trucks is one of the key lynch pins in terms of operational cost control and meeting production targets. The fact that these items are all independent of each other, consume large quantities of energy, and cost a fortune for spare parts adds to the complexity of managing them, as does the fact that every eight or 12 hours they generally have a different person at the wheel. So how do we know they are working well? There are a few standard measures that can be applied to any form of fleet equipment, but it really depends on what it is that we want to know. For example; a truck can be working well from a energy consumption standpoint, but not from an operational standpoint; or it could be working well from the point of view of operations but not from the point of view of cost effectiveness. We could represent these by a range of metrics, each one of them important in their own right, but with no real indication of how each one relates to the other, or of how important they are with relation to other aspects of machine performance. This is at the heart of the problem; when we confuse metrics with KPIs we miss out on a range of additional useful information relating to overall balanced performance of the asset. Let’s look again at the fleet management issues in more detail. As we started to discuss earlier a haul truck has several areas where direct performance matters, these could be:
-Energy performance
-Cost effectiveness
-Productivity
For the sake of this example we will leave out other aspects such as safety and the environment for now. If we take productivity there are a range of performance measures that could be applied within this area. These take on the standard sorts of direct performance measures that you regularly see, e.g. availability, reliability or failure rate (MTBF), cycle time for a specific run, tons during a specific period, average load rates etcetera. All of these are useful indicators of the performance of this asset, but not all of them are directly related to this asset alone. For example, tons during a specific period relates to how the truck is loaded as well as the speed with which it works, load rates are similar in that they rely on other assets to make up the measure. If we were really going to construct a scorecard these figures would be a must-use, but in this case we will work without them. One measure that most people readily understand is that of availability, or the amount of time that an asset is available for duty when it is required to be available. The definition is important because it is no good if an asset is available for work when nobody needs it, particularly if it is not available when they do need it. By itself it tells us a fair amount about the asset; it tells us that it was ready to work when we needed it to work. But the asset could still be performing poorly, even though the availability is high. Let’s take the example of a pump that is required for ten hours and is available for 9 hours. This would give us 90% availability; good for some industries but not so great for others. But what if the pump had actually broken down 20 times over the ten hour period, with each breakdown taking two minutes to correct? If this were the case, then the mean time between failure, or the MTBF would be: 10 hours required20 failures= 30 minutes. So on average the pump would be running for about 30 minutes prior to breaking down and needing attention again. So by itself the availability figure is not telling us the entire story, we need to look at other figures such as the MTBF in this case. Are we done? What other issues could there be regarding availability of the asset? What about quality of workmanship? Not so important if everything is going well, but pretty vital if things are not going well. So maybe there could be a good case for including a measure such as Mean Time To Repair, (MTTR) a proxy for quality or for speed of work. We could also include a measure of rework if anybody could agree on what the definition of rework is. So now we have three metrics that make up our view of the trucks productivity. Although we ruled out load rates and other indicators earlier there is probably another indicator that we could also include; that of Unit Costs. Unit Cost is a great measure in any industry as long as it is defined as the costs of maintenance, operations or both; against the unit of production. Unit Costs are sometimes defined differently in utilities and infrastructure organizations and this can be misleading. In this case the unit costs shows us how much our availability is costing us, availability by itself is great and allows us to have an asset that is contributing to the production targets, but how much is this costing? For example; a truck that is working but cannot take heavy loads due to developing issues with the wheel suspension system, is lowering productivity rates and raising unit costs of production. So now we have four, for ease of understanding these four indicators could be ar-ranged like the graphic below. Here we see that Availability remains the key indicator we use to tell us how we are going in terms of productivity, but it is not the only indicator of productivity. Other indicators are also included in various other perspectives, E.g. unit cost in the cost ef-fectiveness perspective and MTBF in the quality of performance perspective. So we can star to see that a KPI is not just a performance indicator, it is the indicator that tells us the key information we need to know in relation so a specific strategic theme or area, in this case productivity. However, unit cost may end up being the KPI for cost effectiveness. Sound like semantics? It has been my experience that the correct use and definition of KPI’s, as opposed to indicators or general metrics, allows people to go directly to the agreed “most important” indicator of a specific area of performance. And if this is also tied in with a scorecard type approach, as it is in the graphic above, then they will get a picture of how the asset is performing as well as a good indication as to why. There has also been a recent trend away from several indicators and towards one or two significant indicators. If you correctly use a KPI approach, then you can use all of the indicators that your data will support. Every time you look at the scorecard your eye is taken to the KPI, and then it is an easy task to look only at those indicators that are leading you towards why the asset is performing poorly. This sort of graphical dashboard is useful tool for managing large scale asset perform-ance in any situation, particularly given the ease of use of today’s technologies.- (PlantReliability, 21 Sep 07)

Links:

Consulting/Training: http://alfalahconsulting.com

Consultant/Trainer: http://ahmad-sanusi-husain.com

Islamic Investment: http://islamic-invest-malaysia.com

Customer metrics: What should you measure?

By Neil Davey


"Be careful what you wish for," the saying goes... "you might get it!" And this could be particularly apt when it comes to customer data.
There's been a spike in the demand for customer metrics recently. Firstly, an increasing number of CEOs are recognising that non-financial measures such as customer satisfaction are as important to their investors as traditional financial figures, a fact emphasised in Deloitte's 2007 study 'In the Dark'. But equally as significant is the increasing accountability that is being demanded of marketers, with the sector being asked to demonstrate its value now more than ever.

What CEOs and marketers may not have realised until this point, however, is the enormity of the task that faces any firm trying to cut a swathe through the mass of customer data that is at their disposal. Put simply, companies are up to their eyeballs in customer information – and they don't necessarily know what to do with it.

As highlighted in Deloitte's study, for instance, whilst leaders have an excellent idea of what traditional financial figures to use, they are bamboozled by customer data. 87% of companies are happy that their financial measurement is good, Deloitte reports, but only 29% can say the same about non-financial indicators.

And many marketing teams are similarly unacquainted with customer metrics. A 2007 study by VisionEdge Marketing revealed that whilst 78% of respondents track leads to conversion, only a quarter track and measure the rate of customer acquisition and fewer than 10% measure customer lifetime value or customer advocacy. Furthermore, a third of the marketing professionals questioned omit metrics from their marketing plans altogether.

Without a doubt, the sheer volume of customer data that is out there presents a daunting task. It's little wonder that firms are asking themselves what customer analytics and metrics they should – and shouldn't – be focusing on.

A broader focus

If you are looking for a showcase example of a company that has put metrics at the heart of its business, then supermarket Tesco is an obvious choice. However, according to Andrew Jordan chief operating officer of beyondanalysis, it also represents a good example of where customer metric models can frequently go wrong. "There are two fundamental flaws in its model," says Jordan. "Firstly, it relies on a very heavy level of transaction data and secondly it only attaches itself to customers."
An oft-quoted problem associated with transactional data – as with focus on similar financial measurements such as profit margins – is that it encourages leaders to drive their firm using 'the rear view mirror'.

"Purchases, repeat visits, length of call time… many companies track key performance indicators (KPIs) to monitor the successes and failures within the business – including customer satisfaction and churn rates – but the data produced only tells you what has happened and nothing about the underlying drivers of these trends," says Gary Schwartz, VP of product marketing at Confirmit. "The CRM industry is based on examining historical purchase behaviour in order to unlock the secrets and predict purchase behaviour but more often than not, however, repeat purchases are simply a function of lack of other choices!"

A focus solely on customers is similarly misplaced, as it results in the metrics completely omitting anyone who has failed to consider shopping at Tesco or those who have proactively decided not to shop there. "Anyone and everyone has the potential to become a customer, whether they have been a customer in the past or not," stresses Jordan. "So to use metrics that only attach themselves to known quantities is very traditional... very CRM-based. It misses a vital dimension because all you are doing is looking at things like share of wallet and repeat purchases and traditional value. They're all very well, but you've got to start earlier in the journey and understand how these things came about in the first place. A lot of the advice that we give companies is to think in a broader context about how they go about aligning the same metric approach to things like customer acquisition as they do with their own customers."

The internet in particular has created a wealth of data on non-customers for firms to exploit according to Jordan. "I'm referencing the rather murky world of social media, but also the fact that people are now collaborating electronically a lot more and that is creating a very rich stream of data which tells you a lot more about how people lead their lives, why they make the decisions they do, which ultimately inform purchasing decisions."

With the field of 'customer experience' gaining growing prominence, it's no surprise to learn that firms are increasingly looking to apply metrics to 'experiential' aspects. But with so many firms running on ‘command and control' metrics, this doesn't necessarily mean that companies are any better at delivering the experience to their customers that their brand values demand. Indeed, with it could be argued that in many cases the result has simply been that they only deliver what they measure. And without input from – and empowerment of – those employees 'at the coal face', the firm may not even be looking at the most appropriate metrics.

"It's clear that the metrics set at the top of the organisation – usually around shareholder aspirations – dictate the behaviours of that organisation toward customers," suggests Tony Mooney, consulting and propositions director at Experian Integrated Marketing. "These are rarely customer experiential metrics, in our experience. The nearest many organisations get is the use of customer satisfaction surveys and average call answering statistics - neither of which adequately measure customer experience.

"For example, most companies with call centres use average call answering as a KPI. This is merely a hygiene factor and, anyway, is usually inaccurate as it measures call answer times from the point at which the poor customer has made it through several layers of IVR. Of far greater importance than how quickly you pick the phone up is how the call is handled, eg single contact resolution. For other organisations, customer behavioural metrics will be key – such as downgrading and requests to cancel. Understanding the key customer metrics is a process of sound causal analysis – to identify the important drivers of customer behaviour and monitor those. Too many organisations try and manage 'output' metrics and miss the indicators."

Getting the metrics mix right

Clearly the quest for a single customer metric that holds the key to success for every company is a futile one. Behavioural metrics and experiential metrics have an important role to play alongside the more traditional ones for the modern business. But different metrics will hold a different value for different firms. So is there a way to establish the most important metrics specific to your firm?

One approach is the balanced scorecard. The balanced scorecard, arguably the most widely-used management framework of the last 50 years, allows firms to take all the potential metrics available and weight them and then track them over time. The process would, for instance, involve firms drawing up a list of key customer goals - perhaps customer satisfaction, new customer acquisition, customer retention, customer loyalty, fast response, efficiency, reliability or image – and then creating a number of metrics to measure success in the fields – which could consist of a focus on customer satisfaction index, repeat purchases, market share, on-time deliveries, returned orders, new customer acquisitions or perceived value for money.

"The question is whether all customer measures are of equal importance – and if not, how do we decide what we should be focusing on?" says Dana Guthrie of the performance management group at Actuate. "No two organisations have the same strategy, so every company needs to make its own judgement about the most important customer measures for themselves. The [balanced scorecard] process, though, is always going to be the same: a top-down approach of applying your own unique strategy and objectives to decisions about what to measure."

On the face of it a common-sense approach, the balanced scorecard actually involves a rigorous process to select and define the key measures that will ensure successful strategy execution.

Some scepticism of the technique's effectiveness in this area exists, however. "It strikes me very much of management overkill and of trying to design a complex mechanic for something that shouldn't be that complex," says Jordan. "And the problem of doing that, not withstanding the actual process of putting a balanced scorecard together in the first place, is that you'll be continually trying to challenge the validity of the scorecard rather than the validity of the results. The danger is that you will over complicate something that doesn't need to be complicated."

Nevertheless, whilst the balanced scorecard approach is not a guarantee of success when it comes to incorporating customer measures into the performance management mix, it has proven popular – as its longevity attests.
Professor Robert Shaw, though, believes there is a far simpler way that companies can identify the most important metrics and jettison those that are surplus to requirements – by evaluating their value to the decision-making process.
"The first thing is that people need to focus away from the data and onto the question of decision support," he suggests. "Many firms haven't stepped back and asked themselves if the data is actually supporting their decision making. You should ask the question: what are the main applications in this in terms of decisions? You can do audits of how you are applying the data and analytics technology and expertise to answer key decision questions. What comes out of those audits is a great deal of clarity about the value of the technology and data to the decision-making process. And then you can start to prioritise them."
Certainly firms need to take some action to wrestle control of their customer data – a problem that has been especially exacerbated by the internet. "There are a few hundred new metrics available to firms that they can capture that they didn't have a few years ago," agrees Neil Morgan, VP marketing EMEA, Omniture. "It's the biggest change I've seen in consumer marketing. Most traditional business people are struggling to interpret it or action it. The big requirement is to be able to raid this, adapt these metrics and make them useful in a business."
But Shaw insists that there is a dawning realisation amongst some firms that the focus should be on quality – not quantity.

"We used to get a bucket load of data, now it is like having a fire hose pointed at us - and firms don't know what to do with it," he concludes. "A radical rethink is needed. Companies need to take an axe to research and cut the stuff that is not illuminating the decision-making process. The enlightened companies are already doing this. They are asking themselves what decisions they are taking – whether it is to do with direct marketing, or pricing, or how much they advertise, or product/service quality. And then they are asking themselves what they can actually do about something like service quality and how they can measure if that has an effect which ultimately finds its way back to the financial results of the company. And unless it throws some light on the way that service or price or whatever hits the bottom line, then these enlightened companies simply won't be interested in that research."
The rest of the market, however, may yet be rueing the day they wished for more customer data...

Customer Satisfaction: Your Most Important KPI

Contact centers today have three basic goals - to build or retain revenue, to control operating costs, and to grow customer satisfaction. Contact center professionals have gotten very good at measuring revenue and efficiency achievement but still have a ways to go in quantifying customer satisfaction. According to research conducted by the Purdue Center for Customer-Driven Quality, less than half of contact centers have a formal program to measure customer satisfaction. There are many reasons why every contact center needs to have this vitally important information: Satisfying callers -- or more accurately delighting them -- is the primary or secondary goal of virtually every contact center. If we don't know how well agents are satisfying callers then we don't know if we are achieving one of our most important goals. More meaningful agent evaluations - Customer satisfaction cannot be inferred from indirect measurements like KPI performance and quality monitoring. Only the caller knows if he or she was completely satisfied. The very process of asking improves customer satisfaction - People appreciate the fact that you care enough to ask and value their opinions. It makes good business sense - Several studies have demonstrated that it makes good business sense to satisfy customers. It typically costs five to 10 times as much to replace a customer than to keep the ones you already have. According to the Harvard Business Review a 5 percent reduction in customer defections can lead to an 85 percent boost in profits. Understanding Customer Satisfaction Customer delight is a reflection of the sum total of interactions with all customer-facing functions. It is greatly influenced by factors well beyond the control of the agent such as quality of the service or product, pricing, errors in billing, delivery snafus, collection procedures, or the performance of the front line salesperson or service technician. If our intent is to evaluate the perceived quality of the agent interaction in isolation from other influencing factors it is essential to collect this information immediately after the interaction, not days or weeks later when the customer has forgotten much of what happened during the call but still feels upset about the outcome. And if possible, measure customer satisfaction for all customer touch points. For years leading restaurant chains, hoteliers, retailers, and utilities have polled customers at the point of service. They do this by printing an invitation to call a toll free number on your bill or service statement, usually accompanied by a reward such as a discount on your next meal. With data from all touch points businesses can better determine if they are delivering quality customer care on all levels, from the store clerk to the contact center. Methods for obtaining customer feedbackOutbound telephone surveys are the most common method for collecting customer satisfaction data. Telephone surveys provide useful information but are costly, can take weeks to execute, and require a heavy time commitment from contact center management. Postal surveys are less expensive but require even more time to execute. The method is subject to large statistical errors because of small sample sizes and it is impossible to track back to specific agents. E-mail and Web surveys are popular because they are economical, timely, and can be traceable to specific agents. But these methods cannot capture responses from the 30 percent of the public that does not have home Internet access. Also, most companies do not have current e-mail addresses for their callers. The recent introduction of automated post-call IVR surveys has greatly improved the timeliness and accuracy of customer satisfaction measures. Response rates compare favorably with other survey methods and the cost, while high at the outset, is quickly recovered when compared with the cost of repetitive telephone surveys. However, like the other methods feedback can only be secured for the agent interaction. If the wedding gift is delivered a week after the wedding the caller will still be unsatisfied, no matter how well the agent handled the unfortunate situation. On-Demand Customer Satisfaction Surveys In recent years we have seen growing deployment of hosted software solutions in the contact center. Contact centers can now acquire CRM, IVR, call recording, and call routing via the hosted model. With hosting buyers pay for services on an as-needed basis. The hosted model has been used for many years to measure customer satisfaction with hospitality, retail, restaurant, and field service personnel. Hosted surveys offer many advantages to contact centers: There is little or no initial capital investment and there is little or no requirement for internal I.T. support. Customer satisfaction scores can be tracked to specific agents and interactions. Scores can even be imported directly into agent scorecards. Samples sizes are as large as you wish, assuring more accurate information. Volunteered customer comments are recorded. You will know why callers feel they way to do, and in their own voice. Feedback is immediate. Satisfactions scores can be displayed on dashboards and wall boards in real time. Hosted surveys are platform-vendor agnostic. It does not matter which company supplied your call recording or IVR system. You retain complete flexibility. Reports can be tailored to your needs. You can track changes over time, compare different contact centers, even drill down to better isolate causes of extreme satisfaction or dissatisfaction. The hosted model can easily be extended to include other customer touch points like sales staff, technicians, and collections. SummaryEvery call center has well developed metrics that measure achievement of performance and efficiency goals of contact centers but less than half of call centers measure customer satisfaction levels. Many have tried but given up because of budget constraints, demands on management time, and deficiencies in the data collection methods. The hosting option provides a flexible, economical, and highly effective solution that can work with contact centers of all sizes and budgets. - (Destination CRM, 21 Sep 07-Author: Dick Bucci, an associate consultant at The Pelorus Group)