By Evert de Haan and Peter Verhoef
A large and growing amount of firms rely on Customer Feedback Metrics (CFMs) to monitor the customer base and the performance of the marketing department. Examples of these metrics include Customer Satisfaction (CS) and the Net Promoter Score (NPS). Recently, new customer feedback metrics, such as the Customer Effort Score (CES), are gaining traction with a promise to outperform the existing CFMs.
While the positive relationship between customer satisfaction and firm performance, including revenue and profitability, is well documented in academic literature, most findings are mixed for the NPS. In regards to the new Customer Feedback Metrics, such as the Customer Effort Score, third party empirical proof is relatively nonexistent, keeping managers in the dark about the reliability of these metrics. Despite the lack of academic literature and empirical proof, many firms rely on a single metric, specifically the NPS, as their key performance indicator.
Our research team aimed to shed more light on popular Customer Feedback Metrics by investigating the following issues:
- The extent in which different Customer Feedback Metrics are appropriate to monitor the customer base, and
- The effectiveness of using multiple metrics as opposed to using one single metric.
The research performed by our team proved that the NPS is as good as Customer Satisfaction in predicting customer retention. We also found labeling customers as Promoters, Passives, and Detractors works well for many firms. The NPS, combined with information regarding Customer Satisfaction, further improves the ability to monitor the customer base. Using multiple Customer Feedback Metrics is therefore highly recommended.
Table 1. Customer Feedback Metrics (CFM)
| CFM | Measurement |
| CS (Customer Satisfaction) | “All in all, how satisfied or unsatisfied are you with [company X]?” (1 = very unsatisfied, 7 = very satisfied). |
| Top-2-Box CS | The proportion of customers of the firm that gave a score of 6 or 7 on the CS question. |
| Official NPS (Net Promoter Score) | “How likely is it that you would recommend [company X] to a friend or colleague?” (0 = very unlikely, 10 = very likely). Respondents who gave a score of 0–6 are “detractors,” those who gave a 7 or 8 are “passives,” and those who gave a 9 or 10 are “promoters.” Subtracting the proportion of promoters by the proportion of detractors provides the Official NPS. |
| NPS (Net Promoter Score) | This is the average untransformed NPS score (0–10 range) provided by the customer. |
| CES (Customer Effort Score) | “How much effort did you personally have to put forth to handle your request?” (1 = very low effort, 5 = very high effort). |
In our research, we surveyed an extended group of customers from 98 firms across 19 different industries. In this survey we measured three different Customer Feedback Metrics, including Customer Satisfaction, the NPS and the Customer Effort Score. Information regarding these three different Customer Feedback Metrics can be found in the above table.
For Customer Satisfaction and the NPS, we used the untransformed scores as well as two popular transformations. The first transformation, the Top-2-Box CS, indicates the proportion of customers providing one of the two highest scores on Customer Satisfaction at a firm level. In other words, the Top-2-Box CS is the proportion of customers who are (very) satisfied. The second transformation is the official transformation for the NPS; grouping customers into Promoters, Passives, and Detractors. Further detail regarding NPS can also be found in the table above.
Two years after the initial survey measuring the Customer Feedback Metrics, we asked the same customers if they were still customers at the surveyed firm. This allowed us to test how accurately different Customer Feedback Metrics can predict actual behavior of customers. Given the historical strong, positive correlation to overall firm performance and firm value, our team looked at customer retention.
The graph below shows the strength of the relationship between the different Customer Feedback Metrics and customer retention, while controlling for firm- and industry heterogeneity, customer demographics and relationship length. Our research found that all Customer Feedback Metrics are significant in predicting customer retention, since all Customer Feedback Metrics perform better than having no Customer Feedback Metric information (i.e. the bar most to the left in the graph).
Transforming Customer Satisfaction and the NPS do significantly improve the predictions. This is indicated by the higher bars of these two Customer Feedback Metrics compared to their untransformed counterparts. The difference between the Top-2-Box CS and the Official NPS is not significant, so these two Customer Feedback Metrics work equally well in predicting customer retention. When looking at the three bars on the right you can see that combing the Top-2-Box CS with one of the other Customer Feedback Metrics leads to even better predictions. The combination of Top-2-Box CS and the Official NPS leads to the best predictions.
The Customer Effort Score, although statistically significant, is the least predictive Customer Feedback Metric compared to the other predictive measures. This finding contradicts the promises made by the developers of the Customer Effort Score who stated that it would outperform both Customer Satisfaction and NPS. Although this may be the case in some conditions, on a broader level this Customer Feedback Metric performs quite poorly. Therefore, we highly recommend firms and managers not rush to adopt Customer Effort Score, especially as a single metric, until it has been objectively shown that it is a good indicator of future customer behavior and/or firm performance. Customer Effort Score, as an indicator of future customer behavior and/or firm performance, can be proven by independent (scientific) research, or tested by the firm.
In conclusion, we recommend firms to continue using the NPS to track customers and performance, but also include the Top-2-Box CS in the dashboard of metrics. This dashboard enhancement will enable firms to better monitor and predict customer behavior and firm performance. Furthermore, we recommend firms to not only measure these Customer Feedback Metrics, but also link these metrics to customer behavior and firm performance. Doing so will result in a better understanding of the consequences of changes in the Customer Feedback Metrics, and help to make a more educated decision about which Customer Feedback Metrics to include, or exclude, in the dashboard. This approach can better enable firms to financially quantify the impact of marketing initiatives, which ultimately can help improve the position of marketing departments within firms.
The article The Predictive Ability of Different Customer Feedback Metrics for Retention featured in the post was co-authored by Evert de Haan (University of Groningen, The Netherlands), Peter Verhoef (University of Groningen, The Netherlands), and Thorsten Wiesel (Westfälische Wilhelms-Universität Münster, Germany). It is published in the International Journal of Research in Marketing, Volume 32, Issue 2, Pages 195-206.
_____________________________________________________
Evert de Haan is a PhD candidate at the Department of Marketing of the University of Groningen, The Netherlands. In September 2015 he will start as a Junior Professor in Marketing at the Department of Marketing of the Goethe University in Frankfurt, Germany. His research interests concern customer feedback metrics, marketing accountability, the effectiveness of (on- and offline) advertising, the customer’s online journey and the role of mobile devices play in this. He has published in the International Journal of Research in Marketing.
Peter C. Verhoef is Professor of Marketing at the Department of Marketing, Faculty of Economics and Business, University of Groningen, The Netherlands. He also holds a visiting position as professor at BI Oslo Norwegian Business School. He obtained his Ph.D. in 2001 at the School of Economics, Erasmus University Rotterdam, The Netherlands. His research interests concern customer management, customer loyalty, multi-channel issues, category management, and buying behavior of organic products. He has extensively published on these topics. His publications have appeared in journals, such as Journal of Marketing, Journal of Marketing Research, Marketing Science, International Journal of Research in Marketing, Harvard Business Review, Marketing Letters, Journal of Consumer Psychology, Journal of the Academy of Marketing Science, and Journal of Retailing. His work has been awarded with the Donald R. Lehmann award for the best dissertation based article in the Journal of Marketing and Journal of Marketing Research in 2003, the Harald M. Maynard Award for the best paper published inJournal of Marketing, and the Sheth Award for long-term impact of the Journal of Marketing in 2013. He is currently an editorial board member of the Journal of Marketing, Journal of Marketing Research, Marketing Science, Journal of Retailing, Journal of Service Research, Journal of Interactive Marketing, and the International Commerce Review. He functions as an area editor forJournal of Marketing Research and he International Journal of Research in Marketing. He has extensive teaching experience for undergraduate, graduate and Ph.D. students. He is also involved in executive teaching on customer management and is the founder of the Customer Insights Center, University of Groningen. He is department chair of the marketing department.
Great article! This will help me a lot in my future jobs. Thanks so much!
Thank you for your comment, Everto! We’re glad you found this post helpful!
These results are presented as if they are universal, cover all industries, business models and countries. Even in the original NPS article in HBR, the writers noted that NPS was more suited to some industries than others, as follows: “The ‘would recommend’ question wasn’t the best predictor of growth in every case. In a few situations, it was simply irrelevant. In database software or computer systems, for instance, senior executives select vendors, and top managers typically didn’t appear on the public e-mail lists we used to sample customers. Asking users of the system whether they would recommend the system to a friend or colleague seemed a little abstract, as they had no choice in the matter.”
I am curious about the industries covered by this research, and whether or not they have reached conclusions different to those in the original article.
First of all thank you for reading and replying to our post. We fully agree with your statement that the NPS (as well as the other metrics) are more suited in some industries than in other industries. The research conducted by us focusses on 93 business-to-consumer firms across 18 different industries (mainly retail and services). In our paper in the International Journal of Research in Marketing (2015, Volume 32, Issue 2, Pages 195-206, http://dx.doi.org/10.1016/j.ijresmar.2015.02.004), we also demonstrate how the effectiveness differs across the industries that we have studied. Indeed we find that the best metric is industry dependent and that the findings should not be overgeneralized. Our paper can also be found at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2568613.
For practitioners we highly recommend to test which metric is the best predictor for future customer behavior and firm performance in their specific case. This can for instance be achieved by linking metric data with performance and customer data. Doing this will create a better understanding about which (combinations of) metrics are the best predictors, how strong this relationship is and enables practitioners to quantify the long-term financial impact of changes in one of the metrics.
Interesting paper. A couple of things:
– can you expand on your interpretation of the predictive strength number? It seems to me rather low overall, suggesting little predictive strentgh
– you say there is a positive correlation between various CFM and company performance. Is there research you can point me to that shows causality in that direction? Has anyone tested the the opposite hypothesis: higher-performing companies tend to have higher CFMs? In other words, it is quite possible companies that have excellent products/services and are generally easy to work with tend to have more satisfied customers. In that case, the strategy should be to improve products and services, and wait for CFMs to follow. Suggesting that the entire CFM industry is built on a shaky foundation!
Thank you for these very valid question. The predictive strength is actually reasonably high. In our study, the amount of correctly identified customers in terms of retention would have been 50% (a flip with the coin) when we don’t use any information on the customer. This increases to 53.6% when we include age, gender, income and relationship length. This increases even further when we include CFM information, ranging from 55.0% for the CES to 65.1% for Top-2-Box satisfaction (all increases are statistically significant). Although far from perfect, these improvements in predicting customer behavior can be very valuable for firms in terms of deciding which customers to target or give special treatment.
The question of correlation and causality is indeed a very relevant one. Earlier studies indeed tended to look at the correlation of CFMs and firm performance and have drawn conclusions from that. More recent papers test for the causal direction by including both lagged CFMs and lagged performance, and with that try to explain current performance. This is in line with the procedure developed by Clive Granger to test for (Granger) causality. An interesting example of this is a study that has investigated the causal relationship between Customer Satisfaction and market share has been conducted by Rego, Morgan, & Fornell and is published in 2013 in the Journal of Marketing (http://dx.doi.org/10.1509/jm.09.0363). Earlier studies have shown that the correlation between Customer Satisfaction and market share is negative, in contrast to the positive correlation of Customer Satisfaction with other firm performance metrics. By using panel data, Rego, Morgan, & Fornell show that an increase in Customer Satisfaction does actually lead to an increase in market share in the next time period, but that an increase in market share does in turn have negative consequences for future Customer Satisfaction due to an increase in customer heterogeneity. So the relationship is actually more complex than what you would expect by just looking at correlations, and it is indeed very important to take this into account when researching this or when using CFMs in practice.
In our study discussed here we have the benefit that we have data from a lower (customer) level and that we measure both satisfaction and retention at two different periods in time (namely two years apart). We furthermore control for the length of the relationship, since customer who have stayed longer with the firm may have been more satisfied and may also be less likely to switch. Our model furthermore takes into account that customers who have given a higher CFM score are also more likely to participate in the second survey where we measure retention. By doing controlling for all of this, we can indeed say that the CFM has predictive value for future customer behavior, and it is very likely that this relationship is at least partly causal (although there may always be other variables involved which explain part of this relationship).
To summarize, there has indeed been studies that have proven the positive (Granger) causal impact of Customer Satisfaction and other CFM on firm performance and customer behavior. The strength of this relationship is reasonably high but far from perfect, since there are many other factors involved in this, especially when looking at a higher level such as firm performance.