We regularly look for opportunities to benchmark the results of our social media research against the findings of more orthodox consumer surveys. It helps us to fine-tune our approach and to work out how direct questions can be translated into the millions of indirect, unsolicited statements we collect in social media.
Our latest head-to-head with the classical research model involves the study of mobile network customer satisfaction. In March this year the Consumers’ Association released a new Which? report on the UK’s mobile network operators. The research covered much of the same ground as one of our internal social media data sets so we decided to examine how our findings matched up, or not.
The Which? results were based on the experiences of over 6k customers of thirteen contract, sim-only and pay-as-you-go providers. Customers were asked to rate their network provider for a range of service issues and for overall value for money. Customers were also asked whether they would recommend their network to friends and family. The headline conclusion of the Which? report was that smaller, virtual networks delivered a higher level of customer satisfaction than the big network owners EE, O2, Three and Vodafone.
Within the leading group of virtual networks, Giffgaff and Utility Warehouse were the most highly-rated providers. Giffgaff was the only service to achieve the Which? five-star rating for value for money.
Vodafone was the lowest-rated network, with a one-star rating for customer service. One in five customers voted Vodafone as being poor value for money. O2 customers were also unhappy with the value for money they were getting, with one in ten rating it as poor.
Looking at the results of Newton Insight’s study of customer reactions in social media, the overall conclusion of the Which? report stands up. The virtual networks consistently achieve a higher ratio of positive reactions from their customers than the Big Four networks.
Giffgaff and Utility Warehouse are highly-rated by customers in social media, just as they are by the Which? panel. The Newton Insight data has Utility Warehouse ranked above Giffgaff, so reversing the order for the top two in the Which? report. The reason we have Utility Warehouse as #1 is because the network has a superior ratio of positive reactions, although we acknowledge that Giffgaff generates more reactions in absolute terms. It’s also worth noting that in the Which? data, Utility Warehouse achieves a higher average star rating than Giffgaff, even though Which? ranks Giffgaff #1.
Further down the list, the composition of the mid-table is the same in both reports, with Tesco, Asda, Sky and ID Mobile clustered together. The health warning here is that some of these providers have very low levels of customer engagement in social media. We suspect Which? also had to make do with a small volume of responses for Asda Mobile.
One observation which wasn’t really discussed in media coverage of the Which? report was the significance of the findings at sector level. If we pull back the lens and consider the star ratings of the network operators together, Value for Money and Customer Service were the two best-performing indicators. Ease of Contacting was some way behind the two best indicators, which chimes with our own observations in social media, where no response or a slow response are the key drivers of customer irritation.
The sector’s overall Which? rating for Incentives was low, suggesting that customers either don’t care for the freebies as much as the networks believe, or that the networks are not getting incentives right. There were no five-star ratings for incentives in the Which? report and in our social media data incentives were rarely referenced as a motivation to change networks. Perhaps there is an opportunity for one of the networks to put some meaty offers on the table.
All-in-all, the signal from both reports is remarkably similar, even though our approaches are very different. Which? asked 6k people to roll-up their opinions into an overall assessment of satisfaction. Newton Insight’s findings from social media used a more fluid equation based on how 300k customers expressed themselves at the ‘moment of experience’ at any point in the last 12 months.
If you haven’t seen our data before, we segment reactions by emotion, using a variation on the Plutchik model (grab hold of your nearest psychologist to explain it). In the dashboard below, the high ratio of positive emotions expressed by customers of the virtual networks is clear, but in the chart on the right, you can see that most of the virtual networks have only a small share of customer engagement in social media.
Our approach analyses the day-to-day triumphs of customers as well as their frustrations, so we see customers’ likes and dislikes with equal clarity. For the mobile networks, our conclusion is that there are positive elements within the customer relationships for all operators, including those ranked at the bottom of the Which? table. It’s worth noting that Vodafone achieves more positive reactions from its customers than all the virtual networks combined. The problem for Vodafone, and for the other members of the Big Four, is that the good is so outweighed by the bad.
But the big networks have scale on their side and an opportunity to examine what they do well and the areas where improvement is needed. Vodafone, O2 and EE already have the customers, they now need to make them happy.