On the face of it, the results of the airline satisfaction studies released by Newton Insight and Which? look remarkably similar. I have always believed in the potential of social insights to match results assembled from orthodox customer surveys.
In the table below I have compiled results for the nine airlines common to both studies, in order to show how each research company rates these specific airlines against each other.
Looking at the results in more detail, the signal for customer satisfaction is consistent at the top of both tables.
The same airlines occupy the top four positions, although the placing of Delta and Etihad is flipped. I suspect this is a consequence of our different approach to sample sizing and the relatively low exposure of the Which? audience to Delta.
The main difference comes in mid-table, where Which? has Easyjet in fifth place, followed by British Airways.
for Newton Insight, BA occupies fifth spot and Easyjet is close to the bottom. Why the difference? I think there are two things at play here. First, Easyjet is the most represented airline in the Which? survey with almost two thousand responses. This is not aligned with Easyjet’s actually share of passenger volumes. The Newton Insight results have been normalised to account for the difference in passenger volumes between airlines.
The second point is to do with survey approach. Which? asks passengers to roll up their views of an airline in response to direct questions. This approach can influence what passengers consider to be front of mind. That’s not a potential problem unique to Which? It is a general point about the directional nature of question-based surveys.
Finally, each airline, including Easyjet, receives a single score based on a passenger’s recollection of their experiences throughout the year. So memory can influence the outcome.
On the plus side, moment-in-time surveys tell you how people are feeling right this minute. But the next flight means a new result.
Newton Insight measures passenger experiences very differently, by harvesting reactions in real time. This means if a passenger travels with Easyjet five times in a year, we have unique data sets for each journey to accurately reflect the passenger’s mood. As new journeys are enjoyed or endured, our results are updated. There is no problem with memory bias; we collect unfiltered reactions at the moment of experience.
The phasing of the two surveys also influences the overall ratings. Which? offers a single hit of responses for Sep-Oct. I’m not sure if this puts more emphasis on journeys made over the summer and in the second half of the year. These experiences will certainly be more recent in the minds of most respondents.
Newton Insight conducted separate waves for H1 and H2 and included the full twelve months data in our ratings. We should remember that before the events of October, Ryanair had a good first half in 2017. We also have BA slightly higher to reflect the national carrier’s performance both before and after the May bank holiday disruption. United, whilst suffering a hangover from the disasterous publicity generated in April is also recovering slowly.
I’m not disputing the results of the Which? report. As I said, its great to know that we can produce a signal similar to a well-respected survey. But we do need to account for the variances, which is why an explanation is appropriate.