In the previous blog, you have seen how to define the journey and detailed journeys that are the basis to find the drivers for each step in the journey.
The next step is where we start involving customers.
By sending them the questionnaires that are built on the insights from the detailed journey workshops.
In this blog, I’ll walk you through the 4 steps, from building the questionnaires to finding the real, latent drivers, including relevant tips and tricks you’ll need along the way:
But let’s first start with the CX in 1 minute summary of this blog:
1. Develop Questionnaires
One of the first assumptions I immediately like to tackle is that concerning the importance of short questionnaires.
Trust me, we have actual data, therefore no assumptions, showing very little difference in the response rate between a 60-item survey, a 10-item survey or a single question with an open answer survey.
I was surprised as well, even though I had experienced comparable results in my PhD, where I realized a 30% response rate to a 60-item questionnaire (see the end of step 3 before for more response tips).
The surprise for me lay largely in the trend and almost religion we see nowadays, to ask only 1 (often NPS) question and then an open text box explaining the why of the score.
This ‘religion’ is influenced by the reviews and ratings trend.
But ratings and reviews are a completely different ballgame to finding the drivers to really understand how to make a difference in the journey of your customer.
Having said that, I’m not the one to create short questionnaires.
Not when I need to find the drivers in the journey, when I do not know what really matters to my customers.
So you want to keep a completely open mind, and objectively measure all the facets in the detailed journey, making sure you find those items that have the greatest impact on the customers’ satisfaction.
Here are the 10 guidelines to build a questionnaire that is optimized to find drivers:
- Use the sequential order of the detailed journey as a structure
- For each step in the detailed journey, you ask questions
- All questions take the form of statements (1-5 scale)
- The satisfaction question is the only one on a scale of 1-10.
- The satisfaction question concerns that specific journey, rather than their general satisfaction
- Put the satisfaction question at the end of the questionnaire.
- Place yourself in your customer’s shoes to sense what would matter to you if you were them
- Use the brand promise and the emotion in the statements throughout the questionnaire
- Use an average of 5-7 statements for each step in the detailed journey
- Don’t worry if you end up with a quesa questionnaire containing 30 to 60 statements 😉
2. Get the right selections
Building the right questionnaire is the first step in maximizing the chance to find the right drivers.
The second step is to make sure you send those questionnaires to the right target group.
So making the right selections of the participants is crucial.
We have seen that making mistakes in these selections (the so-called queries in IT terminology), will cause bad driver results.
Meaning you have to do it all over again with new selections.
To make sure you get the right selections, you always need a mix of business knowledge and IT knowledge.
The business needs to know how to define a specific journey, IT needs to know how they can match that definition to the CRM system.
To give an example.
Business wants to measure the journey from online application until being interviewed in a social welfare situation.
That means that IT needs to find fields, in the CRM or other core systems, that match the status of being interviewed with having filled out an online application for welfare.
You start with the ideal journey definition from a business perspective.
Then you check with IT what is and what is not possible, and find a pragmatic solution in the event of any difficulties.
You want to make sure to ask your customers as little as possible about where they fit in the journey.
It’s far from professional if you have to ask me, as a customer, whether or not I have made an online application, for example.
“Why would you ask me? You already have my online application I hope…?”
And yes, there will always be some difficulties.
And yes, there are also always pragmatic solutions to be found.
So you make sure you create a definition of the scope for each of the journeys.
And no, not touchpoints but journeys (see this blog to understand journeys versus touchpoints).
When you have discussed this with IT and finalized the definitions, you ask IT to run all the test queries.
Just to make sure there are no strange elements in the queries when you send them out.
And to make sure you have all the right columns to personalize the emails and make additional analyses when relevant.
Since you have now tested the queries, IT can run updated versions very close to sending the questionnaires.
3. Send the questionnaires
The two most important reasons to create a good response rate are the relevance of the questionnaires, and recent experience.
1. Relevant questionnaires
The relevance of the questionnaires goes back to the right selections you make in step 2.
If the CRM registration is not up to date, and you send me a survey about my application for welfare, while I did not apply for welfare but rather for help with my debts, I will not respond.
Even worse, I will start to have serious doubts about your trustworthiness as an organization.
2. Recent experiences
I still see organizations measuring experiences that have taken place way too long ago.
Asking me to fill out a questionnaire that starts with the question: did you have any problems over the past 12 months, is guaranteed to get poor driver results.
How on earth should I remember how my journey really went 12 months ago?
If despite that, I do fill out the survey, the responses will be so average (since I don’t remember the details) that it explains why many organizations become stuck around that 7 score.
But be aware!
The current trend of measuring realtime is also tricky.
Realtime means directly after I have been in contact, after I have applied, etc.
So let’s say that I receive the survey within 1 hour of the fact.
Response wise, this certainly makes sense: more recent than this is hardly possible.
But you’re measuring not my entire journey, but simply my channel experience.
Let’s look at an example.
I have called the contact center and the employee has promised me to send me the form I requested.
If I receive a survey within the hour, I will rate this contact very highly.
But if after several days, I have not received that form, I am very discontented, but you will not be aware of that.
So when measuring journeys, you must make sure that you define the moment when this journey is completed.
Often 1 week after is a good timing.
It should never be longer then 1 month ago.
If you are struggling to get enough respondents when you only select 1 month, here is a solution.
Don’t make the time period longer then 1 month.
Send the survey over several consecutive months until you have enough respondents to draw valid conclusions.
# BONUS RESPONSE TIP AND TRICKS
Email is still the preferred channel for journey research.
And the response rates are still high enough, as long as you stick to the shared guidelines.
When creating the emails, here are some tips that have worked well for us:
- Make sure a person is the sender, not “Organization”
- Create a short, activating subject line (i.e. in the case of the welfare application “Your application! What do you think?”
- Personalize the email (i.e. Dear Zanna)
- Address me personally, not just my name, but also with “you” in the text
- Put the link to the survey or the first question of the survey at the top of the mail
- Include a brief introduction of 1 or 2 sentences, where you mention the journey they have just experienced and for which you’re requesting their feedback
- Let me scan the mail. Don’t fall into the trap of explaining the research in a few paragraphs
- In the email, make sure you inform them that the benchmark results will be shared immediately after filling it out
# BONUS PRE-SUASION EXPERIMENT
One of his many experiments was to conduct market research in a mall.
When people were approached, 29% were initially willing to participate in the research.
This increased to 77%, when starting the approach by asking the question: “Do you consider yourself to be a helpful person?”
Yes of course I do!
And to prove that, I am willing to help you with your market research.
They had the same results when asking for email addresses.
When asked to give their email address to get a free sample for a new soda, 33% were willing.
Then they started with the question: “do you consider yourself to be adventurous, to be someone who likes to try new things?”
After this question, 75.7% were willing to give their email address.
By the way, 97% confirmed that they were adventurous people, which of course is a crazy, but very interesting insight into our psyche…
I’ve used this helpful experiment in email invites as well, and the response was indeed higher then average.
In 1 experiment, the response improved from 5% to 10%.
In another, it improved from an average of 15-18% to around 25%.
I’ve combined several of his insights to find the optimal email invite.
Should you copy this experiment?
I don’t know.
The point I like to make is: experiment, use real data to see what works for you.
Forget assumptions (like everyone assuming the response will be very low on long surveys).
4. Find the drivers
The million dollar question is of course: what does my customer want in his journey to make him very happy with me and to retain him as a loyal customer?
That’s where smart statistics comes in.
But before I share that, first I want to explain the context and the why of finding real, latent drivers.
You’ll then understand why I use this methodology rather than many other methodologies in the CX field.
4a. Latent versus rational needs
This picture is the essence of why many companies are struggling to move past that all-famous 7 or 7.5, while we know that an 8+ is needed to create loyal customers.
Let me walk you through it.
Over 90% of all the research in the customer experience field focuses on our ratio.
For example, asking the NPS question followed by an open text: why do you give that score?
That question gets me to rationalize the score I gave.
I start thinking about it.
Maybe it was a nice employee, a speedy process, a nice website.
I rationalize why I gave that score.
Over 90% of our decision making as human beings, is far from rational.
Most of our decisions are made subconsciously.
This means there can be a big gap between what I’m saying in the open text boxes and what my real, latent needs are.
In the call center setting, most people will tell you they are very annoyed with the IVR (press 1 for xxx, press two for xxx, etc.)
So when you analyze the open answers, you may change that system or even stop it altogether.
That will have a huge impact on your operations.
When we use this latent approach, as I did during my PhD research and repeated this at several organizations, you find that the IVR has very little impact on my satisfaction.
It’s about a likeable employee who listens well, who is friendly, who can answer my question.
So if you don’t know the real, latent drivers of your customers in each of their journeys, you run the risk of focusing your energy on the wrong improvements.
Again, you’ll be stuck at the 7 or 7.5 level.
This also means, that there is a risk in using AI to analyse open text answers.
The current state of AI uses a combination of counting words and sentiment analyses.
But if my answer is my rational answer and not my latent need, the AI is analyzing a whole bunch of rational answers.
On top of that, counting the topics is not giving you insight into the impact of the topic on satisfaction.
Of course, you can assume that something that is mentioned more often is probably more important than something that is mentioned less.
But do you really know whether topic number 1 is twice, 3 times or 4 times more important than topic number 2, 3, etc.?
That’s why I use a scientific approach to find the latent drivers.
4b. How to find the real, latent drivers
I’ve had the ‘luxury’(ahum…) to do a PhD parallel to my job in the customer experience field.
That has led me to understand enough of statistics, to make a useful translation to organizations and put organizations in the driving seat of customer experience.
In essence, I’ve combined three statistical techniques: factor analyses, regression analyses and reliability analyses.
I’m not going to bore you with statistic details, but you can share this with your research company and they should be able to run it for you.
If your research partner is not able to do so, let me know, and I’ll connect you to a research company that does (no, I don’t have any shares in that company).
So to get from the questionnaires to the drivers, you follow these steps:
- At >100 respondents, you export the data (100 is a minimum, the more the merrier)
- You run the 3 analyses to find the top-10 drivers that make the difference
- The explained variance (another crucial statistic) will tell you whether you did a good job during the workshops and questionnaire building.
- You share the top-10 results in very simple language with your colleagues who are responsible for that specific journey.
- You also share the insights of what DID NOT have any impact, so that your colleagues will not waste their valuable time on those topics
- You visualize the top-10 drivers in their position in the detailed journey
Because these results, as you can imagine, are very sensitive and valuable to an organization, I’ve created a fake version just to walk you through how it works.
You should forget a lot about statistics, but there are 4 things you need to remember when you want to accelerate the increase of satisfaction.
A. Knowing you have found the right drivers
We are all customers, employees and members of the general public.
So we all think we know what matters to them.
And it takes a lengthy internal discussion to convince each other to divert from all these assumptions.
The statistics are here to help you solve this problem.
The r2=68% tells you that, in the eyes of the customers, you have found the right drivers.
So no more internal discussions slowing the process down.
Scientifically speaking, 10-20% is already considered high.
From a business perspective and being in the driver seat of customer experience, I always aim for 50% or more.
And using the customer journey mapping technique, we almost always achieve between 50 and 70%, which is extremely high (meaning the journey mapping technique works).
If it’s below 40%, then you have definitely missed something, in the eyes of the customers.
That’s when you should go back to the drawing board with your workshop participants and/or ask customers if you don’t know what you are missing.
B. Knowing what has the greatest impact
One of the greatest challenges in each organization is prioritization.
Making choices is not easy, and very often certainly not based on data-driven decision making.
Using such analyses, you know exactly where to focus your energy.
In this example, improving the letter with instructions on how to apply (b=0,65) is 3 times more effective then improving empathy of the employees (b=0,22).
So you know exactly where to focus.
This of course is a fake example.
In most cases, the soft, emotional human side of the journey has the greatest impact on satisfaction.
But definitely not always, so you have to make sure you know what your customers find important in their specific journey.
C. Knowing the contents of the drivers
In this case, the number 1 driver is already pretty specific.
Accessible service is already a bit more vague. That’s why, on the left side you see exactly what items are part of these drivers.
And these items are always very specific, creating many ideas to improve when you show this to your employees.
D. Knowing which scores need to be improved
As human beings, we are naturally inclined to look at the lowest score and to start improving it.
So if you hadn’t heard anything about these driver analyses, you would look at the last item and start improving it.
But now you know, that improving driver number 1 is three times more effective.
So you should focus your energy on moving the items of driver 1 to a 4 or 4,5 score.
That will have the greatest effect on increasing satisfaction in this journey.
And that’s how you can find the real, latent drivers of your customers, getting you in the driving seat of customer (and employee) experience.
Since most employees are used to thinking in processes or journeys, you then plot these top-10 drivers in the journey.
You can do so, because we have built the questionnaire in the order of the detailed journey.
Making it even more concrete and easy to see where the greatest impact can be made.
Is it only statistics, schematics?
We have proven across several organizations, branches and countries, that the satisfaction of the improved journey increases pretty dramatically (i.e. +0.4, +0.7, +1.1) within a relatively short time (2/3 months).
With an accelerated increase in the business results following organically as well.
Why? Because thanks to these insights, the employees know exactly where they can make the difference!