From 10% to 80% (in the US hospital)
From 9% to 81% (in the Melbourne hospital)
How long are the questionnaires? Long questionnaires are the less likely to get accurate information. Rule of Thumb - more than 3 pages is too much.
Is each question asking about a specific item of information?
Are items within questions related?
Is the information asking for something that the shopper can accurately answer?
Do the questions cover all possible responses?
Will the answers give you information you can work with?
Are the mystery shopper instructions detailed and unambiguous?
Do they follow the 'flow' of the visit?
Are they presented in reporting categories?
Do they use photos & images to show what to look for?
Is there a wide range of question types?
Do they have checkboxes AND radio buttons?
Do they have List+ and List- questions?
Do they have 'NetPromoter' and scale 1-10 type questions?
Do they have hidden/delayed questions?
Do they have business improvement (carry forward) questions?
How they train and certify their secret shoppers. If there is no real training and assessment program in place, then the quality of the secret shoppers, and the quality of the information they provide, cannot be relied on.
How they select secret shoppers. If they rely on a small pool of 'professional' shoppers you are not getting the true customer opinion and experience. While it may be a dream job to be a 'professional shopper', it's not going to get your business real information from real customers. At the opposite end of the spectrum, be wary if they accept all comers. You need a wide range of secret shoppers, but you only want those who are reliable, good observers and can write good reports, so you need a mystery shopping company that has a good selection process, and ongoing monitoring and QA.
How they ensure secret shoppers keep your business confidential. . If their secret shoppers have not all signed confidentiality agreements, then beware. You can be sure that companies who use Job Boards, Shopper Forums and spray emails to assign secret shoppers are not concerned about your confidentiality!
How they pay their secret shoppers. If they are paying shoppers a paltry $3 - $4 per shopping job, or simply reimbursing purchases, they are not giving them the incentive to do a good job. Look for a company that guarantees a minimum of $20 (plus purchase) per assignment.
How they manage their secret shoppers. If there is not a valued shopper program in place then there is no mechanism for guaranteeing the quality of the secret shoppers. The best companies assess the mystery shopper and provide feedback on every shopping assignment, and then use that assessment to determine access to jobs.
Can changes to questionnaires be done quickly and easily?
Do you charge for changes to questionnaires?
Can my existing data be integrated?
Do you have a strategy for breaks in continuity?
Are all reports online within 24 hours?
Are all reports QA'd before release?
Are reports dynamically generated (not static pdfs)?
Are these online graphing and trend analysis tools?
Can you easily change question weightings and apply retrospectively?
We post a job on a Job Board for shoppers to 'grab'. This is the most widely used method, and it is guaranteed to get you a mystery shopper who is not matched to your customer profile, and who has no motivation to do a good job. Score: 1 out of 10!
We have 'career' shoppers, or a 'small pool of expert shoppers'. Again, you are not going to get demographically-matched secret shoppers, or real customer reactions. Score: 2 out of 10.
We email all the eligible shoppers on our database While this is preferable to a Job Board it still does not get you the best shoppers, just the ones who are quick to respond. Score: 4 out of 10.
We select shoppers individually for each job. Excellent! This way shoppers can be demographically matched, selected on merit and they know that doing a good job means they increase their chances of getting more work. Score: 10 out of 10.
What customers expect, like and don't like
Whether they will return and recommend to others
What could have been done to make the experience better
How the competition measures up
Describe everyone in the reception area (minimum description: height / build / sex / hair color/ style/ length ), what they were wearing (including name badges and company shirts / polos ) and describe what each was doing when you walked into the foyer, and how they greeted you.
Describe everyone in the reception area. Shoppers have to remain incognito. The reception area of a large hotel may have 5 or more people on duty. How could a shopper act nonchalantly when they are tracking down and remembering a large number of details of all visible staff?
Height. What if they are sitting down? Or leaning over a desk? The question is inviting the mystery shopper to guess.
Name Badges. May not be visible if someone is more than 3 metres away, or not facing the shopper.
What they are doing. If they are facing the customer and the computer screen is between them how can the customer possibly know? At best it will be a guess. Again, inviting the mystery shopper to be inaccurate.
Was the coffee good? (Yes / No)
What is meant by 'good'?
Does everyone have the same opinion of what is 'good'?
Who is giving the opinion? What do they know about coffee?
Is what they think of as 'good' the same as what a typical customer thinks is 'good'?
The wrong or unclean cup/glass
The wrong type (a latte instead of a machiatto)
Slopped in the saucer, or drips on the cup / glass
Served incorrectly (choc sprinkles on a latte)
Need we say more?