The screener is the last step of the selection process for your user testing participants. With the basic demographics taken care of at the previous steps, you can focus more on the user's behaviour, motivation and any special requirements you need for your research. Here are some examples and guidelines to help you get the right testers for your UX research.
Keep screeners short and simple
Try to limit your screener to the really necessary questions. A longer screener may narrow your potential participant list too much or make your candidate drop out in the process.
Use clear and simple language
Stay away from acronyms and abbreviations even if they're widely used in your area of research. Avoid sentences that could confuse participants.
Avoid leading questions
It's important to leave the desired answer out of the question whenever it's possible. Some participants may look for clues for the “right” answer and will respond in the way that they believe will let them participate in the study. Obscuring the purpose of the question makes things less straightforward so you may need to ask further questions to match against your recruitment criteria, but it should yield more reliable data than giving participants a 50/50 chance of getting the answer ‘right’ by only going with a yes/no option.
Try to avoid yes/no questions whenever possible. Instead of “Do you drink more than two cups of coffee a day?” ask, “How many coffees you drink on an average day?”
- One cup (disqualify)
- Two cups (disqualify)
- Three or more cups (accept)
- I don't drink coffee (disqualify)
Don't ask questions that are available on the user profile
We have a lot of data (demographics) available from our users that you can filter on the Demographics step. We store age, gender, educational background, job title, marital status, city, etc., so most of the time you don't need to create a screener for this information. To give you an idea, here's how a tester profile looks within PingPong.
In some cases, when accurate demographics data is crucial, we still recommend you to add an additional screener for some of the demographic parameters. For example marital status and educational background can change over time and our testers' profile might not be updated. In that case, we recommend screening them additionally.
Always provide clear answers
Avoid questions like: "Do you commute a lot daily?" or "Do you live far from your parents?". Don't be afraid to use single choice instead: How far do you live from your parents?
- I live with my parents
- 0-20 km
- 21-100 km
- 101-500 km
- 501 km+
- I can't/don't want to answer
Utilizing answers like "None of the above" or "Other" can improve the accuracy of the results since participants won’t select the next best answer to proceed with the questions.
Include a non-disclosure agreement (NDA)
Working on a confidential project or an unfinished product, you should include an NDA to ensure the participant doesn’t reveal any critical information before the release. If you need your users to sign an NDA, it’s generally a good idea to add this as a screener question:
Are you willing and able to sign an NDA (non-disclosure agreement) before the session?
- I agree to sign an NDA (accept)
- I do not agree to sign an NDA (disqualify)
- I'm not sure (disqualify)
For signing your NDA, we recommend asking the tester at the beginning of the session to sign your document using PDF eSign solution (like smallpdf).
Always double-check
As with anything, a quick check is always necessary.
- Do any words reflect your own preference or opinion? Remove them if possible!
- Are the answers balanced? Does it account for both the positive and negative experiences the participant might have? It’s important that it does.
- Did you set the qualifying criteria right for every answer? See our examples if you are not sure.
User research screener examples
Targeting users buying online tickets
Which of the following statements apply to you?
- I have never purchased concert tickets online (disqualify)
- I bought a concert ticket online over a year ago. (disqualify)
- I bought a concert ticket online in the last year. (accept)
- I’ve not purchased concert tickets online but am planning to in the next months. (accept)
- I prefer not to say. (disqualify)
Frequency of use questions
It’s important to define frequency in exact terms, instead of “occasionally”, or "regularly”.
How often do you play mobile games?
- Daily (accept)
- Weekly (accept)
- Few times in a month (disqualify)
- Rarely (disqualify)
- Never (disqualify)
Familiarity questions
Aim to provide more options for the testers to choose from, avoid yes/no questions (people might say yes expecting that as the 'correct answer'). You can use these questions to
How would you classify your level of familiarity with blockchains:
- Not familiar at all - have never heard of blockchains
- Slightly familiar - have heard of it but don’t know what it does
- Somewhat familiar - I have a broad understanding of what they are and what they do
- Extremely familiar - I have extensive knowledge on the topic, worked with it
