As a UX designer, I’ve witnessed several projects where UX research was slow and did not fit well into the agile product development process. The usability testing was slow, took a lot of time and was carried out on a ad-hoc basis. It’s not a secret that this was one of the main factors which drove me to start building PingPong.
Testing with one user early in the project is better than testing with 50 near the end. — Steve Krug.
I’d rather do RITE and keep testing throughout the process, than only testing at the beginning or the end. Rapid Iterative Testing and Evaluation (RITE) is a hyper practical usability testing method that focuses on quick, effective and regular usability testing. RITE is breaking down the tedious user testing process into small, iterative testing rounds:
One of the benefits of RITE is building and training your research muscles: you’ll run testing sessions weekly or bi-weekly, and with RITE, UX research can be a fast and effective regular team exercise, rather than a painful solo pursuit.
The history of RITE
The original publication of RITE was published by a team of Microsoft Games employees. They were working on the design and usability of the classic Age of Empires II game and presented the RITE method at the 2002 UPA Conference. You can read the full research paper (Using the RITE Method to Improve Products: A definition and a Case Study) here.
As Researchers you get a lot of fundamental training on what constitutes actionable proof. That fundamental training focuses on 'completing your research' before drawing conclusions. In publishing RITE we wanted Researchers to acknowledge that in the correct circumstances you can make strong and accurate conclusions and 'complete research' quite quickly. Perhaps more importantly, the standard literature (of the time) on usability testing was indexed on accurate identification of problem areas. The key word in the RITE acronym is Iterative. Through iteration you can prove that something is fixed, which we felt was a more important way to think about research findings. Find a thing. That's nice. Find and fix it. Far better. — Ramon Romero
Personal note: I loved the game and it was a huge commercial success too.
The elements of RITE
There are 3 key elements of RITE.
Rapid and Iterative process
The most important part is the Rapid and Iterative aspect. With RITE, you run usability tests on a regular (weekly / bi-weekly) basis where you are able to make quick product changes.
Since these sessions are quite regular, you don’t need to aim to test with 4-6 participants. It’s enough to test with 1 or 2 test subjects.
Classification of issues
As issues emerge from research, you classify each of these:
- Category 1: obvious cause & solution that’s quick to implement (e.g., text changes, re-labeling buttons, rewording dialog boxes, etc.)
- Category 2: obvious cause & solution that’s hard to implement (difficult new features, substantial design and code changes, etc.)
- Category 3: no obvious cause and solution
- Category 4: may be due to other factors (e.g. test script, interaction with participant, etc)
And to handle these issues:
- Category 1 issues get implemented immediately and rolled out for the next test.
- Category 2 will begin implementation and roll out when completed (next round of testing or later).
- For Category 3 and 4, we’ll collect more info on the upcoming sessions to see if they can be upgraded to Category 1 or 2
This simple classification of issues helps the team to move quickly and fix problems efficiently.
Domain knowledge and decision making
The moderator (or usability engineer) carrying out the sessions must have deep domain knowledge and the decision makers of the development team should participate in the research (so they can swiftly make decisions). Your team must interpret the test results quickly and reach quick product decisions.
The problem with classic usability testing approach “You need to test with at least 5 users”
UX research takes a lot of time and this can make it costly. From my experience, running a 5 person usability test will probably take at least a day from the designer/researcher: you need at least 5x30 minutes for the tests, plus breaks, plus preparation time, plus a lot of time for making sense of your research data and reporting this back to your team. This will take at least a full day for an experienced researcher but can easily end up with 2 days. This is a lot of time which requires a lot of commitment and organisational buy-in.
Another drawback when you are only running user testing sessions once in a while is that your research muscle can atrophy. You are simply not being as efficient as someone who is running sessions every week or two. When you only talk to your users once in a while, the entire research process feels much heavier.
When you use the RITE method, you’ll only test with 1-2 people, but you’ll do so much more frequently. Each round of research shouldn’t take more than 2 hours to complete. There’s no extensive reporting required either, because you and other decision makers are involved in the research and make product decisions right after your sessions.
The big win from RITE is that research will ultimately be far more impactful on product’s development: this is true UX research where we get the USER fully involved in the development process, resulting in much higher quality products.
How to get started with the RITE method
To get started with RITE, you need the right ingredients:
- An open and curious mind and willingness to regularly seek out user feedback. If you’ve come this far in this article, that should be you.
- Domain knowledge for the moderator.
- A product / prototype that is relatively easy to change.
- Involvement from decision makers (PM, lead developer).
- Quick and easy access to the end users (so the process won’t slow you down). In PingPong it’s extremely easy and quick to schedule participants even with a very short turnaround time of just a few hours.
- You can start with 1 or 2 user tests scheduled for a time of the week that fits your stakeholders. Make sure you book a specific time for every iteration: weekly or bi-weekly, depending on your schedule. For example, we run our testing session every Thursday afternoon and we quickly discuss the learnings and actions in 15 minutes once it wraps up.
- If you’re running 2 sessions a week, leave some time in between the session so there’s time to update the prototype. This break can be longer or shorter, depending on how quickly you make changes.
- Book the user testing time into your calendar and invite project stakeholders.
- Make sure the stakeholders spectate sessions (here’s how you do it with PingPong).
- After each session, sit down with the decision makers and go through the categorised issues. Make decisions and update the prototype for the next session.
The observers should take great care not to "lead" the test participant. The test participant should be "thinking out loud". The more the test participant freely states their thoughts and feelings about the interface, the more the team of observers will learn. Getting down into detail we usually followed rules such as only the researcher should talk to the user and other observers should simply take notes. We also encouraged the researcher to keep prompting the participant to say what ever came to mind as they used the system. Some very specific suggestions like, if the user says "this is confusing" should be followed by "tell me what makes this confusing for you" would lead the participant to elaborate. Once the participant got used to elaborating and started doing it habitually then no more prompts were needed. — Dennis Wixon
Potential problems with RITE
It’s important to remember that RITE is designed to run usability tests and improve the UX through rapid prototyping and iterative testing.
Therefore, RITE is not the perfect tool for uncovering more complex issues, problems with product concepts or user motivations, pattern identification and addressing deeply rooted issues. These sessions are not open-ended discussions in a longer format where you can run detailed user interviews to conduct research on a much deeper level. For this type of discovery research, the double diamond model can be a much better approach.
Consequently, it’s less ideal for early-stage, product validation type research, when you are trying to figure out the product market fit. Keep this in-mind and make sure you are using RITE accordingly.
Benefits of the RITE method
Design and development tools evolved quickly in the past years, therefore the RITE-method is becoming more and more relevant. It’s easier and quicker to make rapid changes on your product than ever before so the benefits of RITE are becoming stronger:
- Adopting regular and early tests with RITE reduces the cost of fixing issues when compared to identifying them later in the product development process.
- No long and time consuming usability reports are needed. The updated prototype will serve as the new design direction.
- Builds and trains research muscles for the entire team and embraces critical thinking.
- Regular and iterative feedback from users results in quick fixes through classified issues.
- Only a small number of users are needed for each iteration, therefore research sessions are quick and easy to get stakeholders to participate.
- RITE can be carried out on low fidelity prototypes or design concepts to get real user feedback early in the product development process.
- Domain expertise is more important than facilitator expertise: you will do RITE regularly and often so your facilitator expertise will build up quickly. For this reason, it’s great even for user testing beginners.
- Since the team and decision makers are talking to users on a regular basis, RITE helps to rally the entire team around better user experience.
Tips for getting the most from RITE
Aim for short, 30 minute sessions and avoid testing long forms: at the end it’s RITE and not SITE.
The intention behind RITE is to be quick and efficient. So if you’re running RITE on a working product (that’s potentially bigger), break down flows worth testing into smaller components so you can focus on the rapid and iterative aspects of the RITE method.
Get your team involved: find a specific time every week when you’ll run RITE and invite your teammates to Observe the testing session.
Categorize the issues accordingly and follow up on them with the necessary changes on the prototype and with the development team.
After your RITE sessions, quickly run through the findings with your team and make sure everyone is on the same page with the key insights and learnings. Especially focus on Category 1 and 2 issues and make sure Category 3 and 4 issues are documented for future monitoring.
Since you’re running only 1-2 tests per iteration, moderated sessions (supporting the participant to think out loud) will be a much smarter way to run this than unmoderated tests.
Do you need help getting started with RITE? PingPong offers consulting services to get you started, schedule a free discovery call.
