The Various Types of Usability Testing

Why bother with different types of usability testing for your website? After all, a good analyst will be able to interpret web analytics and come up with a solid reason for the various interactions on your website. So, why bother with different types of usability testing?

The reason is twofold. The first reason is that we are not all great and experienced analysts, and an analyst’s career will teach him/her that experience is as valuable and necessary as it is damaging and blinding. The second reason is that it takes a lot of guesswork out of the elimination and bug testing process.

Using web analytics alone to test and fix the usability of your website involves a lot of guesswork. They are educated guesses, but they are still guesses. The more information and evidence gathering tools, methods and techniques you have, then the more likely you are to identify your problems and solve them.

Three Testing Animals

Just like there are automatic cars and manual stick-shift cars, there are only three “types” of usability testing. They are “Moderated in-person”, “Moderated remote” and “Unmoderated remote.” Before explaining what each type of usability test means, let us address why you need other people to test your website.

You may have seen and used a great many usability testing tools on the Internet and wonder why you need other people to test your website. The reason is that it is the only way to test the usability of your website. For example, broken links will lower the usability of a website, and you can run a broken link checker without input from other people. Yet, broken links are a web design and development issue, not a usability issue. A usability test will help you discover why people are not clicking on links.

Here are the three “types” of usability testing:

1. Moderated In-Person: A facilitator will often be co-located with a participant. There are many varieties of this testing type. Getting truly genuine results often requires a lab and two people at a time (participant and facilitator), but this can be expensive and many times a few “good” results are just as useful as a few “perfect” results.
2. Moderated Remote: The participants are in a different location to the facilitator. The process is done in a massive number of ways, from screen sharing to computer monitoring software. This is a far cheaper and more convenient way of testing a website for usability.
3. Unmoderated Remote: This is a where an administrator sets a bunch of tests and people interact with software. When the interactions are complete the software produces results. There are some occasions where this is cheaper than the other methods, such as if you are testing a great many people, otherwise it tends to be a little more expensive that the other testing methods.

Problem Discovery

Use all the tools and all your web design prowess to overcome as many of the bugs and website problems before you introduce your website to the participants that will test it. They will then use your website and identify any problems.

These days, a common problem is device compatibility. Simple things such as text scaling and such are easily identified, but other things such as website tools often spawn usability problems for different devices. Again, these are technical issues that you should try to wrap up prior to having participants test your website.

Consider Your Benchmarks

How usable is the interface? Did your ideas work? Did your solutions fix things? Set a benchmark by which you judge your future efforts. Assess design changes and content changes by considering your benchmarks. Benchmark studies will often require a much larger sample size so as to get tighter confidence intervals. As a consequence, such tests are usually conducted via the unmoderated remote method.

Competitive Testing

This is where you have your users complete a few tasks on your website, on your previous benchmark, and even on your competitor’s websites. You use your results to figure out what is good and bad about your current designs and content.

Each and every usability study will come with some level of artificiality, and it may threaten the validity of your findings. A good way to put your assignments/tasks metrics and scenarios into a usable context is to measure how your website stacks up against your competition and your previous designs.

Eye-Tracking

These tests are both enlightening and rather fun to conduct. You get to see where people look on your website, what order they look at things, and you get to see their sequences of gaze paths.

The analysis of your eye tracking experiments is difficult, and they can be expensive because current technology shows that using web cameras is still unreliable for this test. The participants and facilitators need to be present for moderated in-person testing.

Learnability First-Time-Use Testing

This can be a very tricky thing to test. You have to figure out how easy your website is to learn. When a person uses your website for the first time, they have to learn how to use it. Some things are mildly intuitive, and some things need to be taught or figured out. It can be tricky to analyze these tests because first-time users each have a different history, level of knowledge, and level of learning.

A hardcore Internet user may have an easier time learning the website than a technophobe. Gaining many metrics is easy, but you will have to do a lot of tests to make your analysis easier. Having people repeat the same actions over and over again will often highlight their learning curve.

Websites That Offer Advice on Usability

There are websites and articles out there that teach you how to improve your website usability and they mention things such as broken link checkers, markup checkers, and device compatibility testing. Why would they mention such things if they are not usability issues according to this article?

Technically, they are usability issues, but they are issues that a web designer and web developer should already be dealing with. If you were testing a car to see how fast it could go, you wouldn’t judge it based on its flat tire, you would simply have somebody else fix the tire before you test it. The same is true if you are testing your website for usability, you should have the technical stuff wrapped up before you start testing your website with third-party participants.