“Get out of the buiidling”! It’s so much easier to grab one or two or more real humans
Evidence from 2 or 3 random people is often enough, but really you don’t need for than 5-10..
Can be conducted at any stage
Paper prototype
Online mockup or ‘wireframes’
Actual build
The difference is how hard/painful it is to make changes - which is inversely proportional to how open you will be listening
This could be testing a paper prototype
early, when it is easy to make big modifications,
or later with actual software - when it’s more difficult to make changes
Preparation
Decide on a task(s) for user. “Log in and post a picture”, “Check and update your privacy settings”
Decide on the type(s) of users.
Decide on what knowledge or assumptions they will start with (new user, experienced user, musician, programmer, etc.)
Rules of thumb to follow
Try to get the ‘victim’ to narrate their thought process
Make sure they know that if they are confused, it is the fault of the software, not their fault
If and when they get stuck, engage them in a conversation by giving them small hints
Ask them for a suggestion on how it would be clearer, for example:
“Let’s say you want a plain pizza and you see this screen, what would you do?”
“Oh you can’t decide, well tell me, what are you looking for right now on the screen?”
Running the test
Mistakes are ALWAYS the fault of the software not the user. Make sure they know this and don’t feel like they have to appologize when things go wrong.
Don’t help.
Ask them to narrate their thought process: “I am not looking for a button called security or privacy. Oh here’s one called settings, maybe that’s it. I am going to click it. Hm, I expected to find security related stuff here but instead I see…” etc.
Make notes
Keep it simple
You don’t need a one way mirror, video recording etc etc.
You don’t need 10 subjects. Usually after 2-3 you already know what the problems are