Conducting a usability test

Usability Tests

  • Don’t guess!
    • “Get out of the buiidling”! It’s so much easier to grab one or two or more real humans
    • Evidence from 2 or 3 random people is often enough, but really you don’t need for than 5-10..
  • Can be conducted at any stage
    • Paper prototype
    • Online mockup or ‘wireframes’
    • Actual build
    • The difference is how hard/painful it is to make changes - which is inversely proportional to how open you will be listening
    • This could be testing a paper prototype
    • early, when it is easy to make big modifications,
    • or later with actual software - when it’s more difficult to make changes
  • Preparation
    • Decide on a task(s) for user. “Log in and post a picture”, “Check and update your privacy settings”
    • Decide on the type(s) of users.
    • Decide on what knowledge or assumptions they will start with (new user, experienced user, musician, programmer, etc.)
  • Rules of thumb to follow
    • Try to get the ‘victim’ to narrate their thought process
    • Make sure they know that if they are confused, it is the fault of the software, not their fault
    • If and when they get stuck, engage them in a conversation by giving them small hints
    • Ask them for a suggestion on how it would be clearer, for example:
    • “Let’s say you want a plain pizza and you see this screen, what would you do?”
    • “Oh you can’t decide, well tell me, what are you looking for right now on the screen?”
  • Running the test
    • Mistakes are ALWAYS the fault of the software not the user. Make sure they know this and don’t feel like they have to appologize when things go wrong.
    • Don’t help.
    • Ask them to narrate their thought process: “I am not looking for a button called security or privacy. Oh here’s one called settings, maybe that’s it. I am going to click it. Hm, I expected to find security related stuff here but instead I see…” etc.
  • Make notes
    • Keep it simple
    • You don’t need a one way mirror, video recording etc etc.
    • You don’t need 10 subjects. Usually after 2-3 you already know what the problems are
    • ACT ON WHAT YOU LEARN!