“[Usability testing] blew a Buick-sized crater in my design and architecture – in the best way possible.”
– Adam Friedman and Kalman Gacs, MA Election Stats
Two years ago, Boston PHP and Above the Fold teamed up with Steve Krug to create a hands-on usability testing event: a speed-dating-style series of quick usability tests that gave attendees the chance to facilitate, participate in, and observe several tests in just a few hours. The event was a hit, and on January 16th of this year, we had our long-awaited second date. Once again, our participants – site owners, testers & observers – walked away with new insights… and possibly a phone number or two.
Insider Scoop: Meghan Reilly, Site Owner
After our first usability speed dating experience, we reflected on the unique event format and how much our participants learned from attending the event. For our second date, we decided to get an insider perspective from ATF’s project manager, Meghan Reilly. Meghan signed up to be a site owner, hoping to learn more about conducting a usability test, and to let us know how “usable” the event was for her (It pays to swallow your own medicine, right?). Here’s what she had to say:
What was your overall impression of being a test facilitator?
As a new facilitator, I had to come up with follow-up questions on the fly to probe the testers for additional information. And yet, by the second test it started to feel natural. Running a usability test is so straightforward and easy to learn.
What advice did Steve give that most resonated with you?
Steve stressed that you don’t want to use your unique terminology in the scenarios, because then it just becomes a word hunt for the participant, and I saw that play out in my testing. If I were going to test the same site again, I would definitely rewrite my scenarios to replace unique terms with synonyms.
What was your biggest revelation?
Two things: First, I was stunned at how quickly 15 minutes went by, even when the participant had finished the tasks early. I was also really shocked by the difference in time it took each participant to complete the tasks. Some really got it. Others struggled more.
Second, I was blown away by how much trouble folks had with headings and categories that I found familiar. I never realized that I had drunk of the proverbial Kool Aid: “This is the way it must be. It is known. It is written.” The headings and categories weren’t geared towards the user; they were geared towards the organization that created the site. It’s terrifying, and it causes pain for customer service people who end up writing long, detailed, help documentation to guide users around the site.
Any last impressions you want to share?
If I am this fired up about the results for a site I don’t own, I imagine everyone else is making changes to their own sites as we speak!
Fired-Up Results
From veteran testers to first timers, everyone walked away with a new fire in their belly and new (or remembered) lessons for moving forward:
“As someone who does testing frequently, it was good to be reminded that the last five minutes of the test can be for probing questions – the things you wanted to ask while the participant was going through the tasks, but couldn’t. Knowing that there is this opportunity at the end makes it easier to stay silent during the actual testing.”
– Carolyn Sullivan“You put something on the web, and you don’t know if anyone else understands what you understand, what’s in your head – until you watch someone use it.”
– Gene Babon and Tom Pettingell“I felt vindicated. I watched test participants stumble over designs that I had identified as trouble spots, and things I’d been wanting to fix.”
– Kalman Gacs“Steve reminded us that the goal after testing should be to tweak quickly, not to search for perfect solutions. In the time it takes you to try to be perfect, you’re losing lots of customers because your bad design is still out there.”
– Jim O’Neill
In fifteen minutes you can learn enough about someone to decide if they’re worth a second date. As we heard again at our event, it’s also enough time to learn whether your website or application is usable, and how you can make it better.
What did you learn from the event? Want to attend the next one? Let us know in the comments! And for more on rapid iterative UX design, check out another event that’s coming up on March 1st: Lean Day UX in New York City. Above the Fold will be there, so check back here for our coverage of that event.