Product Testing: Newbies Knows It Best

10/20/2016 02:16 pm ET Updated Oct 20, 2016

As part of my graduate curriculum at Cornell Tech, we were tasked to get our hands dirty. Dirty, as in, speak directly to our customers for some critical, unfiltered feedback before we got too far into product development. The reality of the matter is: the truth is seldom squeaky clean. In fact for us, this time it was far from it.

We spent last week on Roosevelt Island, future home of the Cornell Tech campus, at the Carter Burden Center of Aging to experiment voice-activated technologies with seniors to test the following idea: “How might we provide consumers more control of their health data?”

Any data collected would reside with the consumer, rather than electronic health records that patients rarely access these days.

After testing our technologies, we expected rave reviews but were floored when we heard comments like this one:

“I don’t need this in my home.”

My first thought when we received the less-than-stellar feedback was of course: “What do you mean you don’t need this? We just spent a couple months building this thing for you!”

But don’t worry, I didn’t say it.

You’ve probably heard before, and you’re about to hear again: build em’ something they need. But here’s something you probably haven’t heard:

Test your tech product with someone that doesn’t even use tech.

Some of our participants had never swiped on a smartphone before, let alone written an email. What did this enable exactly?

The authenticity that connects the user to technology.

Rather than asking about the product in its traditional use cases like most customers were, participants at the Center of Aging were requiring us to unlearn everything we had held to be absolute truths about our product in order to answer their questions. Questions that, to us, seemed obvious like:

  • Why am I doing this?
  • What does this mean?
  • Who else can see this?

At the most fundamental level, it allowed us to really understand the “so what” we needed to articulate more clearly to our end-users.

In that process, one of the most illuminating questions we asked them was the following:

“What did our device not ask you that you would have liked to have been asked?

Though we tested with other variations of the aforementioned phrase, that one allowed us to get most directly to the mind’s of our users.

From this surprising, and somewhat disappointing experience, we learned: when going through the product development process, consider getting your product it in front of a total newbie. In other words, not only someone that has never interacted with your technology before, but rather someone that has never interacted with technology, period.

One of our users didn’t even have any contacts programmed into her cell phone. Rather, she carried a list of names and numbers in her purse incase she needed one for reference.

The benefits this kind of testing are two-fold:

First, it rids the user from intended use, eliminating any biases that may exist. And second, more importantly, you’ll find yourself getting to your “lowest technology denominator” that will undoubtedly optimize your user experience for others.

The process we followed to extract knowledge from our users was the following:

  1. One-on-one interview — understand where the user fits within our target demographic
  2. Group usability test — asked for a few volunteers to demo our product in front of the group
  3. Roundtable recap — went around the circle and asked for closing thoughts. We made sure we specifically asked for both constructive (positive) and critical (negative) feedback.

Throughout our process, the roundtable recap proved to be the most valuable of all. Though we allowed for plenty of critical feedback, we made sure to refocus them when they deviated far from providing constructive feedback.

This was largely because when you are asking people to provide you with their feedback, they typically default to their critical feedback vs. sticking to constructive comments.

In all, we learned more from our experimental users than months of hypothesizing how potential users would interact with our product.

Aristotle once said, “The more you know, the more you don’t know.” If we were to construct a corollary from our experience, it would be: the less (your users) know, the more you’ll know as a technologist.

Test often, test widely, but most importantly, test with a newbie.

This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.
CONVERSATIONS