Why The Best Customer Feedback Happens Outside Of The Building


Imagine you’re a product manager at a 30-person startup working on an app for the trucking industry.

The app is sort of like a hybrid between Uber and DocuSign. It connects shippers, the people who need to get their stuff from point A to point B, with truck drivers, the people who drive stuff from point A to point B. Additionally, the app brings all of the associated paperwork online and — at least in theory — makes it easier for truck drivers to send progress updates from the road.

V1 of the app is up and running, and sign-ups are growing faster than expected. There’s just one issue: Usage of some of the app’s core features is low. Specifically, truck drivers are still regularly making calls to dispatchers when they could be doing in-app updates instead.

The challenge: Get these truck drivers to change their ways and use more of your app.

How would you do it? Where would you start? How would you get customer feedback?

Here’s what my wife did.

(Oh, right. Forgot to mention: My wife is a product manager and this is a true story.)

So my wife and her team had already spent dozens if not hundreds of hours talking with truck drivers about the app over the phone, and had even done several user sessions at her company’s office in Boston’s North End. But these didn’t yield them the insights they needed to boost product usage.

They had gotten the sense that they were missing something — that they weren’t seeing the full picture of how truck drivers were interacting with the app, or, for that matter, how truck drivers were interacting with technology in general. And then it hit them:

In order to better understand truck driver behavior, they had to observe truck drivers in their natural environments.

They had to go do customer research at truck stops.

Steve Blank (he created the Customer Development methodology, which helped launch the Lean Startup movement) calls this getting out of the building.

Twitter’s former VP of Product Michael Sippey tells product managers to litereally “get in the van.”

And at Drift, our CEO David Cancel talks about how getting out of the building and talking to customers tells you everything that data and dashboards can’t.

“We want to see their office, who sits next to them, if they’re in a cubicle, what’s in their cubicle, what do they have printed out that’s hung on the wall, what does their browser look like, what are their favorites, what’s on their desktop? You just really want to understand, as a researcher, what are they doing? What does their day look like? And the reason why I care so much about that is because rarely do you get that information when you ask someone a pointed question about features or functions or things that they need.”

So one day my wife and one of her coworkers got in a car and headed off into the heart of trucker country … which in this particular case was in Central Massachusetts, out along the Mass Pike (I-90).

They came armed with $50-worth of doughnuts, an endless supply of informational fliers, and, of course, swag. But not just any swag: awesome trucker swag, which included coolers and vintage-style trucker hats.

At each truck stop they visited, they’d set up a little table, put all the doughnuts and swag out on display, and simply wait for truck drivers to start talking to them.

Turns out, people who spend hours upon hours in trucks by themselves are often willing to talk. And talk. And talk. (Although to be fair, not all of these truck drivers ride alone — many of them have pets. Chihuahuas are especially popular, I’m told.)

My wife and her coworker ended up having so many great conversations that they decided to take their customer research roadshow to a location with even more truck drivers: U.S. Route 11 in Pennsylvania, specifically the stretch between I-76 and I-81 called the “Miracle Mile.” This stretch of highway is littered with truck stops, and 6 to 7 thousand truck drivers drive along it each day.

So my wife and her coworker flew down to Pennsylvania, rented a car, and set a course for the “Miracle Mile.” This time around, they ditched the doughnuts and instead brought burgers and BBQ to the truck stops they visited.

In some cases, they would arrange ahead of time to have particular app users come meet them at particular truck stops. And in other cases, users who just happened to be at the same truck stop as them would recognize the logo (and/or be enticed by food) and come over to chat. But for the most part, the people they talked to weren’t app users (yet) — just truck drivers going about their daily lives.

Whether they used the app or not, all of the truck drivers my wife talked to ultimately helped her to better understand who the app was for.

Instead of picturing her target customer as some nebulous buyer persona, or as some office visitor sitting through a usability session, she could now picture her target customer as an actual human. Someone who had particular tastes and routines. Someone who was pumped to get a free cooler “because a cooler is something I can actually use!” And someone who, as my wife discovered, loves playing games on his smartphone.

The Breakthrough

By law, interstate truck drivers are allowed to drive a maximum of 11 hours in a single day, and must take a break of at least 30 minutes every 8 hours. These regulations help ensure that truck drivers stay well-rested and alert. But a byproduct is that truck stops are often filled with drivers who are just hanging out, killing time until they can legally get back on the road.

And what do truck drivers do to kill time … apart from talking to my wife?

She discovered that a lot of them love playing games on their smartphones. In addition to simply observing truck drivers playing games, one of her go-to questions when she’d visit truck stops was, “What are your favorite apps?” And 9 times out of 10 the truck driver she was talking to would describe (or demonstrate) some fun, colorful online game. “You see, I have this many points,” they’d tell her, “And I just unlocked this level.”

The fact that so many truck drivers were engaging with games on their smartphones never came out during phone conversations or during in-office user sessions. My wife was only able to uncover this by going out into the field, Indiana Jones-style.

Upon returning from the field, she took this new insight and figured out how she could best apply it to the app. The result? A rewards program, which allowed truck drivers to earn points every time they completed certain tasks, like submitting progress updates online. Once they had earned enough points, they could exchange them for prizes, like tablets or gift cards.

Truck drivers could now treat the app less like some boring accounting tool and more like a game — a game that they had an incentive to play. The more they used it, the closer they’d get to winning actual prizes.

And in case you were starting to think that this was just some fluffy, feel-good story about spreading customer happiness, and that there wouldn’t be any numbers behind it … you clearly don’t know my wife. The events I’ve been describing are from about five years ago, and she still knows the numbers.

After rolling out the rewards program to a subset of the app’s users and comparing their behavior to the behavior of a control group, she found that users who were enrolled in the rewards program were 70% more likely to use the app for online interactions (like submitting updates). This is exactly what they had been trying to solve for.

In addition to successfully incentivizing truck drivers to use more of the app, the rewards program led to an increase in driver performance. A few months after the program had been rolled out to the entire user base, my wife found that the on-time delivery rates of the drivers using the app grew from 88% to 97%.

So the rewards program wasn’t only fun for truck drivers, it also helped them operate more efficiently. And that, in turn, made shippers happy, since their shipments were now arriving on-time nearly 100% of the time.

Across the board, the rewards program helped attract new users to the app. The chance to win awesome prizes helped pull in new truck drivers. Meanwhile, the growing number of truck drivers on the app — not to mention their stellar on-time delivery rates — helped attract new shippers.

Business was booming.

About a year after the launch of the rewards program, the startup my wife was working at was acquired by one of the largest logistics companies in the world.

The company cited the rewards program (and the incredible results it produced) as one of their key motivators for making the acquisition.

Drift is a messaging app for marketing and sales. Click here to learn more about we can help you grow your business.

P.S. Join 20,000 of your peers. Subscribe to the newsletter for hypergrowth.

Every Sunday evening we'll send you a roundup of the best content and events from Drift and around the web. Make sure you're ready for the week! Subscribe now.

Subscribe Here