Why You Have to Generate Your Own Data

This is it. You’ve aligned calendars and will have all the right decision-makers in the room. It’s the moment when they either decide to give you resources to begin to turn your innovative idea into reality, or send you back to the drawing board. How will you make your most persuasive case?

Inside most companies, the natural tendency is to marshal as much data as possible. Get the analyst reports that show market trends. Build a detailed spreadsheet promising a juicy return on corporate investment. Create a dense PowerPoint document demonstrating that you really have done your homework.

Assembling and interpreting data is fine. Please do it. But it’s hard to make a purely analytical case for a highly innovative idea because data only shows what has happened, not what might happen.

If you really want to make the case for an innovative idea, then you need to go one step further. Don’t just gather data. Generate your own. Strengthen your case and bolster your own confidence – or expose flaws before you even make a major resource request – by running an experiment that investigates one or a handful of the key uncertainties that would need to be resolved for your idea to succeed.

That may sound daunting if you haven’t tried it. And, you may well ask, how do you do it when you lack a dedicated team and budget?  Fortunately, there’s a fairly systematic way to go about it.

Start by focusing your attention on resolving the biggest question on the minds of the people who will decide to give you those resources. That might be whether a customer will really be willing to use – and purchase – your proposed offering. Or perhaps whether the idea is technologically feasible. Or maybe there’s concern that some operational detail could stand in the way of success.

Once you’ve identified the most important potentially “deal-killing” issue, the next step is to find a cheap and quick way to investigate it. The key here is to find some low-cost way to simulate the conditions you’re trying to test.

For example, for several years Turner Broadcasting System (a division of Time Warner) had been playing with the idea of tying the first advertisement in a commercial break to the last scene in a television program or movie. Imagine a scene of a child landing in a mud puddle followed by a commercial for laundry detergent. Academic research showed this contextual connection had real impact, raising the possibility that Turner could charge a highly profitable premium to match the right advertiser to the right commercial slot. But would the system it used to match its content to advertisers’ offerings be too expensive to make the service profitable? And what if there just weren’t enough scenes in Turner’s library of movies and TV programs that could serve as effective contexts for its advertisers? How could the project team find out?

Instead of speculating, Turner locked a team of summer interns in a room for a few weeks, had them watch movies and television shows, and asked them to count the number of points of context in a select group of categories. Then Turner brought the results to a handful of advertisers, who enthusiastically supported the idea.

Imagine how these experiments changed the meeting. Without them, the team would have presented a conceptual plan full of glaring unknowns. But with these data in hand, they could offer evidence that the idea was feasible and that potential advertisers were interested. Perhaps not surprisingly, Turner ended up launching the idea, named TVinContext in 2008 to significant industry acclaim.

Working out how to generate data to test out an idea at its earliest stages requires some creativity. A mobile device company we were advising was considering a new service that would serve up customized content to consumers based on their mood and location. Would anyone want that? Would they pay for it?

To find out, we had to find a low-cost way to simulate the offering and some way to test people’s interest in something that didn’t actually yet exist. First we worked with third-party designers we contacted through eLance.com to develop mockups of what the interface might look like and worked up a two-minute animated video describing how the service would work. Here’s a screenshot from the video:

MobileCo Video

How could we tell whether the idea resonated with customers? Of course we could show them the mockups and videos and ask them if they liked or didn’t like the idea. But that really wouldn’t tell us whether they liked it enough to use it, let alone pay for it. So we asked customers at the end of the presentation if they wanted to be the first to participate in a beta test of the idea. All they had to do was give us their credit card number, and we’d charge them $5 once the test started. We didn’t actually plan to charge the consumers. Instead, we wanted to know how many were interested enough in the service to part with sensitive data in the hopes that they’d be first in line to access it when commercial trials began. When a significant number of customers were willing to give us credit card details, we knew we were going in the right direction.

One of the most valuable things these kinds of experiments can do is provide dramatically convincing evidence of serious flaws in your idea before you make the mistake of investing serious resources in it. The results from one concrete demonstration is worth reams and reams of historical market data.

For instance, an education company had what at first looked like a really promising idea to improve the quality and efficiency of teacher recruiting. Schools and applicants have long complained that paper résumés aren’t very good indicators of teaching ability and interpersonal skills. What if, the company wondered, we created a service that allowed schools to review short video clips created by prospective teachers showing them in action? Both teachers and schools loved the concept — on paper.

But then the education company tried to get real teachers to create real videos. It advertised the service in a handful of teacher-training colleges and put posts on on-line forums about the service. No interest. The company even began offering $100 for people to sign up. Still no interest.

It turned out that once the opportunity shifted from abstract to real, prospective teachers clammed up. They loved the concept of selling themselves through video, but in reality worried about how they would come across.

Notice how all of these examples involved some kind of prototype. As online tools improve and 3D printing becomes increasingly affordable and accessible, it’s becoming easier and easier to bring an idea to life without substantial investment. For example, a company that manufactures insulin pumps for people who suffer from Type 1 diabetes knew that customers didn’t love the physical designs of current pumps. The company was curious to find out how patients would react to pumps of different sizes and shapes. It worked with a small design shop in Rhode Island to develop a series of physical prototypes that brought the look, feel, and weight of the imagined devices to life. It had insulin pump customers pick up and play with the prototypes and compare them side-by-side with current offerings. The approach allowed the company to get critical feedback before it invested millions in more comprehensive design work.

None of the experiments described above required hundreds of thousands of dollars or hundreds of man hours. And yet they all quickly generated critical data that helped innovators to strengthen – or, in the case of the education company, discard – ideas. When it comes to making your case persuasive, one well-thought out experiment is worth a thousand pages of historical data. Certainly that’s well worth a little extra effort.

Persuading with Data
An HBR Insight Center
This entry was posted in Leadership and tagged . Bookmark the permalink.

Comments are closed.