Step 2.1: Tiny Experiments

Note: this is a series on making a digital health startup.  The first post is here.

In last week’s post, I talked about the tendency of experts to provide well-meaning-but-un-helpful feedback when you pitch your idea.  This week, I’ll talk about a technique for finding out whether people really like your idea: tiny experimentation.

Tiny Experiments

Tiny experiments are a great way to find out if you’re building something useful. If you set your criteria beforehand, it becomes easier to avoid bias, and you’re more likely to get a result that approximates reality.

So what do I mean by tiny experiments?  A tiny experiment is anything that helps you answer a specific question confidentially, as quickly and easily as possible.  For example:

Let’s say you want to make a new lunch restaurant.  It’d be cumbersome and expensive to get a license, rent space, buy equipment, hire staff, plan menus, and open, all before you know whether people actually like your food.

So why not just stand on the street offering samples, and for anyone who seems interested, offer them a coupon to pre-order $10 worth of food for $5.  If they really like your food they’ll do the pre-order, and if they don’t, they won’t.  If you stand on a busy street during rush hour you might interact with 30 or 40 people, and you’ll have a decent sense for your food’s demand.

Let’s say you want to build a new breast pump for nursing moms.  It’d be cumbersome and expensive to design the pump, build it, test it, and get FDA approval, all before you know whether moms want your product.

So why not design a quick 5 minute survey and send it to a few moms?  Describe the pump and then ask a question like: “If this breast pump were available today, and it were free, would you agree to use it for three days in a row?”  Moms are very busy, and they’re only likely to use a new product if it’s something they really want.  If you frame the question in terms of their willingness to experience a mild inconvenience in exchange for the new product, then you’ll find out whether people like it.

This technique, asking if people would use your product if it were free, comes from Steve Blank, a legend in the startup world.  In his essay about the topic Steve discusses the technique for a business-to-business IT product, but really it can be applied almost anywhere.

The key to a good “tiny experiment” is to make it simple and quick, and precise.  Ideally you should get your answer within five or ten hours of designing the experiment.

The Goal of a Tiny Experiment

At a high level the goal is to find out whether people want your product in exchange for some inconvenience. For example: use my product, but pay me X dollars. Use my product, but go through the hassle of integration with your tech team. Use my product, but get your MD to sign on.

Simply asking people, “do you like my product,” is difficult because it’s so abstract. Do I like it? Sure — in some cases. But getting people to make a specific tradeoff in exchange for your product gets you to a more precise knowledge of their desire. And if people still want to use your product, even with the inconvenience, then you’re making something people want!

A variation on this approach, suggested to be my the legendary Doug Ranalli, is to present two different potential products and to ask the customer which idea s/he likes more. Again people often find it hard to say whether they like something in the abstract, but comparisons are way easier. Do I like this telemedicine app concept? Sure. But do I like it more than the medical tourism idea? Now that’s useful information.

Put together a list of 15 or 20 product ideas, and just go out and start comparing 1 to 2. After a while you’ll notice the trend that 2 is more popular than 1, so cross off 1 and now compare 2 to 3. Maybe 3 beats 2, so do 3 vs. 4. Eventually you’ll find a product that beats out the others, and that’s the one you stick with.

Designing Experiments

Ok you know you want to do a tiny experiment – now what?  It’s hard to think of your idea in these terms and to simplify your startup concept into something that can be figured out in a short experiment.

So how do you get started?  Just get started!  Your startup probably has a million implicit assumptions in it; try to write down five of them, and then think of a quick way to prove or disprove the most tenuous assumption.

For example, you’re thinking of making a telemedicine app.  There are probably dozens of implicit assumptions in there, but these are some big ones:

  • Customers want a doctor on their phone.
  • Doctors are willing to speak with customers remotely.
  • You have legal permission to do telemedicine.
  • You can charge a price that makes you profitable.
  • You have the technical feasibility to build the app.
  • etc. and etc.

Here are some quick experiments you can run to test each of these assumptions:

  • Customers want a doctor on their phone.
    • Run a survey of 10 friends; ask them whether they’d be willing to use your app for six months as their exclusive first point of contact with doctors.
  • Doctors are willing to speak with customers remotely.
    • Speak with 5 doctors and ask if they’d be willing to sign up for your “Doctors on Call” list (that you just made up).
  • You have legal permission to do telemedicine.
    • Do a quick google search; surely someone has asked this question before.
  • You can charge a price that makes you profitable.
    • Run another survey of friends, and this time ask if they’d be willing to pre-order two consultations for the price of one.
  • You have the technical feasibility to build the app.
    • Set up a five hour time-boxed Saturday afternoon, and see how much you get built.
  • etc. and etc.

Once you get into the mindset of experiments, you’ll start thinking of them for almost anything.  Where should we throw our birthday party?  Go to five places and see what has the best vibe!  Should I launch a new app for medication adherence?  Make a PPT mockup and ask 5 friends if they’d agree to use it for a week!

The key is just getting started.  Start with a silly experiment, and try it out.  The next will be a little better, a little smaller, or a little more precise.  And eventually you’ll be an expert!

Next week we’ll talk about deciphering the language in a customer feedback interview.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s