S.I.M.P.L.E. before S.M.A.R.T.

For a team new to the principles of "Lean," designing an effective experiment is a daunting and time-consuming process.

Grasshopper Herder Lean Startup Blog offers a more complicated template for designing effective experiments, but they also recommend everyone ignore it and instead design their own experients. 

Some teams love the experimental template and use it regularly, but for teams new to Lean, often designing quick-and-dirty experiments (without using a template) is better: just get the experiment started.

Velocity before Insights

One of the principles we focus on when coaching a new team is based on the Rudder Fallacy. A rudder is useless for changing direction if the boat isn’t moving. So the first thing we have to do to navigate is start rowing.

With an innovation team, it’s better to start running experiments than to worry about whether we’re running the right experiment or a perfectly designed one.

We encourage teams to get out of the building, listen to customers, and run quick landing page tests — anything to get them moving forward.

We’d rather teams run lousy interviews, ask all the wrong questions, and come back confused, versus spend a week designing a “perfect” customer persona and interview guide without actually speaking to any customers. Often, even with terrible interview techniques, a few smart innovators will return with the revelation that they should question their assumptions. And that’s the only insight they need: that they can be wrong, that they don’t know as much as they think they do.

So just as an MVP with customer feedback beats a “perfect” product that has never touched a user, a half-baked experiment beats a perfect one that has never been run.

Starting Simple

We often don’t use our complicated experiment template when working with a new team. Instead, we just talk through the Build, Measure, Learn loop. 

But we do it backwards. Because we’re funny like that.

Learn – What are we trying to learn? What is the one question whose answer would offer progress to our business model?

Examples:

1. Who is our customer?
2. How much will they pay?
3. What features would solve their problems?

Measure – What data, qualitative or quantitative, would help answer that question?

Do we need a yes or no answer to a question? Do we need to see someone make a purchase? Do we need to see a customer’s expression of delight when using our product?

Examples:

1. Percent conversion rate on a landing page
2. Qualitative observations and video of a usability testing session
3. Market research data on the total addressable market

Build – What do we need to do to get that data?

What is the least amount of effort we can put in to get an answer to our question?

Examples:

1. Build a landing page
2. Interview customers
3. Offer the solution as a service instead of a product

This simple formulation can be quickly grasped and used to design a basic experiment, without worrying about the early stop condition, the fail condition, or anything else that would distract the team from getting out of the building.

Learn Simple Template

Which brings us to this simpler template that we designed with only four boxes. Again, we go backwards through Build, Measure, Learn, adding a stopping point called Results & Insights.

1. Learn -> Learning Goal – What one, Specific question about our project should we answer that is Relevant to making progress?
2. Measure -> Data – What Measurable qualitative data or quantitative metrics will we collect?
3. Build -> Plan – How and when will we collect the data in a Timely fashion? Is it Achievable?

With the addition of Results & Insights, we’ve concluded our experiment. The Build, Measure, Learn loop may go round and round, but the experiment will stop once we have data that we can act on.

Simple = S.I.M.P.L.E.

Since our other template is called Learn S.M.A.R.T. (Specific, Measurable, Achievable, Relevant, & Timely), we decided to call this template Learn S.I.M.P.L.E. (Small Insights Mean Progress with Lean Experiments).

https://www.kromatic.com/resources/learn-simple

Now have fun, and get out of the building!

Lessons Learned:

• Out of the building is better than a perfect experiment never launched
• Don’t be a slave to templates
• Start S.I.M.P.L.E. and then figure out how to be S.M.A.R.T.

This post was written by Tristan Kromer (Founder & Lean Startup Coach at Kromatic, and Mentor at the Istanbul Founder Institute), and was originally published on his blog, GrasshopperHerder.com. 

Kromatic helps teams go fast. As lean startup coaches, they work with teams, leaders, and entire organizations to apply lean startup principles and build innovation ecosystems. Kromatic has worked with companies ranging from early stage startups with zero revenue to established businesses with >$10M USD revenue to enterprise companies with >$1B USD revenue.