“The Scientific Method” from https://www.reforge.com/blog/run-growth-team-like-a-scientist
- Question: What do I want to understand?
- Hypothesis: What is my proposed explanation to the question?
- Experiment: Design an experiment to test the hypothesis.
- Observation: Observe the experiment and record results.
- Analysis: Interpret the results.
the qualitative inputs - question, hypothesis, and experiment design - are the most important steps of the process. If you get these steps right, you increase your chances of running successful experiments that produce valid results and actionable insights.
So, what separates the good growth teams (and research labs) from the great ones?
It’s the quality of the questions they ask, the hypotheses they come up with, and how they construct their experiments.
A one-day or one-week in vitro test using a landing page builder to mockup a subscription pricing page, driving traffic to it via ads, and measuring conversion rates against current pricing may have quickly revealed that a subscription pricing model wasn’t appealing to their customer base.
the point is to use an in vitro test to get validation that an in vivo test is worth the additional resources. Tools like Unbounce, Optimizely, and paid advertising make these kinds of in vitro tests easy and fast.
One of my favorite mentors, Anantha Katragadda, told me when I was starting out in growth that if I couldn’t think of a way to fake a product experience in order to get validating data, I simply wasn’t trying hard enough.
To publish a paper, every result must disprove the null hypothesis. This means that you prove, with statistical significance, that the result you are seeing is not due to random chance or biased observation