Run Growth Experiments That Compound, Not Just Convert
Most marketing teams run A/B tests. Growth teams run experiments — and the difference is systematic. An A/B test answers "does variant B convert better?" A growth experiment answers "does this lever actually drive the metric we care about, and if so, how do we amplify it?" The compounding effect of a good growth experiment is a sustainable loop, not a one-time lift.
Prioritize experiments by expected impact × confidence × ease, not by whatever's easiest to build. A 20% improvement in activation compounds faster than a 5% improvement in ad CTR because activation affects every downstream metric.
Higher revenue growth for companies with mature experimentation programs vs ad hoc testing
Harvard Business ReviewOf growth experiments don't produce statistically significant results — velocity matters
Growth team researchMedian base salary for Senior Growth Marketing Managers at growth-stage tech
Industry dataIs this guide for you?
Use this Good fit if you…
- ✓You're running experiments but not seeing compounding growth effects
- ✓Your team runs tests without a clear hypothesis and success metric
- ✓You want to build a systematic experiment roadmap rather than ad hoc testing
Skip Not the right fit if…
- ✗You're pre-product-market fit and need to learn qualitatively before testing quantitatively
- ✗You have insufficient traffic for statistically significant experiments
- ✗You're in a brand marketing role where quantitative experimentation isn't the primary tool
The playbook
Five things to do, in order.
Map your growth model before designing experiments
Acquisition → activation → retention → referral → revenue. Where is your biggest leak? The answer determines where experiments will have the most leverage. Most teams experiment at the acquisition layer when the activation layer has a much bigger hole.
Write a hypothesis in the form: "We believe X will cause Y because Z"
"We believe reducing the signup form to 2 fields will increase activation because our session data shows 60% drop-off on field 3" is a testable hypothesis. "Let's try a simpler signup form" is a guess. The "because Z" is what turns a test result into a learning.
Define success and guardrail metrics before running the test
Primary metric: activation rate. Guardrail metrics: data quality score (guard against low-quality signups), day-7 retention (guard against activating users who don't retain). A test that lifts activation but tanks retention is a failure, not a success.
Run tests long enough for statistical significance at your traffic level
The minimum detectable effect (MDE) at your traffic volume determines minimum test runtime. Under-running tests creates false positives. Use a sample size calculator before you start, not after you see a number you like.
Build the learnings register, not just the results register
"Variant B won at 92% confidence, +18% activation" is a result. "Users who skip the team invite step in onboarding have 40% lower day-30 retention, which means the invite step is load-bearing for retention even though it reduces activation" is a learning. Learnings compound. Results don't.
See the transformation
"We ran 12 A/B tests last quarter and improved signup conversion by 8%."
"Built a growth model identifying activation as the primary constraint (28% of signups reached "aha moment"). Ran 4 activation experiments in 6 weeks: simplified onboarding (no impact), in-app guidance (no impact), early value demonstration (+14% activation, 94% confidence), and team invite prompt (+22% activation, 99% confidence). Implemented both winners. Activation improved from 28% to 41%, driving 18% improvement in monthly ARR."
Questions people ask
How many experiments should we run per month?
At most companies, 4-8 per month is realistic while maintaining statistical rigor. Volume without rigor produces false confidence. Better to run 4 well-designed experiments than 20 poorly-designed ones.
How do I handle experiments that conflict with brand or legal constraints?
Build the constraint list before designing experiments, not after. A growth experiment backlog that ignores brand guidelines or legal requirements wastes the entire team's time. Run all experiment concepts through brand and legal before committing to design.
Ready to put this into practice?
Get personalized coaching for your Marketing & Growth job search — resume, interviews, and offer strategy tailored to you.
