Back to projects
PythonStrategySQLData
A/B Testing Framework
Built an end-to-end experimentation framework for the marketing team, including statistical analysis pipelines.
24Experiments launched
34%Conversion lift
$150KAd spend saved
Project visual
Overview
A reusable framework that allows the marketing team to design, run, and analyze A/B tests without engineering support.
The Problem
The marketing team was running experiments ad-hoc with no standardized process, leading to inconclusive results and wasted budget.
Approach
Created templates for experiment design, built Python notebooks for statistical analysis, and established governance around sample sizes and significance thresholds.
Tools Used
PythonJupyterSQLGoogle AnalyticsStatsmodels
Outcomes
- Launched 24 experiments in first quarter
- Improved email conversion rate by 34%
- Saved $150K in ad spend through better targeting
Learnings
Frameworks are only useful if they are accessible. Documentation and training were as important as the technical implementation.