The Senior Manager, Experimentation, Testing, & Learning owns the quarterly experimentation roadmap and learning agenda that drives measurable improvements in buying group progression. This role runs end-to-end test governance across buying group experience, ensuring tests are designed well, implemented correctly, measured cleanly, and translated into clear scale or stop decisions.
What you will own
Quarterly experimentation roadmap, learning agenda, and test portfolio aligned to brand, events, and integrated program goals with emphasis on buying group progression.
Test intake, prioritization, design standards, and readout templates used across teams.
Scaling criteria and playbooks for rolling out tests that show measurable lift.
Governance for experimentation processes, including decision rules and documentation standards.
Responsibilities
Own and evolve the quarterly experimentation roadmap and learning agenda, aligned to brand, events, and integrated program goals with emphasis on buying group progression.
Manage test intake and prioritization using clear criteria, expected impact, feasibility, time-to-learn, and cross-channel applicability, maintaining balanced coverage across paid, owned, site, and creative.
Define and maintain test design standards: hypotheses, primary and secondary KPIs, success thresholds, guardrails, and decision rules.
Partner with Measurement to define incrementality approaches, holdouts, lift studies, geo tests, platform experiments and standardize efficiency metrics.
Partner with agency and platform teams to ensure clean implementation, consistent exposure rules, and strong QA across channels.
Coordinate with DX on site and landing page tests, including experimentation tooling, tagging, data layer dependencies, and conversion-path measurement.
Partner with creative and agency teams to structure creative tests (message, offer, asset variants) with clear learning objectives and measurement plans.
Standardize how results are reported — readout templates that clearly summarize lift, confidence, segment impacts, tradeoffs, and recommended decisions.
Lead quarterly readouts that drive decisions to scale, stop, iterate, or retest, with documented rationale.
Maintain scaling criteria and update global playbooks when tests show measurable lift and repeatability.