PromptsEdge
FreeGitHubMarketingScripts

A/B Test Planning

Your AI plans, designs, and documents A/B tests, calculates sample sizes, and ensures your experiments are statistically valid. Turn your product changes or ideas into actionable, rigorous test plans with clear hypotheses, metrics, and duration estimates.

Try asking: “Draft an A/B test plan for changing our signup button color”

alirezarezvani on GitHub

Curated by PromptsEdge from a public repo · MIT license. All credit goes to the author.

What is this skill?

Running experiments without a clear plan risks wasted time, misleading results, and missed opportunities. Once your AI has this skill, it can turn any product or marketing change into a well-structured A/B test, covering everything from hypothesis framing to sample size calculation and variant design. This is for anyone who wants reliable answers about what works—product managers, marketers, designers, or founders.

What you can do

  • Frame strong hypotheses, e.g., “Making the CTA button larger will increase signups by 15% for new visitors.”
  • Calculate the exact sample size and estimated test duration for your baseline and expected lift.
  • Design test variants with clear, single-variable changes and document them for implementation.
  • Select primary, secondary, and guardrail metrics tied to business value, such as conversion rate or refund requests.
  • Choose the right test type (A/B, A/B/n, MVT, Split URL) based on your traffic and goals.
  • Allocate traffic and plan for technical constraints, timeline, and analytics setup.

How it works

When you request an A/B test plan, your AI first checks for any existing product marketing context to avoid redundant questions. It then asks targeted questions about your goal, current performance, constraints, and available tools. Using your answers, it frames a clear hypothesis, selects the best test type, and calculates the required sample size and duration using a built-in Python script. The AI documents all steps—hypothesis, metrics, variant specs, traffic allocation, and sample size—so you have a ready-to-review test plan. It also cross-checks calculators and references for accuracy.

Good to know

  • You’ll need Python 3 installed to run the sample size calculator script locally if you want to verify calculations.
  • For the most accurate plan, have your baseline conversion rate and daily traffic numbers ready.
  • The skill outputs human-readable plans and tables; for pipelines, you can request JSON output.
  • For tracking implementation, pair this with an analytics-tracking skill—this skill focuses on test design, not code instrumentation.

Related skills

View all →