LLM Prompts

A/B Test Hypothesis Generator

Generates A/B test hypotheses to guide experimentation.

The method

Use this prompt when planning A/B tests for marketing campaigns, website changes, or product updates. Paste it into ChatGPT and replace the bracketed placeholders with specific details about your experiment. The goal is to create clear, testable hypotheses.

The prompts

Prompt 1
Develop three A/B test hypotheses for [Website Landing Page] with the goal of increasing [Conversion Rate - e.g., sign-ups, purchases, demo requests]. The current landing page features [brief description of current page]. The proposed change involves [describe the change - e.g., different headline, new call-to-action button, altered image, reorganized content]. For each hypothesis, specify the metric being measured (e.g., click-through rate, bounce rate), and explain why you believe the change will impact that metric. Format each hypothesis as follows: 'We hypothesize that [change] will lead to [expected result] because [reasoning].' Provide brief suggestions on how to ensure statistical significance. Also suggest audience segmentation strategies to refine the analysis.
Prompt 2
Imagine you're a marketing expert specializing in A/B testing. Your client wants to improve their [Email Campaign] open rates and click-through rates. The current email campaign has a subject line that reads '[Current Subject Line]' and a body that focuses on [brief description of content]. The client is considering testing different subject lines and calls-to-action. Generate two alternative A/B test hypotheses, each including a proposed subject line, a proposed call-to-action, and the anticipated impact on the email metrics. Each hypothesis should contain the following: 'By changing the subject line to '[New Subject Line]' and using the call to action '[New Call to Action]' we expect to see an increase in [Open Rate/Click Through Rate] because [Explain the reasoning. Focus on emotional triggers, urgency, clarity, value proposition, etc.]' Include guidance on setting up the A/B test within an email marketing platform.
Prompt 3
You are a seasoned product manager looking to optimize user engagement within a mobile application. Users are currently dropping off at the [specific stage in app funnel - e.g., onboarding, checkout, feature usage]. The team suspects the user interface at this stage is confusing or inefficient. Construct two contrasting A/B test hypotheses focusing on UI/UX improvements at this specific stage. Hypothesis 1 should explore simplification of the user interface. Hypothesis 2 should test a more visual or interactive approach. In each hypothesis, specify the exact changes being tested (e.g., number of form fields, use of progress bars, type of animation). Format each hypothesis as follows: 'We hypothesize that [UI/UX Change, be specific] will lead to [improvement in engagement metric, e.g., lower drop-off rate, increased feature completion rate] because [reasoning].' What A/B testing framework (f.e. Bayesian, Frequentist) is most appropriate in this scenario, and why?