LLM Prompts

A/B Test Result Analyzer & Action Suggester

Analyzes A/B test results and suggests actionable next steps based on statistical significance.

The method

Use this prompt after conducting an A/B test. Paste the results, including metrics like conversion rate, click-through rate, and confidence level. Specify the tested variables and the goal of the test. Apply in any LLM.

The prompts

Prompt 1
Analyze the following A/B test results and provide actionable recommendations. The goal of the test was to increase sign-up conversion rates on our landing page. Variant A (Control): Headline 'Start Your Free Trial Today'. Variant B (Treatment): Headline 'Unlock Your Potential: Free Trial'. Conversion Rate: Variant A: 3.2%, Variant B: 4.5%. Click-Through Rate: Variant A: 1.8%, Variant B: 2.1%. Statistical Significance (Conversion Rate): 95% confidence level. Statistical Significance (Click-Through Rate): 80% confidence level. Provide a summary of the key findings and suggest at least three specific actions we should take based on these results. Explain the rationale behind each suggested action, considering both the statistical significance and the practical implications. Consider the impacts on other metrics of interest, like bounce rate and page load time (assume these were negligibly different). Additionally, outline potential follow-up A/B tests to further optimize the landing page.
Prompt 2
You are a marketing optimization expert. I have the following A/B test data: Experiment: Website button color change (blue vs. green). Goal: Increase button click-through rate (CTR). Variant A (Control - Blue): CTR = 2.5%, Sample size = 1000. Variant B (Green): CTR = 3.1%, Sample size = 1000. Statistical Significance: p-value = 0.08. Considering the above data, provide the following: 1. A concise summary of the results. 2. An assessment of whether the difference in CTR is statistically significant and explain your reasoning. 3. Based on your analysis, what action would you recommend (e.g., implement the change, run the test longer, etc.)? Explain the reasons behind your recommendation. Finally, consider any potential risks associated with your recommendation (e.g., brand perception changes, accessibility issues) and offer possible mitigation strategies for each. Assume that the change in button color has no impact on site speed or SEO.