Conversions Matter / DEAL v1.0
Better decisions.
Before more tests.
Define. Evaluate. Act. Learn.
A practical process for deciding whether to investigate, test, fix or do nothing and monitor.
An A/B test is one possible action. DEAL helps you work out whether it is the right one.
Freely available. No email, registration or download required.
Four stages. Explicit gates.
Make the reasoning visible.
Record the evidence, uncertainty and rationale at each stage. New information can reopen an earlier decision at any time.
What decision are we making?
Describe the problem before proposing a solution. Name the audience, journey and context. Separate what you observed from what you think caused it.
Agree the decision owner, deadline, primary outcome, baseline, meaningful improvement and guardrails. Specify what could change, the cost of delay and the consequences of being wrong.
Is the evidence good enough?
Trust: Validate events, missing data, duplicates, consent coverage, instrumentation changes and reconciliation with operational records.
Evidence: Combine analytics, research, support feedback and technical checks. Challenge competing explanations, seasonality and traffic mix.
Feasibility, risk and value: Compare expected benefit, effort, opportunity cost and the value of reducing uncertainty. Check privacy, security, accessibility and compliance with responsible owners.
Choose the right response.
Investigate, Test, Fix or Do Nothing / Monitor. Select the action that fits the evidence and the decision, rather than treating every idea as a test candidate.
For testing, agree the hypothesis, meaningful effect, eligible traffic, sample requirements, duration, allocation, analysis and stopping rules. A test must be capable of informing the decision in time.
What changed our understanding?
Check implementation and data quality before interpreting results. Separate the observation, uncertainty, explanation and next decision.
Capture negative, unexpected and inconclusive findings. An inconclusive experiment does not establish equivalence. A before-and-after improvement alone does not establish causality.
The four choices under Act
A useful outcome is a decision.
These scenarios illustrate the routes. They are examples, not client results.
Investigate
Resolve uncertainty about the problem, cause or measurement. Set a question, owner and time limit.
Example: A conversion drop follows a tagging release. Validate tracking and reconcile orders before changing checkout.
Test
Resolve meaningful uncertainty through a safe, feasible experiment whose result will change an action.
Example: Research suggests delivery uncertainty deters purchases. Test clearer information with trusted metrics and sufficient traffic.
Fix
Address a verified defect or established requirement. Validate the change and monitor for regressions.
Example: A broken payment button blocks customers. Repair it; retaining avoidable harm adds no useful learning. The size of the benefit still requires evidence.
Do Nothing / Monitor
Choose this when change is not justified or value is too low. Assign an owner, review date and reopening trigger.
Example: A small segment fluctuation is within expected variation. Monitor a material threshold rather than launching a low-value test.
Interactive decision guide
Should we
test this?
Five questions to identify your next step. Start with the decision and the evidence.
A guide to judgement, not a substitute for the accountable owner or specialist review. Every route returns to Learn.
Question 1 of 5
Is the decision clear and actionable?
You know the problem, owner, outcome and deadline.
Read the full decision guide
1. Is the decision clear and actionable? If no, return to Define.
2. Can you trust the evidence needed? If no, Investigate and return to Evaluate.
3. Is there a verified defect or requirement? If yes, consider Fix with validation.
4. Would resolving meaningful uncertainty change your action? If no, Do Nothing / Monitor.
5. Is a safe, compliant, useful test feasible within the decision window? If yes, Test. If no, investigate alternative evidence, redesign or monitor.
Close the loop
Learning belongs in the next decision.
A result is useful when it changes what you understand or do. Record the limits of the evidence alongside the result, then return to the relevant stage.
For monitoring, set a review date and a trigger. For a fix, check recovery and regressions. For an investigation, update the explanation. For a test, apply the agreed decision rule and guardrails.
Your decision record
01 Context, decision, owner and deadline
02 Evidence, assumptions and limitations
03 Options, chosen action and rationale
04 Expected outcome and guardrails
05 Actual outcome and uncertainty
06 Learning, next action and review trigger
Part of the toolkit
Use the right framework at the right moment.
DEAL structures the decision process. These complementary frameworks support the work within it.
