What is A/B Testing?
Definition
A/B testing is a controlled experiment that shows two versions of a page, email or ad to randomly split groups of real visitors at the same time, then measures which version produces more of a chosen outcome, such as enquiries or purchases.
Last updated 2026-10-18 · Flowmatix
How an A/B test works
You start with a hypothesis, not a hunch: "Visitors abandon our quote form because it asks for too much up front." Version A is the current form with nine fields. Version B asks for four. Testing software sends each new visitor to one version at random and keeps them there on return visits, so outside factors like weekday, weather or a newsletter going out affect both groups equally.
After enough visitors have been through, you compare the conversion rate of each group and check whether the gap is larger than random noise would explain. If B wins clearly, it becomes the new default and you test the next idea. If the result is flat, you have still learned something: form length was not the problem.
The same method works for subject lines, headlines, pricing page layouts, a call to action wording, or ad creative.
Why it matters for your business
Opinions about websites are cheap and confident. A test replaces "the owner prefers blue" with evidence about what your actual buyers do. Over a year, a steady habit of small, measured improvements compounds: a page that converts better makes every advertising dollar and every SEO visit worth more.
Testing also protects you from expensive changes that feel like progress. Redesigns sometimes lower conversions, and without a control group nobody notices until the leads dry up. A/B testing is one tool inside wider conversion rate optimisation, which starts with research into where and why people drop off.
Common mistakes
- Stopping a test the moment one version pulls ahead. Early leads often vanish as more data arrives.
- Running tests on pages with too little traffic to ever reach a clear answer. Low-traffic sites learn more from user interviews and session recordings.
- Changing five things at once, so a win cannot be traced to any single cause.
- Measuring clicks instead of the outcome that pays the bills, such as booked calls or completed orders.
- Testing trivial details (button shade) before fixing obvious problems in the offer or the page structure.
Frequently asked questions
How long should an A/B test run?
Long enough to cover at least one full business cycle, usually a minimum of one to two weeks, and long enough to collect a sample size you decided on before starting. Ending on a fixed date or sample, not on a good-looking day, keeps the result honest.
Can a small business A/B test with little traffic?
Yes, but pick bold changes with large expected effects, such as a different offer or a much shorter form, and test on the highest-traffic pages or in paid campaigns. Subtle tweaks need volumes most small sites do not have. Our landing page best practices are a good source of bold first hypotheses.
What is the difference between A/B and multivariate testing?
An A/B test compares whole versions. A multivariate test changes several elements at once and measures every combination, which needs far more traffic to produce reliable answers.
How Flowmatix can help
Our conversion optimization work puts this into practice for businesses in Toronto and around the world. Get a personalised quote in about 3 minutes.
Related terms
Ready to see what your project costs?
Answer a few questions and get a personalised plan, a comparison with typical market prices, and a clear next step. No obligation, no sales pressure.