Uncertainty is endemic to the entrepreneurial process. Global trends that emerged mainly during and after Covid-19 reshaped the advertising industry (McKinsey, 2021, October). In this unprecedented moment, where new customer behaviour has emerged, marketing leaders do not have all the answers to make rapid decisions. In addition, prior strategies based on third-party data must be reviewed in light of the deprecation of third-party cookies of Google. These continual upheavals and global shifts like pandemics, digitization, technology transformation and presently armed conflicts have increased the degree of uncertainty that challenged the decision-making process in business (Deloitte, 2021).
In this scenario, experimentation through A/B testing helps growth teams to take the guesswork out and enables data-informed decisions that shift business conversations from “we think” to “we know.”
The use of online A/B testing has helped decision-makers to navigate a contemporary dynamic environment better. Its use has soared among marketers, fueled by a growing appreciation of its value and by the relatively low cost of technology for conducting it.
Why A/B Testing
A/B testing (also known as split testing or randomized controlled experiments) is a method of comparing two versions of a webpage or app against each other to determine which one performs better. A/B Testing is a method almost 100 years old conducted for the first time in the 1920s by the statistician and biologist Ronald Fisher (Gallo, 2017, June).
Several studies suggest that experimentation through A/B testing leads to improvements in firm performance and startups (Koning, R., Hasan, S. Chatterji, A., 2021). When combined, growth marketing and experimentation can help develop innovative solutions minimising costs. In this context, the main goal of my blog post aims to help startup founders or Product/Marketing manager to choose the best A/B Testing tool for their team.
How to choose the proper A/B testing SaaS Tool for your team? Below I have gathered product reviews and thoughts based on my personal experience utilising these tools.
3 Great A/B Testing Software for Startups.
1. Google Optimize
Google Optimize, formerly called Google Website Optimizer, is freemium web analytics and testing tool by Google. Despite being a free tool, it still competes well in terms of quality with expensive tools such as VWO, Optimizely, AdobeTarget, and more. It’s a favourite tool for beginners of A/B testing, to get the initial know-how of experimentation getting started for free. It easily integrates with Google Analytics reports, and its interface is as user-friendly. It is worth mentioning its Redirect test feature available in the Free Version. This is useful to test an entire redesign or two different landing pages by testing two versions of the same page.
PROS
- Google Optimize is very easy to use for beginners with a Google Chrome Extension and WYSIWYG editor that users can use to set up tests.
- A Free-to-use plan is available with advanced features like Personalization, Multivariate Tests, Redirect tests and Simultaneous experiments.
- It is natively integrated with Google Analytics to help the team identify which parts of the website need improvement.
- Targeting options enable to include/exclude visitors based on several conditions
CONS
- Images: It doesn’t store large amounts of data, which means there is no way you can upload new images on the Google server. You need to upload images on a different server;
- Google Analytics Segments: this feature, is useful for personalization, it is only available for premium accounts;
- Limitation of free version: it is possible to run up to 5 tests at the same time and 3 goals making use of the free version as you can see in the screenshot below.
In a nutshell, below I have reported the official breakdown of the differences between the paid version and the free version. The paid version would cost around $150,000 yearly.
2. Optimizely (by Episerver)
Optimizely is the world’s leading experimentation platform, enabling businesses to deliver continuous experimentation and personalization across websites or apps. The company with its HQ in New York was acquired by Episerver in 2020 and currently employs over 320 people.
Optimizely is more than an A/B Testing tool. It is also an experience-optimizing platform. It helps to design experiments engaging cross-functional teams and then create a website to attract and retain this user segment. It also enables you to convert those people into paying customers.
Optimizely offers 5 plans at the present. If you’re looking to conduct experiments on your website, then you’ll want to look at Optimizely Web Experimentation. Below you find a list of Pros and Cons:
PROS
- Cross-functional teams can use Optimizely with little to no engineering support.
- Full stack solution: SDKs code implementation does not reduce performance on the website
- A Server-Side experimentation platform is available and more suitable for technical teams
- A/B and multivariate testing. It enables users with and without technical expertise to make dynamic changes to their websites with a client-side editor to gather immediate results.
- Once you publish an experiment it goes live immediately
CONS
- Pricing: For the smallest plan, you can expect to pay a minimum of $50,000 USD per year.
- Optimizely is an ideal solution for larger companies or well-funded startups that can afford its premium pricing.
- The Free Plan (Welcome Plan) may give the customer access to limited features.
3. VWO (Visual Website Optimizer)
VWO, headquartered in New Delhi, launched in 2009, crossed $25 Million ARR in 2021 and $30 in 2022. With VWO, you can bring your entire conversion optimization program to a single, connected platform—track business goals, analyze visitor behaviour, build data-driven hypotheses, run tests, and personalize content. VWO featuring Full-Stack Server Side give the flexibility to run omnichannel experiments with deep segmentation.
PROS
- Pricing: Starting from $99 per month. This pricing tag can be considered exceptional comparing it with Optimizely.
- Visual Interface: the UX is more user-friendly and the reports are more visually appealing than alternative tools.
- Easy to use. It is a simple step-by-step process for beginners to set up experiments. It is possible to copy & paste the script and create the test in the WYSIWYG.
- Seamless integration: VWO natively integrates Google Analytics, but also a number of third-party tools, to gather insightful data.
CONS
- Limitation of real-time targeting data (behavioural segmentation)
- Website Speed Performance: research conducted by CXL (Abrahamse, May, 2016) about website site speed performance, reported that VWO can have a negative impact on a website’s performance.
Conclusions
Before putting your A/B testing tools to use, my suggestion is to analyse the current needs of your startup team and customer. Selecting the right A/B testing SaaS depends on individual requirements and restraints. Here are some factors you may take into account in your decision-making:
a) The testing scope: do you aim to run only A/B tests, or do you also want the tool to handle multivariate testing or redirect testing?
b) Do you have budget constraints? If not, you might consider going for Optmizely or the premium versions of Google Optimize or VWO.
c) Choose a tool that minimises the impact on your website speed
d) Choose a tool that suits your team’s capabilities and their technical skills.
Did you find this article helpful? Comment below with your opinions! 🙂
Leave a Reply