Fashion retailers face extremely high demands in the fast-growing and innovation-driven e-commerce market: customers expect a huge product selection and informative content such as fitting guides, sizes, and material descriptions – all presented in a design that inspires and creates an immersive brand experience.
A leading premium casual lifestyle brand had redesigned its online shop. It then faced the challenge of determining the performance of the new shop – in terms of technical functionality as well as user experience and conversion. FELD M was tasked with designing a real-world experiment to enable the customer to monitor user navigation and conversion within the newly revamped shop and via the existing platform. To achieve this, we needed to develop a solution to channel predefined percentages of randomly generated user groups to each of the two shop versions for a certain period.
However the proposed A/B test demanded more than regular testing tools. A tailor-made approach was required.
Thinking outside the box
Our approach included cookie logic and server-side delivery of traffic to the two shop versions. This way, we could split the brand’s overall audience into one group that continued to see the old shop version and another group that was steered toward the new version. In Google Analytics 360 – the client‘s web analytics system – we set up two separate views, each showing the data of one user group. Using an application programming interface (API), these data were presented in a customized report that afforded a clear overview of statistical significance during the experiment.
In the first step, 80% of traffic was delivered to the old version and 20% to the new shop for several days. By monitoring 10 predefined KPIs, we were able to determine whether there were any major difficulties with the new shop while limiting its initial exposure. After this first experiment yielded positive results, users were channeled in a 50/50 split to the two shop versions for four weeks. This real-world test delivered robust data on all 10 KPIs, including overall conversion as well as micro conversions, along the entire funnel.
The result
After the 4-week experiment, the customer could go ahead with the shop relaunch, confident that no significant conversion downlift was to be expected.
Have a similar project?
Let's find out together how we can help!
