In part two we look more closely at the segmentation options that help you avoid the single biggest mistake personalization teams make — deciding on the basis of averages across the whole audience.
Mistake #6. Ignoring the funnel stage
One of the first segmentation methods eCommerce companies reach for when they start with personalization is splitting users into new and returning. Personalization algorithms usually know nothing about a new user yet, so showing them popular or discounted products feels like the logical move. Returning users, whose data has accumulated over previous sessions, can already be shown personalized selections based on their preferences — and that is what many companies do. Unfortunately, that choice can be wrong.
Example: a marketer at a consumer electronics and appliances retailer had heard a lot about the effectiveness of the affinity algorithm (it builds a personalized product selection based on a user’s preferences across categories, brands, colour and other attributes). Their hypothesis was that adding a recommendation block with that algorithm to the cart page would raise average order value. Unfortunately, the A/B test showed no statistically significant difference between the control and test groups. The hypothesis was not confirmed.
How to fix it: test different recommendation strategies depending on the funnel stage. With the electronics retailer we suggested going deeper on customer segmentation and testing different mechanics for different types of returning users — those with no purchases and those who had bought in previous sessions. Users without purchases saw a mix of affinity products and recently viewed items; users with purchases saw recommendations built on the “Bought with recent purchases” algorithm. So someone who had previously bought a PlayStation saw a relevant selection of video games in the cart, while the new owner of a vacuum cleaner saw dust bags. The updated logic in the recommendation blocks delivered a statistically significant +5% in average order value for users without purchases and +13% (!) for users with purchases.
Mistake #7. Ignoring context
One of the oldest — and still highly effective — mechanics for lifting conversion and average order value is social proof. You have surely seen those colourful badges on online store products, with their categorical claims along the lines of “In demand”. Judging by recent test results, people still tend to trust other people’s choices and are still more willing to add a product marked “Bestseller” to the cart. Used carelessly, though, social proof can produce the opposite effect.
Example: a fashion and beauty retailer ran a successful social proof test in the cosmetics category. Adding a “Bought X times today” badge to creams and shampoos lifted average order value overall and sales of the brand’s products in particular. The CMO insisted on scaling the mechanic to every other category — and saw conversion drop in womenswear. Clearly, shoppers there wanted to feel unique in their choice of outfit: buying a dress “that everyone has” was not something they got excited about.
How to fix it: test different messages for different categories. Social proof did eventually work for the clothing category once we changed the wording on popular items to “Only X left”.
Mistake #8. Ignoring the traffic source
Across many experiments we at Gravity Field have found consistent patterns in user behaviour depending on how people arrive at the site. In the previous article I described the case of a search-traffic segment for which similar-product recommendations worked best. Even the same traffic type, however, can behave differently depending on the business.
Example: a test of where to place the application form for a bank’s financial product. The hypothesis was that users arriving via a referral link already know everything about the product, so moving the form above the fold would raise visit-to-application conversion. The test delivered only +0.5% in conversion, below both expectations and international benchmarks.
How to fix it: use URL-based micro-segmentation of the audience. We looked at the test results by micro-segment and saw that they behaved differently. Users arriving via referral links from financial aggregators converted +5% better. Moving the form above the fold, on the other hand, cut conversion for users arriving via referral links from CPA bloggers. Using the predictive targeting engine we forecast the uplift for each micro-segment if the campaign were relaunched with new logic: the form above the fold for aggregator traffic and the form on the third screen for blogger traffic. The result was +6% for each micro-segment with the placement relevant to it.
Predictive targeting is a personalization platform engine that makes decisions using machine learning and forecasts the uplift you get from scaling the best “hypothesis to segment” combinations.
Mistake #9. Being afraid to experiment with the cart
The cart and checkout pages are the most sensitive point on the site. If something goes wrong at this stage, all the conversion work done earlier is wasted. That is why product managers are often against adding any extra widgets to the cart. And that is a mistake.
Example: an eGrocery retailer with hundreds of thousands of users whose carts routinely hold 40–50 SKUs. It takes 30–40 minutes on average to fill a cart like that, and in that time some items can sell out — the store is popular. In our experience, session conversion drops more noticeably without replacement suggestions for sold-out items than it does because of bugs.
How to fix it: target recommendation blocks by the contents of the cart. Users with sold-out items need a replacement block above all, while users with only 1–2 items are better served by promotional “checkout aisle” products.
Mistake #10. Relying on prior knowledge alone
In the previous article we wrote about the dangers of shallow “men and women” segmentation, and here we want to continue the theme of behavioural patterns and the treachery of prior knowledge.
Example: a sporting goods store. Its marketers know they have a segment of so-called “kit buyers” — users who come to the store for complete sets of gear. Surely those users should be shown an algorithm-assembled set of products in recommendations? It sounds logical, but the test showed the opposite: adding the recommendation block cut conversion in the kit buyers segment while increasing the number of items per order for everyone else.
How to fix it: practise a data-driven approach. Based on the “hypothesis to segment” data, we started showing kit buyers a selection of similar products. That algorithm — counter to the prior logic of “why would a shopper want ten identical balls?” — was what lifted product page conversion in this segment.
Later, in UX research, we found the answer: kit buyers prefer to build the cart item by item. It is easier for them to pick each piece separately from a selection of similar products, while ready-made sets only scatter their attention.
To sum up: every user sends a great many signals when they enter an online store, during the session and even after they leave. At the moment of entry we already know which traffic source brought them, whether they have been on the site before, what phone they use and even which operating system version. During the session we catch dozens and hundreds of micro-signals — category visits, product views, adds to cart and to favourites, use of search and filters, and so on. And even after the user leaves we keep learning something new about them: whether they collected a click-and-collect order, how many days they have stayed away, whether they unsubscribed from email.
The more factors you take into account in segmentation, the more winning “hypothesis to segment” combinations machine learning algorithms can find. Keep that in mind and get the most out of what personalization can do.
