Client: Mobile App
Specialty: Education
Product: An interactive language learning app that teaches by using verses, quotes and stories
A language app wanted to increase conversions.
We delivered a custom segmentation tool that grows sales exponentially.
The Intro
Picture this — there are exactly 11,000 users on your app at this very moment. And you know everything you need to send your conversion rate through the roof:
- which 3067 of them are only one ad away from splurging on a costlier plan,
- which 554 are ready to buy upgrades without any incentive,
- and those 672 who don’t love their new app and seriously contemplate dropping it.
Miracle right?
Miracle, or our new segmentation technology, developed by applying user behaviour scoring technique derived from machine learning. A couple of minutes with us and you’ll find out just how we’ve created a method of identifying ready-to-pay users while also preventing possible customer drop-offs, let’s dive in!
The Challenge
Our client, an established language learning app, was looking to increase the number of paid conversions and reduce the churn rate, or simply put, inspire users to spend more and keep using the app longer. The first step on our way to succeed was to collect data from the following funnel checkpoints:
- screen views,
- course categories,
- words user chooses, saves or repeats,
- lessons user likes, shares or comments on,
- app usage frequency
Collected information has become the groundwork for all subsequent analytical processes, and in conjunction with statistics-based mathematical models we’ve developed, the prediction of the customer base behaviours (the miracle!) became possible. By the end of the experiment we would promptly identify:
- users who are the closest to make a purchase,
- users with the highest probability of churn,
- users’ interests, to offer them the most relevant content,
- users’ moods based on their messages, comments, reviews, etc.
As a result of this case study, a new tool has emerged. This new user behaviour platform now performs customer segmentation by analyzing the application’s user behaviour logs using scoring and machine learning techniques like neural networks and regression.
The Numbers
The technical side of our client segmentation and user-behaviour scoring solution is just as engaging as the narrational one, let’s take a look!
First things first, the activity data of 22,450 users became the foundation for this project, with information being captured by Amplitude and then transferred over to the MySQL database using our custom-made data connector through Amplitude API. Secondly, our one-of-a-kind machine learning model was launched to learn the patterns of 124 unique user behaviour variables from 20,400 users, and then tested on 2,050 users. A sum of two modelling algorithms — logistic regression and neural network algorithms (multilayer perceptron) — was chosen as the basis of the calculation formula.
The probability of users paying in the next 3 days served as the predictor variable. During our experiment, we found out the prediction accuracy of the logistic regression-based model amounts to 75%, while the prediction accuracy of the neural network algorithms is 79%.
Take a look at some of the actual predictors (analysis variables) we’ve used when building our new and shiny client segmentation solution:
- the number of lessons completed in the first 2 days
- the total number of words learned
- the number of words learned in the first session
- the number of words learned in the second session
- the average number of words learned per session
- the total number of sessions
- the percentage of evening sessions
- the percentage of morning sessions
- the device model
- user rating after the first session
- user rating after the last session
- the average user rating increase between sessions
- the number of pricing plan page views
- the first-query price plan
- which session number led to the pricing plans page
The Solution
Now, let us share some numbers and highlights from probability theory. When our one-of-a-kind model was tested live on 4,600 users who have not participated in the construction of the model previously, three user groups were determined:
- 409 ready-to-pay users
- 1,983 users who were one step away from purchasing
- 2,208 users who were not ready to pay
Users from all three groups were stimulated with a 5% discount for app purchasing using email and push notifications, and shortly the conversion numbers we’ve received reached 72% for group 1, 50% and 22% for groups 2 and 3 respectively.
And that’s how we’ve arrived at the most crucial values:
- First step – the result of the conversion ratio between the user identified as a potential payer (72+50)/2 = 61%
- Second step – the ratio of the first step to ones who are not ready to pay 61% / 22% = 2.78
Just imagine — our model has highlighted the users who were 3 times more likely to pay. 3 times more!
Undoubtedly, we have double-checked the results, comparing it to the control group of other 4,580 users who were presented with even a better purchase discount. Here conversion rate came to 36% which is 1.7 times lower than the test one.
The Results
As a result of our ingenious problem solving and incomparable all-encompassing modesty, our client has received a tailor-made customer segmentation and user-behaviour scoring technology that maximized our client’s return on investment by targeting only those users who actually require incentive ads to finish their purchase.
Moreover, this approach has significantly saved our client’s costs by spotting customers who have decreased in activity and timely stimulating their retention. In addition, our client segmentation model let’s business owners identify customers who are 3 times more likely to pay than the rest of the app users.
This client segmentation and user behaviour technology has also been granted an intelligence aspect — the ability to continuously learn online — which allows our client’s company to improve the prediction accuracy of their findings by bettering from new data constantly.
