The Predicted Time Until Next Order report is a combination of two different modeled attributes—Order Likelihood and Days Until Next Order. The models look at the last 180 days to predict the next 42 days (6 weeks). Optimizely Data Platform (ODP) builds each model custom to the data source, retrains it monthly, and runs new predictions nightly.
Learn how to use the report here.
Model inputs
Models require the following information:
- Each customer's events collected over the last 180 days
- The revenue from the customer to date
- The customer’s order frequency
- The average order value of the customer
How the model predicts
The predictions use a deep sequential neural network trained on a client’s own discrete data events. The model can account for an unknown number and type of events per input, including events proprietary to a customer like a product finder or a fit wizard, according to the period when they occur, without attempting to normalize or reclassify into consistent inputs. The model also works on anonymous data for users who have not made a purchase.
How the model gets evaluated
The platform holds a separate sample of your data when it creates the model to make predictions that then get compared to actual events.
The model is optimized to a measure called recall to find everyone likely to convert. Those with any likelihood to purchase have 40x the conversion rate of those who are unlikely to purchase. "Extremely likely purchasers" can be as high as 100x (or more) the conversion rate of those unlikely to purchase.
Model thresholds
While the model quality is different for every account, we expect the “Extremely Likely” category of customers to convert at a very high rate, followed by "Very Likely," and "Likely." Once optimized for recall, the model ranks customers by likelihood and divides into sevenths, where the first seventh is "Extremely Likely"; the next two-sevenths are "Very Likely"; and the final four-sevenths are "Likely." All other customers are considered "Unlikely" to purchase.
For the data scientists
Here is more technical information:
- The likelihood classifier typically has an AUC greater than 0.9.
- The model is biased to have high recall and low precision because the goal is to assist in identifying strong candidates to message and not just to predict the future.
- The Mean Average Error of our “Days to Convert” metric is around 13 days but more accurate when there is a strong intent signal or the likely conversion is close in time.