Skip to content
AvraAvra
Esc
↑↓navigate↵open⌘Jpreview
On this page

A worked example

One Avra RFM prediction from start to finish — a retailer's churn question, its data, the framing, the backtest, and the weekly list it feeds.

This page follows one prediction from the first conversation to the list a team works every week. The company, its tables, and every name are invented, and the page shows what each step produces, not results: results come from your data, measured on your history. The steps are the ones every Avra RFM prediction goes through.

The decision

An online retailer’s retention team calls customers who look about to stop buying, and gives them a reason to stay. The team can make 500 calls a week, so the real question is which 500. Today, it calls the customers whose last order is the oldest.

That decision sets two things before any data is touched. The list is rebuilt every Monday, and what matters is how many of the 500 people called would really have stopped buying.

The data

The retailer sends five tables, as they are in its systems, through the ingestion pipeline:

Each table tells the model what its time means:

Table Its time Kind Why
customers signed_up_at Event A customer exists from the moment they sign up
orders placed_at Event An order is known the moment it’s placed
order_items placed_at, copied from its order Event A line doesn’t inherit its order’s time, so the extract carries it
products None Static The catalog’s names and categories don’t change
support_tickets received_at Available Tickets reach the warehouse in a nightly sync, so a ticket is known when it arrives, not when it was written

The ticket subjects and product names are text, and the model reads them as language, Portuguese included. Nothing is flattened into one row per customer, and no features are built. See Relational data.

The prediction

With the retailer’s team, the question becomes four parts:

Part For this decision
The records Each customer in customers
Who’s in Customers with at least one order in the 90 days before the prediction date
The outcome No order placed
The window The 28 days after the prediction date, so the answer is known 28 days later

The outcome is a yes-or-no question, so the RFM returns a score for each customer that ranks how strongly their data points to it. The team only needs the ranking, because it calls the top 500. A customer who signs up after a prediction date isn’t scored for that date. See What you can predict.

What one prediction sees

Take the list for Monday, March 2, 2026, and one customer on it. For that customer, the model reads the rows connected to them as they stood on March 2: their orders, the items and products in those orders, and the tickets that had arrived by then. A ticket written on March 1 but synced on March 3 is out of view.

It also reads labelled examples: customers at earlier Mondays, and whether they then went 28 days without an order. An outcome is usable only once its window has closed, so the examples come from February 2 and earlier, whose windows closed by March 2. The window that starts on February 9 doesn’t close until March 9, so the list for March 2 never sees its outcome. See How predictions work.

The backtest

Before anything goes live, the same prediction runs at past dates, every Monday of 2025. Each run sees only what was known that day, and each is scored on what happened in the 28 days after it. The RFM is compared with two baselines: the team’s current rule, oldest last order first, and the simplest one, each customer’s outcome in the previous window.

Because the team works the top 500, the measure that decides is precision at 500: of the 500 customers each list would have put first, how many really stopped buying. The report also shows how the scores rank the whole population, and whether the results hold up across weeks and segments:

The report shows Why it matters here
Precision at 500, each week How many calls would have reached customers who were really leaving
The same for each baseline Whether the RFM improves on the current rule, not only on a naive one
ROC-AUC and PR-AUC How well scores separate leavers from stayers overall. PR-AUC matters because few customers leave in any one window
Results by segment Whether the ranking holds for new and returning customers alike, not only on average
Results over time Whether the ranking stays steady from week to week

The team moves on only if the RFM beats both baselines at 500 calls, week after week. See Evaluation.

The shadow run

Next, the RFM scores customers every Monday on live data while the team keeps calling from its current list. Once each week’s outcomes are known, 28 days later, the two lists are compared. Nothing about the calls changes until the comparison is settled.

Putting the scores to work

Once the team promotes the version behind the production alias, the prediction runs every Monday:

The batch calls the production alias, so a later promotion changes nothing on the retailer’s side. A batch-ended webhook tells the retailer’s job to check the batch it started and import the results. Every week, the team also compares the score distribution with the previous week’s. Once each week’s window closes, it checks how many of the 500 calls reached customers who really were leaving. See Batch inference and Webhooks.

Was this page helpful?