Marketing Micro.Blog

TL;DR: Manton Reece (creator of Micro.blog), and Daniel Jakut (creator of MarsEdit, Mac software for posting to webblogs, including Micro.blog), recently discussed, on their long-running Core Intuition podcast, how Manton could better market Micro.blog and gain more users during the mass-exodus from Twitter that has followed Elon Musk's acquisition and mismanagement of the platform. My top advice is:

1. Rely more on viral and (incentivized) referral marketing than on paid advertising.
2. Go after authors and other (professional) writers, who seem like a perfect fit for the platform
3. Create specific material for people thinking of switching from Twitter to Mastodon, explaining the relationship between Micro.blog, Mastodon, the Fediverse and Twitter, and emphasizing what's different about Micro.blog.
4. Make the client software more obvious on the Micro.blog website; at the moment they're kind of buried in the Help pages.
5. Before you even consider paid marketing, analyse your existing sign-up, conversion and usage data and get set up to do that consistently. If you don't have historical data, start capturing it immediately: history starts with your oldest data (as my friend, David Townsend, taught me).
6. If you do paid advertising, consider not giving incentives as part of the promotion; if you do give discounts, be careful interpreting the results of the campaign.
7. Be aware of negative effects in marketing (which, although relatively unlikely in your case, are more widespread than is commonly accepted). See
   • No (cartoon)
   • Triggered Comparison Shopping (cartoon)
   • Demand Suppression (cartoon)
   • Neither a Response Modeller nor a Penetration Modeller Be (article)
   • The Queen's Hall of Shame (article)
   • How to Lose a Customer, part 1 (article)
   • Target marketing (cartoon)

Background

Micro.blog is an excellent Twitter alternative (inter alia) created by Manton Reece, who is a deep thinker about social media, and has written book about Indie Microblogging. Micro.blog was launched in 2017, as both a blogging platform (with a particular, but non-exclusive focus on title-less, “microposts”, as well as full blog posts), and has expanded to include hosting podcasts, photos, “book shelves” and more.

Micro.blog is a first-class member of the Fediverse, in the sense it supports ActivityPub, meaning that you can follow and be followed by Mastodon users from Micro.blog. Manton deliberately eschews many of the Twitter features that Manton thinks have contributed to its becoming a toxic platform from many users. For example, it has no equivalent of retweeting, does not encourage use of hashtags, does not use an algorithmic timeline, does not show the numbers of followers and account has, does not carry advertising, and has active community management, with Jean MacDonald (@jean@micro.blog) employed as its community manager.

Micro.blog has an open API, allowing people to create accounts and post without paying, and also to link one or more external blogs to Micro.blog, but most users subscribe either to its $5/month hosted microblog plan, or its $10/month Micro.blog Premium plan, which includes podcasting, video, bookmarks and (sending) newsletters.

It's slightly ironic that I am not, in fact, a very active user of Micro.blog, for somewhat obscure reasons, but I am an enthusiast for the platform and have followed its development and growth since before the Kickstarter campaign Manton used to launch it. (The reason I don't use it much as is that I've been taking advantage of its open nature as a way to allow to experiment with hosting a microblog and blog on my own weird tag-based database, microdb, and because I don't work much on that, posting to it is actually a bit of palava for me. I also, already, have blogs on way too many platforms (e.g. this Scientific Marketer Blog on Blogger, the Test-Driven Data Analysis Blog on Github Pages (with Pelican), various List blogs and prose blogs on Eric Bower's fascinating list.sh and prose.sh, SSH/PKI-based platforms under the pico.sh umbrella. All of these effectively syndicate through my micro.blog/njr account.) (Pretty good that you call do all that, huh?)

Marketing Micro.Blog

On Episode 541 of Core Intuition, Manton and Daniel discussed how to market Micro.blog, and mentioned that their community manager, Jean MacDonald, always emphasizes that, if they do, they need to measure the impact (which is obviously right).

As someone who has worked a lot in marketing (not the adtech, surveillance capitalism kind, but the sort of direct marketing most businesses need to do), I have thoughts. Here I'll expand on the TL;DR above.

1. Viral and (incentivized) referral marketing. Social media is viral by nature, and current (active) users are generally the best ambassadors for the platform. I would have thought that further encouraging members to promote the platform by giving them discounts (probably free months/credit on their plan) when someone else signs up for a paid account would be a highly efficient way of using the existing community. I suspect the incentive would not have to be a lot: a single extra month on whatever plan you're on if someone signs up using your code seems plenty.
2. Go after authors and writers. There was some talk on the podcast about celebrities like Elton John, and Manton commented that there are no music-related features on the platform (though as Daniel keeps pointing out, it wouldn't take much to extend bookshelves to record shelves, film shelves etc.)

   But Micro.blog does have a special focus, it seems to me—writing and books. First, unlike Twitter and Mastodon, there is no limit to post length on Micro.blog. If you go past the maximum micro-post length (280 characters), only the first part is shown and you have to click a link to see the full post; and full blog posts with titles just show as links in the timeline. But despite the name, Manton absolutey sees Micro.blog as a full-blown blogging platform. (This confused me for ages, and the name still seems odd, to me, for a platform that supports micro-posts and full blog posts; but naming is hard.)

   More than this, Manton loves books, has built a special Bookshelves feature for tracking what you're reading. Manton has also added the ability to send out your posts as email newsletters, as well as publishing them to the web.

   It seems to me that the ideal people to attract to Micro.blog are writers. I don't know what the best way to do this is, but probably just reaching out to some (and probably offering free accounts, I guess) would be a great start. If a few (more?) novellists, poets, and essayists joined the platform, I would have thought that would both further raise its tone and potentially attract many of their readers.

3. Create specific material for people thinking of switching from Twitter to the Fediverse. Unsurprisingly, there is a help article on Micro.blog called What’s the difference between Micro.blog and Twitter? That's great, and is needed more than ever. But there's not an article called What’s the difference between Micro.blog and Mastodon? I think creating one is important, ideally emphasizing many of same things as the article contrasting with Twitter does, but also more specifically highlighting the significant differences between Mastodon specifically, and the Fediverse more generally, while simultanously getting the message across that you can be a first-class participant in the Fediverse on Micro.blog.

   And of course, that help article shouldn't just be buried in the help: it should be a blog post, linked from the front page, and promoted to high heaven, especially to all the people now writing articles about Mastodon and the Fediverse. (And writers!)

4. Highlight the client software more. Although it wasn't hard to find, I was slightly surprised, today, how long it took me to find the Mac client for Micro.blog, which I didn't have on the particular machine I'm writing on. It isn't in the Mac App store (which is fine), but it also isn't (I think) mentioned on the front page of the site, isn't very prominent in the help, and when you log in on the web, there isn't even an understated call to action suggesting you might want to try a native client.

   This is true on mobile too.

   While there's nothing wrong with using Micro.blog through a web browser, many (most?) people choose to interact with it, as with other social media platforms, through a client app. And there are lots. (Two of Manton's commendable attitudes are (1) that he is a huge enthusiast for the web in general, and the indieweb in particular, and (2) that he is deeply committed to letting a thousand client flowers bloom, and doesn't privilege the apps Micro.blog itself produces.)

   But it wouldn't hurt to let people know.

   It's not like they're trying to hide it. There's a great (anchorless) section of the Micro.blog help site highlighting (at the time of writing) nine apps you can use with the service. But you have to go looking.

5. Measure before you market. As noted above, Jean apparently encourages Manton to put in place good measurement before embarking on (paid) marketing, which is obviously sound advice. As much of the content in other articles on this blog emphasize, these measurements are quite hard, but important.

   Possibly less obvious is that you need a good baseline set of measurements before you start. Micro.blog may already track sign-up rates (for trials), conversion rates, upgrade rates, retention (renewal rates)/churn etc., but I don't ever really remember hearing Manton talk about it on Core Intution, so possibly not. Even if no one at Micro.blog already does this, I would think there's a good possibility enough data is kept to be able to analyse many of these things retrospectively. I would strongly recommend building at least basic stats for these things and going back and graphing them over the whole history of the platform if possible. This will (a) be useful in itself and (b) set the company up for measuring marketing more effectively. (Needless to say, the measurement should be ongoing and ideally automated to produce some kind of report or dashboard, either on-demand or as something computed routinely.)

6. Be careful with incentives. I am definitely not advising against paying for promotional marketing; but I am suggesting that there will probably be a better return on investment from all the things above for lower cost than any likely ROI on direct/promotional paid marketing. If Micro.blog does decide to do promotional marketing—for example, by sponsoring podcasts, as discussed on the show—there are two important things to bear in mind:
   • Even if you use a code to track where people came from, that does not prove that they are incremental sign-ups or sales. Of course, it's quite strong evidence (and even stronger evidence that they probably heard the promotion), but some people will (do!) use promo codes who would have bought anyway.
   • This is particularly true if the promotion comes with some kind of discount or incentive—a 30-day instead of a 10-day trial; 12 months for the price of 10; whatever. For example, if someone I know wants a cloud-based server, I will definitely direct them to Linode (because they're excellent), but I'll point them to one of podcasts with codes offering $100 free credit, because: well, why not? This is fine: I'm sure Linode is happy to give people the $100 to sign them up, just as I'm sure Manton is happy for them to get a discount. But it does affect the measurement of incrementality, and the bigger the incentive (and the $100 from Linode is huge), the more likely it is to affect assessment of incrementality (and, therefore, ROI).
   So if the purpose of the code is to track where users come from, a discount is double-edged: it makes it more likely people will use the code, but also makes the inference you can make from that use less strong. It also slightly biases things towards people who might not really want to pay the full price (though, to some extent, that's just human nature.)

   Needless to say, having good historical measurements allows you also to offset some of this, because if signups are fairly steady you should see the aggregate effect of promotional marketing.

7. Be aware of negative effects. My final caution probably isn't highly salient for Micro.blog, but is important for all marketers to understand: marketing can, and does, have negative as well as positive effects. Unfortunately, there's an Upton Sinclair aspect to this:

   “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” — Upton Sinclair, I, Candidate for Governor: And How I Got Licked.

   Marketers tend to assume that the worst that can happen in a marketing campaign is that it costs money and has no impact. In fact, the truth is much harsher: it is perfectly possible to spend good money to drive customers away. In fact, it's common. Usually, there's a mix, with a few people being put off, and more people being turned on, so that there's a net benefit. It takes fairly inept marketing to generate a net negative effect in overall behaviour.

   One case, however, where this is extremely common is retention activity. I've written about this extensively, for example in this paper on churn reduction in mobile telephony. The basic gist is that people who leave are generally getting poor value for money, poor service, are not using the product or have had a bad experience: often they remain paying customers mostly out of apathy, laziness or because they don't realise they're out of contract. Calling them up and offering them the chance to lock in again often acts as a catalyst for cancellation. (“Really? I'm out of contract. Does that mean I can cancel right now?”) I'm not for a moment suggesting there are hoards of Micro.blog customers only paying subscriptions because they've forgotten about them or can't remember how to cancel. But you never know: there might be a couple!

Uplift Modelling FAQ

[This is a post that will be updated periodically as more FAQs are added, so if you subscribe to the feed, it may keep re-appearing.]

1. Q. What is Uplift Modelling?
   A. Uplift modelling is a way of predicting the difference that an action makes to the behaviour of someone. Typically, it is used to predict the change in purchase probability, attrition probability, spend level or risk that results from a marketing action such as sending a piece of mail, making a call to someone, or changing some aspect of the service that the customer receives.
2. Q. Uplift Modelling sounds like Response Modelling. How is it different?
   A. Ordinary "response" modelling actually doesn't model a change in behaviour (even though it sounds as if it should): it models the behaviour of someone who is subject to some influence. Uplift models instead model the change in behaviour that results when someone is subject to an influence—typically, how much more that person spends, how much less likely (s)he is to leave etc.
   Mathematically, a response model predicts something like

   P (purchase | treatment)

   ("the probability of purchase given some specific treatment", such as a mailing), whereas an uplift model predicts

   P (purchase | treatment) – P (purchase | no treatment)

   ("the difference between the probability of purchase given some specific treatment and the corresponding probability if the customer is not subject to that treatment").
3. Q. Uplift modelling sounds like Voodoo. How can it possibly know the change in behaviour of a single individual?
   A. Uplift modelling can't know the change in behaviour for any individual, any more than a normal model can know the behaviour of an individual in a future. But it can predict it. It does this by looking at two groups of people, one of which was subject to the marketing action in question, and the other of which was not (a control group). Just as it is standard to measure the incrementality of a campaign by looking at the overall difference in purchase rate between the treated group and an otherwise equivalent control group, uplift modelling models the difference in behaviour between these two groups, finding patterns in the variation.
4. Q. Does Uplift Modelling Really Work?
   A. Uplift modelling can work, and has been proven to do so with in-market tests. Uplift models are harder to build than conventional models, because they predict a second-order effect—usually the difference between two probabilities. This means that the error bars tend to be larger than for conventional models, and sometimes there is simply not enough signal for current techniques to model accurately. This is especially true when, as if often the case, the control group is small.
5. Q. When does uplift modelling predict different things from non-uplift models?
   A. It's perhaps easier to say when they predict the same thing. This is usually when there is essentially no behaviour in the control group. For example, if a set of people purchase product X after a mailing, but no one purchases it without the mailing, and uplift model should predict the same thing as a conventional response model. Their predictions are most different when the variation in the change in behaviour opposite from the variation in the underlying behaviour. For example, suppose the background purchase pattern (the one you see if you don't do anything) is that mostly men by product X, but the effect of a marketing action is to make more women buy it, but fewer men, even though still more men than women buy when treated. In this case, uplift models will make radically different different predictions from "response" models. A response model will concentrate on the fact that more men buy (when treated) that women; but an uplift model will recognize that women's purchases are increased by the treatment whereas men's is suppressed.
6. Q. How do you measure the quality of an uplift model?
   A. Standard quality measures for models (such as gini, R-square, classification error etc.) don't work for uplift models as they are all based on comparing an actual, known outcome for an individual with a predicted outcome. However, since a single person can't be simultaneously treated and not-treated, we can't make this comparison.
   There is, however, a generalization of the gini measure called Qini that has some of the characteristics as gini, but which does apply to uplift models. This has been described in the paper referenced as [1].
7. Q. What are the main application so of uplift modelling?
   A. So far the biggest successes with uplift modelling have been in the areas of customer retention and demand generation (cross-sell and up-sell, particularly).
   The state-of-the-art approach to customer retention is to predict which customers are at risk of attrition (or "churn") and then to target those at high risk who are also of high value with some retention activity. Unfortunately, such retention efforts quite often backfire, triggering the very attrition they were intended to save. Uplift models can be used to identify the people who can be saved by the retention activity. There's often a triple win, because you reduce triggered attrition (thus increasing overall retention), reduce the volume targeted (and thus save money) and reduce the dissatisfaction generated by those who don't react well to retention activity.
   The other big successes have come in the area of cross-sell and up-sell, particularly of high-value financial products. Here, purchase rates are often low, and the overall incremental impact of campaigns is often small. Uplift modelling often allows dramatic reduction in the volumes targeted while losing virtually no sales. In some case, where negative effects are present, incremental sales actually increase despite a lower targeting volume.
8. Q. Are there any disadvantages of uplift modelling?
   A. Uplift modelling is harder and requires valid controls groups to be kept, which have to be of reasonable size. Experience shows that it is also easy to misinterpret the results of campaigns when assessing uplift, especially when it is first adopted. Adoption of uplift models usually results in reductions in contact volumes, which is sometimes seen as a negative by marketing departments. An uplift modelling perspective also often reveals that previous targeting has been poor, and sometimes brings to light negative effects that had not previously been identified.
   There is also some evidence that uplift models also seem to need to be refreshed more frequently than conventional models, and there are clearly cases where either data volumes are not adequate to support uplift modelling or where the results of uplift modelling are not significantly different from those of conventional modelling. Anecdotally, this seems to be the case in the retail sector more than in financial services and communications.
9. Q. How does uplift modelling relate to incremental modelling?
   A. It's the same thing. Various people have apparently independently come up with the idea of modelling uplift, and different statistical approaches to it. There is no broad agreement on terminology yet. Names include
   • uplift modelling
   • differential response analysis
   • incremental modelling
   • incremental impact modelling
   • true response modelling
   • true lift modelling
   • proportional hazards modelling
   • net modelling.
   These are all the essentially the same thing.

References

[1]	Using Control Groups to Target on Predicted Lift: Building and Assessing Uplift Models, Nicholas J. Radcliffe, Direct Marketing Journal, Direct Marketing Association Analytics Council, pp. 14–21, 2007.

41 Timeless Ways to Screw Up Direct Marketing

Screw Up the Control Groups

1. Don't keep a control group
2. Make the control group so small as to be useless
3. Raid the controls to make up campaign numbers
4. Use deadbeats as controls
5. Don't make the controls random
6. Use Treatment Controls but not Targeting Controls

Alienate or Annoy the Customer

7. Trigger defection with retention activity
8. Use intrusive contact mechanisms
9. Use inappropriate or niche creative content
10. Insult or patronize the Customer
11. Mislead or disappint the Customer
12. Fail to Listen to the Customer
    • Bonus marks for refusing to take "no" for an answer
    • Double bonus marks for calling the customer back after she's said "No thanks, I'm not interested" and hung up.
13. Overcommunicate
14. Make it hard for the Customer to do what you want.
    • (Overloaded websites, uncooperative web forms, understaffed call centres, uninformed staff and failure to carry stock to meet demand are but a few of the ways to achieve this).
15. Intrude on the Customer's Privacy
16. (Over)exploit the customer

Misinterpret the Data

17. Confuse "responses" with "incremental responses" (uplift)
18. Confuse revenue with net revenue
19. Take Credit for That Which Would have Happened Anyway
20. Double Count
21. Believe the name of a cluster means something
22. Believe the data
    • The fact that the computer says it's true and prints it prettily, doesn't mean it is.
23. Disbelieve the data
    • The fact that the data doesn't show what you hoped, thought or expected doesn't mean it isn't so.

Screw Up Campaign Execution

24. Discard response information.
25. Revel in unintended discounts and incentives
26. Use dangerous, insulting or disrespectful labels
    • Yes, I really have known an airline label its bottom-tier frequent fliers "scum class", and a retailer label a segment of its customer "vindaloonies".
27. Ship mailings with the internal fields filled in instead of external ones (see also 26)
    • Dear ScumClass, . . .
28. Direct people to slow, hard-to-navigate websites, IVR Systems or overloaded or poorly designed call centres
29. Fail to inform your front-line staff about offers you're sending to customers
30. Fail to ensure your company can fulfill what the marketing promises
31. Use a name that's very visually similar to a better known name that has negative connotations for many people.
    • I frequently get mail from Communisis, but I never read their name correctly.

Mismodel or Misplan the Campaign:

32. Confuse statistical accuracy/excellence with ability to make money
33. Model the wrong thing
34. Use undirected modelling instead of directed modelling
35. Assume equivalence of campaigns when important things have changed.
36. Ignore external factors (seasonality, competitor behaviour etc.)
37. Fail to make dates relative
38. Contaminate the training data with validation data
39. Screw up the Observation Window (Predict the Past, not the Future)
40. Ignore changes in the meaning of data
41. Fail to sanity check anything and everything

The Two Controls

Neither A Response Modeller nor a Penetration Modeller Be

In The Fundamental Campaign Segmentation I introduced a theoretical classification of customers according to how their behaviour is affected by a marketing action such as a mailshot. Today I want to look at a softer, more practical version of that segmentation, and use this to look at the rather serious shortcomings of both response models and penetration models.

The “hard” version of the segmentation that I talked about previously depended on knowing whether each customer buys if treated, and also whether she buys if not treated. Such knowledge allows us to talk of “Persuadables”, who buy only if treated, “Sure Things”,1 who buy either way, “Lost Causes”, who don't buy either way, and “Boomerangs”, who only buy if not treated. But of course, we can never know which segment anyone falls into, because we can't both treat and not treat someone.

What we can do more easily, both conceptually and in practice, is to estimate the probability of someone's purchasing when treated, and the corresponding probability of purchasing when not treated. This suggests a “soft” version of the Fundamental Segmentation as follows:

The horizontal axis shows the probability of purchase if the customer is not treated, while the vertical axis shows the corresponding probability under treatment. A customer who is completely unaffected by the campaign will lie along the centre of the white stripe (the “leading diagonal”). The more positively the action affects the customer's purchase probability, the further toward the top left she will lie on the graph, and conversely, if the action actually reduces her probability of purchase, she will lie further towards the bottom right corner.

Given this framework, it is natural to define soft versions of the segments with boundaries parallel and perpendicular to the diagonal representing “completely unaffected” customers. The key point is that our returns are proportional to the distance from that diagonal (which represents the uplift). Distance from the diagonal towards the top left corner quantifies positive incremental impact (extra sales generated), while distance below and to right represents negative effects (destruction of incremental sales).

The Problem with Penetration Models

Let's look at penetration models through the clarifying lens of our Fundamental Segmentation in its “soft” form. Penetration models simply look at the proportion of people who purchase a product without any particular stimulation. In other words, they model purchase probability without treatment—exactly what we are plotting on the horizontal axis. So let's look at what happens if we target some campaign action on the basis of such a penetration model. The usual practice is to target everyone with a “propensity” above some cutoff, i.e. everyone to the right of some vertical line on the graph. One such is shown on the diagram below.

As the diagram makes fairly plain, the problem with penetration models is that they tend to target all of the Boomerangs, thereby actively driving away business, as well as wastefully picking up all of the Sure Things, who would buy anyway. Perhaps more surprisingly, they don't even encourage us to target all of the Persuadables. This is because penetration models are built on an untreated population and have absolutely no information about what would happen if we stimulated these customers.

Clearly, if our goal is to maximize the return on a direct marketing campaign, the last thing we want to do is target on the basis of a penetration model.

The Problem with Response Models

So if penetration models don't look to be doing anything very close to what we want, what about traditional “response” models? The difference between a penetration model and a response model is that where the former is built on an untreated population, the latter is built on a treated population. Assuming, again, that some threshold value is picked for probability of purchase if treated, the impact of targeting with a response model is shown below.

The good news is that the “response” model (so called) does something closer to what we want, while still being far from ideal. On the positive side, it does capture all the Persuadables—the people whose purchase probability is materially increased by our marketing action. On the less positive side, it also targets all the Sure Things (who would have bought anyway) and a good proportion of the Boomerangs, for whom our efforts are counterproductive.

How Uplift Modelling Does the Right Thing

The problem with both penetration models and response models is that they misdirect our targeting efforts by modelling the wrong thing. If our goal is to maximize the incremental business we generate, we would like to target the Persuadables and no one else. If we model uplift—the increase in purchase probability resulting from treatment—this is exactly what we are able to do, regardless of exactly how much “uplift” we require to break even. This is illustrated below.

Attrition, Churn and Lapsing: Retention Targeting

If our goal is retention, the story is similar, but with some intriguing (and perhaps unexpected) differences.

The soft version of the Fundamental Campaign segmentation for retention presented previously is exactly as you would expect, and is shown here.

The Problem with Attrition Models

The standard approach to model-driven targeting of retention activity is to model the probability that a customer is going to leave and then to target valuable customers thought it identifies as having an unacceptably high risk of leaving.

For applications such as cross-selling, we saw that a response model (built on a treated population) was a significant advance on a penetration model (which is built on a non-treated population). As we shall see, this pattern is reversed for retention.

If we have an established retention programme, the likelihood is that most, if not all, high-risk customers will be subject to retention activity. As a result, when we build our attrition models (churn models), these will usually be based on a population that is largely or entirely made up of customers who have been (at least in the high-risk segments). We can see the effect of targeting on this basis on the overlay below.

It might at first seem strange that a retention model built on a treated population has exactly analogous failings to a penetration model for demand stimulation: it misses some of the Persuadables, (customers who can be saved, or at least made more likely to stay, by our retention activity) while targeting all Sleeping Dogs (who are driven away by it) as well as many who will go regardless of our actions (the Lost Causes).

As the next overlay shows, if we are going to target on the basis of probability of leaving, it should at least be the probability of leaving if we do nothing. Ironically, this is easiest to achieve if we have a new retention programme, and therefore an untreated population.

So while for demand generation, a response model built on a treated population is somewhat better than a penetration model built on a treated population, this situation is reversed for retention activity: we are better off targeting people based on their probability of leaving if untreated than the corresponding probability when subject to a retention action. And it is particularly important to get this right for retention because most retention activity actually triggers defection for a significant minority of those we try to save (and occasionally a majority).

Uplift to the Rescue

Needless to say, once again the real solution is to target based on that which actually affects campaign profitability, the incremental effectiveness of the action, or the uplift. We can see this clearly on our final diagram.

As I said before, it is important that we model that which is important, and if our goal is to maximize the incremental impact of our campaigns, that is uplift.

Footnote

1In fact, the name I used for Sure Things in the previous post was “Dead Certs”, but it seems that is a bit of UK-centric idiom (“cert” being an abbreviation of certain); Sure Things appears a bit more international, i.e. American. ↩

Uplift and Lift: Not the Same Thing at All

A lot of this blog is about Uplift. You may wonder why I use an ugly, slightly redundant term like uplift when marketers normally talk simply about lift. The reason is that they are completely different.

Lift is a measure of targeting effectiveness that tells you how much better than random your targeting is. The idea is simple. Suppose you get a 1% purchase rate if you target randomly, and a 3% purchase rate if you target only segment A. Then the lift is 3% / 1% = 3. If you have a score (say, purchase probability) then you can plot lift as a function of volume targeted (starting with the best customers), and it will look something like this (assuming your score allows better-than-random targeting):

Lift curves show the same information as Gains Charts (with which they are often confused) but display it in a different way. And they always end at 1 because of course if you target the whole population, you get the same response rate as if you target randomly (on average).

In contrast, uplift directly measures the (additive) difference between the expected outcome in a treated group and control group. For a binary outcome, such as purchase, this is

P (purchase | treatment) – P (purchase | no treatment).

For a continuous outcome, such as spend, this is

E (spend | treatment) – E (spend | no treatment).

So in summary,

• Lift measures the effectiveness of targeting. It quantifies how much better the outcomes are in a target group than they would be in a randomly chosen group (multiplicatively: 2 = twice as good).
• Uplift quantifies the effectiveness of treatment. It measures the difference in outcome between a treated group and an equivalent non-treated group (additively: 10% uplift means that if a person's probability of purchase without treatment is 5%, with treatment it's 15%.) Strictly, uplift for binary outcomes is measured in percentage points, while for continuous outcomes, e.g. incremental spend, uplift is simply the difference in expected spend between the treated and non-treated group.

Response Model

The Fundamental Campaign Segmentation

I am going to attempt to change forever the way you think about marketing campaigns, but I need your help: I need you to suspend disbelief for a few paragraphs,

Let's think about a direct marketing campaign intended to stimulate purchase of a particular product—Product X. We'll make it very clean, by assuming our goal is simply to get people to purchase Product X within one month of the campaign drop.

We get to choose whether to include each customer in the campaign, and each customer either does or does not purchase Product X. Of course, customers can buy Product X even if they're not included in the campaign. As rational marketers, our goal is to generate incremental sales, i.e. ones that would not have occurred without the campaign.

Now let's segment these customers into four groups as follows.

• The Persuadables. These are the people who only buy the product if we include them in the campaign. They are our true targets.
• The Dead Certs. This group buy the product whether or not we treat them. Think of the people who queue for days to be the first to get a new games console.
• The Lost Causes. Like the Dead Certs, the Lost Causes are completely unaffected by the campaign—the difference is that whether or not we treat them, they don't buy.
• The Boomerangs. We have to accept, at least in principle, the possibility that there is fourth group—people who only buy if we don't include them in the campaign. You may be skeptical that this last group exists, but for now at least accept their logical possibility. One name for this group is the Boomerangs, because—metaphorically—they come back and hit you in the face if you target them.

I'm not a great fan of Boston Boxes, but this does seem like a case screaming out for one, so here it is.

If our goal is truly to maximize the return on our marketing investment, and if we assume for now, that there are no side effects (warm fuzzy feelings generated by being given an offer, even if you don't take it up—that sort of thing), it is clear that the ideal targeting strategy would be to treat only the Persuadables. That way, if you treat 100,000 people, you get 100,000 incremental sales, which is perfect. (I know, I know, people could buy two. Or Seventeen. And they could tell their friends. And you could win the lottery three weeks running…)

If you've hung in this long, there are a couple of obvious big questions. One is whether the “Boomerangs” actually exist. The other is whether we can identify these groups, even if we accept, in principle, that they must (or might) exist. I'll tackle the first of these today; the other will be the subject of a separate, future post.

Boomerangs, Sleeping Dogs, Triggered Cancellation and Negative Effects

There is one case in which the existence of Boomerangs (people for whom the marketing campaign has the reverse of the desired effect) is both simple to demonstrate and easy to understand—the case of retention activity. My company, Portrait, has seen numerous cases of retention campaigns that increase the rate of customer loss in certain segments, and in some cases actually increase it overall. This is not as strange as it may seem.

The mainstream state-of-the-art approach to customer retention for contract-based products (e.g. phone contracts, insurance policies, and fixed-term loans) is first to build a model of how likely people are to fail to renew. In the communications industry, such models are called “churn models”, while in financial services they are normally known as “attrition models”. In the simplest case, people with a high modelled propensity to leave are targeted with a retention action. A more sophisticated approach weights the attrition probability with an estimate of customer value, so that retention activity focuses more on revenue (or profit) at risk than on customers per se.

It's obvious that there is a strong correlation between a customer's level of satisfaction with a service and his or her likelihood of renewing. So modelling attrition probability is pretty similar to predicting dissatisfaction, i.e. the customers most commonly targeted by the standard approach are those who are unhappy with the provider.

Many of us, however, are lazy. Even though I might hate my phone provider or insurance company, there's a good chance that I won't, make the effort to dig out a number and call them up to cancel. It's just too much hassle.

But what if that company takes the trouble to call me up just before my renewal is up? Needless to say, this removes a crucial obstacle to my cancelling, and there's a very good chance I'll say “Ah, I don't suppose just cancel right now is there?” Especially if the call itself annoys me, as is the case for many people.

We have overwhelming evidence that such triggered cancellation is a real and common phenomenon. Boomerangs definitely exist in many, if not most, retention campaigns, though I tend to think of this segment more as “Sleeping Dogs” who are best left to lie. (Though my colleague, Neil, tells me that in Liverpool the phrase is “Don't let sleeping dogs lie.” What can I say?)

But even where such Sleeping Dogs do exist, unless the retention action is spectacularly inept there will be other segments in which the retention efforts can bear fruit. The trick is to separate the wheat from the chaff—or in this case, the Persuadables from the Sleeping Dogs.

Here is the corresponding Boston Box for retention campaigns

For demand generation campaigns—cross-selling, up-selling, deep-selling etc.—negative effects are usually smaller. However, we have compelling evidence that they do exist. This matters because even a company happy to lavish attention on customers whose behaviour will not be positively affected by it should think twice before actually spending money to drive business away.

As you might expect, more intrusive contact mechanisms (particularly phone calls) swell the numbers of Boomerangs, as do campaigns that are narrower, more divisive or risqué. One of our clients claims he always sees negative effects in the last one or two deciles.

Summing it up: The Incremental Gains Chart

From one perspective, the Dead Certs (people who will buy whether we treat them or not) and the Lost Causes (who won't buy whatever we do) are equivalent: our action has no impact on either group. Thus if we think only in terms of our impact, a simpler segmentation is into the group we affect positively (the Persuadables), the group for whom our intervention has no effect (collectively, the Unshakables), and the group we affect negatively (the Boomerangs, or Sleeping Dogs).

If we assume that we can predict the change in purchase probability for each person, we can then plot an incremental version of a gains chart to produce the “Italian Flag” diagram, as below.

Zone 1, (the green stripe), consists of the people we can affect positively—the Persuadables. If we could identify them perfectly, this would be a straight line, that comes to an abrupt halt, but with any real model, the best we can do is identify them probabilistically, hence the curve. Obviously Zone 1 is where we spend money to make money---a reasonable proposition. Zone 2, the white stripe, is all the people who are largely unaffected by our action—a mixture of the Dead Certs and Lost Causes. Here the line would be perfectly horizontal if we could identify these groups with certainty. In Zone 2 we spend money for no appreciable return. Zone 3, red, is the group where we have a negative effect—the Boomerangs if we're selling things, and the Sleeping Dogs in a retention context. This is the disaster zone, in which spend money to drive away business. There is no rational case for targeting here.

Moving to this slightly more nuanced view, where we deal in probabilities rather than known outcomes, we can recast anyone whose probability of purchasing is increased by our action as a Persuadable; those whose purchase probability is largely unaffected as an Unshakable, and those whose probability of purchase is reduced as a Boomerang.

The trouble with conventional so-called “response” models is that, despite their name, they don't model response at all: they model purchases pure and simple: they don't look at the untreated population in the model-building process. So it's not that they simply conflate the Dead Certs with the Persuadables: they will actually also tend to recommend targeting people whose likelihood of purchase is reasonably high if treated, but even higher if left alone. Quite simply, “response” models aren't: they model the wrong thing.

In future posts, I'll discuss how we might go about segmenting people for real, and how we might model the change in response probability (the uplift) to allow us to target on that basis.

Response