article

Why Counting Counts When Mitigating Fraud Risk

One, ah ah ah. Two, ah ah ah.

What does counting have to do with stopping online fraud and abuse? As it turns out—a lot.

Counting is at the heart of mitigating financial fraud, spam, scams, account takeover, and many other forms of online abuse. One could go as far as to say that systems for managing fraud risk at scale are, in large part, big counting machines.

In this article, we:

1. explain this fundamental connection between counting and fraud detection;
2. survey the many varieties of counters and their applications;
3. provide practical guidance for incorporating counting into your mitigation strategy.

Fraud Risk Experts Count

To understand how automated fraud prevention systems use counting, let's begin with an example of manual review by a fraud domain expert.

In his insightful article, "4 Steps to Identify Fraud in Shopify," Alexander Hall provides the following example of a suspicious order:

The customer, John, placed an order on 08/02/2020 with an IP address from New York, the billing information is Y-verified, but the order is intended to be shipped to Hong Kong. The CVV matched, but this is the 8th card used on this account, and each order has been flagged.

Among the risk factors Hall highlights, note that "the 8th card used on this account" describes a count. We are counting the number of unique credit cards linked to this Shopify account over its history.

Unpacking this fraud risk factor, we see that it:

• captures information about not just the current order but past orders too;
• spans the full known history, in this case, rather than considering only a recent time window;
• centers on the order's account, although we might be interested in counts centered on other attributes as well;
• counts the number of unique credit cards, though, as we'll see, other counters are useful too.

We'll break down these various dimensions of counters in later sections. For now, here are a few more examples to give you a sense of the diversity of counters we might create:

Common Fraud Risk Factors: Greedy, Lazy, and Impatient

In Hall's example, the large counter value, 8, raises a red flag. And indeed, more often than not, large counter values tend to be more suspicious than small ones. There are plenty of exceptions in which high counter values indicate, for example, tenure, loyalty, and high customer value. But it is so common to add risk for high counts that the industry has adopted terms like velocity checks and rate limiting to describe thresholds on high counter values.

Why do fraudsters tend to run up high counter values? I submit that it is because of three inherent characteristics of fraudsters that we can turn against them:

1. Greedy: When a good customer spends their own money, they purchase only what they need or want. A fraudster spends someone else's money, so they spend without limit.
2. Lazy: To remain hidden, fraudsters could place their orders from different accounts, devices, IP addresses, etc. But doing so takes a lot of work. By using the same order attributes, again and again, they run up high counts for those attributes.
3. Impatient: Once a fraudster has found a repeatable process that succeeds, they will exploit that hole as quickly as possible before it is discovered and closed.

We use financial fraudsters to illustrate the point. Still, these characteristics are true of scammers, spammers, phishers, and trolls, which is why counting is so broadly-applicable to stopping abuse.

Aggregate Signals

This article is Part 2 in a series on data enrichment, which we defined in Part 1 as:

Data enrichment transforms and combines raw data elements to create meaningful fraud signals.

where a fraud signal is an attribute that helps distinguish good from the bad. The risk parameters described in Part 1 are what we might call basic signals based only on the current order.

Counters are more advanced because they allow us to also incorporate data from past orders. To make sense of that large volume of data, counters aggregate values over their history to boil things down to a single value.

Counting Comes In All Shapes and Sizes

Now that we've covered what counters have in common, let's talk about the choices we have when defining counters to suit our various use cases. We will use Scowl, Sumatra's data enrichment language, to make the examples precise.

The Aggregator

First, we must choose which aggregating operation to use. The simplest is: Count, which increments by 1 for each new event: 1, 2, 3, ... and so on. A Sum increases by a given attribute, such as the order price. And a CountUnique aggregate increments each time a previously unseen value is encountered, like a new device or payment method. Here are a few examples:

-- Total number of orders
Count()
-- Total dollars spent
Sum(price)
-- Number of unique IP addresses used
CountUnique(ip)

While different fraud service providers support different classes of counters, most popular platforms today support at least these three essential types. Scowl supports several additional counter types, which we'll leave to a future discussion.

The Entity

The previous examples are impractical because they increment for every order, regardless who made the purchase. In practice, counters are tied to individual entities we want to risk. This entity will most commonly be an account identifier, which in Shopify would be the Customer ID. Updating our examples:

-- Total number of orders BY THIS ACCOUNT
Count(by customer_id)
-- Total dollars spent BY THIS ACCOUNT
Sum(price by customer_id)
-- Number of unique IP addresses used BY THIS ACCOUNT
CountUnique(ip by customer_id)

Some fraud platforms limit counters to be defined on the customer entity only. Fraudsters exploit this limitation rather easily by simply creating many different accounts to carry out their fraud.

Other platforms support a few different entities that can be selected from a pre-defined list, e.g., card number, IP address, or customer. By enforcing rate limits on several entities, these systems better exploit fraudsters' inherent laziness and are typically more effective.

Scowl was designed to support use cases across Trust & Safety, including users browsing anonymously. For that reason, the language supports arbitrary user-defined counters. Any attribute may be used as an entity:

-- Total number of orders BY THIS EMAIL ADDRESS
Count(by email)
-- Total dollars spent BY THIS IP AND BROWSER TYPE PAIR
Sum(price by ip, user_agent)
-- Number of unique IP addresses used BY THIS CREDIT CARD
CountUnique(ip by card_number)

The Window

So far, our examples have incorporated the entire event history as far back as the data permits. These lifetime or eternal aggregates, as they are sometimes called, certainly have their place in a fraud solution. They are particularly effective at identifying good customer behavior, which is as important as risky behavior when sorting good from bad.

Time-windowed aggregates consider only the related events that occurred in a fixed time period prior to the evaluated event. For example, we might care about how an account behaved in the last 5 minutes or 10 days. Time-windowed counters allow us to impose tight thresholds that catch greedy, impatient fraudsters while letting good customers through. Some examples:

-- Order by the email address IN THE LAST 3 HOURS
Count(by email last 3 hours)
-- Dollars spent by this ip IN THE LAST 2 DAYS
Sum(price by ip last 2 days)
-- Unique IP addresses used by this credit card IN THE LAST WEEK
CountUnique(ip by card_number last week)

As for time window support across platforms, their flexibility and power vary. Some platforms limit the history for "lifetime" counters to, say, a couple of months. Many offer a handful of choices, e.g. 1 hour, 1 day, and 1 week for windows. Fraudsters have become savvy about these commonly-used windows and will often time their purchases to exploit them. For example, they may make purchases every 25 hours to allow a daily counter to clear.

Tactics and Recipes for Mitigating Fraud Risks

As with all signals, counters may be presented as features to a machine learning model or used directly in rules— most commonly threshold rules (aka rate limits or velocities). As discussed previously, while human fraud experts are terrific at crafting the signals themselves, thresholds are best computed from statistics of the data.

There are several advanced techniques fraud fighters can employ with counters that are worthy of their own in-depth discussion.

Here are two of those example tactics to mitigate fraud risks:

• Stacking: You compute the same aggregate over several time windows with different thresholds to make reverse-engineering the logic more challenging.
• Trending: You divide the count over 10 minutes by the count over the past 10 hours to see if the trend has shifted recently.

For examples of counters as part of complete fraud logic, check out Scowl's Recipes page.

Fighting the Fraud Fight with Real-Time Machine Learning

By turning bad actors' greed, laziness, and impatience against them, counting plays a crucial role in an effective fraud mitigation strategy. Aggregate signals look beyond the current order to incorporate past behavior, which is a powerful tool for uncovering suspicious activity and other fraud risk factors. Counters can be defined with various aggregator types, entities, and time windows, with varying support across platforms.

At Sumatra, we aim to be nothing short of industry-leading in empowering fraud fighters to work with counters. In addition to SCOWL's expressiveness, the Sumatra platform provides a suite of capabilities that, to our knowledge, don't exist elsewhere in identifying fraud risk factors. This includes historical simulation to validate candidate aggregates and logic on demand and warm start to backfill new counters for fast and accurate deployment. If you want to get the most out of counters in your solution to mitigating fraud, we invite you to get in touch.

In this article, we have expanded the concept of data enrichment to include data from historical orders. In the next installment, we will expand this concept further to include behavior from other parts of your ecosystem, such as signups, logins, browsing, and more.

Until then, keep fighting the good fight.