Most fraud teams discover coverage gaps in their stack the hard way: a chargeback rate spike, an alert from their acquirer, or a support queue flood of "I didn't make this purchase" tickets. A structured audit of your fraud controls -- done proactively, before an incident -- lets you find the gaps before attackers do. This is the process we use when onboarding new customers and the one we recommend teams run internally at least once per quarter.
Step 1: Map your transaction flows and control coverage
Start by documenting every transaction entry point your platform has: checkout, guest checkout, subscription renewals, stored-value top-ups, P2P transfers, refund initiation, withdrawal requests. For each flow, document what fraud controls are applied: which rules fire, whether there is a scoring layer, what the manual review trigger conditions are.
This mapping exercise almost always reveals asymmetries. The main checkout flow has 12 rules and a scoring layer; the subscription renewal flow has 2 rules and no scoring. The P2P transfer flow has velocity checks; the refund initiation flow has none. Attackers are good at finding the flows that have less coverage -- mapping your coverage before they do is the goal.
Common gaps found in this mapping: refund abuse paths (requesting refunds on orders that were delivered), add-payment-method flows (which often have less scrutiny than transaction flows), and guest checkout (which lacks account history signals and is frequently targeted for that reason).
Step 2: Pull your chargeback data and analyze reason codes
Chargeback data is the ground truth for what fraud is reaching your platform. Pull the last 90 days of chargebacks and group them by reason code. Reason code 4837 (no cardholder authorization) and reason code 4863 (cardholder does not recognize) are the primary indicators of card-not-present fraud. Reason code 10.4 (Visa CNP fraud) and 37 (Mastercard unrecognized transaction) are the equivalent codes on those networks.
For each cluster of chargebacks, ask: what did these transactions have in common? Same merchant category, same time of day, same BIN range, same device fingerprint cluster? The pattern in your chargebacks tells you what your current controls are missing. If you have 40 chargebacks all from the same device fingerprint cluster and no rules firing on device velocity, you have a clear gap to close.
Also track chargeback-to-transaction rate by merchant category, by new vs. returning customers, and by transaction amount band. These breakdowns often reveal that a specific segment (new users, digital goods, high-value) has a materially higher chargeback rate than your platform average, pointing to where your controls need to be stronger.
Step 3: Audit your rule library for dead rules and threshold decay
Rules accumulate over time. Every fraud incident generates pressure to add a new rule; few rules are ever removed. After a year or two, a typical rules library has rules that are genuinely doing work, rules that fire rarely because the pattern they were written for has changed, and rules that fire frequently but produce mostly false positives because they were written to catch a specific incident and the threshold was never recalibrated.
The audit for this is straightforward: pull the fire rate and decline rate for every rule in the last 90 days. Rules that fire on fewer than 10 transactions in 90 days are candidates for review -- either the pattern they were written for is no longer occurring, or the threshold is too high to catch current variants. Rules with a fire rate above 1% of your total transaction volume and a chargeback rate below 0.1% on the transactions they decline are probably generating false positives worth investigating.
Threshold decay is the other thing to look for. A rule that was calibrated to catch fraud at $100+ may need recalibration if the average fraud amount in your chargeback data has shifted to $200+. Thresholds set at launch should be reviewed against current data at least annually.
Step 4: Test your velocity check coverage against known attack patterns
Take the attack pattern from your last significant fraud incident and simulate what your velocity checks would have done against it. If the incident was a card testing attack that spread 500 attempts across 50 IP addresses over 5 days, your per-IP hourly velocity check would not have fired. Would your per-device velocity check have fired? Your per-card decline-rate check?
If you have not had a significant incident, use the card testing anatomy described in our earlier post as a reference scenario: 612 transactions across 12 sellers, distributed over 5 days, with 8 device fingerprints and variable IP addresses. Walk through each control layer and ask: which of these signals would have triggered a rule? Which would have required a scoring layer?
This simulation exercise reveals which attack patterns are in your coverage blindspot. The goal is not to build rules for every simulated scenario -- that leads to the rule library bloat problem. The goal is to identify the structural gaps: attack types that no combination of your current controls would have caught, which points to where you need scoring-layer capabilities.
Step 5: Measure your false positive rate
The second half of the audit is measuring the cost of your controls, not just their coverage. The false positive audit method is described in our earlier post on the false positive problem -- briefly, it involves shadow-approving a sample of would-be-declined transactions and tracking their chargeback outcomes. If your false positive rate is above 50% on declined transactions, your controls are costing you as much in declined legitimate revenue as they are protecting you from fraud.
Also measure review queue backlog if you have one. A manual review queue that has a 48-hour average time-to-decision is a real problem: transactions held for 48 hours generate support tickets, abandoned purchases, and negative reviews. If your review queue is the bottleneck, the priority is reducing the volume entering it by improving the accuracy of automated scoring so fewer transactions need manual adjudication.
What to do with the audit findings
An audit produces a prioritized list of gaps: uncovered transaction flows, pattern blind spots, threshold decay, false positive sources. The prioritization question is: which gap is most likely to be exploited in the next 90 days, and what is the cost if it is?
Uncovered high-value transaction flows with no fraud history are not necessarily low priority -- they may have no fraud history because attackers have not targeted them yet. A flow with no controls is a target waiting to be found. Coverage gaps should be addressed before chargeback pressure makes them obvious.
If the audit reveals that your stack has limited behavioral and graph-level coverage -- the categories that rules cannot reach -- that is a signal that a scoring layer would provide the most marginal coverage improvement. We are glad to walk through the findings with any risk team and discuss where our scoring layer would fit into the gap map. Contact us at [email protected] if that conversation would be useful.