Fraud Detection Decision Lab
Two logistic regression models written from scratch in NumPy score 283,726 card transactions, 0.17% of them fraud. Move the threshold and set the cost of a review to see exactly what each choice catches, misses and costs.
283,726Card transactions scored
0.17%Share that are fraud
57 of 70Frauds caught on test
0.741AUC-PR vs 0.0017 baseline
01The problem
The problem
Only 0.17% of transactions are fraud. A model that flags nothing scores 99.8% accuracy and catches no fraud at all. The real question is: at a given review cost, how many frauds do we catch, and how many good customers do we bother?
02The approach
The approach
- Logistic regression written from scratch in NumPy: sigmoid, binary cross-entropy, gradients and optimiser.
- Model A uses class-weighted cross-entropy. Model B trains on SMOTE-resampled data.
- Thresholds are chosen on the validation set. The test set is scored once.
- Models are compared on AUC-PR and F2, which reward catching fraud, rather than accuracy.
03Results
Results
Held-out test set: 42,557 transactions, 70 of them fraud.
| Model | Frauds caught | False alarms | Precision | AUC-PR |
|---|---|---|---|---|
| A · weighted BCE | 55 of 70 | 16 | 77.5% | 0.724 |
| B · SMOTE | 57 of 70 | 28 | 67.1% | 0.741 |
The trade-off: Model B catches two more frauds but raises twelve more false alarms. Which is better depends on what a missed fraud costs against an analyst’s time, so the app makes cost an input.
04The Decision Lab
The Decision Lab
Set the cost to review one alert and the cost of a missed fraud, move the threshold, and watch catches, misses and total cost update. The app also compares the two models side by side and includes a transaction explorer.