In 2006, Wake County, North Carolina uncovered a $5 million transportation fraud scheme. An auto parts company had been submitting thousands of fake invoices, carefully kept just under the approval threshold and in collusion with county employees. The techniques that caught them are tools any data professional can use to spot dirty data, catch anomalies, and truly know what is happening in a data set.
In this session, we will rebuild the investigation step by step using SQL Server and Python. Starting from a database you have never seen before, we will work through a variety of techniques in order to uncover the same kinds of patterns the real investigators found. You will walk away with Jupyter notebooks you can clone and run against your own data Monday morning.
Click here to access the slides for this presentation.
The slides are licensed under Creative Commons Attribution-ShareAlike.
Click here to access demo code for this presentation.
The source code is licensed under the terms offered by the GPL.
If you are not familiar with Jupyter notebooks, I have a guide on how to install Jupyter on Windows. This will allow you to try the notebooks out on your own.
Here are links for the individual data sets used in this talk. Please note that the versions I have archived are for specific points in time, so the data format may change, and the data itself will likely have changed. For the actual data sets I use, check out the demo code above.
Benford's Law is one of the most interesting numeral analysis findings, given its wide-ranging and unexpected appearances in data sets.
Although I did not use most of the techniques in this section, I want to provide links for people interested in learning more.