The datasets database is a utility we're always maintaining. It contains known police datasets. Each dataset has a status, and can potentially be scraped.
Scrapers are bits of Python code designed to collect information from a Dataset. Scrapers are best understood and defined by the Dataset they are scraping.
Structure
Datasets are maintained and can be easily viewed in DoltHub. The same tables are in a Hadoop PostgreSQL mirror, which is a better back end for Scrapers.
Hierarchy
1.
Country
The United States has ~18,000 Agencies across all regions.
2.
Region
A state, county, municipality, or other region containing multiple police agencies.
3.
AgencyA specific police organization, typically containing many Datasets.
4.
DatasetA URL where one type of police data can be found.
5.
ScraperA bit of code which downloads everything it can find on a Dataset.