Comment on page

May 1, 2021

Attendees
Josh
Mitch
Jeff
Eddie
Topics
Topic
Notes
Downloadable scrapers package
Proof of concept: https://github.com/EricTurner3/pdap-intake-ui/tree/overhaul​
Chain of custody
Where
When
Who
With what scraper code
What granularity do we want for audit history? Cellwise
We need auth—anything else is too easy to spoof
Data collision
Treat each timestamp as its own piece of data
Datasets
Maintain source info / keep it up to date
Auth
Keybase (infra)
Django built-in
GitHub for scraping (pub/privkey)
Medium-term SSO
Incentives
Dolt bounties
Paid scrapers—pay for a quick project (fiverr, freelancer)
Someone paid has ~guaranteed expertise. Collect feedback from professional consumers 
$
We're covering a few licenses, but need more donations. Considering Patreon
Support
​EFF, ACLU, NFOIC​
1.volunteer time
2.volunteer money
3.introduce us
4.use the data
Scraper utilities
​https://pdap.ericturner.it/datamap/​
​https://pdap.ericturner.it/datamap/schema​
Data centers
We have NY and SF data centers in DigitalOcean, but they don't talk to each other
Security
​https://pdap.atlassian.net/browse/PDAP-162​
1.perimeter
2.secrets manager
Who can make data PR approvals?
Front end
​https://github.com/Police-Data-Accessibility-Project/gatsby-pdap-frontpage/pull/17​
​

June 5, 2021

April 24, 2021

Last modified 2yr ago

Topic	Notes
Downloadable scrapers package	Proof of concept: https://github.com/EricTurner3/pdap-intake-ui/tree/overhaul
Chain of custody	Where When Who With what scraper code What granularity do we want for audit history? Cellwise We need auth—anything else is too easy to spoof
Data collision	Treat each timestamp as its own piece of data
Datasets	Maintain source info / keep it up to date
Auth	Keybase (infra) Django built-in GitHub for scraping (pub/privkey) Medium-term SSO
Incentives	Dolt bounties Paid scrapers—pay for a quick project (fiverr, freelancer) Someone paid has ~guaranteed expertise. Collect feedback from professional consumers
$	We're covering a few licenses, but need more donations. Considering Patreon
Support	EFF, ACLU, NFOIC 1. volunteer time 2. volunteer money 3. introduce us 4. use the data
Scraper utilities	https://pdap.ericturner.it/datamap/ https://pdap.ericturner.it/datamap/schema
Data centers	We have NY and SF data centers in DigitalOcean, but they don't talk to each other
Security	https://pdap.atlassian.net/browse/PDAP-162 1. perimeter 2. secrets manager Who can make data PR approvals?
Front end	https://github.com/Police-Data-Accessibility-Project/gatsby-pdap-frontpage/pull/17