Searchβ¦
β¬
back to pdap.io
π
Welcome to PDAP!
β‘
Volunteer Activities
Process overview
Maintain Datasets
Write Scrapers
π€
Mission components
Key Principles
Data Intake
Data Storage
Data Access
π
Tools & Resources
Terms & Definitions
Internal tools
DoltHub
Splunk
Third-Party Resources
π£
Updates
Blog
Working sessions
Current Progress
π
Meta
About
Policy
Legal
β
Elsewhere
Discord
GitHub org
DoltHub org
Patreon
Powered By
GitBook
Data Intake
Running Scrapers and submitting the generated Extractions.
Current status
We're still in the iteration and case study phase. To submit data:
1.
Run an approved Scraper from the
Scrapers Repo
to get an Extraction.
2.
Share your Extraction in Discord. We'll work with you to write and publish a case study.
3.
We'll work with data consumers to understand how we're doing.
4.
When we have learned from case studies, we'll solidify the process.
Requirements
1.
We know when data was submitted.
2.
We know which scraper was used to generate the data.
3.
Nice-to-have: we know who submitted the data.
Since we have git version control on our scrapers, we can audit any piece of submitted data to understand how it was gathered.
Mission components - Previous
Key Principles
Next - Mission components
Data Storage
Last modified
4mo ago
Copy link
Edit on GitHub
Contents
Current status
Requirements