📖
Police Data Access Point Docs
pdap.ioGitHub
  • 👋Welcome
  • ⚡Activities
    • Label new Data Sources
      • Labeling events
    • Volunteer for Data Requests
    • Search for Data Sources
    • Publish data
    • Web scraping
    • FOIA requests
    • Advocate for open data
  • 🔬About
    • Search the PDAP database
    • Terms & definitions
      • What is a Data Source?
      • Terminology
    • Database details
      • Data Sources data dictionary
      • Agencies data dictionary
      • Requests data dictionary
      • Record Types taxonomy
      • Hidden properties
    • GitHub
    • Hugging Face
  • 📡API
    • Introduction/Getting Started
  • 🛠️Tools & Resources
    • Related projects
    • Resources for using data
    • Using LLMs like ChatGPT
  • 🔁Meta
    • Internal Tools (Retool)
    • Internal dev resources
      • GitHub issue template
      • GitHub pull request template
      • Product changes checklist
      • ☑️Production QA Checklist
      • Retool
    • Operations
      • Staff resources
        • Meeting Minutes
          • 2021-07-14
          • 2021-06-16
          • 2021-03-14
          • 2020 11-21 Tech Stack discussion
          • 2020-09-30 Leadership Cadence
          • 2020-10-14 Leadership Cadence
          • 2020-10-21 Leadership Cadence
          • 2020-10-28 Leadership Cadence
          • 2020-11-04 Leadership Cadence
          • 2020-11-12 Leadership Cadence
          • 2020-11-18 Leadership Cadence
          • 2020-11-25 Leadership Cadence Notes
          • 2020-12-02 Leadership Cadence Notes
          • 2020-12-09 Leadership Cadence Notes
          • 2020-12-12 Working Session
          • 2020-12-16 Leadership Cadence
          • 2020-12-30 Leadership Cadence Notes
          • 2021-01-06 Leadership Cadence Notes
          • 2021-01-13 Leadership Cadence Notes
          • 2021-01-20 Leadership Cadence Notes
          • 2021-01-27 Leadership Cadence Notes
          • 2021-02-03 Leadership Cadence Notes
          • 2021-02-10 Leadership Cadence Notes
          • 2021-02-17 Leadership Cadence Notes
          • 2021-02-24 Leadership Cadence Notes
          • 2021-03-03 Leadership Cadence Notes
          • 2021-03-10 Leadership Cadence Notes
          • 2021-03-16 Leadership Cadence Notes
          • 2021-03-27 database working session
          • 2021-03-31
          • 2020-12-1
          • 2021-01-23
          • 2021-04-10 Meeting notes
          • 2021-04-17 Meeting notes
          • 2021-04-21 Leadership Cadence
          • 2021-04-28 Leadership Cadence
          • 2021-05-05 Leadership Cadence
          • 2021-05-12 Leadership Cadence
          • 2021-05-19 Leadership Cadence
          • 2021-05-26 Leadership Cadence
          • 2021-06-02 Leadership Cadence
          • Decision log
        • Brand assets
      • Legal
        • Public records access laws & precedent
        • Legal Data Scraping
        • State Computer Crimes laws
      • Policy
        • Impartiality resolution
        • PDAP Access
        • PDAP Privacy Policy
        • Password Management
        • Personally Identifiable Information
    • Community calls
      • October 17, 2023
      • February 22, 2023
      • February 1, 2023
      • January 20, 2023
      • January 5, 2023
      • October 25, 2022
      • September 22, 2022
      • August 23, 2022
      • October 2, 2021
      • September 25, 2021
      • September 11, 2021
      • September 4, 2021
      • August 7, 2021
      • July 27 Dolt Bounty retro
      • July 17, 2021
      • July 10, 2021
      • June 26, 2021
      • June 19, 2021
      • June 12, 2021
      • June 5, 2021
      • May 1, 2021
      • April 24, 2021
    • Newsletter
    • Join our Discord
Powered by GitBook
On this page
  • Support for unprocessed data
  • Stabs x Pythonidaer working session
  • Random
  • Miner pool
  • Volunteer / Community Management

Was this helpful?

Edit on GitHub
  1. Meta
  2. Community calls

August 7, 2021

PreviousSeptember 4, 2021NextJuly 27 Dolt Bounty retro

Last updated 1 year ago

Was this helpful?

Richard got hadoop up and running

Dolt could be stored there

We should not run scrapers on the hadoop box because there's no authentication right now, they should be run on digitalocean if anywhere.

VPN access info is available for gsuite folks

Dev is in New York, Production is in San Francisco. They can't talk to each other at the moment without a VPN. Keeping our infrastructure in one place may be worth looking at in the future :shrug:

We could use Lambda, maybe not worth considering a change at the moment.

Stabs x Pythonidaer working session

Generated a scraper.py file, success!

We need to explain the path to Hadoop when it exists

Scrapers can contribute to documentation as they learn

Random

Zenhub is nice

Feedback welcome on the front page (pdap.io)

Main site is down, moving to simpler HTML/CSS for the two pages: Home and FAQ

Could be CMS driven / as accessible as possible

Miner pool

Data itself could be stored as NFT

Volunteer / Community Management

We don't have anyone to do this. We should better get to know our volunteers, and keep in touch with people in a meaningful way.

Right now the only place is Discord, how can we be more inclusive there / tie GitHub / DoltHub contributions to Discord in some way. Be clear about who's contributing!

How can we make it clear that there's always a seat at the table for new people?

Stabs to make "how many people made PRs in github and dolthub this month" show up in Discord somehow via web scraping?!

On the front page rewrite, Josh + team to write copy letting people know they're welcome

Lots of updates made to the readmes.

is a good metaphor to think about—our scrapers could run like that. People could use their compute power. Like reverse torrenting. Or something.

🔁
Josh to try Webflow → HTML quick prototype
Distributed mining pool
Support for unprocessed data
datasets