# Welcome This is a guide for using the Police Data Access Point and associated tools, and contributing data or code. ## What we do We're cataloguing sources of data in a database at [**https://pdap.io**](https://pdap.io) so people can find information about local police systems. ## Get Involved {% hint style="success" %} People of all skill levels can help with our mission! {% endhint %} The quickest way to contribute is by labeling (annotating) new Data Sources. Head to to get started. To support us financially, [you can donate here](https://pdap.io/contribute.html). We're a non-profit, so your donations may be tax-deductible. ## Codebase To contribute code, [start with a Good First Issue](https://github.com/orgs/Police-Data-Accessibility-Project/projects/25/views/1). [Our github organization](https://github.com/Police-Data-Accessibility-Project) contains our code repositories. We have a handful of apps working in conjunction to identify, collect, document, archive, and share sources of data. ### Primary repositories * The [Data Sources App repo](https://github.com/Police-Data-Accessibility-Project/data-sources-app) is the back end for our Data Sources API. * The [Data Source Identification repo](https://github.com/Police-Data-Accessibility-Project/data-source-identification) houses our code for collecting sources for the database. * The [pdap.io repo](https://github.com/Police-Data-Accessibility-Project/pdap.io/) is our primary front end for working with the API. # Label new Data Sources Growing our Data Sources Database ## Why we're here **Labeling potential Data Sources helps us make a more complete database!** Our source collectors crawl the internet using a variety of strategies, and volunteers catalogue the sources they find. One day, we may also be able to train machine learning models to identify useful data. For more background and a technical perspective, check out the [Data Source Identification repo](https://github.com/Police-Data-Accessibility-Project/data-source-identification). ### How to start Reach out to or [use this volunteer form](https://airtable.com/appcYa6x4nS7W8IR3/shrk9c5sBsBr3cdJJ) with "data labeling" selected. We'll give you instructions for accessing the labeling interface! #### Content warning This content contains annotations related to the criminal legal system, which may include discussions of topics such as crime, legal proceedings, incarceration, and violence. Additionally, part of our work is labeling websites which may appear to be related but do not pertain to the criminal legal system, and might contain anything at all. *It's ok to take a break, or do something else.* # Labeling events A guide for labeling events, or "annotation parties" ## Goals of each event * Train a small group on data labeling * Add as many data sources to the database as we can * Set people up to label on their own time * Refine our data labeling tools ## Structure of activities ### In advance 1. Participants RSVP to the event, held on Discord or in-person. 2. Participants create a PDAP account and are sent invites to the Retool organization. 3. Organizers give participants the appropriate permissions in the back-end. 4. If anyone has location or subject matter interests, organizers give them instructions for following the appropriate search. ### At the event 1. Organizers welcome participants and kick off popcorn-style introductions. 2. Organizers screen-share the process of: 1. selecting a batch 2. labeling for relevancy 3. labeling for record type 4. labeling for agency 3. Organizers open the floor for questions. 4. Participants are encouraged to begin labeling, and are free to ask questions about specific labels or parts of the process as they come up. 5. Labeling continues for an hour; participants are encouraged to continue after the event at their own pace. 6. Organizers use the Final Review screen to approve or adjust labels in real-time, giving feedback as necessary. # Work on Data Request projects ## Opening a Data Request See our [guide for opening requests](/data-requests). ## Finding open Data Requests * You find [all open requests here](https://github.com/orgs/Police-Data-Accessibility-Project/projects/26). * If you want to contribute, leave a comment on any request's GitHub Issue. ### Follow a search Click "Follow" (you will be prompted to sign in). You will get periodic updates with new requests in the area. ### GitHub Communication about open Data Requests happens on GitHub Issues. We recommend you make a free account! You don't need to know anything about code or open source to leave comments in GitHub. ## Sign up as a volunteer If you want to help but aren't sure where to start, you can sign up as a volunteer. PDAP staff will reach out when requests for data come in! Head to the [volunteer form](https://airtable.com/appcYa6x4nS7W8IR3/shrk9c5sBsBr3cdJJ). # Find new Data Sources Our database is a good start, but there's more data out there. Help us find it! ## Quick start If you found a good source of data, you can [submit it to our database](https://pdap.io/data-source/create)! ## Finding data in your area Checking to see what data is currently published in your area is a great way to move our project forward, one city at a time. 1. [Search for your chosen agency or area](https://pdap.io) in our database. 2. Search the agency homepage and related sites for data about the agency. When you find one, make a new [Submit Data Sources form](https://airtable.com/app473MWXVJVaD7Es/shrJafakrcmTxHU2i) submission. 3. Fill in the form for each valuable data source. This should take 1–5 minutes, and you will get faster as you go! ## Submitting bulk data If you already have several of Data Sources in a spreadsheet—maybe from a previous project, or because you collected them in bulk—you can submit them to us in any format. 1. Head to the existing [Data Sources table](https://airtable.com/shrbnadIQyefJhI1D) and download a CSV to get the most recent format. This will help us import and approve your Data Sources more quickly. 2. Submit spreadsheets to the "[Submit bulk Data Sources or Agencies](https://airtable.com/shrbnadIQyefJhI1D)" form. ## Inaccuracies & missing options If you discover a mistake or missing agency, you can email and we'll correct the data. # Publish data ## If your data is already online If you have a link to published data, that's great! You can [**submit it to our database**](https://airtable.com/app473MWXVJVaD7Es/shrJafakrcmTxHU2i) where it will show up in searches. We'll trigger automatic archives, and people who subscribe to data in the area will be notified. ## If you need to publish data first If you have public records or data you want to share more broadly, there are a few ways to go! Make sure to [**share your published data when you're done**](/activities/submit-data-sources). ### Options for sharing data {% hint style="warning" %} If this is overwhelming, don't worry—every case is different, and we can provide advice. Email for a free consultation. {% endhint %} * [GitHub](https://github.com/) is a free platform intended for storing open-source code. It can be a great choice for sharing a small dataset, especially if that comes with a Python notebook of analysis or a web scraper which generated the data. [Here is a demo](https://github.com/Police-Data-Accessibility-Project/github-actions-demo) of an automated web scraper which collects Oakland's Calls for Service data, for free. * [JKAN](https://jkan.io/) (for smaller agencies) and [CKAN](https://ckan.org/) (for larger ones) are two open-source data portals which need to be set up by someone with some technical skills. * DocumentCloud can be used for publishing PDFs, and has additional tools for recognizing text (OCR) and annotation. You will need to [request verification](https://airtable.com/shrZrgdmuOwW0ZLPM) to publish documents. * Your town or region may have an open data portal ([here's an example from Pittsburgh area](http://www.wprdc.org/)) which may be interested in publishing the data you have. * A service like [Dropbox](https://dropbox.com/) can be used to share folders of documents in bulk, with fewer tools for keeping them organized or labeled. ## Making published data accessible Sometimes, published data is not really *accessible*—maybe the data is rendered as a visualization on a dashboard, or you want to calculate a sum or average value from a 250-page HTML table. That's where [web scraping](/activities/web-scraping) comes in. # Web scraping ## Our approach to web scraping ### Which data are we scraping? {% hint style="info" %} [https://github.com/Police-Data-Accessibility-Project/scrapers](https://github.com/Police-Data-Accessibility-Project/scrapers/) {% endhint %} Public records about the police system, which can include sources from police, courts, and jails. ### Contributing Philosophy Scraping can turn cumbersome records into useful data. When someone wants to use records but they're in a difficult format, scraping is often the answer. Typically, it's only worth writing a scraper if you have a use case for the data already, and you can't easily download what you need. **What question are you trying to answer?** ### Where the community fits in Our target users are the thousands of people already using police data. We can support their work by connecting the community of PDAP volunteer scrapers with real, impactful projects. If you don't have your own ideas about what to scrape, head to [our volunteer page](https://pdap.io/volunteer) to see if any open requests catch your interests. You can also find local groups working on the criminal legal system. ## Immediate goals After hundreds of hours of user research, we have determined that these are how we will add value in the police data landscape. * **Track independently scraped data** in our database. Prevent duplication of effort by showing people what's already out there. To submit data you've scraped, [start here](/activities/submit-data-sources). * **Connect people** with web scraping skills to community members trying to make better use of police data without technical expertise. [Volunteer to respond to requests here](https://airtable.com/shrS4PAZTYVT1zSq8). * **Build open-source tools** in the [Scrapers repo](https://github.com/Police-Data-Accessibility-Project/scrapers) to make running a scraper on-demand easier for people who don't know what "CLI" means. * **Scrape data sources** and agency metadata via our [Data Source Identification pipeline](https://github.com/Police-Data-Accessibility-Project/data-source-identification). Especially important are Data Sources with a record type of "List of Data Sources." ## How to contribute We're still in the iteration and case study phase. If you want to learn something about the police, you can write a scraper to parse, normalize, or get deeper information from our Data Sources. If you don't have scraping skills, you can use the [#data-exchange channel in Discord](https://discord.com/channels/828274060034965575/1006564024894378106) to find someone who may be able to help. 1. Run a Scraper you wrote, or one from the [Scrapers Repo](https://github.com/Police-Data-Accessibility-Project/scrapers), to get an extraction. 2. Share your extraction and what you learned in Discord. 3. We'll all learn about the criminal legal system from the experience, and brainstorm ways our tools could better facilitate your work. 4. Repeat! ## Not immediate priorities #### Aggregation and hosting of scraped data * Data is most often useful in its own context, and scraped data is usually small enough to fit on free-tier hosting. After you publish a dataset, we can list it in our database! * It's not an immediate priority to make a big database to store scraped data in a normalized format. Comparing and combining data is its own research project. It's almost everyone's first thought when they hear about our project (ours too). Our research tells us access, organization, sharing, technical skills, and communication are the bottleneck for people using the data. * Aggregation is incredibly complex, and involves more than just mapping properties. So much context is needed before data from two departments can be compared. * Publishing and vouching for extracted data, and documenting its provenance so it can be audited, is a big project. We only want to undertake this work for data we *know will be useful.* #### Automated scraper farms * It's not an immediate priority to automate the running of all the scrapers in our shared repo. The main reason: *this is not what our users are asking us for.* We plan to archive the sources, and facilitate sharing of scraper code. If we have a stable archive, scraping can be done on-demand. #### Scrape every data source! * Scraping is hard work, and there are hundreds of thousands of potential data sources out there. For many applications, data doesn't even need to be processed to be useful—it just needs to be findable. We don't need to scrape things unless it's clearly adding value. # FOIA requests ## What are FOIA requests? [The Freedom of Information Act](https://en.wikipedia.org/wiki/Freedom_of_Information_Act_\(United_States\)) (FOIA, pronounced "FOYA") has created the springboard for residents to request data from any level of government. Not all documentation can be disclosed to the public, but FOIA Requests are the place to start. ## Can I help by making FOIA requests? Sure! PDAP receives requests for help collecting data from agencies across the United States. Sign up to [volunteer for data requests](/activities/volunteer-for-data-requests) and we'll connect you with an impactful FOIA project. If you have your own idea, you can [make a data request](https://airtable.com/app473MWXVJVaD7Es/shrbFfWk6fjzGnNsk) to see if others want to volunteer! If you have successfully made a FOIA request through a service like MuckRock (or put the results online in some other way), the URL of the completed request is a valuable data source. [Submit it here](/activities/submit-data-sources#submit-data-youve-collected), and the community will thank you! ## Our advice for a successful request ### Use [MuckRock](https://www.muckrock.com/) to make requests {% hint style="info" %} MuckRock [has a great FAQ](https://www.muckrock.com/faq/) specific to their platform. {% endhint %} This is not a necessity, but MuckRock automatically follows up, finds the right agency, helps you navigate refusals, etc. ### Do some groundwork * You can use [MuckRock's guide](https://www.muckrock.com/place/united-states-of-america/) to see a high-level interpretation of your state's open records law. * Check whether there's a provision that records are provided in the same format they're kept. * Call in advance. Tell the human on the other end what you plan to request and ask them if they have guidance for making the request as easy as possible for them to grant. * Find out **what they call the record** you're looking for. Requesting the name of a specific, existing table will make things easier. * If you develop a relationship with the agency, you may be able to informally request and access the records without a FOIA process. * Check for "required" forms. Technically, you shouldn't need anything but the request—but complying with these "requirements" is a good way to make sure your request is granted. ### Making the request * **Specify a time range**, if that will work for your purposes. Instead of phrases like "any policies..." or "all arrest records..." ask for "current policies" or "the most recent 5 years of arrest records". * Add a line requesting that **if any part of the request is denied, the agency should proceed with the rest of the request** and cite the provisions in the state that allow them to deny the request. * If you don't know what the records are called, describe them generally—if you guess something too specific, the agency can deny your request with no obligation to alert you of similar records by a different name. * If you're looking for records about one person or incident, consider requesting a broader set of data to keep records in context and make them as useful as possible for other researchers. You may also want to compare your incident report with others. ### After the request Once you make the request, it's important to watch for communication from the agency. * If you request something like "all arrest records from the past 5 years", the agency may deny your request because they are most often asked to share single arrest records. If you can get through to them, a polite phone call can help. Explain that you'd like to minimize the work on their end, and ask why an export or listing in the format of their database won't work. * Check your email periodically for updates. MuckRock handles some of the notifications and responses for you, but if an agency rejects your request, it's up to you to fix and address it. * If the request is completed, [submit it to our database](https://airtable.com/app473MWXVJVaD7Es/shrJafakrcmTxHU2i)! This will allow more people to find and use your hard-won records. # Advocate for open data ## How to advocate for open data in your area {% hint style="info" %} You can positively impact your local systems by becoming an expert on your local data ecosystem, and pushing for more transparency. {% endhint %} ### 1. Identify transparency allies and people already using data about police systems. This could be city council members, journalists, activists, public defenders, researchers, municipal employees, or residents with a question to answer. You can also [join our Discord](https://discord.gg/wMqex8nKZJ) to find other people doing similar work. ### 2. Find out what kind of data would be topical/useful. A community can have different priorities at any given time—is everyone talking about traffic stop policy? Would better access to court case data answer several questions at once? ### 3. Find out where local data could be improved. Is anything missing, like properties that would make the data more useful? Are there records which should be published, but aren't? You can use our database to help keep track, [make a search](/features/using-data), and [submit data sources](/activities/submit-data-sources). ### 4. Uncover reasons why data is not currently published Publishing data requires time, money, and expertise. Understanding opposition to open data will help sharpen your strategy, and help your local government to be more transparent. Is someone already responsible for open data where you live? Their perspective is invaluable. ### 5. Challenge barriers to open data This could involve making a [FOIA request](/activities/foia), [writing a web scraper](/activities/web-scraping), or including [language about open data](https://phila.legistar.com/LegislationDetail.aspx?ID=5007829\&GUID=33E8CA43-B311-4219-A5A5-BBE600D14D55\&Options=ID%7CText%7C) in legislation. * Do [any other agencies](https://data-sources.pdap.io) publish useful data? If they can do it in `neighboring town`, why not here? * Does your town already have an [open data ordinance](https://apps.pittsburghpa.gov/cis/ProposedPittsburghOpenDataOrdinance.pdf) which requires publishing this data? # Terms & definitions # What is a Data Source? A Data Source is a web page, file, database, or filing cabinet somewhere which contains records about a criminal legal system agency (law enforcement, courts, corrections). Often, they are published by the agency itself. Many more records are hidden behind records request processes. Data Sources can be records describing an agency's activities, like **traffic stops** or **use of force policies**. They describe agencies in the criminal legal system like the `Pittsburgh Bureau of Police` or `Allegheny County Jail`. ### What kinds of Data Sources are best? {% hint style="info" %} Finding all the Data Sources for your hometown is a human-sized project that can make a real impact. [Start here](/activities/data-source-annotation). {% endhint %} If it's about a police, court, or jail system, we are probably interested in keeping it in our database. A guide to our [relevant record types can be found here](/about/data-dictionaries/record-types-taxonomy). This includes [FOIA'd documents](/activities/foia), web URLs, and independently collected or published records. # Terminology | Term | Our definition | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Agency | A police department or organization, like "Aurora Police Department". Agencies often have parent-child relationships to one another. | | API | An Application Programming Interface, which allows computer programs to communicate with each other. [More on Wikipedia](https://en.wikipedia.org/wiki/API) | | criminal legal system | Law enforcement, courts, and corrections. Our focus is on the United States. | | data accessibility |

This is a scale which we're still defining.

  1. The records should exist somewhere, but we need to locate them.
  2. We know where records can be accessed.
  3. There is historic data available in a stable archive.
| | data custody / provenance |

Who collected and published the data?

agency\_described (which agency is the data about?)

originating\_entity (who generated the records?)

supplying\_entity (who is publishing the records?)

Sometimes these are all the same entity; sometimes they are all different.

| | Data Source | A URL pointing to a place on a police website where public records may be scraped, like "police-agency.com/arrest-reports". [Read more here](/about/terms-and-definitions/what-is-a-data-source). | | Data Source archive | A raw, unprocessed HTML archive of a Data Source at a specific time. | | metadata | Packaged with data (like a Data Source or scraper extraction), metadata is information about when and how the data was collected. | | public records | Some information is required by federal, state, or local law to be public. Governments keep several types of public records, and make them publicly available to different degrees. | | scraper / data scraper |

A bit of code responsible for collecting an Extraction from a Data Source or Archive. Check out the GitHub repo. For more about our philosophy, start here.

Colloquially, "scraper" may refer to a person writing a Scraper.

| | scraper extraction | The result of running a Scraper is an "extraction", usually intended to further parse or process an HTML page or PDF into more usable data. | # Database details # Data Sources data dictionary {% hint style="info" %} To see which options are available for select fields, consult the [submission form](https://airtable.com/shrJafakrcmTxHU2i). {% endhint %} ### Required properties for submission `submitted_name`, `submitter_contact_info`, `record_type`, `agency_supplied` (+ other "[provenance](#provenance)" properties, if "no") ## What is it? | Property | Type | Description | | --------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | name | string | Uses `submitted_name` if present or concatenates `record_type` + `" for "` + `agency_described`; can get weird when one or both are not present | | submitted\_name | string | Required for individual Data Source submissions for clarity. | | record\_type | string | What kind of data is accessible from this source? For more info, see the [Record Types taxonomy](/about/data-dictionaries/record-types-taxonomy). | | tags | array | Are there any keyword descriptors which might help people find this in a search? Try to limit tags to information which can't be contained in other properties. | | description | string (textarea) | Information to give clarity and confidence about what this source is, how it was processed, and whether the person reading the description might want to use it. Especially important if the source is difficult to preview or categorize. | ## Agency | Property | Type | Description | | ------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | agency\_described | array (foreignkey based on an agency's ID within Airtable) | To which criminal legal system agency or agencies does this Data Source refer? | | agency\_aggregation | array | If present, the Data Source describes multiple agencies. Can be an item like `local` or `county`. | | state | string | 2-character ISO code, related to the associated Agency object, if present. | | county | string | Related to the associated Agency object, if present. | | municipality | string | Related to the associated Agency object, if present. | | agency\_type | string | Related to the associated Agency object, if present. | | jurisdiction\_type | array | Related to the associated Agency object, if present. What is the highest level of jurisdiction for the agency? Can be an item like `local` or `county`. | ## Provenance *Where did it come from?* | Property | Type | Description | | ------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | agency\_supplied | boolean | Is the relevant Agency also the entity supplying the data? This may be "no" if the Agency or local government contracted with a third party to publish this data, or if a third party was the original record-keeper. | | supplying\_entity | string | If the Agency didn't publish this, who did? | | agency\_originated | boolean |

Is the relevant Agency also the original record-keeper? This is usually "yes", unless a third party collected data about a police Agency.

| | originating\_entity | string | If the Agency was not the original record-keeper, who was? | ## Access & format | Property | Type | Description | | ------------------ | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | source\_url | string | A URL where these records can be found or are referenced. | | readme\_url | string | A URL where supplementary information about the source is published. | | access\_type | array | Array items can have values such as `Web page`or `API` | | record\_format | array | What format(s) are the records in natively? Array items can have values such as `CSV`, `JSON`, `XML`, `RDF`, `RSS`, `HTML table` and others | | detail\_level | array | Is this an individual record, an aggregated set of records, or a summary without underlying data? | | size | string | The file size on disk of all the data at this source, if downloaded. | | data\_portal\_type | string | Some data is published via a standard third-party portal, typically named somewhere on the page. | | access\_notes | string | Is anything special required to access the data? | | last\_cached | date | When was this last archived by our [automated archives app](https://github.com/Police-Data-Accessibility-Project/automatic-archives)? Formatted as YYYY-DD-MM. | ## Coverage & retention | Property | Type | Description | | ------------------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | coverage\_start | date | The earliest date covered by this source, if known, in the format YYYY-DD-MM. | | coverage\_end | date | The date at which updates stop, in the format YYYY-DD-MM. | | source\_last\_updated | date | The date this source was last updated, in the format YYYY-DD-MM. | | update\_frequency | array | How often is this data source updated? | | update\_method | array | Are records replaced (`Overwrite`) or added (`Insert`)? | | retention\_schedule | array | How long are records kept? Are there published guidelines regarding how long important information must remain accessible for future use? | | number\_of\_records\_available | integer | How many similar pieces of information are available at this source? | ## Meta & utility | Property | Type | Description | Default value | | ------------------------------ | -------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------------- | | scraper\_url | string | The url of any web scraping efforts associated with this Data Source. | | | url\_status | array | The status of the `source_url`, including options like `ok` , `none found` , `broken` | "ok" | | approval\_status | array | Set manually by the PDAP team; statuses include: `approved` `rejected` `needs identification` | null | | data\_source\_created | datetime | The date this source was first created in our database. | Date of data source submission, in the format YYYY-DD-MM. | | agency\_described\_linked\_uid | string | The Airtable-generated UID of an associated Agency | | | airtable\_uid | string | The Airtable-generated UID of this particular data source | | # Agencies data dictionary {% hint style="info" %} To see which options are available for select fields, consult the [submission form](https://airtable.com/shrzxLdSsYmBvIWMH). {% endhint %} ### Required properties for submission `submitter_contact`, `submitted_name`, `homepage_url`, `jurisdiction_type`, `agency_type`, + geographic info dependent on the `jurisdiction_type`. ## Identification & web presence | Property | Type | Description | | ------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | name | string | If a `state` is present, concatenates `submitted_name` + `" - "` + `state_iso` to help differentiate agencies with similar names in some Airtable menus. Otherwise, uses `submitted_name`. | | submitted\_name | string | What does the agency call itself? | | agency\_type | string | Options include `police`, `court`, `jail`, and `aggregated`, which usually represents coverage of multiple types of agency. | | contracting\_agency | link to other records from `agencies` table | If the jurisdiction or agency contracts its police services to another agency, it can be selected here. | | multi\_agency | boolean | True for utility agencies, used to geolocate data sources which cover multiple agencies. For example, a source about the "Allegheny County Aggregated" multi-agencies contains records about all agencies in Allegheny County. | | defunct\_year | integer | If present, denotes an agency which has defunct but may still have relevant records. | | homepage\_url | string | A URL for the homepage of this agency. | | no\_web\_presence | boolean | True when an agency does not have a dedicated website. | ## Geographic | Property | Type | Description | | ------------------ | --------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | jurisdiction\_type | string | What is the highest level of jurisdiction for the agency? Can be an option like `local` or `county`. | | state\_iso | string | 2-character ISO code. Does not apply to federal agencies. | | county\_fips | string (not an integer because of leading 0s) | Unique 5-digit code to identify counties. Does not apply to state or federal agencies. | | county\_name | string | Official County name. Does not apply to state or federal agencies. | | municipality | string | Official municipality name. Does not apply to county, state, or federal agencies. | | zip\_code | string (not an integer because of leading 0s) | Postal code for the agency's headquarters. | | lat | float | Geographic latitude for the agency's headquarters. | | lng | float | Geographic longitude for the agency's headquarters. | ## Meta & utility | Property | Type | Description | | ------------- | ------ | --------------------------------------------------------- | | airtable\_uid | string | The Airtable-generated UID of this particular data source | # Requests data dictionary {% hint style="info" %} To see which options are available for select fields, consult the [submission form](https://airtable.com/shrzxLdSsYmBvIWMH). {% endhint %} ### Required properties for submission `submitter_contact_info`, `request_notes`, `location_described` ## Form submission | Property | Type | Description | | ------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | coverage\_range | string | A range of dates or years covered by this request | | data\_requirements | string | Details the data must have, like "case numbers" or "incident location". | | location\_described | string | Is your research focused on a specific department? Are you comparing two counties or cities? | | submission\_notes | string (textarea) | What are you trying to learn? Are you trying to answer a specific question, or complete a specific project? Is there anything you've already tried? | | submitter\_email | string | How should we contact you to respond to this request? We won't share this with volunteers without your permission. We will never display it publicly or with volunteers without your permission. | | target\_date | string | When would you like to see this request filled? Are you working on a deadline? | ## Management & display | Property | Type | Description | | --------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------- | | archive\_reason | string | Why was this archived? | | data\_sources | link to `data sources` | Airtable link to relevant sources by airtable ID | | date\_created | datetime | When this was created | | date\_status\_last\_changed | datetime | When the value of `request_status` last changed | | github\_issue\_url | string | Link to the GitHub issue associated with this request. Added manually in Airtable, by integration in v2. | | id | autonumber | Used for display in Airtable. | | internal\_notes | string | Internal notes by PDAP staff about the request. | | pdap\_response | string | Public notes by PDAP about the request. | | record\_types\_required | array | Multi-select from our [taxonomy](/about/data-dictionaries/record-types-taxonomy). | | related\_requests | link to other `requests` | Manually linked. Will be deprecated in favor of GitHub. | | request\_status | array | Source of truth: GitHub. Not updated in Airtable. | | sources\_airtable\_uid | array | The UID(s) of Data Sources associated with the Request | | withdrawal\_reason | array | Describe why issues were withdrawn, for metrics tracking. | | work\_required | array | Types of work that might be needed to respond to the request, such as "data location" or "web scraping" | # Record Types taxonomy Examples and descriptions of the Record Types we're collecting. {% hint style="info" %} We use the `tags` property to add color to a Data Source if there's not a specific enough Record Type. {% endhint %} ## Police & public interactions | record\_type | description | example | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | Accident Reports | Records of vehicle accidents, sometimes published so that people involved in an accident can look up the police report. | | | Arrest Records | Records of each arrest made in the agency's jurisdiction. | | | Calls for Service | Records of officers initiating activity or responding to requests for police response. Often called "Dispatch Logs" or "Incident Reports" when published. | | | Car GPS | Records of police car location. Not generally posted online. | none found | | Citations | Records of low-level criminal offenses where a police officer issued a citation instead of an arrest. | | | Dispatch Logs | Records of calls or orders made by police dispatchers. | | | Dispatch Recordings | Audio feeds and/or archives of municipal dispatch channels. | | | Field Contacts | Reports of contact between police and civilians. May include uses of force, incidents, arrests, or contacts where nothing notable happened. | | | Incident Reports | Reports made by police officers after responding to a call which may or may not be criminal in nature. Not generally posted online. | | | Misc Police Activity | Records or descriptions of police activity not covered by other record types. | | | Officer Involved Shootings | Case files of gun violence where a police officer was involved, typically as the shooter. Detailed, often containing references to records like Media Bulletins and Use of Force Reports. | | | Stops | Records of pedestrian or traffic stops made by police. | | | Surveys | Information captured from a sample of some population, like incarcerated people or magistrate judges. Often generated independently. | | | Use of Force Reports | Records of use of force against civilians by police officers. | | | Vehicle Pursuits | Records of cases where police pursued a person fleeing in a vehicle. | | ## Info about officers | record\_type | description | example | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- | | Complaints & Misconduct | Records, statistics, or summaries of complaints and misconduct investigations into law enforcement officers. | | | Daily Activity Logs | Officer-created reports or time sheets of what happened on a shift. Not generally posted online. | none found | | Training & Hiring Info | Records and descriptions of additional training for police officers. | | | Personnel Records | Records of hiring and firing, certification, discipline, and other officer-specific events. Not generally posted online. | | ## Info about agencies | record\_type | description | example | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | | Annual & Monthly Reports | Often in PDF form, featuring summaries or high-level updates about the police force. Can contain versions of other record types, especially summaries. | | | Budgets & Finances | Budgets, finances, grants, or other financial documents. | | | Contact Info & Agency Meta | Information about organizational structure, including department structure and contact info. | | | Geographic | Maps or geographic data about how land is divided up into municipal sectors, zones, and jurisdictions. | | | List of Data Sources | Places on the internet, often data portal homepages, where many links to potential data sources can be found. | | | Policies & Contracts | Policies or contracts related to agency procedure. | | ## Agency-published resources | record\_type | description | example | | --------------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | | Crime Maps & Reports | Records of individual crimes in map or table form for a given jurisdiction. | | | Crime Statistics | Summarized information about crime in a given jurisdiction. | | | Media Bulletins | Press releases, blotters, or blogs intended to broadly communicate alerts, requests, or other timely information. | | | Records Request Info | Portals, forms, policies, or other resources for making public records requests. | | | Resources | Agency-provided information or guidance about services, prices, best practices, etc. | | | Sex Offender Registry | Index of people registered, usually by law, with the government as sex offenders. | | | Wanted Persons | Names, descriptions, images, and associated information about people with outstanding arrest warrants. | | ## Jails & Courts specific {% hint style="info" %} Other record types can describe jails and courts, especially from the "info about agencies" category. {% endhint %} | record\_type | description | example | | --------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | | Booking Reports | Records of booking or intake into corrections institutions. | | | Court Cases | Records such as dockets about individual court cases. | | | Incarceration Records | Records of current inmates, often with full names and features for notification upon inmate release. | | # Hidden properties These columns contain information about our users or process. They live in our database, but should not be returned in API responses or otherwise made publicly accessible.
tableproperty to hidereason/notes
data sourcessubmitter_contact_infoPII
data sourcessubmission_notesintended to be private
data sourcesprivate_access_instructionsintended to be private
agenciessubmitter_contactPII
data requestssubmitter_contact_infoPII
data requestsinternal_notesvisible to those with write access
volunteersnamePII
volunteersinternal_notesvisible to those with write access
volunteersdiscordPII
volunteersgithubPII
volunteersemailPII
# Search the PDAP database {% hint style="info" %} [Visit our website](https://pdap.io) to use the search app. {% endhint %} This is our core feature, and will always be open to everyone. Our database **makes it easier to find existing data**, enhancing our ability to answer questions and complete research projects about police systems. Ask a question → **search for data** → analyze and report. ## Search tips * **Expect missing data.** Most of our data is passively collected based on the needs and interests of our users. If you're looking for a specific record from a small town, we probably don't have it documented yet. Make a request and we'll help you out! * **Use the** [**Record Types taxonomy**](/about/data-dictionaries/record-types-taxonomy) to understand how we categorize things. ## Searching with API You can also [use our API](/api/introduction) to browse the database. # Follow a Search When you make a search or find a location on the map at , you can click "follow" to get updates. We'll send a notification each month if we find any new sources in the area you followed, or if any new [Data Requests](/data-requests) have come up. # Sign Up / Sign In In 2025, we released the User Accounts feature, allowing anyone to create a free account and log in. Click the User Icon in the navigation menu of to sign up or sign in. ## Benefits of user accounts * [Follow searches](/features/follow-a-search) to get notified when new data is added * Make Requests to get help finding data or with a larger research project * Submit new Data Sources to our database, making it better for everyone * Participate in [Data Labeling](/activities/data-source-annotation), another way of adding sources to the database. # Data Requests Using data to answer questions is a multi-disciplinary process. Data Requests are a major way our community can find each other and collaborate toward shared goals. They can be as simple as "I'm looking for some missing data" or as open-ended as "I'm beginning a research project and could use advice". ## Opening a Request There are buttons littered around the homepage, search, and map for opening a new request if data is missing. When a new Request is created, it goes into an approval queue for moderation. Once a Request is approved, a GitHub issue is automatically created and [added to this project](https://github.com/orgs/Police-Data-Accessibility-Project/projects/26). We use GitHub to organize and work Requests for a few reasons: * it has built-in project management and discussion features we now don't have to make ourselves * many data-skilled folks have a GitHub account and it's free to create one * we feel right at home in GitHub as an open-source project * Requests are kept anonymous by default, but are open for collaboration and the requestor can sign up for notifications. ## Working on Requests See our [volunteer guide for data requests](/activities/volunteer-for-data-requests). # Introduction/Getting Started ## Base URL > ← builds from main branch > > ← builds from dev branch ## Endpoints & Models Navigate to either base URL above to see the API and model reference for v2 API on either the `main` or `dev` branch. This documentation assumes you are familiar with programming languages such as Python, as well as the big-picture functionality of the API. For a more thorough breakdown of the functionality of an API, consider documentation such as Postman's [Beginner's Guide to APIs](https://www.postman.com/what-is-an-api/#how-do-apis-work). Reach out to or make noise in [Discord](https://discord.gg/wMqex8nKZJ) if you have questions. ## Overview The PDAP API is how internal and external users programmatically access information and make changes to the PDAP Data Sources database. ## Getting Access Logged-in and authenticated users can get access to the API. ## Authentication The majority of API routes require one of two forms of Authentication: API Tokens, and JSON Web Tokens (JWTs). A limited number of endpoints allow one or the other. #### API Tokens * API tokens are long-lived tokens used for endpoints with low security needs that do not modify existing data, such as `GET` requests. * API tokens can be passed to accepting endpoints using `Basic` authorization (i.e., `Basic `). * API tokens can be obtained by using the `/api-key` endpoint and generating login credentials * API tokens are long-lived, and users can only have one active at a time. #### JSON Web Tokens (JWTs) * [JWTs](https://en.wikipedia.org/wiki/JSON_Web_Token) are short-lived encrypted tokens used for more secure endpoints * JWTs can be passed to accept endpoints using `Bearer` authorization (i.e. `Bearer `) * An `access_token` and `refresh_token` are both generated by the `/login` and `/login-with-github` endpoints * The access token is the token to be used for most endpoints * Separate single-purpose JWTs are used for specific endpoints, such as `/reset-password` ## Rate limits Different endpoints have different rate limits to them, with endpoints like `/data-sources/{resource_id}` `GET` Being more generous than endpoints such as `/login`. This helps prevent abuse of the system, and should minimally impact normal usage. ## Examples {% tabs %} {% tab title="Python" %} **GET Agencies** ```python import requests base_url = "https://data-sources-v2.pdap.io/" api_key = "YOUR_API_KEY_HERE" # Replace with your actual API key url = f"{base_url}agencies" # Create the Authorization header headers = { "Authorization": f"Bearer {api_key}" } # Make the GET request with the Authorization header response = requests.get(url, headers=headers) ``` {% endtab %} {% tab title="JavaScript" %} #### GET agencies ```javascript const axios = require('axios'); const baseUrl = "https://data-sources-v2.pdap.io/"; const api_key = "YOUR_API_KEY_HERE"; // Replace with your actual API key const url = `${baseUrl}agencies`; // Create the Authorization header const headers = { 'Authorization': `Bearer ${api_key}` }; // Make the GET request with the Authorization header axios.get(url, { headers }) .then(response => { console.log("Search results:"); response.data.forEach(item => { console.log(`Agency: ${item.name}`); }); }) .catch(error => { console.error("An error occurred:", error.message); } }); ``` {% endtab %} {% endtabs %} # Related projects {% hint style="warning" %} PDAP is neither officially nor unofficially aligned with the missions of any of the projects listed here. We are interested in talking to people using police data for any purpose. If you have experience to share, reach out to . {% endhint %} ## Collecting Data ### Transparency / reform focused * * * * * * * * * * * * * * * * * * * * * [https://americanequity.org/](https://americanequity.org/index.html) ### Facilitating records storage for police * ## Lists of projects * * ## Police data users * News agencies, researchers, and lawyers. * * * * * ### Projects using data * * ## Collecting different types of data * * * * # Resources for using data ## Using & analyzing data [Common abbreviations in criminal legal system records](http://amerusa.net/resource_documents/CriminalRecordAbbreviations.pdf) [QA process for data analysis](https://source.opennews.org/articles/qa-process-confidence-data-stories/) [CPE's guide on analyzing traffic stop data](https://policingequity.org/data-collection-insights/33-cpe-toolkit-stop-data-collection-guidebook/file) [RATH](https://rath.kanaries.net/) by Kanaries for no-code exploratory data analysis [Splunk](https://www.splunk.com/) is an intelligence analysis tool to make sense of vast amounts of data. ## Collecting data [Compendium of state open records laws](https://www.rcfp.org/open-government-guide/) [ScrapingBee for headless websites](https://www.scrapingbee.com/) [Best practices for data collection](https://inspectelement.org/best-practices-data-collection.html) from Inspect Element [Finding undocumented APIs](https://inspectelement.org/apis.html) from Inspect Element [Automatic DocumentCloud scraping](https://www.muckrock.com/news/archives/2022/may/24/release-notes-keep-an-eye-on-your-favorite-agencie/) which requires a verified MuckRock account. If you don't have one, you can use ours! Reach out. [The Berkeley COPWATCH "People's Database" guide](https://www.berkeleycopwatch.org/people-s-database) for creating a community accountability tool ## Processing data [Crosswalker](https://github.com/washingtonpost/crosswalker) is for joining columns of text data that don’t match perfectly. [CJWorkbench](https://github.com/CJWorkbench/cjworkbench), a spreadsheet-like program with powerful tools for data journalism. [PDF processing and OCR](https://github.com/chadday/nicar_ocr) from Chad Day for NICAR [Useful file types: Parquet, SQLite, FlatGeobuf](https://observablehq.com/@asg017/nicar23-lightning-talk-tipsheet-3-file-formats) from Alex Garcia [Frictionless data](https://frictionlessdata.io/) for transforming and describing messy datasets, making them more interoperable, and creating a pipeline [Kyle Walker's guide](https://walker-data.com/umich-workshop-2023/python/#1) for using Python to do spatial mapping and analysis ## Learning Python [Consuming APIs with Python](https://realpython.com/python-api/#consuming-apis-with-python-practical-examples) from Realpython [An intro to your first notebook using Python & Jupyter](https://github.com/palewire/first-python-notebook) from Palewire [A mini Python and Jupyter bootcamp](https://github.com/ireapps/pycar) from NICAR [Python scraping](https://github.com/oxylabs/web-scraping-tutorials/tree/main/python) from Oxylabs # Using LLMs like ChatGPT The usability of Large Language Models (LLMs) like ChatGPT has made simple coding and scripting tasks far more accessible to inexperienced coders. We have some thoughts about making the most of these tools, while being aware of their limitations. ## Limitations {% hint style="info" %} If you ask it to count how many vowels are in a sentence, it will probably fail. If you ask it to create a small python program which counts how many vowels are in a sentence, it will likely succeed. {% endhint %} ChatGPT leverages sophisticated pattern recognition, predicting what comes next in a sentence based on vast training data, akin to an advanced auto-complete system. The "predicting" part is critical: ChatGPT doesn't "know" anything: it's making educated guesses based on the contents of its training data. For topics where it has a large amount of training data, its responses are more likely to be reliable and broadly accurate. However, for tasks like counting, mathematical operations, or logical reasoning, it often falters because these tasks require strict internal logic rather than probabilistic pattern matching. To give an example: A 1st grader would know that 2 + 2 = 4. An LLM does not know that -- instead, it knows that, if you start with "2 + 2 =", the next character is usually "4". For a mathematical problem that a simple calculator could solve but which has little precedent in its training data (e.g. 172 / 9), it will probably stumble because it has no data to draw on. It is also likely to perform more poorly the more niche a topic is (because it likely has less material for that topic in its training data). However, it probably will not indicate that it knows little about the topic, and will try to provide a coherent- and confident-sounding answer, leveraging information it thinks is relevant from its training data (regardless of whether that training data actually *is* relevant). That confidence and coherence can make it tempting to trust what an LLM is saying, especially when you don't know much about a topic; that's why it's important to supplement whatever an LLM tells you with information from a more reliable source, and to take what an LLM says (especially on more niche topics) with skepticism. ### Example: counting vowels Without a program, GPT struggles to do basic math:
The actual number of vowels in the sentence is 13 (14 if you count 'y', as ChatGPT did). When it is allowed to write a program, it produces a more sensible answer: ``` # Let's count the number of vowels in the given sentence. sentence = "The rain in Spain stays mainly on the plain." vowels = "aeiouAEIOU" # Count the vowels in the sentence vowel_count = sum(1 for char in sentence if char in vowels) vowel_count ``` ## How we use LLMs Our community includes many people who are not career software developers, but writing scripts to solve problems out of necessity. ### Troubleshooting foundational software setup Have you ever tried to set up a local dev environment? Tried to figure out which kernel your jupyter notebook is using? Tried to untangle a github branch that has conflicting commits? What if you don't even know the terminology you need to search? Chances are, you aren't the first—but finding an answer brings needles and haystacks to mind. ChatGPT can be a great resource for parsing the problems you are having and providing troubleshooting steps. Before waiting for a response from a busy software engineer or unforgiving forum, try asking basic questions: > I can't figure out why my linter `black` is using a different python version than the project I'm linting. I'm on a mac. > This can occur if multiple Python versions are installed on your system and `black` is using a different version than your global `python3`. > > Here’s how to troubleshoot and fix it: ### Making templates and mockups > i have a spreadsheet with an address column, and i want to put the data on a map with basic colorcoding using bokeh. what would a basic template python notebook look like? > Here’s a basic template for visualizing addresses on a map using Bokeh with color coding: ### Checking syntax If you're learning SQL or Python, basic errors that are hard to find can have disastrous results. Pasting some code into an LLM can quickly help you diagnose issues. > What's wrong with my syntax? \ ### Editing written content If AI-written content is poisoning the internet (if you can't be bothered to write it, why would you expect anyone to read it?), the antidote is simplicity and clarity. However, LLMs are good at *editing* and *simplifying*. > Please rephrase "Facilitating the democratization of information and empowering stakeholders through seamless data accessibility" for a third-grade reading level. > "Making it easy for people to get the information they need." It can also help analyze the sentiment of a word or phrase, to make sure you are communicating the message you would like to impart. > what does it say about my org if we use "locality" instead of "municipality" to refer to that concept? > Using "locality" instead of "municipality" can suggest that your organization prefers a broader, more inclusive term. "Locality" encompasses any local area or community, not necessarily defined by strict legal or administrative boundaries. This might imply your organization values flexibility, inclusivity, and perhaps a more community-oriented approach. It can also indicate a preference for simpler, more universally understood language. ### Finding open-source software solutions Chances are, the problem you're solving has been encountered by others before. LLMs can be surprisingly creative in combining obscure open-source packages to solve problems. > I have log forwarding with my hosting provider. What is a simple free or self-hosted log viewing package? # Internal Tools (Retool) We use Retool for "internal" tooling, such as **submission approval** and **data labeling.** {% hint style="info" %} Access the dashboard {% endhint %} #### **Why Retool?** Building a front end app takes time and effort we don't want to spend unless it needs to look nice or we know *exactly* what we want. Retool is great for prototyping workflows, before they are solidified into a self-hosted web app. #### **What are "internal" tasks?** * A task which requires a higher level of context, trust, or access * A task which is meant to support our users or mission | Retool App | Purpose | Who gets access? | | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ---------------- | | Data Request Intake | Modifying and approving (or rejecting) submitted Requests, so they can be seen by the community | PDAP staff | | Data Request Edit | Modifying community requests | PDAP staff | | Data Source Intake | Modifying and approving (or rejecting) submitted Data Sources, so they can be seen by the community | PDAP staff | | Data Source Edit | Modifying Data Sources | PDAP staff | | Source Collector & Labeling Interface (see [Data Source Identification](https://github.com/Police-Data-Accessibility-Project/data-source-identification)) | Triggering the Source Collector to create batches of collected data; human-labeling data which has been collected. | PDAP staff | # Internal dev resources # GitHub issue template ### Context Brief description. Is this part of a parent issue? ### Requirements * Required functionality. ### Tests * Anything we should test for? How should it be tested? ### Docs * What docs should be updated? Link to related docs changes in the PR. ### Open questions * Any uncertainties? Anything we're flexible about? # GitHub pull request template #### Fixes * Issues being closed by this PR #### Description * A brief description of what this does. Include screenshots, if anything should change visually. #### Testing * Steps to test this PR, if there are any required. * With endpoints, include a screenshot or API response to show what correct behavior looks like #### Performance * If you're adding or heavily modifying a script, estimate how long it should take to run. #### Docs * Accompanying docs PRs, if applicable #### Breaking Changes * What functionality, if any, might be broken by this change, which could impact other developers. For example, an environmental variable changing. # Product changes checklist ## Data Sources database ### record\_type changes If you're adding or modifying record\_types, do the following things: * [ ] Make the intended changes in Airtable * [ ] Update the submission forms * [ ] Make the intended changes in the [record types taxonomy](/about/data-dictionaries/record-types-taxonomy) ### schema changes If you change the schema, do the following things: * [ ] Make the intended changes in Airtable * [ ] Make the intended changes in the relevant [data dictionary](/about/data-dictionaries) * [ ] Make an issue in the [Data Sources Mirror](https://github.com/Police-Data-Accessibility-Project/data-sources-mirror/) repo describing what you did so it can be updated * [ ] Make an issue in the [Data Sources App](https://github.com/Police-Data-Accessibility-Project/data-sources-app/) repo describing what you did so it can be updated * [ ] mostly the relevant endpoints # Production QA Checklist A checklist of basic manual tests to be performed immediately prior to and after the release of changes to production ## Overview This is a checklist of basic sanity checks to validate functionality of the web app immediately prior to and following a release of changes to production. This checklist is designed to be completed within 10 minutes or less -- it is not a substitute for more substantial automated testing as much as a way to manually validate critical functionality quickly. ## How To Use 1. Run through this checklist in the dev environment as a final precondition to releasing changes to production. 2. Run through the checklist *again* on the production domain immediately afterwards to confirm nothing has broken. ## Basic Checklist * [ ] Using Chrome, open [developer tools](https://developer.chrome.com/docs/devtools/overview) and navigate to the "Console" tab to enable visibility of client-side errors. * [ ] Navigate to the domain and confirm the home page loads as expected without error * [ ] Perform a search in the database using a common keyword such as "Prison" or "Pittsburgh". See that results are returned without error. * [ ] For a given result, click on "More Details" and see that results are loaded. * [ ] Click on the following links and confirm they load the expected page without error: * [ ] "Data" * [ ] "About" * [ ] "Donate" * [ ] "Docs" * [ ] Recommended: perform other unexpected behavior specific to the code being tested, to try to "break" the site. For example, entering invalid search terms or trying to navigate to URLs without proper auth. Confirm the web page handles it gracefully, documenting cases where it does not. * [ ] Perform the above in both the desktop version and a mobile device ## Expanded Checklist
Expand for Checklist * [ ] Check `sign-in` page * [ ] Can a user successfully sign in using GitHub? * [ ] Can a user successfully sign in using a password? * [ ] Check Map * [ ] Does the map load? * [ ] Are states colored more or less heavily depending on how many data sources they have? * [ ] Can you click on a state and have it zoom in? * [ ] Can you click "View All" and be taken to the appropriate search page? * [ ] Can you click on one of the counties viewable in the dropdown and be taken to the appropriate search page? * [ ] From a county level, can you zoom out and have the view revert to the state level? * [ ] Can you zoom out further and have the view revert to the national level? * [ ] Can you click the `loop` icon and have the view revert to the national level? * [ ] Are counties similarly colored based on how many data sources they have? * [ ] Can you click on a county and zoom in further to see the "pins" of different locations? * [ ] Can you click "View All" and be taken to the appropriate search page? * [ ] Can you click "Suggest missing data" and be taken to a New Request (possibly after sign-in)? * [ ] Can you click `Follow for updates` and get it confirmed? * [ ] Can you unfollow something you're following, from the map view? * [ ] Can you click on a locality viewable in the dropdown and be taken to the appropriate search page? * [ ] Search Page * [ ] Does the page load all data types, populating `local`, `county`, `state`, and `federal`? * [ ] When clicking on any of those categories, do the searches automatically navigate to searches of those that geographic level? * [ ] When clicking a data source, are you taken to the appropriate `Data Source` page? * [ ] `Search Location` Box * [ ] When checking any `Types of Data` subtype and clicking `Update search`, does the page filter results to *only those* `Types of Data` *selected*? * [ ] When checking one of multiple `Types of Data` checks and clicking `Update search`, does the search now update to only the `Types of Data` subtypes remaining? * [ ] Does the search typeahead work? Can you load new results, click on one, and be taken to a new version of the page? * [ ] `data-source` page * [ ] When clicking `next`, are you taken to the next relevant data source? * [ ] "Relevant" implies that all data sources should connect first to other data sources with the same record type, then the same locality, then the same county, then the same state * [ ] When clicking `next` and then `prev`, are you taken to the same data source page you started at? * [ ] Does clicking `View Data Source` take you to the appropriate link? * [ ] Are all relevant data source information pages properly populated in the following boxes: * [ ] `Access & format` * [ ] `Provenance` * [ ] `Coverage & retention` * [ ] `Data Source Meta` * [ ] Are the following attributes properly populated? * [ ] `Agency` * [ ] `County, State` * [ ] `Agency Type` * [ ] Home Page * [ ] Clicking on the profile icon should take a user to their profile * [ ] Additional Links * [ ] `About the data` * [ ] `"Data Source"` * [ ] `start with the docs!` * [ ] All `Recently Added Data Sources` * [ ] Have a functioning link * [ ] The question mark takes them to the relevant `Data Source` page * [ ] `Programs` * [ ] `Open a Data Request` * [ ] `Sign up for an account` * [ ] `Take Action` * [ ] `docs.pdap.io` * [ ] `Locate sources` * [ ] `Help add Data Sources to our Database` * [ ] `Submit a source you found` * [ ] `Collaborate on Data Projects` * [ ] `Volunteer for Data Requests` * [ ] Good first issues * [ ] Each individual link takes the user to an open issue * [ ] `View more Good First Issues` takes the user to the appropriate view * [ ] `Praise for our work` * [ ] `Alliance for Police Accountability` * [ ] `Alliance, NE` * [ ] `Dr. Kyla Bourne` * [ ] `Examples` * [ ] `shared it in our database` * [ ] `this article in Public Source` * [ ] web scraper `?` mark * [ ] `We made it open-source` * [ ] `Software Roadmap` * [ ] `sign up for data labeling here!` * [ ] Footer Navbar * [ ] `GitHub` * [ ] `Discord` * [ ] `Newsletter` * [ ] `Docs` * [ ] `Publish data` * [ ] `Email` * [ ] `Guidestar Badge` * [ ] Profile Page * [ ] A user should be able to click `Unfollow` and unfollow a given search. On refresh, that search should not appear. * [ ] A set of `Recent Searches` should be viewable by the user, correlating to searches the user performed. * [ ] The user should click `Regenerate API Key` and receive a new API key popup * [ ] The user should either have a `Link account with GitHub` button or an indicator that the user's account is linked with GitHub * [ ] Clicking the `ResetPassword` button should take the user to the `change-password` page * [ ] Clicking the `Sign-out` button should take the user to the `Sign-In` page *
# Retool Retool is a low-code web app which we use to design applications for internal use. In our case, we use them to easily and safely modify internal data, such as data requests and data sources. ## Resource Intake Pages These pages are for reviewing recently submitted resources which have yet to be approved. These pages will display entries of unapproved resources. The reviewer will then have the option of either accepting or rejecting/archiving them. The following Resource Intake pages exist: * [Data Source Intake](https://pdapinternal.retool.com/apps/6e3f5704-7365-11ef-a7a0-2ff6a691f56d/Data%20Source%20Intake) * [Data Request Intake](https://pdapinternal.retool.com/apps/033f3aea-7ac3-11ef-8e85-5b7af40d9169/Data%20Request%20Intake) ## Resource Edit Pages These pages provide a view of a larger number of resources, with reviewers having permission to examine and modify these. Some filter and pagination options are available. The following Resource Edit pages exist: * [Edit Data Sources](https://pdapinternal.retool.com/apps/60948a44-9646-11ef-8664-ab94ecae4e51/Data%20Source%20Edit) * [Edit Data Requests](https://pdapinternal.retool.com/apps/cae619ea-98b6-11ef-9967-ff6f0daf774f/Data%20Request%20Edit) # Operations # Staff resources Keeping the lights on. # Meeting Minutes # 2021-07-14 ## 90 days General themes: * money * volunteers * bounties * messaging ### Goals * ready for the next bounty * policy discussion about the next one * know what we want (datasets) * list of people to contact / press release / announcement strategy e.g. Stanford * debrief with The Bounty Hunters and close their blockers * **focus the efforts on a smaller geographic area to unlock more** * capture the list of things that are pending 1023 status * update the website * updated documentation about making datasets submissions * get 100 patrons / recurring donors * get a person involved who's able to talk / market / network / outreach effectively on behalf of PDAP * formally intro + brainstorm partnership with Launchcode + support the Bounty Hunters # 2021-06-16 ## Attendees * Josh C * Josh L * Eddie * Stabs * Alec ## DoltHub bounty * Morale is somewhat low * Added an [approval process](https://github.com/Police-Data-Accessibility-Project/PDAP-app/blob/main/README.md) to see if it helps us approve PRs faster ## Roadmap * Exists: * People on the GitHub team have access to ZenHub: * More explanation here: ## Needs * Security * Scrapers & python knowing or wanting-to-learn people * For more volunteers and potential donors to see this page: # 2021-03-14 {% file src="/files/-MbWzhOcHbh9dkcY5I08" %} # 2020 11-21 Tech Stack discussion Broad strokes: we determined that the MVP fulfills the pieces not currently achieved by Splunk—which could serve as the entire front end at very small scale. This means **data integrity** and **depth** are the core of the value added by PDAP. ## Parts **Ingestion** → **Archival Storage** → **Search & analysis** ## Core principles, aka value add Data stewardship Transparency Discipline “Librarian” ## Insights "I want police data." Make a query → **get information** Enter search via UI (selects) or code → present specific data api (json), chart, csv "I want to be able to analyze the police data I found." Analysis tools or analysis that is done for you *Find extremes in the data automatically* "PDAP needs to verify data." * Guard the submissions process * Credibility score for each type of data "PDAP needs to be like a librarian." * Nonintrusive * **Pedigree** * Legally captured? * Multiple sources? * Anonymity? "What does it mean to verify data?" "We need to be able to get data out of cold storage." * Eventually it'll be too much data for Splunk ## Workflows powered by Splunk **Query data → Analyze data → Export insights** ***Upload data*** **→ Analyze data\***\ \**The user agrees that we can keep the data, and provides information or verification about it.* **Save an Analysis → Share the Analysis with someone else** **Save an Analysis → Revisit it with updated data**\ *Alert user if an analysis changes based on updated information* ### Specific abilities granted by Splunk * easily write regex * accept any type of data * oddly / non-delimited * many file types * faster analysis / searching on the server rather than locally * automatically find "interesting fields" * search * analysis ## Workflows not supported yet **Supply data to PDAP by volunteering or other sources** **Verify submitted data → Request more info from a submitter** **Provide an unprecedented breadth of data** **Safely archive historic data for the foreseeable future** **Understand the categorization structure**\ *We need to make sure the structure is future-proof, and establish policies for sortation that cannot easily be corrupted.* # 2020-09-30 Leadership Cadence ## Date 30 Sep 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Bree Aronin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f752e054d09f70076e70472?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Goals * Discuss PDAP focus ## Discussion Notes * Reevaluate current situation * If we consider what we know now, what would we have changed coming into the project? * We had good momentum, would be good to know better way to keep it going * Being able to dig down into objectives quickly in order to orchestrate goals * We focused too heavily on leadership right away, when maybe objectives would be higher priority * A lot personal learnings for some * Being given direction for a few of us would have taken us pretty far, and perhaps for a lot of other volunteers. A lot of us didn’t know what we needed to do, and we did lack a structure with deadlines because of decentralization * Being able to scope to focus in on goals and targets * Decentralized nature of the project makes it important to centralize information; a lot of relevant dialogue/information exchange occurs that gets lost * Will help with folks taking on things when they know where to look for it * Volunteers that had longevity be given the longer term stuff * People would come in and overtask themselves and become overwhelmed * Some volunteers have high expectations and want to be told what to do * Folks wanted to come into the project to reap the “glory” and when it wasn’t panning out, they fell off * Stanford Ignite didn’t quite have the impact we expected it to, and we put too much stock in it Project Updates: * Contacted A\&P, apparently they got us documents back on 9/11 that we should have proofread ## Action items * Proofread articles of incorporation @11:59 PM ET - Everyone 03 Oct 2020 * Confluence membership for folks - [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * Newsletter deadline [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) , [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) , [Bree Aronin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f752e054d09f70076e70472?ref=confluence) , [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence)14 Oct 2020 * Meet up to talk about reengaging with volunteers 07 Oct 2020 - [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Asset inventory 14 Oct 2020 - [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Decisions * # 2020-10-14 Leadership Cadence ## Date 14 Oct 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * Emily Crose ## Goals * Discuss FOIA and board membership logistics ## Discussion topics | **Topic** | **Presenter** | **Notes** | | ---------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **FOIA** | |

  • Primer questions:

    • What do we do with FOIA?
    • How do we want to approach Stonewalling
    • What can we even do to compel valid responses from FOIA requests?
    • Is there a way to source the information around which agencies fulfill/evade requests
    • What are the gaps that we need information for that FOIA can fulfill?

    • What does our attitude need to look like when we interface with the other end of these FOIA requests?

      • FOIA is an activist endeavor - how far as a group should we take that at this moment in time?

  • Comparing federal agencies vs non-federal FOIA requests

    • Federal agencies

      • Generally consistent regarding the kind of information that we get

    • State and local

      • Complete dismissal/non response for requests

      • Invalid excuses on not being able to fulfill requests

        • I.e. “non-existing” documentation for involvement/support of OPERATION LEGEND will require the generation of new documents to fulfill the request; law enforcement doesn’t “have to comply” because they cite that FOIA only applies to already existing documents
      • Paywalled behind large pre-processing fees
      • Requests for what should be readily available information (such as mission statements) are rejected in FOIA requests

  • There’s value in the process of making FOIA requests, in addition to the content

    • They hold law enforcement accountable

    • Response/non-response can be tabulated as a data point

      • Metropolitan police departments have not been reliable in fulfilling requests
    • We can cross reference FOIA sourced data points with NIBRS/state-IBRS data points for inconsistencies

  • Current state of PDAP support for FOIA

    • For FOIA requests to take, there’s a need for legal representation that PDAP currently doesn’t have

      • It’s been mentioned that we can seek this legal representation out if it’s not A\&P
      • We’ll have more pull once we’re incorporated and not simply a group

    • Volunteer data opportunity - get someone to make requests and keep count of who fulfills/evades requests

      • Identify what the expectations are for a request
      • Document actual experience and how it differs from expectations

  • Direction

    • Current issues with Muckrock preventing us from socializing the FOIA requests to the rest of the group

      • May require some follow up with Muckrock
    • As we flesh out the NIBRS/OIBRS comparisons, we’ll have more information on what is missing and what we could potentially have FOIA fill in
| | **Board** | |
  • We’ll generally want folks that will be active participants and consistently around
  • How do we make sure we stay diverse, and what does that look like for us?

  • Leadership to DM Eddie for those interested

    • Full Name
    • Address
| | **Splunk** | |
  • Alec taking the lead on interfacing with reps from Splunk
| ## Action items * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence)[Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) Send Eddie a DM about board positions. Full name and address * Making a FOIA request that is in line with what the FBI requests of local jurisdictions ## Decisions * # 2020-10-21 Leadership Cadence ## Date 21 Oct 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Goals * Come up with a few tasks to focus on for Technical Working Session ## Discussion topics | Item | Notes | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Slack Standup |
  • Doesn’t really get used. Should we turn it off? Change the frequency?
| | Board membership filing |

  • Paperwork around incorporation was handed over to A\&P last week

    • Though the incorporation and nonprofit application are serial, it's assumed that A\&P is working on both in parallel

  • We wanted to get minimum number of folks to get simple majority

    • As soon as incorporation happens we can adjust the board membership as we see fit

    • Current positions:

      • President - Kristin
      • Chair - Eddie
      • Treasurer - Alec
      • Secretary - Josh
| | Splunk |

  • Splunk is an incredible initial solution for aggregating the data

    • Helps us make informed analysis on the data that we have
    • Helps us visualize raw trends
    • Provides marketing real estate for the work we’re doing; a lot of the work we’ve been doing so far isn’t particularly “visual”, so this bridges the visual gap
    • Minimal constraints around uploading the data for free

  • First Splunk meeting went well. We’ll need to address a few questions around the technical aspects of the partnership

    • Self hosting or cloud?

      • What does the payment plan need to look like?

        • Cloud storage
        • Higher data velocities
        • Any computation
      • Self hosting would be enterprise with a 10 GB limit
      • We’re currently using their cloud for free, just a few limitations around how much/often data can be uploaded
    • Given our needs and in terms of the offerings, what would be best for us?
    • Data transformation offerings/advice for incongruent field values
  • Questions around limitations on users/licenses/credentials
  • Next meeting this Friday at 2:30 PM CDT

  • Goal of one visual product with caption per week?

    • It’s incredibly important to show the measure of our work, even in bits and pieces, to the public at large
| | Assign newsletter chunks |

Location of blurb in Newsletter outline

Did not get a chance to talk about this at length

| | Open-Source CRM? |

  • We wanted to spin up some sort of CRM to help the internal comms team

    • We’ll de-prioritize this (but ultimately leave it to the team on what they want to do)
| | Infrastructure |
  • Reaching out to Dave but we’ll touch base with Richard and strap down a plan on how to move forward
| ## Action items * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) Check in with [Daniel Melles (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1c815b7070d4001b1c4b17?ref=confluence)22 Oct 2020 * Develop more Splunk visualizations/dashboards * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) to shorten Standup 27 Oct 2020 ## Decisions * Adjust the weekly slack bot to use a single question to query for topics to talk about for the cadence * Open-Source CRM not a current priority # 2020-10-28 Leadership Cadence ## Date 28 Oct 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * ## Goals * Decide on someone to reach out to Louisville Courier Journal * ## Discussion topics | Item | Presenter | Notes | | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | KSP Citation Database | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) | | |

Splunk Feasibility

|

Alec Akin (Unlicensed)

| | | [Minimum Viable Product](https://pdap.atlassian.net/wiki/spaces/product/pages/64258090/Minimum+Viable+Product) | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Some pieces of the MVP outline that need some thought/documenting

  • Need to modularize the overall product diagram
| ## Action items * ## Decisions * # 2020-11-04 Leadership Cadence ## Date 04 Nov 2020 ## Participants * [Garrett (Unlicensed)](https://pdap.atlassian.net/wiki/people/557058:64a82314-0c02-44ac-bb64-b60ad7937b5c?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) ## Goals * ## Discussion topics | Item | Presenter | Notes | | ----------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A\&P Status | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |

  • Paperwork from A\&P still pending; got some movement from A\&P but nothing definite

    • Eddie to prioritize contacting donors to help fund legal work to fast track A\&P application
  • 05 Nov 2020 Received additional paperwork from A\&P to continue the process; Eddie filled out the necessary information.
| | Splunk | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Splunk account currently frozen as trial period expired

    • Critical to conversations with donors and volunteers to be able to show something tangible and visual
    • We’ll be continuing with leveraging Splunk as our focus for MVP
| ## Action items * Eddie to reach out to donors to see if he can get sponsorship for fast tracking the application status * Alec to touch base with Splunk to see if they can extend the trial mode ## Decisions * # 2020-11-12 Leadership Cadence ## Date 12 Nov 2020 ## Participants * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) ## Goals * Formative documents update\ Data update (who, what, where, how) * Fundraising * Volunteer/tasking update ## Discussion topics | Item | Presenter | Notes | | ---------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Filing update | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |
  • Updated A\&P last week - no negative feedback
  • Anticipate end of December before incorporation
| | Database tools | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) |

  • Splunk

    • Good tool, concerned about the price
    • May not be the perfect tool, but we have experience and ownership with Alec.

    • Discussion next week on alternate open-source tools

      • Look for what is popular and common.

  • Custom Stack

    • Do we have time to accomplish this?
    • More flexibility by using open-source software
| |

Fundraising

| [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |

  • Questions to consider

    • Are there people we would not accept funds from?

      • Worried about strings attached to money
    • Would/can we accept anonymous donations?
  • As an educational non-profit, we need to spread out our funding sources.
  • We will need to plan/budget for potential donations.
| | Project Management | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Working withFormer user (Deleted) for project management and Confluence development

    • Breaking down into tasks with time estimations
| | Volunteer engagement | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Working on a message to volunteers

    • Thank you
    • Call to action
  • Thank Constantine for providing direction for new volunteers
| ## Action items * Follow up on alternate tools for database management/analysis [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Decisions * # 2020-11-18 Leadership Cadence ## Date 18 Nov 2020 ## Participants * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) ## Goals * Compare/contrast alternatives to splunk * High level picture conversation of how splunk/alternative fits in actual MVP ## Discussion topics | Item | Presenter | Notes | | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Splunk alternative - ELK stack |

Alec Akin (Unlicensed)

Richard Ji

|

ELK (Elasticsearch, Logstash, Kibana)

  • Open source, free
  • Wealth of information for reference

Neither ELK nor splunk do particularly well with long term data storage; will still need some form of data lake solution

We’ve been considering splunk as a springboard in order to transition into something more flexible and maintainable.

Splunk resolves:

  • Immediacy
  • Security
  • Outsource maintenance and high availability for the most part
| | Planning | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |
  • Still working with A\&P to continue process

  • Once we’re unblocked, where does the money need to go?

    • Splunk
    • ???
| | Interim work | |
  • Alec has a small, personal Splunk instance we can work off of as we wait for incorp/501c3
  • Could be a good time to spike out some learnings for outstanding problems whether we use splunk or not (long term data storage, exposing API for public use, determine how to validate data integrity, serving up data back to the public for free)
| | Accuracy of PDAP data arch diagram and data ingestion diagram | | Still fairly accurate![](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fpdap%2F-MbsiVpKX_z6z4FbW9B-%2F-MbskI7FEdFU2jk3IpqM%2F138706967.png?generation=1623376090989413\&alt=media)![](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fpdap%2F-MbsiVpKX_z6z4FbW9B-%2F-MbskI7HoprzwS4z7cKY%2F138739735.png?generation=1623376090912604\&alt=media) | | | | | ## Action items * Working session for white boarding architecture for splunk/alternative with MVP [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * Working session for some splunking learning [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * Reach out to folks for documenting emails [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Decisions * # 2020-11-25 Leadership Cadence Notes ## Date 25 Nov 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Goals * ## Discussion topics | Item | Presenter | Notes | | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Chief Technology Officer | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) | | | Incorporation and 501c3 status | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |
  • Eddie reached out to A\&P asking about timelines on incorporation and 501c3 status
  • A\&P hopefully received our “pressure” for timeline
  • Need to brush up a few details about IRS filing
  • Likely 2 or 3 more weeks until filing
| | Software offerings for non-profit discounts | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Most software companies end up using a third party to confirm non-profit status before offering discounts
  • Likely legal grey area for both parties if you promise a “pending” non-profit status
| | Confluence Standard Trial Ended | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • What we lose going from Confluence Standard to Freemium:

    • Restricted to 10 users (matches our current Jira restriction)
    • Space and Page permissions
    • Allowing anonymous access to our Confluence site
  • With 501c3 status we can get either a free Server (self-hosted) installation or 75% off of Cloud licenses

  • Possible next steps as interim until 501c3 status:

    • Ride the free plan out and continue developing/improving our project documentation, but not be allowed to share pages publicly
    • Pay for a couple of Standard ($5/month) accounts. They'd be designated "content editors"

    • Get a Server deployment for 10 users. Would be $10 total, as an upfront cost.

      • Pros: Cheaply build out our Confluence and be able to share it all publicly.

        • Can also tack Jira onto our Server for another $10.
        • Transitions well if we decide on the free Server w/our 501c3

      • Cons: Would have to pay for our hosting

        • Time getting things set up
        • If we decide on the 75% discount for cloud, we would have to migrate our data back
  • Good idea to think about not spending money trying to predict what PDAP might be in the future
| |

Industry Connection and Board Diversity

|

Eddie Brown (Unlicensed)

|

  • Eddie made a connection with Burke Brownfeld

    • Very interested in the outlook of our project and how we are trying to achieve change
  • Eddie would like to tap into him as a resource for identifying diversity for the PDAP board
| | PDAP Architecture Workshop Review | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) | [2020 11-21 Tech Stack discussion (Josh C's notes)](/meta/operations/staff/meeting-minutes/project-home-2020-11-21-tech-stack-discussion-josh-cs-notes) | ## Action items * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) Reach out to Atlassian by 29 Nov 2020 * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) Reach out to Louisville Courier-Journal by 29 Nov 2020 * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) Work to determine how different industries approach data/tip validation * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) Announcing CTO 25 Nov 2020 * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) and [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) poking around at using Splunk for developing schema and guiding ETL scripts ## Decisions * Continue with Atlassian Freemium # 2020-12-02 Leadership Cadence Notes ## Date 02 Dec 2020 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Matthew Pierson](https://pdap.atlassian.net/wiki/people/5f7e4b14021acf0076f6cceb?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Goals * Check in on existing Action Items * Summarize current technical direction ## Discussion topics | Item | Presenter | Notes | | ---------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A\&P Update | |
  • Representation hasn’t come back to us in regards to the status
| | Summary of yesterday’s technical discussion | |

  • Two experiments around being able to accommodate scraper data ingestion

    • Utilizing google drive
    • Standing up an SFTP server
  • Storage is a relatively simpler question to answer; we want to focus instead (for now) on having an entry point for scrapers to drop off data they’ve collected
  • We plan to reach out to scrapers that have existing data and collect it
  • We want to be mindful of cloud costs; we want to research restrictions/protections around common security/load concerns
| | Blocker around free license of JIRA/Confluence | |
  • We won’t be able to have a public facing presence with these tools because we’re on the free license
  • There are some workarounds, possibly leveraging google sheets
| | | | | ## Action items * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) will work on a POC for public facing workaround for JIRA * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) flesh out public drop area in google drive, research scaling * ## Decisions * # 2020-12-09 Leadership Cadence Notes ## Date 02 Dec 2020 ## Participants * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Goals * Talk about shopping list on tech products * Update on our incorporation and 501c3 filing * Identify new tasks associated with operating our entity ## Discussion topics | Item | Presenter | Notes | | ---------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Incorporation | |
  • Working out the details around receiving donations for being incorporated in Delaware
  • We have an entity number for tax purposes
  • Optimistically we’d be ready to start accepting donations, more realistically it’ll be a couple of weeks from now
  • 501c3 status may take 6 months to a year but we can backdate funds received
  • Entity tied intimately to Eddie (social and address) currently
| | Shopping list | |
  • Google Apps/Suite, Atlassian products, AWS
  • We want to make projected budget last 6 months to a year
  • Employ personnel to be held accountable for the work

  • Process around how we receive/disperse funds

    • Tooling?
    • It’s important to delegate financial accounting to a third party to figure our financials out (i.e. Paypal, Venmo, etc)
  • Accounting software

  • Once funding comes in, no personal accounts related to PDAP full stop

    • In-use accounts need to be inventoried and accounted for
  • Centralized password/account management

  • Domain name(s)

    • Finalize domain
    • Consider other endings
| | Public facing Splunk | |
  • Need to follow up with Splunk reps
| | Website | |
  • Need to make a decision on domain
  • We can redeploy the website on a pdap specific AWS account
| | Hype engine | |

  • Identify what things we’ll need to have in order to embrace massive flow of volunteers again

    • Scrapers
    • Website
    • Infrastructure/database
    • Processes for each bucket of folks
    • Documentation

  • How do we push back against outside pressure

    • Vendors
    • Bad actors
  • How do we engage and maintain engagement with volunteers
| | OIBRS \| NIBRS discrepancies | |
  • Tech difficulties around MS Access DB
  • NIBRS substantially lacking OIBRS information during initial reconciliation process
| ## Action items * Refine #leadership channel membership list [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Dig up old documentation around incorporation/non-profit status benefits (e.g. Techsoup) [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) & [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) work out ownership of LinkedIn account * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) to follow up with Splunk reps ## Decisions * # 2020-12-12 Working Session ## Attendees * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) ## Goals * Come up with shopping list * Work on individual action items ## Notes * Shopping List * [Google Workspace accounts](https://workspace.google.com/pricing.html) * Possibly splunk, but more likely a VPS service instead (i.e. Digital Ocean) * Dev/Playground server * Scraping server * Website server * SFTP server * Donation software ## Shopping List | **Service** | **Use** | **Cost** | **Cost Type** | **Approx. Current Predicted Cost** | **Approx. Post Non-Profit Predicted Cost** | **Initial Year Cost** | **Notes** | | ------------------------------------------ | ---------------------------- | ------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Google Workspace Business Starter/Standard | File Management | $6/$12 | User/month | $36/$72 | $36/$72 | $720 for 10 users | | | Duo MFA | Privileged Access Management | $3 | User/month | $30 | $30 | $360 for 10 users | Nice to have, not critical yet | | Bitwarden Teams | Password Management | $3 | User/month | Free/$30 | $30 | | Nice to have, not critical yet | | Jira Standard | Project Management | $7 | User/month | $63 | $15.75 | $840 | | | Confluence | Project Wiki | $5 | User/month | $50 | $12.5 | $600 | Nice to have, not critical yet | | Slack | Remote Comms | $8/$6.67 | User/month or User/month billed yearly | $17,142 | $2,572 | | | | DigitalOcean | VPS/Hosting | $5 | 1 server/month |
  • Webserver - $60/year
  • Scraping/util server - $240/year
  • TFTP server - $60/year
  • Splunk - $960/year \*assuming self hosting
  • DB server - $180/year
| | $1500 |

Server for each at least:

  • Webserver - frontpage site
  • Scraping server
  • TFTP server
  • Splunk server
  • DB server
| | Domain Name | Public facing host name | Variable; $12 | Per year | $12 | $12 | | | ## Accounting Software | **Name** | **Cost /user** | **Cloud/Self Host** | **Pros** | **Cons** | | ----------------------------------- | ------------------------------------------ | ------------------- | ----------------------------- | -------------------------------------------------------------- | | [Akaunting](https://akaunting.com/) | **Free**, may need to purchase add-on apps | Cloud |
  • Open source
|
  • Unsure about viability for donation tracking
| | | | | | | ## HR Software | **Name** | **Cost/user** | **Cloud/Self Host** | **Pros** | **Cons** | | ------------------------- | ------------- | ------------------- | -------- | -------- | | Hubspot with Integrations | | | | | ## CRM Software * [Open source options](https://crm.org/crmland/open-source-crm) | **Name** | **Cost/User** | **Cloud/Self Host** | **Pros** | **Cons** | | ---------------------------------------------------------------------------------------------------------------------------- | ------------- | ------------------- | --------------------------------------- | --------------------------------------------- | | Hubspot | | | | | | [SuiteCRM](https://suitecrm.com) | | |
  • Open source
| | | [Insightly](https://pdap.atlassian.net/wiki/pages/resumedraft.action?draftId=218202113) | $29 | | | | | [Spreadsheet solution](https://docs.google.com/spreadsheets/d/12S3hIhJUC8O-dqwNnEHvFCMfWtnkZSzmCf04rhg92q8/edit?usp=sharing) | Free | Cloud |
  • Available immediately
|
  • Probably not the easiest UX
| ## Data Compliance | **Name** | **Cost/User** | **Cloud/Self Host** | **Pros** | **Cons** | | -------- | ------------- | ------------------- | -------- | -------- | | | | | | | | | | | | | # 2020-12-16 Leadership Cadence ## Date 16 Dec 2020 ## Participants * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Goals * Come up with plan of action for determining our liability for volunteers ## Discussion topics | Item | Presenter | Notes | | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Legal form for volunteers | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |
  • Will need to formalize a part of the onboarding process
  • Worth running it past A\&P, but doesn’t need to run through them
  • Probably won’t want anything boiler plate (our needs are pretty specific)
| |

Corporation

|

Eddie Brown (Unlicensed)

|
  • Eddie working on setting up bank account
  • Telling people that donations aren’t currently to nonprofit, but full expectation is there
  • Once 501c3 is granted, benefits are dated back to the filing date
| | Digital Ocean | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Working through application
  • Might have a significant chunk of hosting costs covered
| | Working Session | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Technical-oriented Tuesday cadence will be replaced by a Saturday working session

Currently 3pm PT on Saturday

| | Shopping List | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Working on finalizing estimated costs for various tools
| | Video Call Notetaking Bot | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Looking into a transcription bot
  • Fireflies, Perfect Recall
  • Looks like we can probably get away with Freemium for now
| | Opportunity to formally restructure org | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Might discourage people from applying their skills if they think that someone with a formal title is more qualified than them
  • Some sort of the decision chain of command is generally positive
  • Especially without paid staff, a fairly flat structure might be worthwhile
  • More accountability? Ownership of your “work”
  • Are we in fact organizers? We aren’t full professionals, brainiacs, etc.
  • Does a formal structure actually improve volunteer longevity?
| | Google | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Benefits of adding people into our org?

    • Protects people from liability of using personal email etc. for PDAP activity

    • Cleaner access to Drive, Calendar, etc.

      • Being in the same Google Org makes it much more streamlined to distribute access

Can set up a Google Voice number to avoid giving out your personal number

| | Re-energizing PDAP | |
  • Kristin interested in rallying the the troops, in mass, once we get $$$ in the bank
  • Are we actually ready to get people excited again?
  • Alec: Would like to have a better control over website, storage infrastructure
  • Josh: We need docs for the 4 “always open” tasks first.
| ## Action items * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) Draft or delegate a PDAP Liability Form for onboarding * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) P.I.I. by state research project * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) Add Richard, Will, Kristin, and Matt to Google * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) Send out note encouraging use of PDAP emails for communication and other asset accounts * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence)[Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence)[Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence)[Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence)[Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence)[Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) Review this document ## Decisions * Adding Project Leadership to Google Workspace * Use PDAP emails for external communication and other assets (Slack account, e.g.) * Test out Fireflies on Saturday # 2020-12-30 Leadership Cadence Notes ## Date 30 Dec 2020 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Goals * Update on our incorporation and 501c3 filing * Discuss coordination on new domain transition * Discuss Slack standard plan expiration ## Discussion topics | Item | Presenter | Notes | | ---------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Corp bank account | Eddie |
  • Holidays have delayed setting up the corp bank account but it’s in review now as of today
| | Additional leadership | Eddie | | | Partnership intermediary agreement | Eddie |
  • Eddie Brown (Unlicensed) looking into PIAs to help facilitate cooperation between us and government agencies

  • Federal partnership

| | Slack expiration | Josh |
  • We’re getting good utility around messaging history and as a tool for communication

  • Slack pricing model is very expensive with the current size of members

    • We’re likely to let the license lapse considering this issue

  • We’ll make use of the communication tools we have; may potentially stand up a separate Slack site that we do pay for for intra communication

    • Will increase overhead in managing content/news between the two, but Slack has been pretty useful
| | Digital Ocean | Alec |
  • Currently working on setting up account(s) with the appropriate corp billing information
| ## Action items * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) to step [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) through current tools and processes. * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) investigate how [policedap.org](http://policedap.org) is working right now (???) * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) to look into shared ownership of all existing tools currently used ## Decisions * # 2021-01-06 Leadership Cadence Notes ## Date 06 Jan 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Goals * Static site progress ## Discussion topics | Item | Presenter | Notes | | --------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Corp Finance Account | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |
  • Wells Fargo hasn’t replied on the bank account creation
| | PDAP Frontpage | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Working through end to end deployment of pdap-frontpage

    • Connected project Github to Digital Ocean
  • Updated code files
| | Digital Ocean Access | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) | | | Google Workspace | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Enabled 2FA on Google Workspace

    • Please enable by 11:59 PM 08 Jan 2021
| | Core admin tool ownership | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) | | | Additional personnel search | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) |

  • Search to include underrepresented persons to involve in both the core leadership group and/or board of directors

    • Potentially pending finalization of corp financing account
| ## Action items * Add our domains to Digital Ocean [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence)[Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Follow up with Wells Fargo [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f2205e570fb250022c01aaa?ref=confluence) ## Decisions * Open up all PDAP related Google Workspace folders/files to @policedap.org members # 2021-01-13 Leadership Cadence Notes ## Date 13 Jan 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Goals * Discussion * Admin consolidation with core tools * Contact Subreddit * Financing progress * [pdap.io](http://pdap.io) site ## Discussion topics | Item | Presenter | Notes | | ---------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Admin consolidation | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Updated payment method/address for Google Workspace
  • Consolidated signed incorporation docs
  • Need to get in touch with r/DataPolice to adjust info
| | [pdap.io](http://pdap.io) progress | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • It’s up!
  • Uses static site features that makes it free for DO
  • New repo for it found in gatsby-pdap-frontpage
| | Scraper proof of concept | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • portia - non-technical scraping solution
  • We’ve got a demo in leadership channel working
| | Wells Fargo account | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Application was submitted for review but it timed out

  • Re-submitted application with corrected corp invoice information

    • Hopefully it’ll get expedited according to Wells Fargo instructions
  • Eddie Brown (Unlicensed) to seed funds for initial licenses
| | Volunteer resources | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |

  • Working on slack integrations

    • Adjusting Greetbot for new wave of volunteers
    • Requires removing some existing apps

  • Volunteer management and approval form is ready to go; pending legal approval

  • User attribution to scraped data - keybase.io
| | Core Team/Board page on site | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Would be good to have an area showcasing core team to contact in terms of

    • Customer relationships
    • Resource to help orient folks
| ## Action items * Get in touch with rubbermilitia get access to the r/DataPolice [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * Update site with most up-to-date copy [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * Take down AWS site for [policedap.org](http://policedap.org)[Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Edit code on pdap-frontpage to determine if it populates [policedap.org](http://policedap.org)[Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * Research what API integration will look like with [keybase.io](http://keybase.io)[Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * Model out a team page for the site [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Decisions * # 2021-01-20 Leadership Cadence Notes ## Date 20 Jan 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) ## Goals * [pdap.io](http://pdap.io) site status * Finances ## Discussion topics | Item | Presenter | Notes | | ---------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Change corp address to Delaware | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |

  • Financing is tedious in Illinois, much easier to do it in Delaware

    • Overhead with moving the official address is much less
  • Eddie Brown (Unlicensed) in talks with the powers that be in Delaware non-profit helpers
  • Board meeting to officialize change of address to facilitate our incorporation and non-profit operations
| | Splunk | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Splunk won’t be providing a discounted license (either on-prem or cloud) until the IRS gives us the 501c3 status
  • Our focus will be on collecting/aggregating/storing data anyway so this isn’t a setback
  • We always have the tedious free version
| | pdap.io | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Mailchimp account currently on pdap.io is an account we don’t have access to, so subscription button has been stripped out
  • Josh Lintag will be working on replacing that or adjust it to be something different pending conversations
| | Volunteer | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |

  • Getting Greetbot updated in Slack again

    • Some minor blockers around getting that set up again that have been unblocked
  • Close to being ready around pointing folks over to the new documentation
  • Primary goal has been to be able to tell 1 story for initial volunteer contributor
| | Skillsets around DB/scraping facet | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • We need to pull in folks that have those skillsets as it is critical to current/upcoming work
  • DB choices/architecture is needed
| ## Action items * Signed up with Rob Brand (agent in DE) and Dana with DE Alliance for Non-Profit Advancement [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * Change address on Google Workspace [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Setup board meeting to officialize change of address and update documents [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) ## Decisions * # 2021-01-27 Leadership Cadence Notes ## Date 27 Jan 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Goals * Removal of inactive accounts on Google Workspace * Google drive volunteer access * Site subscriptions * Address change * Logos ## Discussion topics | Item | Presenter | Notes | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Logos | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Currently exchanging information with a trademark lawyer through A\&P
| | Registered Agent | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Haven’t heard from Rob Brand yet; will be following up
| | DANA | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Will be having a meeting with DANA with our opening pitch; meet and greet
| | Removal of inactive accounts in Google Workspace | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Removed Garrett’s account to minimize costs
  • Moved over any data from that account to Josh Lintag 's account for posterity
| | Google Drive accesses | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Work with Former user (Deleted) on ideal behavior of documentation access
  • We may build out a Data Classification doc to keep track of what levels of access certain types of data will have
| | Address change | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Pending Rob Brand’s registration of Change of Agent form; officialize by board decision last Saturday
| | Site subscriptions | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Now pointing to correct MailChimp account
| | Domain change for google workspace accounts | | | | Documentation and volunteer flow | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |
  • Will Beaumont quality testing through volunteer flow

  • Front page development changes that hasn’t been pushed yet.

    • Work through it in working session
  • Still need a golden sample of a scraper
| | Finance | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Bank accounts setup, can apply financing to our accounts
  • We’ll be reimbursing folks
| ## Action items * Reach out to existing scrapers (UCR peeps) and get samples of scraper code two add to list of examples on main repo [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Reach out to [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) around finance stuff [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) ## Decisions * Hour change of cadence to be an hour earlier # 2021-02-03 Leadership Cadence Notes ## Date 03 Feb 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Matthew Pierson](https://pdap.atlassian.net/wiki/people/5f7e4b14021acf0076f6cceb?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Goals * Change of Registered Agent * Google Workspace invoices ## Discussion topics | Item | Presenter | Notes | | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Workspace invoices | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) | | | Registered Agent | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Pending some paperwork on Rob Brand’s end
| | Trademark logos | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Still waiting on Paul
| | Fundraising on website | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Eddie Brown (Unlicensed) looking at options, no real blockers in terms of corporation constraints
  • Paypal seems simplest; we’re considering costs and work needed to drop the button on the site
  • Reach out to Kristin to see what the social media blitz will look like and prep for it
| | Website copy | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |
  • Check over the site and determine what needs to stay in/drop from the site
| | Infra work | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Still working on skilling up, will be using some available, free resources to figure out how to deploy all this stuff
| | Law data ecosystem (like case data, data consumed by law firms ) | [Matthew Pierson](https://pdap.atlassian.net/wiki/people/5f7e4b14021acf0076f6cceb?ref=confluence) |
  • Existing data providers have a couple of standard sources they pull their data from, aggregate it and resell it

  • Will generally source and buy data instead of scraping it themselves; there’s no guarantee of thoroughness

    • We could eventually be those people they buy data from; just a thought, we’ll put a pin on it
| | Generate milestones | |

  • What do we have left to get “started”?

    • We’re generally great to go as an initial push but it’s not scalable
  • We need a list from business, product, and tech
| ## Action items * Enroll in PayPal [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * Create initial epics and stories for technical work to determine FOC (full operational capability) [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence)[Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * Get scraper samples up somewhere [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence)[Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Decisions * # 2021-02-10 Leadership Cadence Notes ## Date 10 Feb 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Goals * Chugging along infra * Scraper hub info ## Discussion topics | Item | Presenter | Notes | | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Registered Agent | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |

  • Still pending some paperwork on Rob Brand’s end

    • Rob is waiting on …
| | Trademark logos | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Still pending on Paul, Eddie to follow up
| | Fundraising on website + finances | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Eddie Brown (Unlicensed) created a Paypal account, waiting on Paypal tot get back regarding non profit discount
  • Add additional account holders to Paypal account
  • Haven’t gotten responses from Kristin nor initial donor

  • Will be working with garnering some donations

    • DANA
    • Get resources from contacts, LinkedIn
    • Non-profits that help other non-profits

    • Grants

  • As soon as we get non-profit status, donations tax-deductibility can potentially be backfilled. Need additional research to look into the details

    • Paypal can autogenerate receipts
| | Website copy | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Former user (Deleted) reaching out to Miles to get context on foundational questions around website code
  • Working through copy
| | Infra work | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Working on initial subtask for building out HDFS
  • Continue fleshing out stories to facilitate forecasting
| | Scraper hub | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • In talks with representatives around engineering, non profit perks
| | [policedap.org](http://policedap.org) domain | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |
  • Following up with Dave around domain transfer
| | Crypto | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Rolling out our own crypto could potentially be useful

    • Use cases include security, tracking
  • Possibly reward contributors
  • Possibly professionally motivate
  • Shouldn’t raise the bar of participation
  • Crypto contracts to automate transactions with volunteers
| ## Action items * Reach out to IBM about blockchain implementation/technologies and look at other blockchain providers [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) to work on blockchain market surveys * ## Decisions * # 2021-02-17 Leadership Cadence Notes ## Date 17 Feb 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Goals * HDFS update * Prod website update * Trademark * Crypto ## Discussion topics | Item | Presenter | Notes | | ----------------------------------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | HDFS update | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • WIP, adding documentation and tests to harden the work
  • Pivot from this work to work on website
| | Website | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Copy changes up to production website
  • We have evidence of people looking through it and subscribing to the mailing list
  • Will require PayPal button for donations, should be simplistic
| | Trademark | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Good progress, Eddie has been working with A\&P
  • Costs to trademark logo/mission will be \~ $500
| | Paypal account | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Josh and Eddie have full management access
  • Working with PayPal around non-profit discounts
| | Change of address/ registered agent | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Still waiting on the state to get back to Rob in order for us to move forward
  • We’ll enter a contract with him as soon as that happens
| | Donors | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Kristin will be reaching out to promising donors
  • Buffed up slide deck will be very helpful
| | DANA | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Will be getting in contact with DANA to ask about core services
| | Technical expertise/blockchain | [Alec Akin (Unlicensed)](https://pdap.atlassian.net/wiki/people/5f1e64ee2aa25000286fc7fc?ref=confluence) |

  • Connections with a mentor, possible avenue for expertise

    • Blockchain tech resource

  • Eddie working through blockchain resources, will layout some options on direction

    • Let’s hammer down some constraints to be able to narrow focus and direct efforts to see what we want this solution to look like
| | Copy work/documentation | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |
  • Reddit wiki pointing to one place
  • Working on the slide deck
  • Possible rework with the website in order to make it more accessible to an open source treatment
| ## Action items * ## Decisions * # 2021-02-24 Leadership Cadence Notes ## Date 24 Feb 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Goals * PDAP CEO staffing ## Discussion topics | Item | Presenter | Notes | | ------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | CEO staffing | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |

  • Got about 3 dozen applicants via LinkedIn

    • Will take some time to filter through
| | Infrastructure | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Commitment on fixing up an HDFS cluster this weekend Josh Lintag
| | Potential incubator | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • We have an opportunity to participate with an incubator, details pending
| | Website | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) | | | Trademark | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Potential costs are steep, we’re planning on delaying the need for the trademark. Eddie to follow up on that
| ## Action items * ## Decisions * # 2021-03-03 Leadership Cadence Notes ## Date 03 Mar 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) ## Goals * Funding ## Discussion topics | Item | Presenter | Notes | | ----------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | CEO staffing | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Going through CEO applications, will need to start getting feedback on individuals
  • We’ve already rejected multiple applications
| | Funding | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Eddie reaching out to kin for helping with someone that has experience fundraising previously
  • We’ll seek out federal grants and try to meet this year’s deadline but may miss it (most close in March)
| | Billable dev team | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Reaching out to contact, pending conversation on what statement of work will look like for various amounts

  • Contact has previous experience doing the work we’re wanting

    • Includes potential dev and hosting costs
| | HDFS cluster | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • It’s up there, may require some debugging to hammer out kinks
| | Documentation + Website | [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) |
  • Focusing on finalizing docs and volunteer flows
  • Looking at website to determine a better direction in terms of maintenance and data-driven (CMS)
| | | | | ## Action items * Reach out to M+R [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Do the friends and family tour [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) to reach out to more family members (contact at NPR) * Reach out to LaunchCode [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) ## Decisions * # 2021-03-10 Leadership Cadence Notes ## Date 10 Mar 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Goals * ## Discussion topics | Item | Presenter | Notes | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Title changes | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |

  • Learnings from DANA

    • Titles generally are \*Directors i.e. Executive Director vs CEO
| | Registered Agent | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Rob has us enqueued for our taxes, so that should be good to go
| | LaunchCode | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Following up on opportunities with LaunchCode, potentially partner with inmates/former inmates
| | Axon | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Have pending meetings with higher ups with Axon
| | Dev team | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Likely need to own our own Azure environment for them to work out of
  • .Net Core tech stack
  • Potentially have a draft of SOW by end of the week
| | Marketing/News |

Eddie Brown (Unlicensed)

Alec Akin

|
  • Alec has a connection that is interested in writing up a story for us
  • Kristin can do a media blitz as soon as we’re ready
| | Data validation | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Reached out to volunteer scraper of the Washington state UCR data and filling out the metadata for the scraper code and data
| | M+R reach out | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • First intros made, pending conversations
| | Recovering accounts | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Alec following up with Dave still about policedap.org

    • Have existing documents that direct folks to policedap.org which is problematic
| | Website | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |

  • Josh C. generally overlooking work around the website

    • Potentially shifting away from Gatsby
    • There’s a few things that may require updates
| | Donation design considerations | [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) |
  • Current flow to donations are not as intuitive as it could be
  • Zach’s gonna take a stab at it
| | Grantwriting update | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) |

  • Richard reached out to Benjamin

    • He’s very interested and will be looking into it
| ## Action items * ## Decisions * Changing titles to include \* Director vs Chief \* # 2021-03-16 Leadership Cadence Notes ## Date 16 Mar 2021 ## Participants * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) * Dawnielle Robinson * Kristin Trynski ## Goals * PDAP Reorganization * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) stepping back * Extend core volunteers - tech talent * Open up Jira/Confluence? * Immediate needs how to cultivate * `tools to facilitate mass data collection by volunteers` * `tools to help us process and store data` ## Discussion topics | Item | Presenter | Notes | | ------------------ | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Executive Director | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |
  • Dedicated position to help raise funds

  • Interviews start tomorrow

    • 1 no; 7 interviews setup
| | Consultation fees | [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) |

  • Two folks we’d want to consult with eventually, financially limited atm

    • Fundraising mentoring
    • Board mentoring
| | PDAP reorg | [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) |
  • Josh Lintag still on the board but will be relinquishing most current responsibilities. Will be around for the most part to help with transitioning and knowledge transfer

  • Extend core volunteers. Former user (Deleted) & Josh Lintag to compare volunteer notes

    • Dolthub could ameliorate the need for more core volunteers. Alec Akin to follow up. Validation work could also be offloaded to volunteers
  • We will be opening up Jira/Confluence to facilitate work for the public pending we address immediate concerns around data submission
| | Donations | Kristin |
  • Determine something a bit better than PayPal to make visible folks that donate

  • Look into options like Stripe or similar solutions to GoFundMe

    • Lower the friction around donating
  • Visual of donation goals and how much left to reach those goals
| | Uncle Jeff | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Pending questions in email chain that need to be answered
| ## Action items * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) to follow up with Dolthub and determine if a partnership is possible * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) to deploy Jira/Confluence standard plan * [Zach Goulson](https://pdap.atlassian.net/wiki/people/5f1f8319ef11df0025869e21?ref=confluence) update front-page site to latest versions ## Decisions * # 2021-03-27 database working session ## Date 27 Mar 2021 ## Participants * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) * Mitch Miller * [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) * Kristin Tynski * Jeff Jockisch * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) ## Goals ## Discussion topics | Item | Notes | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

dolt

  • Review the DoltHub proof of concept for dataset catalogue

    • does not yet include sources from @stabs or form submissions
    • still need to remove the form and point people to dolthub, if this proof of concept looks good
|
  • why is dolt better than a daily backup?
  • lowers barrier to entry → can work without much automation or maintenance.
  • documents chain of custody.
  • overhead—how much time do people spend managing commits and PRs vs. actually scraping?
  • ease of use for contributors is a primary objective if we’re going to scale
  • scale—we’re building a unique database of databases in addition to scraped data
  • free
  • dolt can be used to identify & track sources → the translated data could be sent to
  • bounties only work for data, not scrapers/infrastructure.
  • bounties allow people to contribute with minimal social engagement.
  • Using Dolt as a POC for storing data in a version controlled fashion
  • How do we scale this?
  • Resolving version control may slow things down
  • Automatic scrapers pushing into a branch to merge to master becomes a bottleneck; not applicable atm
  • Edit history is tracked
  • Why dolt instead of daily backups? Or our own history tables?
  • Dolt can be used as a little more than an audit log for data sources
| |
  • Do we need a global table for unique\_data\_properties → we make SQL INDEXes of that based on data\_types? Or are we better off creating tables for each data\_type? How do we decide?
|
  • We should identify which data points are most consistently available across data sources → this gives us targets to hit.

  • metadata fields always need to be added in data discovery

    • parent child relationship between types and properties
    • the query will be ugly but this is how it’s best designed for large code

  • mongodb may work as a metadata repository

    • we’d need something to talk to both
| |

  • Define the next steps for keeping structure parity between Datasets, Data, and Scrapers

    • How should these structures relate to one another?
|
  • Scrapers need a config file for where they’re sending data
  • Dolt repos may need a path back to the scraper
  • dolthub ≠ where people provide scraper code
  • Tiered approach for data properties: 1. NIBRS 2. Store CSV…anything in between?
  • We can decide which tiers we’re actively ingesting based on how often they’re available.
| | Enterprise features / structure down the road |
  • Risk: github or dolthub fails
  • We’ll want a backup DB, then an API.
  • Sooner rather than later we need to back up our data for better disaster recovery.
| | Documentation: read the docs vs confluence |
  • Richard is writing documentation for backend design & workflows
  • it’s in markdown
  • Confluence is a stopgap for collaborating and getting on the same page
  • What does implementation look like?

    • Does it work for open source?
| | People bringing drop-in scrapers |
  • Scrapers (the humans) often bring their own scraper library with them → we should enable them to slot in to the extent it’s possible

  • Messaging scrapers: what are we targeting?

    • We need to make sure we’re clear about the breadth / scope of data.
  • Fragmenting the target makes it hard for us to progress. We want to focus on NIBRS format data to validate it against what’s available from the FBI & broaden context
| | data to not collect |

At one point this was our list, is it accurate?

  • CaseNum
  • FirstName
  • MiddleName
  • LastName
  • DOB
  • DefenseAttorney
  • PublicDefender
  • Judge
  • ArrestingOfficer
  • ArrestingOfficerBadgeNumber

Consider: not collecting data we don’t know is legal to have.

  • the government made the data public. If it’s public, it’s not considered personal.
  • We need to be careful which source the data is coming from. If it’s not a direct source, we may be subject to different restrictions. Will third party aggregators have
  • With the bounty program we can mandate that they include certain proofs in their submission

Whenever we decide on a property to collect, we need to justify it / provide rationale for the decision

| | Are we collecting data aggregated by third parties? |
  • Each aggregator has their own format
  • Do we want to throw it away?
  • Discrepancies in the data aren’t necessarily “red flags”; agency’s maybe aggregating according to different criteria so there maybe discrepancies according to that unknown criteria
  • We do want aggregate level data for validating the record level data that is being returned to us
  • For now these are stored as source\_type = “third\_party”
| | Data types to prioritize |
  • arrest reports
  • traffic stops
  • incident reports
| | Scraper approval |
  • Multi-tier approval is possible with github
  • How does wikipedia do it?
  • Base scraper approval is most urgent
| | ETL framework |
  • Should extraction / ETL be a required part of a scraper?
  • Mitch Miller is using python to create a framework which should be flexible enough to meet our needs
  • ETL should not go directly into the scraper, but it does need to be closely related.
| ## Action items * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) find out how much overhead (time) is involved in an end to end dolt scrape → public * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) add data format (NIBRS) column to dataset catalogue * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) Identify minimum data properties to meet NIBRS data format, has this somewhere already * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) Document data\_types priority [in scrapers readme](https://github.com/Police-Data-Accessibility-Project/Scrapers/pull/43). Arrest Reports, Traffic Stops, Incident reports. What’s most available. What most easily paints the full picture. Tiers. * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) validate workflow: **localized, raw data is stored in dolt → it could be aggregated / ETL’d to a centralized server.** dolt is the audit/transparency. * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) Expose basic roadmap in documentation with backup DB → API * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Draft a policy and rationale for “fields not to collect” → **A\&P** * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Draft a policy and rationale for mirroring dataset websites → **A\&P** * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) import dataset catalogue [form submissions](https://docs.google.com/spreadsheets/d/176f0pTxlIyvBWqXmJCPPlh2zRrcMZNgojB2fJOBBhGw/edit#gid=901781374) and deprecate form * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) Get @stabs [base scraper](https://github.com/Police-Data-Accessibility-Project/Scrapers/pull/38/files) approved **Mitch Miller** is making an ETL framework ## Decisions * @stabs need to be recognized. Let’s be sure to celebrate their hard work # 2021-03-31 ## Date 31 Mar 2021 ## Participants * [Former user (Deleted)](https://pdap.atlassian.net/wiki/people/5f8f95be40588b0077ed830a?ref=confluence) * [Eddie Brown (Unlicensed)](https://pdap.atlassian.net/wiki/people/5fd63e354d2179006ecbcb80?ref=confluence) * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Discussion topics | Item | Notes | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Dolt |
  • works as POC
  • requires scraper
  • Richard is waiting for a bug fix from dolt and will merge his branch after that
| | Scrapers |

  • Former user (Deleted) add “convert data to a known type or create a new type” to scraper instructions

    • This is a good amount of work because all would need to be kept up to date

  • Former user (Deleted)We should use github to store the DDL scripts and reference it in dolthub, should make a folder for them

    • people would reference these → their columns would be kept up to date
  • Richard Ji basic Python style guide for scrapers
| |

What data to collect

|
  • We need a decision / definition of “what we really want”

  • The decision from last meeting is something like

    • Accept all data that is up and out publicly
    • Tier and sort and prioritize data based on good or consistent formatting, but accept all legal public data. Omission is not a good start to our process if our goal is “a source of truth for police data.”
    • Only surface PII
    • Classification should not be a barrier—we only need to classify what is
  • Former user (Deleted) draft a policy → publish
| | Business / professional things |
  • Eddie working with Denice Ross on New Jersey data that’s not public. Someone needs to explain to Eddie what’s going on with New Jersey data → he can use that as more specific rationale for more investigation
  • Eddie ensuring our taxes are paid.
| ## Action items * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) to dig deeper on what blockers were for getting New Jersey data + communicate to Eddie * [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) basic python style guide for beginner scrapers # 2020-12-1 {% file src="/files/-MbWyFOvmuUetAhLiD96" %} # 2021-01-23 {% file src="/files/-MbWzcetEX1wPRSvDgC7" %} # 2021-04-10 Meeting notes ## Date 10 Apr 2021 ## Participants * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) * Eddie Brown * Alec Akin * Eric Turner * Stabs ## Goals * Answer schema questions to unblock [dolt implementation](https://pdap.atlassian.net/browse/PDAP-60). * Decide whether tableplus or DBeaver would be useful in the dolt pipeline ([discussion here](https://policeaccessibility.slack.com/archives/C014Q3ZT2GG/p1617822153049800)). ## Discussion topics | Item | Notes | | ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Datasets |
  • UUIDs have been added in dolt
  • may want to remove hyphens to save space (currently mysql built in)
  • Once things are more stable it’s probably worth doing some views

  • Eric wrote a script to keep CityProtect datasets up to date as well as commit the dataset to the dolt db

    • Fork then merge is a necessity
    • We could put these on a chron job or just manually one them—once for each portal type
    • These update quarterly, so automating that is not our biggest problem.
    • 355 agencies in cityprotect for bulk downloading, avg 10 CSV files
  • We’ll need to get data out of datasets
  • Clones and forks become problematic just to make simple additions
| | Dolt x Scale |

Alternative for down the road:

Risk: dolt is young and missing advanced SQL features. Dolt is not our advanced analytics tool. Dolt is not our deep storage layer.

Databases can’t currently reference each other, and repos are 1:1 with databases.

Data is unlimited but current technical cap is \~200gb

Can Dolt be an intake tool and not grow?

This may not even be our presentation layer because of the low amount of data it can store

  • one way around this might be checking data out of one dolt repo into another, preserving paper trail. does dolt support this / is it reasonable?
  • Otherwise priority ↑ for
| | Where are we storing our data properties? |

Is breadth an issue in Dolt? Would this table be prohibitively gigantic?

SQL column limit \~1000, mysql 4000. we should be good.

| | Hosting | If we went with mongodb directly we could bolt it into our DigitalOcean | | Tableplus / DBeaver |

context

Tableplus free has some limitations (open tabs)

| | Scrapers | For now—people should write whatever they want / is most convenient for them. If we want to refactor, we can figure out what the priority is on it + do it later. | | Server access | We don’t use passwords, just keys—send it to Alec if you need access. | | OCR | Stabs tried \~5 different ones and none of them worked well enough. Alec’s tried tensorflow | ## Action items * proof of concept for separate dolt DB (check whether DBs can easily reference one another) [![](https://pdap.atlassian.net/secure/viewavatar?size=medium\&avatarId=10318\&avatarType=issuetype)PDAP-144](https://pdap.atlassian.net/browse/PDAP-144) Done * proof of concept for `global_properties` table referenced by data\_types views [PDAP-121](https://pdap.atlassian.net/browse/PDAP-121?src=confmacro) - JIRA project doesn't exist or you don't have permission to view it. * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) to make request in dolthub discord for one:many repos:dbs * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) reach out to 3 major cloud providers + mongodb for sponsorships * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Point database volunteers in the direction of [Dolt SQL viewer integrations](https://github.com/dolthub/docs/tree/gitbook-dev/content/integrations) * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) add scraper philosophies to readme (do what you want, we can always refactor later) ## Decisions * Dolt is not our advanced analytics tool. Dolt is not our deep storage layer. # 2021-04-17 Meeting notes ## 44RDCXDSDDDate 17 Apr 2021 ## Participants Mitch Miller [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) Jeff Joskisch [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Discussion topics | Item | Notes | | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Dolt / Databases |

  • Lots of changes being made in datasets

    • agencies
  • Does Dolt support SQL COPY
  • Richard has MongoDB creds for anyone who would like to experiment
  • How slow is Dolt? Breakingly? We’re going to have \~2 million records.
  • We can host a mirror on our server and run an instance if people want the same data quicker / without the dolt UI
| |

Podcast fame

|

Jeff was interviewed on Privacy Please releasing this week (wednesday) and mentioned PDAP

| | OCR |

Tensorflow has been suggested

We may need a lot of training data, which we don’t have.

  • Is there a way we could slowly start to feed this stuff to tensorflow now?

Requirements:

  • We need to be able to comma delimit things on the way in

There’s no harm in the meantime with publishing unedited PDFs

  • may inspire contributors to help with OCR
| | FE |

There are some folks ready to work on stuff for when we have data

For now it’s pretty small and people could clone it and run it locally

Miles is converting gatsby to JSX which will make iteration easier

| | mongodb | We should have a template if we’re using Mongo | | Docker compose file | Mitch is working on a docker compose file for dolt and mongo, which will be helpful as we get our ETL framework together | ## Action items * [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) to ping Mitch in slack when sql-server POC is done (nearly) * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) policy / rationale for PII → docs (this is a high priority) * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) make shitty base tables from examples of other data types * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Do meeting notes in Docs next time so they can be shared # 2021-04-21 Leadership Cadence ## Date 21 Apr 2021 ## Participants [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) @Eddie Brown [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Discussion topics | **Item** | **Person** | **Notes** | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Tax Filing | @Eddie Brown |
  • Josh Lintag Completed registered agent duties and tax documents are filed
| | Fundraising Update |

@Eddie Brown

Alec Akin

|

  • Alec Akin and Eddie are looking for fundraising opportunities

    • Josh Chamberlain What is the relationship/experience with people approached for fundraising?

      • People say yes initially, but don’t follow through with promise

  • Alec Akin sign up for a Patreaon

    • Merch?

      • This is a big time commitment to create a top level experience
    • Name levels of support and recognize on website
| | Executive Director Search | @Eddie Brown |
  • Michael: No ED, but fundraising role
  • Stephanie: Awaiting her response on acceptance

  • Challenges

    • Getting people to commit to the board: Are we targeting the right people (Alec Akin is this right for your thoughts?) Structure/Size
    • Is there a misunderstanding about our needs and a “typical” board seat
    • Club House: Is this an opportunity to drive volunteerism? Visibility for PDAP.
| | Volunteer Cycle | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |
  • Website looks good
  • Conversations with other Volunteers is great

  • Look for data (which is missing)

    • People fall off at this point
    • Our critical shortfall
    • Damages trust with our organization
    • In progress and we are close!
| | Tech Update | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • Eric Turner Has made significant contributions and are advancing our efforts.

    • We have several data sets: counting both agencies scraped and data sets
    • - Map proof of concept of our current data

  • @miles Has made major FE contributions

    • Simplified for copy additions using Markdown
  • We have PDAPed (new verb)!
| | Volunteer Appreciation | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |

  • Can we add this to the site to provide recognition?

    • Funding
    • Volunteer contributions
  • Github is a great way to provide recognition and legitimacy for open-source
| | Outside Organization Recognition |

Alec Akin

Will Beaumont

|

  • Tech soup

    • Need 501(c) status
    • Gives access to Tech Soup resources

  • Amazon

    • Allows employees to see group in volunteer application

      • Can donate money and time
| | Documentation | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • Finalizing descriptions of our tools and process

    • Kanban to Github is how most open-source work

      • Ability to prioritize/re-prioritize publicly is important
    • Working to make available to the public
  • Would like Alec Akin to check verbiage

  • Alec AkinandRichard Ji working on our DB security posture/architecture

    • Risk analysis/risk matrix - what are our biggest threat?

      • Focus on board
      • Other groups
| | Treasury | @Eddie Brown |
  • We have the funds to cover our startup costs
| ## Action items * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Reach out to Patrick about fundraising * @Eddie Brown Circle back with Executive Director position * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Ask Kristin about Club House * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) List organization in Amazon employee volunteer portal * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Status update to @kristin and product roadmap to public * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Exploring open-source Kanban (Github, others…) # 2021-04-28 Leadership Cadence ## Date 28 Apr 2021 ## Participants [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Past Action items * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Status update to @kristin and product roadmap to public * [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) Exploring open-source Kanban (Github, others…) * @Eddie Brown Circle back with Executive Director position ## Discussion topics | **Item** | **Person** | **Notes** | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Kanban | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |
  • No great solutions for open source
  • Migrating to Github; It meets our needs, especially with the level of detail we need for new volunteers
| | Gitbook | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |
  • We have open source approval and will not need to pay for Github
| | Digital Ocean | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Awarded $1000 in funds for PDAP
| | Scrapers/Product | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • New data\_type table name to allow intake (base table)

    • Arrest reports, etc.
  • Eric Turner is close to having a working data schema; closer to saying yes to data
  • Helping gather databases and organizing what that we need to scrape
| | Club House | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) | | | Other Help/Funding source | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Paperwork from IRS is required to go forward
| | Security |

Alec Akin

Richard Ji

|

  • Secrets management from Keeper

    • PK management
    • Good enterprise solution/easy to use
  • Working to apply for wildcard cert for website security badge

  • VPN: Very important to maintain security as our group grows. It is another layer of security.

    • Wireguard VPN: high technical barrier

    • Wirespeed VPN: $5/cpu/month

      • working through limitations with our 2 data centers: 1 NY/ 1 SFO

        • currently we require 2 VPN for each data center
      • alternative is to migrate data to one center for simplicity
  • Looking to grow security team
| | Database Update | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) |

  • Postgre SQL database setup sent to Eric Turner

    • Support
    • Track records
    • Stable
    • Dolt had blocks and we have moved away
  • Interested in modeling

  • Looking into containerization for scrapers

    • A way for us to allow others to donate computing resources to extract data using scrapers
    • Makes it more difficult to block us
| | Executive Director | @Eddie Brown |
  • No response from candidates
  • Moving on for the time being
| | A\&P | @Eddie Brown |
  • A\&P charged $900 for non-profit incorperation
| ## Action items * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) List organization in Amazon employee volunteer portal * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Continue to engage w/ Patrick about fundraising * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Club House feedback from Kristin # 2021-05-05 Leadership Cadence ## Date 05 May 2021 ## 05 May 2021 Participants [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) @eddie brown [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Past Action items * [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) List organization in Amazon employee volunteer portal * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Continue to engage w/ Patrick about fundraising * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Club House feedback from Kristin ## Discussion topics | **Item** | **Person** | **Notes** | | -------------------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Registered Agent | @Eddie Brown |
  • Contacted to verify that we have met the filing deadline
  • Thinks we have not filed
| | Executive Director | @Eddie Brown |
  • Stephanie Cord-Miller is being considered
  • Wants to meet further with the board to understand our capability
| | Roadmap Update | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • Get data to the place were it can be shared

    • Define the scheme

  • Dolt proof of concept complete (blue star)

    • We need PostgreSQL

      • Alec Akin digital ocean has an option for hosting PostgreSQL

        • Richard Jimost interested in newest version, digital ocean offers a past version for stability
  • Scrapers want to scrape and not deal with data admin

  • Alec Akinand Josh Lintag: What does it take to publish a proof of concept

    • Focus on getting bare bones up
    • Server to accept bulk data files

  • Proof of concept for data mirror, local scraping, project status view available

    • Need to be be careful because chain of custody is not available in these demos

    • Should we wait for VPN?

      • Alec Akinwe are ok to continue, SSH key probably provides adequate protection right now
      • Richard Jiprobably ok, we should run a server update soon to reduce exposure
  • Migrating from Jira to Github
| | Amazon employee volunteer portal | [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) |
  • Waiting for IRS to complete to avoid rejection by Amazon audit team
| ## Action items * [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence)VPN/firewall setup * @Eddie Brown Need PDAP accounting for service # 2021-05-12 Leadership Cadence ## Date 12 May 2021 ## 12 May 2021 Participants @Stephanie Kord Miller [Josh Lintag](https://pdap.atlassian.net/wiki/people/5f20c61fc9c094001c5d32ca?ref=confluence) [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) @eddie brown [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Discussion topics | **Item** | **Person** | **Notes** | | ---------------------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Stephanie Kord Miller | @skm |
  • Small/med product “try before you buy”

  • Strategic planning

    • Needs

      • Org structure

    • Roadmap - get the shared vision to paper for next 12 mo

      • Tasks

        • Go to market
        • Where do we want to target
        • Drive volunteer-ship
        • Who do we hire?
        • Where to go?
      • Organization

      • Revenue Strategy

        • Fundraising is a piece

      • Budget

        • Eddie - this is what kills us/drives back

  • First steps/project ideas

    • How do we get money?

      • Use Kristin’s agency
      • What do you do when you aren’t a 501c3?

    • How much is needed?

      • Initial thought

        • 20k/mo for 2x ppl
        • 10k/mo for data bounties
| | Challenges w/ waiting on IRS | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • There is a space to cross when waiting for non-profit status to get…what can we do in the interim
  • Holding up progress get tech licenses (need official status)

  • @skm: have we checked partnering with other 501(c)3 to provide resources?

    • We were concerned about signing up for something we didn’t understand

      • @skm Who would be the right partners?
| | Cliff notes on past work | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • Formalizing how we interface with the community

    • 2-sided market: data miners (scrapers), data users (researchers)

    • Decentralized nature of work is part of the work

      • People show up for a few weeks and then drop off--how do we harness?
    • Petrean or lots of small dollar donations

  • @skm

    • Identify the audiences you are serving

      • Engineers/technologists = volunteer workforce

        • How do we grow, engage, entertain
        • Grow our volunteer to speed product roadmap velocity

      • Journalists/research/foundation = Consumers

        • What does partnership look like?
        • How do we monetize (we need to make money--people are doing it)?
        • Corporate sponsorship? PR benefits, data access
        • How much do you get for free?
        • Allow people to pay for a build/state of data?
        • University relationships w/CS departments
    • Build amplification, awareness, audience
    • Fundraising is the fastest revenue - understanding this is important for our audience and our brand
| | Challenges | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) |

  • Concern is over making a strong foundation to build on

    • We need to be ready for people to show up

  • MVP

    • Want to move fast to get toward foundation
| | Next steps | @skm |

  • Spend 2 days

    • Want’s to consider fundraising questions vs larger revenue model
    • Does the investment look/feel right?
| | Ben Email Address | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) |
  • Official email for legitimacy when applying for grants
  • Granted by Josh Lintag
| ## Action items * @skm submit proposal 17 May 2021 to consider fundraising questions vs larger revenue model; discuss 19 May 2021 # 2021-05-19 Leadership Cadence ## Date 19 May 2021 ## 19 May 2021 Participants [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) @eddie brown [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Discussion topics | **Item** | **Person** | **Notes** | | --------------------------------------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Fundraising questions vs larger revenue model | @skm |
  • Needs more time for topic
| | Podcast “Why don’t we know” | @eddie |
  • Recording Thursday morning 27MAY
| | Trusted agent | @eddie |
  • Taxes has been filed
| | Patreon | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • Setup in Josh’s name to get started

    • Looking to setup into PDAP account proper
    • Need backup person for redundancy
| | General Feedback | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |
  • Too many insider terms
  • Why are we doing it is more important than the how (technical details)

  • Why is the data important!!

    • Why is information power
| | Other success/ideas on how to model ourselves | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • During development we should consider focusing down to sharpen our project

    • Focus on inward growth?
    • Show how powerful it can be in one community
  • Next step grow to larger goals
  • Consider starting with Minneapolis? 26 May 2021 might be possible to achieve
| | Dolt Hub/database | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |

  • We are a collection of different data - dolt is not necessarily build for this

    • Solution is to try and achieve our needs in the fewest number of data sets

    • Focus on single agency before growing

      • Try not to spend all of our $dolt on one agency
| ## Action items * Need backup account for Patreon [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) # 2021-05-26 Leadership Cadence ## Date 26 May 2021 ## 26 May 2021 Participants [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) @eddie brown [Will Beaumont](https://pdap.atlassian.net/wiki/people/5e9c6021ca2a1d0c2e249bab?ref=confluence) ## Discussion topics | **Item** | **Person** | **Notes** | | -------------------------- | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Registered agent | @eddie |
  • IRS kicked back form
  • We got credit for doing/in the process of returning
| | Exec Director | @eddie |
  • No updates, will check in this week
| | Security Update | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) | | | Update | [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) | | | Update | [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) |
  • Dolt has been easy to work w/

  • Verify patreon is up to speed

    • Testing interface/copy
  • Minneapolis test case underway to showcase PDAP
| | Update | [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) |

  • ETL Library

    • Lookup agency, data type id

    • Working on pipeline

      • Req for builds to pass in order to approve

  • Dolt - July bounty expires

    • Only Katie can do PRs

    • No schema changes--bounty req

      • Need to create a branch until bounty is over
      • Eric Turner notifying #tech\_general
| | Podcast “Why dont we know” | @eddie |
  • Podcast theme accidental secrets
  • Recording 27 May 2021
| | Treasury | @eddie |
  • Balance \~$300
| | Social Time | [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) |
  • Do we want to be deliberate about team building

  • Varies with individuals

    • Some people are open to the idea
    • Be conscious of time constraints
  • Will allow space outside of meetings using hangout room
| ## Action items * [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) push no PR policy for Dolt bounty to #tech\_general # 2021-06-02 Leadership Cadence ## Date 02 Jun 2021 ## 02 Jun 2021 Participants [Josh Chamberlain](https://pdap.atlassian.net/wiki/people/6068f9e790e3950069fbaaf4?ref=confluence) [Eric Turner](https://pdap.atlassian.net/wiki/people/6069da262b469c007014d7fa?ref=confluence) [Alec Akin](https://pdap.atlassian.net/wiki/people/60319bf02a42cc0069af9ac8?ref=confluence) Eddie Brown [Richard Ji](https://pdap.atlassian.net/wiki/people/5f8f95be0e068b00766b6903?ref=confluence) ## Discussion topics | **Item** | **Person** | **Notes** | | -------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Slack v Discord v Teams??? | Josh |

Discord is approachable for the crowd we are trying to work with

Discord has drop in audio and video rooms

Teams has lots of security / features

Patreon x Discord are tightly connected

Threads is the million dollar question

People want fewer slack channels

We’re going to migrate responsibly in the near future, creating announcements and removing channels to help guide people to Discord.

We get to start with a clean membership slate.

| | Risk matrix | Alec | Will be putting in work on these now, working with Richard to talk about roles. | | Securing environments | Alec | | | DoltHub bounty | Eric |

Dolt team has been doing an amazing job supporting us

First PRs submitted

| | Podcast | Various |

One idea: recording a conversation between PDAP team

Another idea: getting in touch with podcast hosts we like

| | Dolt → PostgreSQL | Richard | Finishing the code and dealing with edges for making a mirror of Dolt db in Postgres | | Hadoop servers | Richard |

We need a place to store files / arbitrary format storage

Looking at other tech options that may be better long term than Hadoop

| | Blog | Josh | [Exists](https://docs.pdap.io/updates/blog). Everyone is welcome to contribute. We should peer edit for each other as best practice. | | | | | ## Action items * # Decision log Create decision | Decision | Status | Stakeholders | Outcome | Due date | Owner | | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | ----- | | [Use Google Meet for video conferencing needs](https://app.gitbook.com/wiki/spaces/PDAP/pages/240943147/Use+Google+Meet+for+video+conferencing+needs) | DECIDED | | Retiring using Josh's Zoom. | 19 Dec 2020 | | | [PDAP will not directly pursue advocacy positions](https://app.gitbook.com/wiki/spaces/PDAP/pages/371097602/PDAP+will+not+directly+pursue+advocacy+positions) | DECIDED | | PDAP’s direction isn’t dictated by partisan politics | 14 Jun 2020 | | | [PDAP Domains](https://app.gitbook.com/wiki/spaces/PDAP/pages/240975878/PDAP+Domains) | DECIDED | | Buy pdap.dev and pdap.io domains for a year. We won't wait for existing domain (policedap.org) to be available and move forward with new ones. | 19 Dec 2020 | | ## Decisions Record important project decisions and communicate them with your team. Create decision # Brand assets ## General Our goal is to be welcoming, trusted, and focused. Everyone who abides by our community guidelines is encouraged to participate, and we should be friendly—but we're mostly here to get work done and advance our mission. ## Type The open-source Google Font [Public Sans](https://github.com/uswds/public-sans) is our official web font. For monospace, we can use [Source Code Pro](https://fonts.google.com/specimen/Source+Code+Pro). For print, we can use the free Google Font [Libre Franklin](https://fonts.google.com/specimen/Libre+Franklin). It's a predecessor of Public Sans, and may look more refined in print. We shouldn't use any other fonts. ## Design system [This repository](https://github.com/Police-Data-Accessibility-Project/design-system) is the starting point for building a new PDAP front end app or microservice. ## Colors ![](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MbniT2-Ph2AwP_DYs8x%2F-MbnlutQ17_4hysQG8Mu%2FScreen%20Shot%202021-06-09%20at%2010.36.55%20PM.png?alt=media\&token=60ad4d6e-13de-4794-8f45-d82da9ab0499) ```css --off-white: #fffbfa; --light-gray: #bfc0c0; --medium-gray: #515052; --gold: #d5a23c; --byzantium: #512a4f; ``` ## Logos & Marks We don't have the font used for the original logo but we have [vector](https://www.google.com/search?q=vector+vs+pixel) copies of it, so it can be infinitely scaled and converted to any color. Don't add drop shadows or outlines. ### SVG {% hint style="info" %} Infinitely scalable. Use these whenever possible, especially if you need to change the color or insert it into a print design. {% endhint %} ![logo.svg](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MgfZT-cCDrQgtjV4G2N%2F-Mgf_2DykDkw-CtZn6b8%2Flogo.svg?alt=media\&token=dc6e71cf-c0db-4016-a0a0-c939cb5c8f6a) ![acronym.svg](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MgfZT-cCDrQgtjV4G2N%2F-Mgf_4unu8XUZ2Z9JqTP%2Facronym.svg?alt=media\&token=2ff30fc8-ac64-4f50-b92c-1552c73f702a) ![lockup.svg](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MgfZT-cCDrQgtjV4G2N%2F-Mgf_76oqjkyY8lOlgo3%2Flockup.svg?alt=media\&token=47b77919-8a33-4c9d-a7b4-8a9aa635b4c2) ### PNG {% hint style="info" %} Conveniently exported to pixels. Use these for things like social media profile pictures. {% endhint %} ![logo.png (extra space added for circle crops)](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MbniT2-Ph2AwP_DYs8x%2F-MbnkFiYgEwbKzYbq2in%2Flogo-space.png?alt=media\&token=f6684875-3eab-448d-aec8-ae2141f920cc) ![acronym.png](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MkoERZeaP-3n6Z5ehrb%2F-MkoFJNEHWJAVUEj-nwt%2Facronym.png?alt=media\&token=0e4c1b32-35d7-4645-b0bc-8e2493ef39d1) ![lockup.png](https://2918691127-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MXyolqTg_voOhFyAcr-%2F-MkoDSTeYffTIClrgwVg%2F-MkoDi3VJup9zD6ESRAU%2Flockup.png?alt=media\&token=00e7bcde-4c47-47f9-b786-41d8dbc7cc22) ## Inverted logos & marks {% hint style="info" %} These are invisible on a white background, so they have been attached as files. {% endhint %} {% file src="/files/9Of2u3sVvkYSYdjEXM1t" %} {% file src="/files/Mrsgd4RB0F3KUB5Qmk93" %} {% file src="/files/8V606NuSLW0KTROJUMnO" %} {% file src="/files/gTLRbDCVgZcX6HVZlyC0" %} {% file src="/files/oafyxZuLBBvXWC7ifigy" %} {% file src="/files/fQNG9aEO5eSp0IeNT24f" %} # Legal # Terms of Service ## API Terms of Service ### 1. Application Usage These Terms apply to all access and use of our services, including through our web applications, mobile apps, and direct API access. Additional features available through our applications are subject to these same terms. ### 2. Acceptance of Terms By accessing or using this API (the “Service”), you agree to be bound by these Terms of Service (“Terms”). If you do not agree, do not use the API. ### 3. License & Usage You are granted a limited, non-exclusive, non-transferable license to access and use the API. You must comply with all applicable laws and regulations. Abuse, excessive requests, or attempts to disrupt the service may result in termination of access. ### 4. Restrictions You may not: * Use the API in a way that violates any laws or infringes upon third-party rights. * Reverse-engineer, modify, or attempt to bypass API access controls. * Resell or redistribute the API without prior written permission. * Sell, license, or commercialize access to the API or its data without prior written permission. * Create multiple accounts or use other methods to circumvent rate limits, access controls, or usage restrictions. ### 5. Data & Privacy * We collect only the information you provide during account creation (email and password) and basic usage analytics as described in our [Privacy Policy](https://docs.pdap.io/meta/operations/policy/pdap-privacy-policy). * We do not share your personal information with third parties. * You are responsible for securing any data you access through the API and complying with applicable privacy laws when using or sharing such data. ### 6. Rate Limits & Availability * The API may have rate limits, and excessive usage may result in temporary or permanent access restrictions. * We do not guarantee uninterrupted service and reserve the right to modify, suspend, or discontinue the API at any time. ### 7. Disclaimer of Warranties * The API provides links and references to public records and data maintained by various sources. We do not store, modify, or verify the underlying data. * We make no representations or warranties about the accuracy, completeness, timeliness, or reliability of any data accessed through these sources. Users are responsible for verifying the accuracy of data for their specific use cases. * The API is provided "as is" and "as available" without any warranties, express or implied. ### 8. Limitation of Liability We shall not be liable for any direct, indirect, incidental, or consequential damages arising from the use or inability to use the API. ### 9. Modifications to Terms We reserve the right to modify these Terms at any time. Continued use of the API after changes constitute acceptance of the updated Terms. ### 10. Termination We may suspend or terminate access to the API at our discretion if these Terms are violated. ### 11. Governing Law These Terms shall be governed by the laws of The Commonwealth of Pennsylvania, United States. For questions, email . # Public records access laws & precedent ### Pennsylvania * Allegheny Co. v. Hailer, B., et al 1469 CD 2021 ([docket](https://ujsportal.pacourts.us/Report/PacDocketSheet?docketNumber=1469%20CD%202021\&dnh=DsPX7RpJ4jYowwXHaZ2C0A%3D%3D), [brief](https://www.rcfp.org/wp-content/uploads/2021/06/2022-07-27-Allegheny-County-v.-Hailer-Principal-brief-of-appellants.pdf)) * Autopsy reports are public # Legal Data Scraping Data scraping for an ethical source of truth ## Questions to ask before you begin: * Is the material I intend to scrape protected by copyright? * Does the website I intend to scrape require authentication? * Will the scraping compromise individual privacy? * Will scraping reduce the value of the original data? * Am I in violation of the terms of use/service of the website I intend to scrape? * Will my scraping activity overload, damage, or otherwise adversely affect a server? **The answers to all of these questions should be “no”.** If you cannot answer the question, or are unsure, you should not proceed. Further discussion on each of these questions can be found below. **Is the material I intend to scrape protected by copyright?** Copyright, a form of intellectual property law, protects original works of authorship including literary, dramatic, musical, and artistic works, such as poetry, novels, movies, songs, computer software, and architecture. Copyright does not protect facts, ideas, systems, or methods of operation, although it may protect the way these things are expressed, for example, the creative arrangement of otherwise non-copyrightable facts. The [U.S. Copyright Office](https://www.copyright.gov/help/faq/faq-protect.html) provides additional information on the scope of copyright protection. When scraping data, you should limit scraping to only non-copyrightable facts, and should only collect parts of pages required for the purpose. **Does the website I intend to scrape require authentication?** Scraping should be strictly limited to information that is presumptively accessible to the general public, and which does not require authentication to access such as a username and password. If a website requires authentication to access data, that data should not be scraped. For further reading on this issue, the U.S. Court of Appeals for the Ninth Circuit’s 2019 decision in [hiQ Labs, Inc. v. LinkedIn Corp.](https://law.justia.com/cases/federal/appellate-courts/ca9/17-16783/17-16783-2019-09-09.html) provides a helpful analysis. **Will the scraping compromise individual privacy?** While information available in public records is public by its nature, be aware if the information being scraped reveals personal data that is generally understood to be private, such as social security numbers and bank or credit card information. This may also include personal data that can identify an individual person, such as name, email, phone number, and address. If in doubt, omit such personal data from the scope of the scrape. **Will scraping reduce the value of the original data?** Again, scraping should be strictly limited to information that is presumptively accessible to the general public. This limitation is important to ensure that original data being scraped is not diminished in value as a result of it being scraped. **Am I in violation of the terms of use/service of the website I intend to scrape?** When you login (e.g., through a username and password) and/or expressly agree to the terms of use/serve of a website, you are entering into a contract with the website owner, thereby agreeing to their rules. These rules may explicitly state that data may not be scraped from the website, and failing to adhere to such terms may put you in breach of that contract. As noted, however, your scraping should be strictly limited to information that is presumptively accessible to the general public, and which does not require authentication. [*Recent court rulings*](https://www.eff.org/deeplinks/2022/04/scraping-public-websites-still-isnt-crime-court-appeals-declares) *have confirmed the legality of web scraping regardless of Terms of Service. Ethics dictate that following Terms of Service is still best practice.* **Will my scraping activity overload, damage, or otherwise adversely affect a server?** Do not harm the website you are scraping. This includes, for example, using a reasonable crawl rate and assuring that the volume and frequency of queries you make so not burden the website’s servers or interfere with the website’s normal operations. Respect the delay that crawlers should wait between requests by following the crawl-delay directive outlined in the robots.txt file. Where possible, strive to limit scraping to a time of day when the website is unlikely to be experiencing heavy traffic, for example, early in the morning or night. All PDAP scrapers and crawlers must also abide by State and Federal computer crimes statutes, including those collected [here](/meta/operations/legal/state-computer-crimes-laws). # State Computer Crimes laws A quick guide to legal codes in each state. | State | Code | Public Access Link | Notes | | -------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | Alabama | §13A-8-110 - 19 | Alabama Digital Crime Act. | Alabama Computer Crime Act (13-8-5) was repealed in 2012 –> Was replaced with the AL Digital Crime Act Civil Suits are permitted | | Alaska | 11.46.484 | “Criminal Mischief in the Third Degree” | | | Arizona | 13-2316 | Computer tampering; venue; forfeiture; classification | No misdemeanor computer crimes but there are felony computer crimes | | Arkansas | § 5-41-103 & 5-41-104 | “Computer Fraud” // “Computer Trespass” | | | California | CA Penal Code Section 502 | California “Computer Crimes” | | | Colorado | § 18-5.5-102 | “Computer Crime” | | | Connecticut | Computer crimes: Sec. 53a-251 through 53a-256; Internet crimes: Sec. 53-451 through 53-454 | Computer crimes; Internet crimes | | | Delaware | § 935 | “Misuse of Computer System Information” | | | Florida | ss. 668.801-668.805 | | | | Georgia | Article 6 | Computer crimes | | | Hawaii | HI Rev Stat 708-895.5 to 708-895.7 | 1st Deg Unauthorized Computer Access // 2nd Degree // 3d Degree | | | Idaho | ID Code § 18-2202 (2012) | Computer Crime | | | Illinois | 720 ILCS 5/16D‑3 | “Computer Tampering” | | | Indiana | IC 35-43-1-4 | “Computer Tampering” | | | Iowa | IA Code § 716.6B | “Unauthorized Computer Access” | | | Kansas | KS Code 21-3755 & 21-5839 | Computer Crime, Computer Trespass // Unlawful Acts Concerning Computers | | | Kentucky | KY Code 434.840 - 860 | “Unlawful Access to a Computer” | No civil provision | | Louisiana | LA Rev Stat § 14:73.1 et seq | Computer Related Crime | | | Maine | Title 17-A, Chapter 18: | Computer Crimes | No civil provision | | Maryland | § 7-302 | Unauthorized access to computers and related material | | | Massachusetts | Ch. 266 §33A | Obtaining computer services by fraud or misrepresentation | No civil provision | | Michigan | 752.791, et seq. | Fraudulent Access to Computers, Computer Systems and Computer Networks | | | Minnesota | 609.87, et seq. | “Computer Crime” // Definitions // Computer Theft // Unauthorized Access // Facillitating Access | | | Mississippi | § 97-45-1 (sec 1-33) | Computer Crimes and Identity Theft | | | Missouri | 537.525, 569.095, et seq. | 537.525: Tampering –> Civil Action Provision Tampering with computer data // Tampering with computer equipment // Tampering with computer users | | | Montana | Mont. Code Ann. § 45-2-101, § 45-6-310, § 45-6-311 | Definitions // Definitions of computer use // Unlawful use of a computer | | | Nebraska | 28-1343.01 - 1348 | Computer Crimes Act | | | Nevada | NRS 205.473 - 513 | Unlawful Acts Regarding Computers and Information Services | Civil Action allowed per NRS 205.511 | | New Hampshire | 638:16 - 19 | Computer Crime | | | New Jersey | NJ Rev Stat § 2C:20-1 | Definitions // | | | New Mexico | § 30-45-1 through 30-45-7 | “computer abuse” // “unauthorized computer use” // definitions // | | | New York | Article 156 | Offenses Involving Computers | | | North Carolina | Article 60 § 14-453 - 58 | Computer-Related Crimes | | | North Dakota | CHAPTER 12.1-06.1-08 | Computer fraud - Computer crime | Civil Action permitted | | Ohio | 2913.04 | Unauthorized use of property - computer, cable, or telecommunication property | | | Oklahoma | §21-1952 through §21-1958 | | | | Oregon | ORS 164.377 | Computer Crime | | | Pennsylvania | 18 Pa. Cons. Stat. §7601-7661 | Computer Offenses | | | Rhode Island | RI Gen L § 11-52 | Computer Crime | Civil Action permitted | | South Carolina | Chapter 16 | | | | South Dakota | S.D. Cod. Laws § 43-43B-1 to § 43-43B-8 | Computer Programs | | | Tennessee | 39-14-602 | | | | Texas | Sec. 33 | | | | Utah | Title 63D | | | | Vermont | VSA 4101-4107 | | | | Virginia | 18.2-152.1, et seq | Virginia Computer Crimes Act | | | Washington | 9A.90 | | Civil Action permitted | | West Virginia | Article 3C | | | | Wisconsin | 943.7 | | | | Wyoming | WY Stat § 6-3-501-505 | Computer Crimes | | # Policy # Impartiality resolution {% file src="/files/UkGhXgd4xLqCtb9atJGW" %} # PDAP Access ## General
TierDescriptionTeam / Access
AdminHigher-context contributors who require write access to do their work.

Members are granted access by Organizers.
  • GitHub repo write as needed
  • Discord PDAP Staff role
  • @pdap.io email if needed
Organizer

Keeping the lights on and making sure we're able to operate and access resources if one or two people leave.

Members are the board, staff, and people appointed to the PDAP nonprofit corporation by the board.

  • GitHub team admin
  • Keybase
  • LastPass
  • Crypto wallet
  • Finances
  • All Admin items
  • GSuite
## Data Sources form #### Some known risks * Spam: someone uses the form or API to submit way too much stuff in an attempt to overload us * Sabotage: someone uses the form or API to submit data that is rude or harmful to our mission #### Some options * Require auth with email address (manually or with an app) * Require auth with something fancier like Keybase * Manually approve all entries before they're made public #### Current solution The risks aren't a problem, yet. In the meantime we're going to: * Collect email addresses on the form, optionally * Set up a moderation queue to approve submissions before they are made public * Not embed the form into a live site, only sharing the link with individuals who ask by contacting staff in any way (explaining this process in the docs) **Sharing un-approved Data Sources, sometimes** When we share a public view [like this one](https://airtable.com/shrZgN3ZWviwXpPjx) with a specific user, we will show un-approved sources which fit into the existing filter set. The goal is to help people see the results of their submissions immediately. There is some risk of vandalism, and in general we don't but it's mitigated by these factors: * To the rest of the world (on our public view), it looks like we only show approved data sources, so the incentive for vandalism is low. * We're sharing these geographic-specific views with people who we think are serious users, and they may share it with their network; not many people will see these un-approved data sources. * We're still listing them as "not approved", so we are not misrepresenting them. * In the future we'll be able to further restrict who is able to view un-approved Data Sources; maybe they can opt in, only see sources submitted by themselves or others in their community, etc. ### GitHub Actions We often use GitHub Actions to automate tasks. The pattern for new volunteer-submitted automated utilities is that we: 1. Create a new repository where the code will live, or a new directory in an existing repo 2. Ask the volunteer to submit their code to the new repo, without worrying about automation 3. Wire up the automation ourselves, once we ensure the code meets the standard # PDAP Privacy Policy We only collect the information you provide to create your account (email, password). We encrypt passwords, use minimal cookies, use Plausible for privacy-first analytics, and don't share your data with third parties. # Password Management Ensuring PDAP keeps tools and services in its ownership. #### What’s the policy? The following must be true for all services: 1. Eddie Brown (Board Chair) has top-level "admin" or "owner" access. 2. At least 3 people from the PDAP team (defined by Discord account) have admin or owner access, to the extent it’s possible. “Admin or owner access” confers the ability to manage users/members, and billing. # Personally Identifiable Information Also known as PII {% file src="/files/OXIhyBcTARpo4bLVaMJJ" %} ## Defining PII #### [2 CFR § 200.79](https://www.law.cornell.edu/cfr/text/2/200.79) > \[...] for example, first and last name, address, work telephone number, email address, home telephone number, and general educational credentials. The definition of PII is not anchored to any single category of information or technology. Rather, it requires a case-by-case assessment of the specific risk that an individual can be identified. Non-PII can become PII whenever additional information is made publicly available, in any medium and from any source, that, when combined with other available information, could be used to identify an individual. This definition of public data is also supported in [HiQ Labs v. LinkedIn](https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn#cite_note-1). ## Goals We want people to feel sound ethically and legally when they contribute to PDAP. To that end, these are our goals: 1. Draw a “[bright line](https://en.wikipedia.org/wiki/Bright-line_rule)” from our [Data Sources Database](broken://pages/nmpsayHbm28zgI94Cqdh) to the source material. We are not altering public records, we are helping people find them. 2. Avoid aggregating already-public PII in a way that would make it more usable (e.g. turning arrests from different jurisdictions into a unified database). [Citation](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2678420). * We don't want to become or facilitate a database of people impacted by the legal system. 3. Strive for completion, and not be accused of editorializing by omission. * The board decided [submitting PII is allowed](broken://pages/-MYXSzQD2zS0CQR8q_P9). * We can track whether a given Data Source contains PII. * PII collection is not required. If we aggregate data, we can choose to omit PII like `name` and `address` because those properties do not become statistically useful in aggregate, only dangerous. 4. Transparency in our published resources and information. * Our open-source tools are transparent in how they work. * Documentation about published data and tools should be clear about any properties which are collected—or left behind—and why the decision was made. # Community calls # October 17, 2023 Context: We're professionalizing our front end: design system, component library, tailwindUI. Agenda: * what's the vision for how our front end should be structured relative to our API and users? (capturing tasks / ideas) * what's been done so far? * what's up next? * are there open questions? ### Goals * dev environment: branch in github auto-deploying * has its own environment variables * def FE → dev API * set a standard for secrets / environments * as needed access * don't expose tokens in the front end / repos * JWT validated by the API against a secret key in DO * refresh tokens prevent access * token ⭤ user ID pair * what about public front end views (no auth)? * API can use permissions to prevent access/action (roles) * the FE client can use its own `client` token * logged in user credentials would override `client` token * local, dev, prod all use same methodology * consider an isolated back end service for users/OAuth * better security, separation of concerns * * let's not use Supabase auth to database, it should be the API ### To Do * [ ] research OAuth to estimate time / more specifically plan * [ ] set up refresh tokens * [ ] sketch roles/permissions # February 22, 2023 ## Updates from PDAP We've submitted for grant renewal from Heinz, and are actively seeking additional sources of funding. We're getting a slow and steady flow of `requests` and `volunteer offers` via the forms on our [front page](https://pdap.io). ### Open projects * We need a jail roster scraper for Allegheny County, to help local folks investigate who's being kept in the county jail, and for how long. * Our friends at Big Local News are looking for help scraping some data about officer-involved shootings from California. We're trying to cover [23 agencies](https://docs.google.com/spreadsheets/d/1f_qWiXWTcAsUNMtOQlrv0sYG4hgLfVGzupVH3Ek12Z0/edit#gid=0). * We're using machine learning to identify data sources. We have an algorithm, and need to generate more training data for the next iteration. # February 1, 2023 ## Updates from PDAP Josh is still primarily working on grant application and collaboration with groups local to Pittsburgh using or generating data. Jacob is working on long-term software projects related to more permanent Data Sources. ### Known projects in need of help * Research on Southeast Arkansas ([thread in #data-exchange](https://discord.com/channels/828274060034965575/1063264064719044688)) * scheduling a working session when we know more * we may have a connection in AK * San Francisco court scraper [issue here](https://github.com/biglocalnews/court-scraper/issues/179) (Jacob's been doing research and groundwork; ready to hand off or finish) * Research on Detroit Police Department (discussing new information today) * OPD would give the requestor the ability to request complaint and calls for service data, providing it as a Pandas dataframe * Machine learning identification of URLs * More notes about this project in recent community call notes * Scheduling a working session next week * Stocking wrote a crawler which PDAP can fork + keep in our repo to regex ID URLs → JSON labels; craeft offered to host stuff on a personal box for us to access/use # January 20, 2023 ## Updates from PDAP * Josh is working on * Local projects in Allegheny County, PA * Journalists, activists, student groups, etc. who are trying to do things with data * Applying for the next round of grant funding * SE AR research project * Jacob is working on * our [long-term software projects](https://github.com/Police-Data-Accessibility-Project/planning/milestone/1), making our Data Sources exist in an app separate from Airtable * groundwork for a San Francisco court scraper (for a requestor) * putting [URLs in a pile](https://github.com/Police-Data-Accessibility-Project/data-source-identification/issues/2) for analysis ### Known open projects in need of help * Research on Southeast Arkansas ([thread in #data-exchange](https://discord.com/channels/828274060034965575/1063264064719044688)) * San Francisco court scraper [issue here](https://github.com/biglocalnews/court-scraper/issues/179) (Jacob's been doing research and groundwork; ready to hand off or finish) ## Call notes * Discussed [ACPP's map](https://www.grieftoaction.org/#/map) of Allegheny County boroughs/agencies * they may want to update it based on our db # January 5, 2023 ## Updates from PDAP * The new [https://pdap.io](https://static1.squarespace.com/static/627c29c0bcb7f5543e9f8ed1/t/627d35687db3d00c1c66b432/1652372841540/072621_2020_PMP_report.pdf) front page points people to the new "request data" workflow. * We've been doing this informally already—we'd like to get more requests, so we can better understand the kinds of things people are looking for and make more of an impact. * We're making good progress on turning our Airtable tools into a home-grown app. There's now a [public repo](https://github.com/Police-Data-Accessibility-Project/data-sources-mirror) which mirrors our Data Sources database as CSV and JSON. * We're working on ways to identify URLs en masse. * GitHub issue: * A volunteer wrote a sitemap scraper, which locates potentially useful URLs given a list. It's an open PR still under review. We want to get it merged soon: * We did a Doccano labeling exercise in an attempt to train a machine learning algorithm to identify content based on URL. We still need to experiment in order to close the loop on this. ## Call notes * linklabel regex script: * (C++) scans massive amounts of URLs for keywords * early bottleneck: commoncrawl URL database is \~4TB * to do: * publish the script + regex library * generate lists of URLs * get a good list of regex keywords set in advance * crunch through the URLs * ??? * Elasticsearch * Supports [stemming](https://www.elastic.co/guide/en/elasticsearch/reference/current/stemming.html) * Commoncrawl storage * craeft offered to spin up a 4-5TB linux server to hold mass amounts of URLs * goal: people can get batches of URLs off the server # October 25, 2022 {% file src="/files/NDt1oaKUK2V8DEwxwwzj" %} # September 22, 2022 {% file src="/files/0hkFFQdNGhc8ZrfRwwCc" %} # August 23, 2022 {% file src="/files/6jyW6NT3iZXUNfLqEnTe" %} # October 2, 2021 ## Scraping We need to firm up support for unprocessed data, this issue needs some work to solidify what we need: ## Azure setup Risk: abuse that eats through our credit or destroys our data. is a way to facilitate data analysis with an encrypted / private middle layer. Could be used for address/name removal/hiding. ## Guidestar Our profile is here: We're at gold, and to get platinum will need an independent evaluation, likely by a consultant. ## Social media / outreach Josh working on additional "language guidelines" for talking about PDAP, in addition to the [brand assets](/meta/operations/staff/brand-assets). Alec secured new Instagram and Twitter accounts that we can get verified and start using. --- [Next Page](/llms-full.txt/1)