We have assembled executive orders and proclamations on coronavirus policies from major cities, and all US states and territories to create a machine readable data set for the public. Our interns have created a data set of these policies by type, date, and content as detailed in this codebook spreadsheet. Our data is updated regularly, and txt files are also available for those doing research and creating their own database. To learn more about the details of these notes, we have created a webinar.
The COVID-19 pandemic has unveiled unprecedented challenges for the majority of the countries in the world. In the United States, an estimated 90% of Americans experienced lock-downs and social distancing orders.1 As the novel coronavirus disease 2019 pandemic threatens countries throughout the world, the United States became an epicenter of the outbreak.2 Systematically tracking and assessing policy responses to the public health crisis in each state is essential to better understand how to manage each outbreak and slow the growth of positive cases. Within the current healthcare system, good care is almost unobtainable for many individuals; eligibility is limited, and coverage is costly and too uncertain, resulting in an increasing number of uninsured individuals.3 While researchers and health care professionals are racing to develop a vaccine, officials are implementing social distancing, stay at home orders, restricting travel, and closing businesses and schools to try and slow the spread of the outbreak.4 With positive case numbers on the rise, what responsibilities do governments have? What are the roles of policy makers? How can we ensure the safety and security of citizens throughout the country?
BroadStreet and The COVID-19 Data Project Policy Track have partnered with Temple University’s Center for Public Health Law Research (CPHLR) to look longitudinally at executive orders, health directives, proclamations, and policies related to COVID-19. The COVID-19 Data Project has adapted CPHLR’s process, enabling up-to-date collection of order releases for major cities and counties, the United States as a whole, and each state and US territory, conversion into machine readable formats with visually quantifiable and qualitative data. On a weekly basis, new releases and their original sources are gathered, converted, read, and coded for the quality assurance process. Through a review and coding process, aligned with the scope criteria set in place by the CPHLR, the teams of the Policy Track clean and prepare the data for quantitative analysis and further research projects.
Starting on April 6th, 2020, Temple University’s CPHLR began collecting, coding and analyzing state statutes, regulations, emergency declarations, and mitigation policies related to the COVID-19 pandemic. The project was divided into phases, with the Phase 1 longitudinal dataset covering legal action from January 20, 2020, through June 1, 2020, for the following states: Alabama, Alaska, Arkansas, California, Colorado, Connecticut, Georgia, Hawaii, Iowa, and Kansas. The start date of January 20, 2020, was selected as it is the day prior to the first case of COVID-19 confirmed by the Centers for Disease Control and Prevention on January 21, 2020. Data from the following sources was used to examine, research, and collect relevant state actions: American Enterprise Institute COVID-19 Action Tracker, American Network of Community Options and Resources COVID-19 State Policy Resources, ASTHO Coronavirus Disease 2019 (COVID-19) Responses Hub, Boston University School of Public Health COVID-19 US State Policy Database, Council of State Governments COVID-19 Resources for State Leaders, Kaiser Family Foundation State Data and Policy Actions to Address Coronavirus, National Conference of State Legislatures State Action on Coronavirus (COVID-19), and The National Governors Association (NGA).5 After beginning Phase 1, the CPHLR team quickly recognized the need for help as the number of legal actions increased dramatically and rapidly. With help of BroadStreet volunteers, some of whom were also students at Temple University, a partnership was formed to collaborate and work side by side. Phases 2 through 4 ramped up including the remaining 40 states, the District of Columbia, and Puerto Rico, with the COVID-19 Data Project Policy Track teams helping to identify which orders were within scope and assisting in building the longitudinal dataset.
The Policy Track teams meticulously reviewed the COVID-19 statutes, regulations, emergency declarations, and mitigation policies for three of the ten Phase 2 states: Michigan, Wyoming, and Washington; and assisted the CPHLR team in converting the documents into machine readable format. The leaders of the Policy Track developed a Google Sheet to record, analyze, and code the documents in line with CPHLR’s protocol (see complete process below). Comparing the CPHLR scoping notes with the Policy Track document, the Collaboration Team was able to identify outliers and adjust the process to create the final process document. The Policy Track teams proceeded with the updated process adjustments when working with documents for the remaining states and territories. Beginning in September, the teams also began including documents for state capitals and major cities.
The data is structured in wide format. The orders are in alphabetical order by location. The first order is a Public Health Order for Adams County, Colorado and it includes information limiting gatherings, alcohol sales, and imposing a curfew. This is how the first order would be represented: 05000US08001, County, Adams County, Colorado, Health Order, 0, 11/07/2020, 12/07/2020, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0. Each row of data begins with the identification number, followed by the GeoType to which the order applies and the name of the location.
In order to analyze and generate data each week, the COVID-19 Data Project Policy Track organized into teams. Four of the teams complete “Data Bursts,” a team meeting where individuals work independently and simultaneously to review and convert the documents, generate the dataset following CPHLR’s protocol, while exchanging ideas about how to resolve any issues that arise. Each Burst Team works on a range of a preformatted spreadsheet, unique to each week. The Flight Team, the fifth team, contributes to the data analysis process throughout the week by navigating preparatory and finalization steps. To ensure a smooth and manageable Data Burst for each team, recently released documents are collected and converted into Google Doc format; this conversion allows the teams to easily edit the documents and efficiently collaborate with one-another throughout the data analysis process.
Before the Data Bursts can begin each week, several tasks must be performed to equip the teams with the necessary documents and data. First, the COVID-19 related official documents for each state and territory are gathered from The Council of State Governments website and the website of each location. Federal orders are gathered from The National Archives Federal Register website. All cities orders are gathered on a spreadsheet for tracking updates. Specific dates of the gathering(s) are listed on a Policy Track document, which identifies the documents to be analyzed for a given week. Gatherings occur at least once per week. After the gathering for the week is complete, a leader in the Policy Track organizes the documents by state onto a shared spreadsheet. On the shared spreadsheet, the leader enters their name, the source from which they gathered the document(s), and the date each document was gathered.
On the shared spreadsheet, the documents for each week are separated onto tabs for ease of use during the Data Bursts. As weeks are completed, the completed tabs are combined and passed to the Quality Assurance team for further review. For organization and management purposes, a count of the total number of documents analyzed each week and a count of the total number of documents per location is maintained.
The original format of the documents is either HTML or PDF. The conversion of the documents into Google Doc format is essential to the steps completed during each Data Burst. The HTML formatted documents are converted by copying and pasting the text into a blank Google Doc. The majority of the documents gathered are in PDF format, which requires a more complex conversion process. The PDF documents must be downloaded, converted to a Microsoft Word Document (which requires specific software), uploaded to Google Drive, and then opened as a Google Doc. Once the documents have been converted into the Google Doc format, the direct links are incorporated to the shared spreadsheet for the Teams to access. However, both conversion methods generate spelling and formatting inconsistencies which must be corrected manually. Manual correction may include “auto spell check,” image removal, border removal, and/or hand-typing documents in full. When spelling and formatting has been completed on the documents, it is labeled as “Cleaned” for Teams to recognize.
The deep read is the main part of the process which all Policy Track team members are trained to complete. The review and coding, or “deep read,” of the documents following the CPHLR protocols is the main step of the data analysis process. The deep read includes the analysis and mark-up of the documents and the entering of key details and binary variables pertaining to the inclusion of “in-scope” topics. The marked-up document provides the Track leaders and the Quality Assurance (QA) team members with important information and creates a track record of the logged binary variables.
Each Data Burst includes the deep read of the COVID-19 documents and conversion of the documents to a machine readable format. Each Burst Team Lead hosts a regular, weekly meeting with their team for a 90-minute Data Burst. The Data Bursts are completed in a team-setting in order to provide a group space for working and to reinforce the community’s bond. It has been found that the analysis of these complex documents is most successful when individuals can actively communicate.
During the Burst, each team member tracks and documents their work on the shared spreadsheet. The shared spreadsheet is formatted to guide individuals through the analysis of each document, identifying content that is within scope by following the scope descriptions on the spreadsheet. The individuals place a 0 (zero) in cells to signify a document does not contain content about the given scope topics; a 1 (one) is placed in cells to signify a document does contain content about the given scope topics. Cells that have a 1 also have a comment attached to the cell to describe the content found in the document for that topic.
The scope topics and descriptions have been created by Policy Track leaders based on CPHLR’s “protocol notes.” The topics are grouped into color-coded categories, which allows for the mark-up of the document(s) to align with the binary code and attached comments on the spreadsheet. This alignment enables the QA team to easily confirm the content is applicable to the scope topic. Any ambiguous information in the documents is noted and a detailed review is conducted by the QA team to determine if the content is applicable to the scope topics.
The following are the categories of scope topics:
The individuals may also enter notes and/or miscellaneous information about the document that may be of interest to researchers and reviewers. At the conclusion of the Data Burst, the Burst Team lead reviews the work completed by the team members for completion. At the conclusion of each week, the content created by the teams is reviewed by the QA Team before integration into longitudinal dataset performed by a Policy Track leader.
The QA Team completes an in-depth review of the analysis completed by the Burst Teams each week. The QA Team uses a combination of the completed spreadsheet(s) and the mark-up of the individual documents to review the completed analysis. Their process includes review of:
The review status of each document is labeled on the shared spreadsheet to track and monitor the completion of QA. When the QA Team has completed their in-depth review, a Policy Track leader is alerted and the shared spreadsheet is integrated into the longitudinal dataset. This dataset includes the code of all documents that have been completed, grouped by location.
The most recent release of this document occurred on June 4th. Previous releases of this data and document have occurred monthly since August 2020. As we continue to release COVID-19 data these release notes will be updated to document methods. We feel it is essential to document the good and bad of the process so future pandemics can benefit from what we have learned.
When using data images, downloaded data, or shared document formats, please attribute BroadStreet as well as the original source, when applicable. For examples and more information, review this article which answers the question "How do I cite BroadStreet?".
Allison Muir, MHA
In February 2021, Allison Muir created this webinar to discuss the details of the policy data.
1. Chan AT, Brownstein JS. Putting the Public Back in Public Health — Surveying Symptoms of Covid-19. N Engl J Med. 2020;0(0):null. doi:10.1056/NEJMp201625
2. Monroe M. Florida emerges as world’s new epicenter for COVID-19. TheHill. Published July 8, 2020. Accessed July 20, 2020. https://thehill.com/homenews/state-watch/506492-florida-emerges-as-worlds-new-epicenter-for-covid-1
3. Fuchs VR. Health Care Policy After the COVID-19 Pandemic. JAMA. Published online June 12, 2020. doi:10.1001/jama.2020.10777
4. Gostin LO, Wiley LF. Governmental Public Health Powers During the COVID-19 Pandemic: Stay-at-home Orders, Business Closures, and Travel Restrictions. JAMA. 2020;323(21):2137-2138. doi:10.1001/jama.2020.5460
5. Coronavirus Resources. National Governors Association. Accessed August 6, 2020. https://www.nga.org/coronavirus-resources/