On May 1st, 2020, our cumulative case totals for Suffolk County, NY show a decrease of 686 cases. Objectively, this does not make sense; the cumulative case totals are cumulative. They should increase, or remain constant. They should never decrease. Some small variation from this is normal as counties will occasionally report a decrease of one or two cases if the further investigation revealed that they were reporting a false positive, or if the cases were reassigned to a different county. A decrease of several hundred cases suggests something more peculiar occurred.
The Centers for Disease Control (CDC) and Council of State and Territorial Epidemiologists (CSTE) were quick to publish guidelines on reporting COVID-19 cases. In short, they used two terms for cases: confirmed or probable. Confirmed cases were individuals who had returned a positive molecular test, such as RT-PCR, to detect active infection. Probable cases were diagnosed in other ways. One of the tests used in diagnosing probable cases was antibody tests, which detected evidence of contact with the virus at some point in time. However, an infection could have occurred anywhere from days to months in the past. At no point were antibody tests alone considered diagnostic evidence. Ultimately, diagnosis of infection required expressing symptoms or epidemiologic linkage as well.
At BroadStreet, the COVID-19 Data Project is interested in tracking how the virus is spreading throughout a population as a function of time. As a result, we tried to exclude positive antibody tests from our data wherever possible. If one is researching to determine how policy in a given month is effective in reducing the rate of infection, then providing information for new cases in the database could be more helpful.
This brings us back to Suffolk County. At some point in April, their health department began including positive antibody tests in the same figure as positive molecular tests. This became apparent on May 8th when our quality assurance team noticed that the positive cases reported by the New York State Health department differed from the county health department by over 3000 cases. Further investigation revealed that Suffolk County had been including a category of people in their case figures that New York State was not: the positive antibody tests.
The COVID-19 Data Project worked to isolate and remove those antibody test results from the data when possible. After that work, the only artifact of their presence in our data was the decrease of 686 cases on May 1st.
Neither health department is wrong in their choice of including or excluding positive antibody tests from their reported figures. Instead, the choice depends on why and how the data is being utilized. If you want to look at how far the virus has spread in a community, surveillance tests (such as antibody tests) can help create a fuller picture than using molecular tests alone. If the interest is in seeing how quickly the virus is spreading right now, including people who had contact with the virus at an unknown point in history is not relevant.
Ideally, both sets of data would be easily obtainable. The user could then sort through and decide to include or exclude categories according to their research purposes. Many health departments are either not prepared for or not cognizant of the demand for accessible and transparent data.
COVID-19 data can be used in real-time to analyze and optimize our responses to the pandemic. Our goal at Broadstreet is to aid this process. We use a transparent methodology to collect our data. We assemble case definitions from every state health department for anyone using our data to see. We aim to empower members of the public and researchers alike by creating an accessible, machine-readable dataset tracking the COVID-19 pandemic.