INSUBCONTINENT EXCLUSIVE:
Eli Cahan
Contributor
Share on Twitter
Eli Cahan is a medical student at NYU on leave to
complete a master in health policy at Stanford as a Knight-Hennessey Scholar
His research addresses the effectiveness, economics, and ethics of (digital) health innovation
More posts by this contributor
How the coronavirus outbreak will stress-test startups
Will unreliable research bury your
healthcare startup?
Day after day, the burden of COVID-19 caused by SARS-CoV-2 mounts further
As of this writing, nearly 400,000 patients worldwide were confirmed for the disease, including over 46,000 cases spanning every state in
this country.
In tandem with this mounting burden, due to numerous fumbles over the past 10 weeks, the United States has faced significant
bottlenecks in the production of diagnostic testing and imposed substantive red tape to deter testing
As more diagnostic tests have come online, the rate of &confirmed cases& — the key performance indicator (KPI) being monitored by
decision-makers across the country — continues to accelerate.
Why businesses love KPIs
Leaders of startups are familiar with the attention
paid towards KPIs such as profit margins, burn rates, net dollar retention rate, and customer acquisition costs
These metrics, when chosen appropriately, allow leaders to continuously take the pulse of their companies and take action in response.
Along
these lines, Phil Nadel has writtenon TechCrunch that founders &cannot hope to grow a company in any meaningful way
without…KPIs…[b]ecause KPIs, if constructed correctly, give management and potential investors a cold, analytical snapshot of the state
of the company, untainted by emotion or rhetoric (emphasis added).
Conversely, when misconstructed, misconstrued or overlooked, KPIs can
cause organizations to crash and burn
This phenomenon has been deemed &surrogation,& or reflecting the potential for critical thinking around strategy to be subverted by
elevation or suppression of a single number
For example, maximizing profit margin can hurt the quality of goods/services, and maximizing lifetime value (LTV) can negatively impact
customer experience by, say, encouraging the use of shady cross-selling or up-selling techniques that are harmful in the long-term to
customers (aswas the caseat Wells Fargo after the financial crisis).
Taken too far, wrongly-selected KPIs can cause organizations to suffer
profoundly, such as with Uber patchy quality standards for driver recruitment leading to innumerable controversies around customer
And while consistently poor quarterly failures in the boardroom can lead to organization crises, they do not bear remotely the same stakes
as public health crises.
The primary KPI for COVID-19 has been an unreliable figure in the United States
During the COVID-19 pandemic, the
primary KPI tracked by United States leaders has been the number of confirmed cases of the disease
Yet epidemiologists monitoring the outbreaks have become increasingly frustrated with the meandering pace of actions taken by United States
The core of the mismatch between actions desired by scientists and those (not) taken by policymakers lies in misinterpreting the
&denominator.&In other words, misconstruing the pandemic primary KPI.
The denominator refers to the formula from which mortality rates are
calculated — the number of deaths divided by the number of cases
While this seems to be straightforward algebra, the devil is in the details
Since the beginning of the outbreak, the Chinese government has waffled on their definition of &confirmed cases&: deciding ultimately to go
with a definition (positive only if laboratory-confirmed, regardless of symptoms or other tests) that may suppress the real number
Analogously, diagnostic testing snafus in the United States suggest that the number of &confirmed cases& here is simply not reliable
Both of these stand in stark contrast to countries such as Taiwan and South Korea, both of which swiftly diverted resources to scale up,
broaden, and fully report testing.
Predictably, these snafus have led to diagnosis of disproportionately severe cases thus far in the United
States with the exception of NBA players and movie stars, only highly symptomatic individuals have been eligible to receive testing (due to
clinical red tape), able to receive testing (due to shortages), and subsequently confirmed
This selection bias of severe cases implies that overall case number — the true denominator — is markedly underestimated
Accordingly, since the beginning of February, epidemiologists have been vocal that &simple counts of the number of confirmed cases can be
misleading indicators of the epidemic trajectory.
Relying on confirmed cases as the primary KPI may have delayed and misled critical
action
A timeline of COVID-19 progression in the United States illustrates the impact of using confirmed cases as the country primary KPI
for public health response.
On January 22, President Trump stated that &we have [SARS-CoV-2] totally under control, it one person
[confirmed].& On February 25, with 53 confirmed, he claimed that ''the coronavirus…is very well under control in our country
We have very few people with it.& The following day, the president declared that ''the risk to the American people remains very low.& On
March 6, with 227 confirmed, he shared that &I think we&ve done a tremendous job of keeping [the number of confirmed cases] down.& And just
one week ago on March 17, the president praised West Virginia as the single state without any cases, hailing &Big Jim, the governor…must
be doing a good job.
By the end of that day, the praise no longer held
Reports later emerged that the reason West Virginia had no cases for so long was related to the fact that the state was ill-equipped, and
perhaps resistant, to performing testing that might come back positive.
With confirmed cases on United States soil as the North Star for
decision-making — despite ''the system blinking red& on other dimensions — early decisions to take vital preventive actions were punted
It wasn''t until March 13 (over seven weeks after the first confirmed United States case) that the president declared a national emergency
And even as 70 million Americans were under lockdown as of March 20, 47 states maintained few restrictions for social distancing (which
remains the only intervention currently with proven effectiveness against COVID-19)
Those under lockdown permit leaving for &essential& tasks, which itself is fuzzily-defined.
Moreover, emerging evidence suggests that mild
(&subclinical&) cases — exactly the ones that have not been tested — may be driving community spread of the virus
Nonetheless, focus on confirmed cases as principal KPI has elicited few interventions against these &below the surface& cases
Domestic travel restrictions by foot, bike, car, bus, train, and plane remain mostly limited
Spring breakers continue to party at Miami Beach hookah shops and Nashville honky-tonks.
Simultaneously, numerous universities — sensitive
to the fact that college dormitories are amongst the highest risk settings for contracting an infection — suspended classes and evacuated
However, these moves overlook the fact that college students are amongst the least likely to manifest symptoms
As such, by seeking to prevent any confirmed cases on their campuses, these universities may have ushered the virus into the homes of
parents and grandparents in innumerable local communities across the United States
Alternative KPIs for COVID-19 can help get the United
States back on track
If the wrong KPI created this mess, then the right KPI(s) are urgently needed to begin fixing it.
For starters, risk
stratification is desperately required
Rather than resorting to black-and-white measures of infected versus uninfected, risk spectra can better characterize the threat faced by
specific individuals, communities, states, and countries.
For example, mortality risk can be quantified and monitored
Elderly individuals and those with pre-existing medical conditions are at the highest risk of severe infection
By deriving and pooling individual risks, the relative threat to communities can be evaluated: encouraging high-risk communities to take
more immediate and more proactive preventive action
Florida, for instance, could benefit from this kind of measurement: 27% of the state residents population are elderly, but thus far
shockingly few individuals have been tested for coronavirus in retirement communities like the Florida Keys (where 75 people had been
assessed as of March 24 amidst the height of Spring Break).This could lead to protective policies like visitation restrictions and hygiene
guidelines.
Additionally, transmission risk can be a useful KPI
Certain demographic groups, like students and healthcare workers, are at much higher risk of propagating the virus — with or without the
Individuals with exposure to the virus in confined spaces (such as cruise ships or airplanes) also have much higher transmission risk
Scoring metrics could be adapted from other infectious diseases to help policymakers better visualize and prevent transmission in their
This could encourage prospective procedures like contact tracing and symptom monitoring.
Designating mortality risk (for vulnerable
patients) and transmission risk (to vulnerable patients) as KPIs could provide much more granularity to decision-makers than can confirmed
Including these collectively on a dashboard will generate insights and stimulate further actions than relying on a single narrow, fallible
KPI.
And of course, these are only two (reductive) examples of potentially useful metrics for public health responders
Far more diverse forms of metrics are possible
These are often pioneered, unsurprisingly, by startups — such as in the case of Kinsa Health, a producer of smart thermometers
The company temperature geo-maps are already providing a leading COVID-19 indicator for local decisionmakers, as they have previously for
the seasonal flu (compared to CDC models).
Kinsa fever map could show just how crucial it is to stay home to stop COVID-19 spread
In
healthcare and beyond, new KPIs will prove critical moving forward
As far as COVID-19 goes, the steps our country needs to take are fairly
An analysisby researchers at Imperial College London COVID-19 Response Team suggested that near-universal social distancing measures are the
only tool available to prevent &hundreds of thousands of deaths and health systems (most notably intensive care units) being overwhelmed
many times over.& Countries that continue to be ravaged by the virus (such as Iran and Italy) enacted social distancing interventions
As the trajectory of the infection curve in the United States closely mirrors these countries (despite probable underestimates of case
burden), more universal measures are sorely needed
Better healthcare KPIs can help us understand the next steps we need to take.
As far as the startup community goes, there is much to learn
A proverb in medicine advises: &during a cardiac arrest, the first procedure is to take your own pulse.& Since startups may face numerous
crises (in the business sense) amidst the economic convulsions of COVID-19, composure will be essential to guide decisions under pressure
Meaningful KPIs are the stethoscopes, tourniquets, and barometers that enable startups to take their own pulse—and to rapidly commence any
resuscitation that may be required.
Another medical proverb dictates that &an ounce of prevention is worth a pound of cure.& Prescient
leaders of startups would be well-served by seeking the tools for prevention sooner rather than later.
