WHO’s estimates of the upper
and lower bounds of health system
performance differ in two important
ways from most analyses
of what health systems actually
achieve. The first is that a “frontier”
is meaningful only if no country
can lie beyond it, although at least
one must lie on it. The frontier or
upper limit is therefore estimated
by a statistical technique which
allows for errors in one direction
only, minimizing the distances between
the frontier and the calculated
performance values. (The
lower bound is estimated by the
conventional technique of allowing
errors in either direction.) The
second is that the object is not to
explain what each country or
health system has attained, so
much as to form an estimate of
what should be possible. The degree
of explanation could be increased
by introducing many more
variables. If tropical countries show
systematically lower achievement
in health, because of the effects of
many diseases concentrated near
the equator, a variable indicating
tropical location would raise the explanatory
or predictive power. Similarly,
if outcomes are worse with
respect to equality in ethnically diverse
countries, a variable reflecting
that heterogeneity would explain
the outcomes observed.
The difficulty with the attempt to
explain as much as possible is that
it leads to a different frontier, according
to every additional variable.
There would be one for tropical
countries and another for colder climates;
one for ethnically mixed
countries and another for those with
more uniform populations; and so
on. If performance were measured
relative to the frontier for each type
of country, almost every health system
might look about equally efficient
in the use of resources, because
less would be expected of some
than of others. Every additional explanation
would be the equivalent
of a reason for not doing better. This
is particularly true of explanations
related to individual diseases: AIDS
and malaria are major causes of
health loss in many sub-Saharan
African countries, but to include
their effects in the estimation of the
frontier means judging those countries
only according to how well
they control all other diseases, as
though nothing could be done
about AIDS and malaria. This is the
reason for estimating the frontier
according to nothing but expenditure
and human capital, which is a
general measure of society’s capacity
for many kinds of performance,
including performance of the health
system.
The measures of attainment draw
on data referring to the past several
years, to make the estimates more
robust and less susceptible to
anomalous values in any one year.
The measures of expenditure and
human capital are similarly constructed
from more than one year’s
data. Nonetheless, both the outcomes
and the factors that determine
potential performance are
meant to describe the current situation
of countries. They do not take
into account how past decisions and
use of resources may have limited
what a system can actually achieve
today – which could also be a reason
for poor performance – nor
do they say how quickly a poorly
performing system might be expected
to improve and come
closer to the frontier.
This way of estimating what is
feasible bypasses two particularly
complex issues which are well illustrated
by control of tobaccorelated
mortality and disability.
One is that many actions taken by
health systems produce results
only after a number of years, so
that resources used today are not
closely related to outcomes today.
If a health system somehow persuaded
all smokers to quit and no
one to take up the habit, it would
be many years before there was
no more tobacco-induced disease
burden.1 The other is that no
health system could reasonably
be expected to bring smoking
prevalence down to zero any time
soon, no matter how hard it tried.
Determining how to evaluate
progress rather than only a health
system’s current performance is
one of many challenges for future
effort