The following is mirrored from its source at:
http://www.thebulletin.org/issues/1993/may93/may93Gofman.html
------------------------------------------------------------------------------
Beware the Data Diddlers
by John W. Gofman
The Bulletin of the Atomic Scientists
May 1993
* The proposal
* Un-knowledge
* The rules of research
* Follow the rules
* Everyone wins
* Violating the rules of research
Permit me to put the question starkly. Although there is ample
evidence that nuclear pollution presents health risks, how can we
properly assess the degree of risk when the governments that have
unleashed the poisons also sponsor virtually all the health
research concerning nuclear radiation?
That conflict-of-interest problem has a common-sense solution. We
-- and "we" refers mainly to individuals and nongovernmental
organizations -- must insist that independent "watchdog
authorities" be established to monitor the work of those who may
have a vested interest in underestimating the health risks that
may be attributed to nuclear radiation.
These watchdog authorities will not be responsible for conducting
the actual health-risk studies. Rather, they will be charged with
insuring that the raw data used in health-risk studies are
obtained and maintained as objectively as possible.
It's a common-sense idea. But like many simple ideas, it will be a
tough sell.
Valid conclusions about the health consequences of various
pollutants rest upon databases that can be trusted. If a database
is false -- either from careless work or from intentional bias --
it will cause innocent analysts of the data to fill medical
journals and textbooks with misinformation. As surely as a skewed
foundation compromises the building erected upon it, a false
database turns all users into purveyors of possibly deadly
information, no matter how honest they may be. The accuracy of a
database is the key to every conclusion that emerges from it. The
health consequences of false databases can vary from trivial to
tragic.
Although the principles discussed on these pages focus on
radiation research, they apply with equal vigor to databases on
dioxins, pesticides, mercury, and other major non-nuclear
pollutants.
Because it seems just a matter of common sense to have independent
watchdog authorities, many people simply assume that they already
exist. But they do not. Not for radiation research, and not for
any other pollutant. The current situation regarding databases is
unacceptable. Indeed, future generations might characterize it as
criminally negligent.
The proposal
The Chernobyl database is already under construction by the
International Program on the Health Effects of the Chernobyl
Accident (IPHECA), which will operate through the World Health
Organization. The main sponsors of the IPHECA study -- with $200
million suggested for its initial stages -- are the governments of
the United States, Britain, Russia, France, Germany, and Japan.
These governments also sponsor nuclear programs -- either nuclear
power, nuclear weapons, or both.
The conflict-of-interest is self-evident, and it is not limited to
IPHECA's Chernobyl study. Nearly all radiation research is
sponsored by governments that fiercely defend and promote nuclear
energy. I believe that they recognize their goals are not aided if
the public comes to believe that radiation is harmful -- even at
low doses, and even if slowly delivered.
The current situation in radiation research is a bit like relying
on the tobacco industry to conduct all the research on the health
effects of smoking. Fortunately, we don't do that. Since the
1950s, thousands of independent studies regarding the impact of
smoking on health have been organized. In effect, the scientists
who have conducted these tobacco studies have acted as watchdogs
vis-a-vis the tobacco industry, which takes a generally benign
stance toward the products it produces.
Similarly, scientists with the mandate and the financial
wherewithal to act independently must "watchdog" the building of
radiation databases. I propose that we start with Chernobyl:
* IPHECA should fund a team of independent scientists who will
work inside the Chernobyl study. They would have the
authority to make sure that the essential rules of research
-- as detailed on page 42 -- are observed. They also would
have the right to publish their own views as an integral part
of every IPHECA document.
Their assignment would not be to dictate a uniform analysis
of the data. Rather, it would be to insure that the database
itself can be trusted and that dissenting views about its
handling and its meaning are not punished or silenced.
* The watchdog scientists would in no way be answerable to
governmental authorities. Rather, they would represent the
public's right to a "second opinion" on matters of radiation
and human health. An independent watchdog supervisory group
could be appointed by environmental and other citizen groups
to select and oversee the actual watchdog efforts.
* Watchdog authorities must be permanent. Although the work
that goes into the preparation of databases is most intense
in the early phases, input is necessarily added for many
decades, as the health of participants is followed up.
Watchdog authorities must operate for the entire duration of
a given study, because massive violations of the rules can
occur even late in a study. (See "Violating the Rules of
Research," page 43.)
Un-knowledge
During a lifetime in biomedical research, I've concluded that it
is hard to prove anything about anything. There are sampling
errors, confounding variables, and necessary equipment that has
not yet been invented. The path to understanding is only darkly
lit, and the stones strewn along the way are numerous.
Acquiring truth about health and biology is difficult at best. In
contrast, acquiring falsehoods about health and biology seems to
be inordinately easy. Consider the legions of peer-reviewed
professional journals that have carried "evidence" favoring
various pharmaceutical, dietary, surgical, or physical therapies
for almost every problem -- and the number of disappointments when
the initial glowing reports are undermined by later reports
suggesting no health benefits at all.
I call such information "biomedical un-knowledge" -- shorthand for
all the findings that are the opposite of what is true about
health and disease.
When I was younger, I assumed that no one wanted "un-knowledge." I
no longer make that assumption. Consider the "wish-list" that
nuclear energy-promoting governments seem to have for the
radiation research they sponsor. It goes something like this:
* Best of all would be a finding that a little extra radiation
improves human health, a sort of invisible Vitamin E. This
speculation has a name: hormesis. Indeed, some of its most
avid proponents are already writing about the need to treat
society in general for "radiation-deficiency disease." The
second international conference on radiation hormesis -- with
some 250 speakers and participants -- was held in Kyoto,
Japan, last July.
* The next best finding would be to determine that there was a
threshold dose below which no harm occurs. The "safe doseno
risk" claim has become exceedingly common after the Chernobyl
accident. For example, the U.S. Energy Department, in its
1987 report on the probable health consequences of Chernobyl,
assigned a "zero risk" to some 500 million people exposed to
low doses by fallout -- if all doses below a half rad are
harmless.
In a condensed version of the Energy Department's report
(Science, December 16, 1988), the assertion that Chernobyl
might induce zero extra cancers for persons exposed to low
doses was repeated ten times in six pages. That's a fair
definition of overkill. There was no mention of the powerful
evidence and logic that argue against the threshold
speculation.
* If hormesis and thresholds are not successfully sold to the
general public, the next best finding would be to claim -- as
is often done -- that a dose of radiation is far less harmful
if it is received slowly over time than if the same dose is
received all at once.
As a scientist, I have always taken these wishes and speculations
seriously. I have spent years testing them with the existing
evidence and with logic. I, too, would prefer for radiation to be
harmless. Who would not?
Unfortunately, evidence and logic do not support the wish list. On
the contrary, evidence and logic suggest that low-dose ionizing
radiation may well be the most important single cause of cancer,
birth defects, and genetic disorders.
The rules of research
There are nine fundamental rules of research that any
organization studying the effects of radiation on human health
should be required to follow. They may seem unexceptionable, yet
they have often been ignored:
1. Groups must be comparable. An essential condition for
discovering the effects of radiation is reasonable certainty
that exposed and non-exposed groups are so similar that the
sole reason the groups might be expected to experience
different rates of disease and disorder is their exposure to
radiation.
2. The groups must have experienced an actual difference in
dose. If the rate of disease in two groups is being
compared, it is essential to establish with reasonable
certainty that the groups actually received appreciably
different accumulated doses. If, in fact, the groups
received nearly the same total amounts of radiation, a study
is predestined to find no provable difference in disease
rates between the two groups.
3. The difference in dose must be sufficiently large. The
dose-differences between compared groups must be large
enough to allow for statistically conclusive findings
despite the random variations in numbers and in population
samples. Analysts can cope with the random fluctuations of
small numbers both by assuring sufficiently large
dose-differences between compared groups, and by assuring
large numbers of people in each group.
4. The dose must be carefully reconstructed. Obviously,
analysts will reach false conclusions if supposedly
non-exposed individuals in a database actually received
appreciable doses, and if people exposed to supposedly high
doses received lower doses than the database indicates. When
it comes to Chernobyl, weather patterns caused a very
considerable variation in the spread of contamination, which
makes this scientific pitfall a real possibility unless
careful and objective dose-reconstruction is substituted for
assumption. Fortunately, there are several dosimetry
techniques that can reduce uncertainty about dose, even
decades after the event.
5. Dose analysts must be rendered demonstrably objective.
Analysts who estimate the dose subjects have received should
have no idea of the medical status of the individual or the
group to which the individual belongs. Health-status data
and dose-related data must never appear in the same file.
Analysts must do their work "blind" to protect the database
from accidental or intentional bias concerning the
relationship between radiation dose and health.
6. Diagnostic analysts' objectivity must be confirmed. To be
credible, studies must be designed to guard against bias at
every point. If any study of the effects of Chernobyl is to
be valid, this principle must extend to all analysts,
physicians, and technicians who diagnose the health status
of study participants. They must not know whether an
individual's radiation dose was high or low, and they must
be denied information (such as place of residence) that
would allow them to form an opinion about a likely dose. It
is crucial that teams of "special experts" not be allowed to
alter diagnoses at a later date-"unblinded."
7. No retroactive changes in input data must be permitted after
any results are known. In a continuing study, one must not
be allowed to alter, delete, or add retroactively to the
original input when response results become available.
Deciding to revise original data creates an opportunity to
falsify the cause-effect relationships (if any) between dose
and response. Any study is suspect if retroactive changes
are made in diagnosis or dose, if cases are shuffled from
group to group, or if any data or cases are suddenly dropped
or new cases are added "as needed" from some reserve.
8. Data should not be excessively subdivided. Even the largest
databases can be rendered inconclusive and misleading if
analysts subdivide the data into too many categories or
subsets. If analysts hope that a study will find no provable
effects even if such effects exist, the outcome can be
arranged by creating a "small-numbers problem," which will
prevent all or nearly all of the study's results from
meeting any test of statistical significance. Moreover,
excessive subdivision increases the frequency of finding
random effects, which may pass the test of statistical
significance, but which are nevertheless false. Selective
preservation of such false findings can be used to support
an intentional bias. Any excessive subdivision of data
should be viewed with suspicion.
9. No pre-judgments are permitted. Both pre-judgments and
hypotheses are ideas held at the outset of an inquiry.
Pre-judgments are assumptions, often unstated, that can
cause investigators to ignore or discard particular
evidence. In contrast, hypotheses are tentative
explanations, openly stated, that invite challenges from
other investigators.
--J. G.
Follow the rules
My work on the risks of low-dose radiation has been controversial.
Some scientists say they agree with me. Many say they do not. But
whether I'm right or wrong about the low-dose question is
irrelevant in evaluating the watchdog proposal. The watchdog idea
serves the interests of objective, scientific inquiry. It does not
promote the interest of any particular point of view regarding the
possible outcomes of specific studies.
It is fortunate that we do not often have disasters of the
magnitude of Chernobyl. But the tragedy of Chernobyl will be
compounded many-fold if we squander the opportunity to learn
everything possible about the health risks associated with it, as
well as its ecological consequences.
Creating a trustworthy database from an event such as the
Chernobyl accident is a profound obligation of the world's
scientific community. If research on the radiation consequences of
Chernobyl is poorly designed or biased, or both, the false
conclusions will nevertheless enter the professional literature
and the textbooks.
Research that exaggerates the health hazard of radiation will be a
disservice to humanity. But if researchers underestimate the true
health hazards, the misinformation will be literally deadly,
because it will result in great increases in "permissible doses"
and unnecessary and preventable human exposures to radiation in
the environment, in occupations, and in medical treatment.
To help prevent the production of false databases and false
"findings," either through bias or scientific error, medical
science has developed the already noted rules of research. These
rules are acknowledged implicitly or explicitly throughout the
epidemiological literature.
The key to creditable and credible research in the health effects
of radiation is full obedience to these rules. Strict adherence to
the rules simply eliminates issues regarding conflict-of-interest
and scientific misconduct. No one needs to raise such issues -- if
impeccable adherence to the basic rules can be demonstrated. But
whenever such adherence cannot be demonstrated, society would be
ill-advised to accept the purported scientific "findings."
These assertions are not intended to impugn the motives or the
work of most scientists who prepare or analyze radiation
databases, although some misconduct exists in every field. But as
already noted, if the rules of research are violated in building
databases, no analyses of the data can escape the poison. The
first obligation of every objective scientist is to question the
believability of raw data before he or she uses them.
Everyone wins
Perhaps IPHECA itself should have thought about introducing the
watchdog concept, as a means of obtaining credibility for its
studies. However, IPHECA has not done so.
Although I don't expect IPHECA's scientists to endorse the
watchdog concept publicly, I suspect that most of them will
privately welcome it. Most scientists, after all, would prefer to
be honest and to do impeccable work.
Under a watchdog authority, everyone wins. The scientists win by
being liberated from humiliating pressures to please their
employers. The victims of nuclear accidents win by having
objective, reality-based evaluations of the harm or lack of harm
they have been exposed to. And of greatest importance, humanity
gains by being freed of the specter of 100 years of biomedical
un-knowledge about the health effects of radiation.
The watchdog proposal establishes a system that rewards and honors
truth-telling, instead of punishing it by loss of employment.
Further, the watchdog remedy requires no technological
breakthroughs to make it feasible.
The only requirement: A firm insistence that it is necessary, an
idea whose time has truly come. We cannot expect governments to be
the source of such insistence. The demand must come from all
segments of society, not just scientists. The sooner that people
create a ground swell of support for such a concept, the earlier
it will become a reality.
I urgently invite all individuals, groups, and organizations to
let me know if they will permit themselves to be included on a
list of endorsers of the watchdog concept.
Last December, the concept received emphatic support from the
international jury of the Right Livelihood Award Foundation in
Stockholm. Subsequently, I have received very favorable reactions
to the proposal from a variety of scientific, environmental, and
political figures in Ukraine, Belarus, and Russia.
The dollar costs of watchdog authorities over the decades will not
be trivial, per database. But the work of the watchdogs is at
least as important to ordinary taxpayers as is the work of the
governmentally sponsored teams underwritten by their taxes. I
suggest that funding for independent watchdog authorities should
come from the same budget that supports governmentally funded
investigators. For the sake of discussion, I suggest five percent
of the budget.
For the Chernobyl database, if the initial budget for IPHECA is
$200 million, five percent would be $10 million. If five percent
of IPHECA's budget (whatever its ultimate size) is transferred to
an independent watchdog authority, I suspect that taxpayers -- or
at least those who were aware of it -- would rejoice.
Why are the nuclear-committed governments so ready to conduct
health studies concerning Chernobyl? And Chelyabinsk? And why do
they continue to govern the studies of Hiroshima and Nagasaki?
They claim, of course, that they conduct such studies for the sake
of truth, and for the benefit of all humanity. But at the
grassroots, ordinary people may judge the sincerity of such claims
by how positively governments respond to the watchdog proposal.
Making the proposal is relatively easy, of course. The hard part
will be building a critical mass of international support to make
it a reality.
The sponsors of current research on radiation and other types of
pollution may fight vigorously behind the scenes to kill the
watchdog idea. And after the watchdog proposal is accepted --
soon, I hope -- people must still remain vigilant. They must
insure that independent experts are not -- or do not become --
sheep who wear a watchdog costume. In the end, we are all
watchdogs. We owe future generations at least that much.
Violating the rules of research
The atomic bomb survivors database is updated and managed by the
Radiation Effects Research Foundation (RERF) in Hiroshima. RERF
is sponsored by the U.S. Energy Department and the Japanese
Ministry of Health. In 1986, RERF virtually replaced the existing
database, which had been used for about 21 years. New doses were
assigned to survivors, many of the study's participants were
suspended, and the remaining participants were shuffled into new
groupings (cohorts). The reason given for these retroactive
alterations was a set of revised dose estimates for both neutrons
and gamma rays, from the 1987 publication, U.S.-Japan Joint
Reassessment of Atomic Bomb Radiation Dosimetry in Hiroshima and
Nagasaki: Final Report.
This revision is a violation of a basic research rule -- that no
retroactive changes in input data are permitted after any results
are known. Parallel analysis with both old and new dose
estimates, without an assault on the cohort structure of an
ongoing study, is an acceptable practice to take account of new
dose estimates, but RERF did not use it. Instead, its maneuvers
with the A-bomb study database create a potential to revise the
results to fit a preferred outcome. In contrast, credible
epidemiological research makes elaborate efforts to avoid this
possibility.
I complained about the rule violation to Dr. Itsuzo Shigematsu,
the RERF chairman. In a letter to me, he replied in part:
"Concern about bias does not appear to be justified. We shall,
however, be sure to consider the necessity, and if so, the
feasibility of dual analyses of our data [altered and non-altered
input]."
RERF is already revising its revised database, which is called
"the basis for any future amendments to the A-bomb survivor
dosimetry that may be desirable." Perpetual, retroactive
alteration seems to be RERF policy.
Meanwhile, RERF provided me with the old "unshuffled" data (the
original database) as well as its new "shuffled" data through
1985 for an independent examination. (This examination is
described in the fourth chapter of Radiation and Chernobyl, This
Generation and Beyond, forthcoming.) For radiation-induced
cancer, rates were calculated using both old and new
dose-estimates with the unshuffled cohorts. Both analyses show
that low-dose radiation is more harmful per dose-unit than high
doses of radiation. By contrast, other analysts using only the
new dose estimates with a shuffled cohort, have published a curve
suggesting that low dose radiation is less carcinogenic per
dose-unit than high-dose radiation.
Similar results were found when the database was shuffled and
instances of radiation-induced mental retardation were examined.
Serious mental impairment occurs when an eight to 26-week-old
fetus receives an appreciable dose of radiation. The unaltered
database suggests that there is no "safe" dose or threshold. But
RERF analysts have frequently suggested that the atomic bomb
survivor study indicates a possible threshold. Almost all the
difference in interpretation can be explained by RERF's choice of
the retroactively altered database; key cases of mental
retardation have been moved from one dose category to another.
The retroactively altered database produces data on cancer and
mental retardation rates that are more favorable to nuclear
polluters than the unaltered database. Database "shufflers"
should be required to demonstrate that no bias was introduced by
their research rule violations, particularly because their
sponsors are not neutral observers. In a matter of such
importance to human health, these rule violations should not be
tolerated at all.
Different rules of research have already been broken in the study
of Chernobyl health effects. By 1989, three years after the
Chernobyl accident, a number of complaints about health problems
in Belarus and Ukraine had reached the press. At the request of
the Soviet government, the International Atomic Energy Agency
(IAEA) organized a study, led by Dr. Shigematsu. In May 1991,
Shigematsu announced that the IAEA's international experts had
found no relationship between illnesses in Belarus and Ukraine
and the release of radiation from Chernobyl.
The IAEA study received a great deal of attention in the public
press, but it was sharply criticized in scientific journals -- as
it should have been. It was flawed in a number of ways. The study
broke more than one of the rules.
IAEA teams sampled seven "contaminated" and six "uncontaminated"
or "control group" villages. The researchers then selected a
radiation-exposed population that had received doses that may
have been less than one rem higher than the doses of those in the
control group -- if they were any higher at all. Under such
conditions, the search for meaningful results was doomed from the
beginning, and the IAEA's conclusion was predictable from the
start. The IAEA did not compare effects in "control" villages
with villages where far higher doses were received (in many of
the villages that were not included in the study, doses exceeded
20 rem).
If only small differences in dose are examined, careful dose
reconstruction is critical. But the IAEA researchers did not do
this adequately. For several days, the Chernobyl 4 reactor burned
graphite, and short-lived and intensely radioactive nuclides were
released, which made later estimates of exposure based only on
levels of cesium 137 unreliable. Using cesium 137 levels rather
than careful measurements of actual biological doses contributed
to inherently questionable conclusions.
--J. G.
See Also: Section 2: The Atomic Bomb Survivors -- A Study and Its
Alteration, from Radiation-Induced Cancer From Low-Dose Exposure,
An Independent Analysis (1990) by Dr. Gofman; especially,
"Chapter 5, A Growing Problem: Retroactive Alteration of the
Study."
Copyright Š 1993 Bulletin of the Atomic Scientists
Reprinted for Fair Use Only.
http://www.ratical.org/radiation/CNR/bewareDD.html (hypertext)
http://www.ratical.org/radiation/CNR/bewareDD.txt (text only)
http://www.ratical.org/radiation/CNR/bewareDD.pdf (print ready)