The Role of Independent Analyses in Research on Toxic Agents
If we let "XYZ" be the generic name for some specific chemical or physical toxic agent -- explicitly excluding ionizing radiation -- we can briefly examine the role of independent analyses in research on the carcinogenicity of XYZ. One opinion is that conflict-of-interest creates no problem in scientific research. A different opinion, however, is that conflict-of-interest in research constitutes a legitimate concern and legitimate topic for open discussion.
Position of Medical Journal Editors :
The International Committee of Medical Journal Editors considers conflict-of-interest so important that the Committee recently moved to bring the topic into full view when the Committee amended its "Uniform Requirements for Manuscripts Submitted to Biomedical Journals." The new passages on conflict-of-interest are quoted by Lancet in its issue of September 14, 1985, page 595 (we are omitting other new passages on true authorship, for instance):
In every article, "one or more statements should specify . . . (c) acknowledgements of financial and material support, and (d) financial relationships that may constitute a conflict of interest . . . Financial or material support from any source must be specified. If a paper is accepted it may also be appropriate to include mention of other financial relationships that raise a conflict of interest, but initially these should be outlined in the covering letter."
"Manuscripts must be accompanied by a covering letter. This must include: . . . (b) a statement of financial or other relationships that might lead to a conflict of interests . . . "
After quoting these and other new passages from the "Uniform Requirements," Lancet explicitly states that an author's financial sponsorship may be a potential source of bias when Lancet says:
"On conflict of interest, firm guidelines are even more difficult to formulate; as a first step, perhaps authors should ask themselves whether they would be embarrassed if financial or other potential sources of bias came to light after publication."
Sponsorship of XYZ Research :
Both logic and observation confirm that most people who need to expose other people to XYZ (either directly or environmentally) have a preference for analysts who say such exposures create a negligible amount of cancer -- or better still, none at all.
Therefore, in terms of protecting human health, it would be inherently unsafe if the XYZ community sponsored and thus controlled nearly all research on the carcinogenicity of XYZ. Scientists in the XYZ field would quickly learn the need for prudence about anything which would upset such sponsors -- if the scientists wished to have their grants renewed, their papers published, their nominations to XYZ advisory committees approved, and generally wished to have a comfortable future in their field.
If You Own the Consensus, How Can You Lose?
In such a situation, one predictable result of the funding would be the extreme scarcity of "boat-rockers" and the extreme abundance of sponsor-friendly and self-censoring XYZ experts.
Similar statements from the latter about the cancer-hazard from XYZ exposure would indeed constitute the overwhelming consensus in the field. Moreover, due to the very wide distribution of grants by the XYZ community, the consensus would appear to arise from a great variety of disinterested sources: Medical centers, schools of public health, schools of veterinary science, departments of environmental sciences, epidemiology, biostatistics, physics, biology, toxicology.
Nonetheless, an "overwhelming consensus of XYZ experts" might be artificial, under the circumstances.
Who Controls the Input?
The situation would be even more menacing, in terms of undistorted estimates of XYZ carcinogenicity, if the XYZ promoters also tightly controlled the raw data which analysts must use to reach any conclusions at all. To the extent that only sponsor-friendly insiders would be enabled to generate, collect, sort, or revise the actual observations, then anyone using those data -- dependent and independent analysts alike -- would be at the mercy of XYZ partisans.
With all the revelations during recent years -- in field after field -- about falsified research, falsified safety-testing, falsified performance-testing, falsified cost-reports, and falsified pollution-reports, there is nothing far-fetched about the prospect in XYZ research that both dependent and independent analysts could get deceived by falsified databases (see Chapter 24, Part 1). However, chances are high that only the independent analysts would ever raise such a question. The sky almost always falls on anyone who does (Nova88).
The Role in One Sentence :
The role of independent analysts in research on toxic agents is this: They try to find out if evaluation of XYZ toxicity is the same -- when conflict-of-interest is missing -- as the evaluation when conflict-of-interest is present.
It must be emphasized that the presence of conflict-of-interest does not automatically make every dependent analysis wrong. Of course not. And it must be equally emphasized that the absence of conflict-of-interest does not automatically make every independent analysis right. Independent analysts can suffer from other biases, and we can make occasional mistakes as readily as anyone.
Substance, Not Source :
When I was entering science in the late 1930s, it seemed to be understood that what matters in any analysis is its scientific merit -- its substance, not its source. Readers do not need a "double standard." The first question about independent and dependent analyses alike is: Do the authors show how they proceed, step-by-step, from the raw data to the arrival at their conclusions? If they do not, they are expecting you to care more about the source than the substance.
Complaints about Independence :
Taught by fifty years of observation, we know that many members of the XYZ community use innuendo to cope with independent analysts: "But some of their methods are not the methods we use," or "Their work is not peer-reviewed.
Such complaints are meant to suggest that the independent methods and results are wrong. But it is not good enough for critics to cast aspersions over a whole analysis unless they are prepared to debate its substance on a scientific level.
When independent analysts take an independent look at a problem, they well may use independent methods. A presumption that independent methods are inferior to the conventional approach would be an unwarranted bias. After all, independent methods may be more appropriate and superior to conventional approaches, especially if conventional approaches were chosen in order to miss or obscure an unwelcome reality.
With regard to peer-review, the peer-review system itself is under serious criticism for inattention to scientific standards (see, for instance, Renn86, Renn88). In any case, XYZ committees can and do self-publish their own work without any control or veto-power from independent peers. Likewise, independent peers may publish their work in channels located beyond the veto-power of the XYZ community.
Whether a report is by an XYZ committee or by an independent analyst, what will count in the end -- with regard to human health -- is the work's scientific content. The rest is noise.