The Cochrane Collaboration and Pharmaceutical Money
- Briefly describe the Cochrane group
- Review published articles assessing the methodological aspects of their work
- Discuss the pros and cons for the Cochrane Group of accepting pharmaceutical sponsorship
- from http://www.cochrane.org/docs/descrip.htm
- international non-profit and independent organization
- makes ?systematic reviews? of trials available
- founded in 1993
- named for epidemiologist Archie Cochrane
- major work is Cochrane Database of Systematic Reviews
- published quarterly as part of The Cochrane Library (http://www.cochrane.org/reviews/clibintro.htm)
- Organizational structure:
- composed of various review groups (CRGs) (http://www.cochrane.org/contact/entities.htm#CRGLIST)
- groups are coordinated by an editorial team
- from http://www.cochrane.org/docs/commercialsponsorship.htm
- 26-31 October 2003 Barcelona meeting
- Discussed the effects of commercial sponsorship
- recognized this to be a complex issue
- Understand the problem to be _actual_ OR _perceived_ influence from commercial sponsorship
- from The Cochrane Collaboration: Evaluation of Health Care and Services Using Systematic Reviews of the Results of Randomized Controlled Trials
- Cochrane's treatise Effectiveness and Efficacy
- war experiences convinced him that placebo effect and human resilience were responsible for the majority of the benefit of untested/unproven treatments
- "the layman's uncritical belief in the ability of the medical profession at least to help if not to cure" may be unjustified
- Cochrane believed that the layman was asking for doctors to do something, even when nothing could be done.
- Pharmaceutical companies convinced doctors that there was always something they hadn't tried
- "Between 1951 and 1968, requests for pathology tests increased three times, x-ray units of work nearly doubled. The patient expected the doctor to do something to help him: the more the better. The doctor wanted to help, and he could usually think of some new drug he had not tried (ably abetted by the pharmaceutical companies) or of some new diagnostic test (ably assisted by medical research) he had not tried out."
- Cochrane was of the belief that if it wasn't proven scientifically, it probably had no advantage over placebo
- Felt that the RCT would open up a whole new world
- Obstetrician Iain Chalmers implemented first Cochrane-influenced system: a register of perinatal RCTs
- Chalmers implemented extensive procedures of hand-searching journals 'n' shit to make sure that all journals were looked through and all trials were collected
- Randomized trials were collected instead of non-randomized collection methodology to avoid selection biases, which are very potent
- In fact, everything done is to avoid selection biases
- diethylstilbestrol is a good example of selection biases gone terribly bad
- 1950s studies with bad methodology were reported at the same time as randomized control trials
- the bad-methodology studies basically said that everything was peachy-keen, since they compared to "historic" control groups
- simultaneously, RCTs were saying that diethylstilbestrol was not helpful and was probably harmful
- RCTs were largely ignored for the more enthusiastic bad-methodlogy studies, and doctors kept prescribing bad drugs
- In 1969 diethylstilbestrol was shown to cause adenocarcinoma of the vagina among children of mothers treated
- could have been averted if the RCTs were more accessible.. maybe..
- Chalmers implementation was good, except that it lacked the "last mile" -- no summary of the data was given, and the utility to physicians was not large
- Reviews answer this problem, but are problematic in that they are not held up to the same standards
- Enter the "systematic review"
- Chalmers published several books based on reliable evidence
- Cochrane said: "I now have no hesitation whatsoever in withdrawing the slur of the wooden spoon from obstetrics, and I feel honoured by being associated, even in an indirect way, with such an important publication."
- Paper format yielded to online format
- This method's utility widely recognized, so movement towards large-scale adoption of it proceeded in governments, etc.
- NHS founded Cochrane centre to make changes like with the perinatal shizznat
- Is there any guarantee that accessibility of evidence will equate with changes in healthcare practices?
- e.g., antenatal corticosteroid use in preterm babies.
- revealed to be useful.
- Implemented after cochrane review was released
- Slowly resulted in change in that particular practice
methodological aspects of CC
- from Quality of Cochrane reviews: assessment of sample from 1998
- found "major" problems with 29% of 53 reviews published in 1998
- problems consisted of
- conclusions biased in favour of interventions
- problems with methods
- insufficient stylistic or typographical controls
- concludes that the Cochrane collaboration still has some ways to go yet.
- from letter: Quality of Cochrane reviews is better than that of non-Cochrane reviews
- points out that only 52% of all entries at "DARE" database in York University had effectively "systematically assessed the validity of the included studies"
- agrees with the conclusion that the Cochrane Collaboration can improve, but says it's comparatively better than other studies.
- from Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
- is a tongue-in cheek common sense attack on RCTs and systematic reviews.
- shows that no method is completely free of problems
- from Descriptive survey of non-commercial randomised controlled trials in the United Kingdom, 1980-2002
- from Clinical Arithmetic
pros/cons of CC accepting pharmaceutical sponsorship
- from Reports of Randomized Trials in Acute Stroke, 1955 to 1995 - What Proportions Were Commercially Sponsored?
- 25 of 46 trials about acute stroke were sponsored by pharmaceutical companies
- super-shitty reporting of commercial sponsorship
- from Pharmaceutical industry sponsorship and research outcome and quality: systematic review
- studies were more likely to find outcomes favouring sponsor company
- studies were equally rigorous to those not sponsored
- Strange findings attributed to inappropriate comparator groups
- from Cochrane launches global consultation on drug company sponsorship
- no conclusion was reached at the October 2003 conference
- complex consultation of members is under way
- results are supposed to be published by february
- Seems to be an understanding that this is the way of the world
- Professor Gordon Guyatt of McMaster University is now in favour after having been a long-time critic of pharmaceutical influences
- his decision correlates with a recent $150 000 (Â£90 000; â‚¬130 000) grant from Merck Sharpe & Dohme, which makes an osteoporosis drug
- from Cochrane plans to allay fears over industry influence
- Vasiliy Vlassov from the Russian branch of the Nordic Cochrane Center, cited an example from two years ago, when a drug company sought to sponsor a review of its product, provided that it was favourable.
- Martin Offringa, a joint director of the Dutch Cochrane Center, told the BMJ of examples where drug companies had offered funding for reviews, to make their products look "evidence based" for marketing purposes. When the Dutch centre declined and asked instead for generic funding for teaching reviewers, the funding was refused, Dr Offringa said.
- from Reinvention starts here
- laments "academic medicine"'s fall from grace
- RCTs are not as well funded as previously, especially with non-commercial funding bodies
- from Is academic medicine for sale? (western1/uwo)
- drug companies need access to patients more than clinicians
- if it were in fact about clinicians, they would buy them outright
- john norton (psychiatrist) speaks of the hidden pressures that hit him when he spoke ill of a drug by a sponsor company
- could indicate that drug companies may not *directly* influence results, but create this atmosphere
- Stephen A. Young mentions the pressures for prestige etc embedded into the medical system from the outset.
- Firlik & Lowry - nobel aspirations; academic pressures; prestige desires all also influence paper writing and lead to unethical behaviour.
- Angell - drug influence promotes drug research
- fine, give money.. but the terms have to be right!
- human arrogance
- flat earth
- Everything that can be invented has been invented. --Charles H. Duell, Commissioner, U.S. Office of Patents, 1899.
- Cochrane challenge
- stemmed from belief that human body is resilient
- war experiences
- "the layman's uncritical belief in the ability of the medical profession at least to help if not to cure" may be unjustified
- rise of RCTs
- cochrane structure
- rise under chalmers
- acceptance of challenge
- Chalmers buying into systematic reviews
- founding of cochrane collaboration by NHS of britain
- culmination when Cochrane said: "I now have no hesitation whatsoever in withdrawing the slur of the wooden spoon from obstetrics, and I feel honoured by being associated, even in an indirect way, with such an important publication."
- before cochrane/systematic reviews
- diethylstilbestrol is an example of selection bias
- based on belief that if it's not RCT, then it's not worth it
- concept of systematic reviews
- attempt to take standardized set of criteria and apply them
- based on concept that there is not a big enough population (N) in any one given study
- includes *all* studies; published and unpublished
- division of groups, etc.
- paper showing "major" problems with ~ 29% of 53 cochrane papers in 98
- paper showing "major" problems with ~ 48% of 480 DARE papers
- paper poking fun at RCTs and showing they're not always mandatory
Pharmaceutical companies suck!
- Predicting the future:
- "Between 1951 and 1968, requests for pathology tests increased three times, x-ray units of work nearly doubled. The patient expected the doctor to do something to help him: the more the better. The doctor wanted to help, and he could usually think of some new drug he had not tried (ably abetted by the pharmaceutical companies) or of some new diagnostic test (ably assisted by medical research) he had not tried out." - archie
- Cochrane believed that the layman was asking for doctors to do something, even when nothing could be done.
- Speaks of a general mentality of people, even doctors!!
- inherit conflict of interest must be clear!! Not picking on it; just want to exclude those things that can be excluded
Archie and Iain take on the Flat Earth Squad: The Cochrane Collaboration and commercial sponsorship
The Earth was flat for a thousand years, and the sun revolved around the earth for two hundred more. To have asked one of the inhabitants of Flat Earth to imagine a world that was round revolving around the sun is like asking us to imagine a world without capitalism or oxygen. Even half a millennium after the life imprisonment of Galileo at the hands of the church for his claims that ours was a heliocentric universe (1), it is obvious that we continue to make the same mistakes, born of hubris and perpetuated by it. The Earth is still flat, and we refuse to listen to the heretics who would say otherwise.
Archie Cochrane is one of the heretics. During World War II, Cochrane was assigned to twenty thousand patient prisoners with diphtheria, typhoid and other serious illnesses. Only four of his patients died. After his experience, Cochrane concluded that his patients' recovery was due to the â€œrecuperative power of the human bodyâ€ (2) and not the practise of medicine. In his 1972 treatise Effectiveness and Efficiency, Cochrane went on to say that â€œthe layman's uncritical belief in the ability of the medical profession at least to help if not to cureâ€ was unwarranted and even unjustified (quoted in (2)). It was an affront to those practitioners of medicine marauding as scientists. Cochrane had called their bluff, and in so doing issued a challenge that would change the Flat Earth of medical research into a round one.
The problem was simple: medical studies of the time were not rigorously controlled, had serious methodological problems, were unsuitable for answering the problem, or were prone to bias (2). Cochrane intimated that the only way to achieve results beneficial to patients would be to eliminate all sources of bias, which could be done by systematically analyzing all useful experiments conducted on a particular question (i.e., via randomized controlled trials). Using this method, one could prove the efficacy of a treatment. If it could not be proved scientifically, it probably held no advantage over a placebo (2).
Iain Chalmers was the first to accept Cochrane's challenge. Chalmers implemented a register of perinatal randomized controlled trials (RCTs), for which his team searched (by computer and by hand) all published and unpublished work and documented RCTs relating to perinatal care, even surveying researchers in the field to discover unpublished work on the topics of interest. The register was a research feat, but it lacked the was completely inaccessible to physicians (2). For it to be useful, the data had to be objectively analyzed and presented in a usable format.
Enter the systematic review. Systematic reviews allowed for available research to be objectively analyzed and digested into a presentable format. Using methodology such as inclusion of all published and unpublished RCTs and a strict series of criteria for analysis of the data, Chalmers and his cronies were able to assemble a reliable assessment of whether an intervention was scientifically proven (and therefore useful). This effort culminated in the 1989 publication of Effective Care in Pregnancy and Childbirth. Chalmers' success convinced Britain's National Health Service to adopt and promote evidence-based medicine as the preferred standard for assessing treatments and research (3). In 1993, the Cochrane Centre was founded (4) and â€“ through the Cochrane Library's Cochrane Database of Systematic Reviews (CDSR) â€“ has become the largest purveyor of systematic reviews in the world (2) (4).
The Cochrane Collaboration consists of various Cochrane Review Groups, each of which is focused on a particular aspect of research and is coordinated by an editorial team (5). Each group follows predetermined guidelines in searching, acquiring and analyzing research in the field of interest, and then compiles and releases a systematic review based on these guidelines (2) (5). Reviews are subsequently used in clinical practise and policymaking, though accessibility to good science does not necessarily correspond to policy or practise changes (2).
The Cochrane Collaboration is by now held in high regard amongst physicians and policy makers the world over. This influence has led to examinations of the accuracy of Cochrane systematic reviews in relation to other studies and reviews. In one study on Cochrane reviews from 1998, Olsen et al. (6) reported that there were major problems in 29% of the 53 Cochrane reviews investigated. Of the 53, it was reported that the results did not warrant the conclusions in nine (17%), the conduct or reporting of the reviews was unsatisfactory in twelve (23%), and another twelve (23%) had stylistic issues. Olsen et al. (6) concluded that the Cochrane Collaboration still had room for improvement, though they pointed out that reliance on the Cochrane Library's materials was less problematic than reliance on anecdotes or unscientific reviews. Olsen et al. (6) also noted that the solutions to most of the issues raised were in the Cochrane handbooks and guidelines, and that many problems with Cochrane systematic reviews could be avoided if the guidelines were adhered to more tightly. Letters responding to Olsen et al. (6) were quick to point out that other databases, such as DARE, are reported to have major problems in almost half of their reviews (7), and that other appraisals of Cochrane literature after 1998 found a smaller proportion with problems (8). Ultimately, the conclusions reached by all were that the Cochrane Collaboration's work is methodologically superior to other bodies of systematic reviews.
One immutable problem with Cochrane systematic reviews is their insistence on RCTs. In an elegant criticism of the randomized control trial dogma, Smith and Pell (9) asked whether RCTs are necessary to safely conclude that parachutes are an appropriate intervention for â€œmajor trauma related to gravitational challengeâ€. Smith and Pell (9) accurately point out that there has never been an RCT investigating the issue, and that such a fact would make the use of parachutes as good as placebo according to die-hard proponents of systematic reviews. This analysis of the applicability of RCTs demonstrates the limitations inherent to RCTs, limitations long recognized by sociologists and psychologists. Still, if RCTs are possible, there seems little uncertainty of their comparative usefulness.
One problem facing the validity of scientifically-conducted studies has always been money. In the eighteenth century, it was generally recognized that elite society had a vested interest in how various scientific investigations turned out (10), toiling diligently to ensure favourable outcomes. Today, â€œbig pharmâ€ is the new threat (11) (12). Menaced by red ink, institutions have rushed to form alliances with pharmaceutical companies, claiming they facilitate technology transfer and keep institutions fiscally viable, both claims that are refuted (11). While conferring no real societal benefits, the current system of pharmaceutical sponsorship has repeatedly been shown to influence and bias trial outcomes (11) (12) (13) (14). For example, almost all studies sponsored by pharmaceutical companies find that company's product to be the best treatment (11) (14). Why not? Researchers heavily invested in the companies whose products they research are hardly motivated to deliver results that would damage their stock portfolios.
With non-commercial funding for research decreasing, it is little surprise that, late last year, the Cochrane Collaboration began discussing current policy relating to the commercial sponsorship of Cochrane systematic reviews. As it stands, there are quasi-guidelines on how to address conflicts of interest and commercial sponsorship (15). The issue, however, is that Cochrane investigators are increasingly ignoring these guidelines, or otherwise perceive them to be antiquated. The perception of increasing collusion with the pharmaceutical industry, resulting in biased reviews, has motivated members to reevaluate the guidelines on commercial sponsorship and reach consensus on a new set that would allay public fears and face the realities of the new economy (15).
Ultimately, the members of the Cochrane Collaboration know how to maintain the integrity of their reviews â€“ no commercial sponsorship. However, non-commercial funding for RCTs has dropped in recent years (16) (17), and austerity budgets the world over are leaving health ministries ill-equipped to fund research sufficiently. This has led to many well-reasoned proposals on accepting corporate sponsorship without it interfering with trial integrity. One such method is channelling sponsor money into an amorphous fund, such that researchers would be more distant from the sponsors, and sponsors give money to research that is not necessarily tied to their products (11). This measure would help allay fears about corporate tampering in research. However, even if an organization attempts such a fund, it remains to be seen whether sponsors would consider such a program useful (i.e., profitable) enough to participate in. So the question remains: Can the Cochrane Collaboration maintain integrity where no one else has?
Comfort is a false god. And yet, we pursue it so passionately that we deceived ourselves into believing in a flat earth, a geocentric universe, and our own superiority. Today, we pursue Comfort once more. Instead of enduring the pain of non-gala luncheons or second-rate resort getaways, medical practitioners and researchers have chosen the path of most comfort and least resistance. The Cochrane Collaboration started on the noble premise that truth must be the only monkey. Less than two decades after its conception, the quest for provable truth may be diluted in exchange for money. The only question left is: Will it?
- Wikipedia. Galileo Galilei. http://en.wikipedia.org/wiki/Galileo_Galilei (accessed 22 January 2004).
- Dickersin K, Manheimer E. The Cochrane Collaboration: Evaluation of Health Care and Services Using Systematic Reviews of the Results of Randomized Controlled Trials. Clinical Obstetrics and Gynecology. 1998; 41(2): 315-331
- Wikipedia. Evidence-based medicine. http://en.wikipedia.org/wiki/Evidence-based_medicine (accessed 23 January 2004)
- The Cochrane Collaboration. What is the Cochrane Collaboration? http://www.cochrane.org/docs/descrip.htm (accessed 23 January 2004)
- The Cochrane Collaboration. Cochrane Entities. http://www.cochrane.org/contact/entities.htm (accessed 23 January 2004)
- Olsen O, Middleton P, Ezzo J, GÃ¸tzsche P, Hadhazy V, Herxheimer A, Kleijnen J, McIntosh H. Quality of Cochrane reviews: assessment of sample from 1998. British Medical Journal. 2001; 323: 829-832.
- Petticrew M, Wilson P, Wright K, Song F. Quality of Cochrane reviews is better than that of non-Cochrane reviews. British Medical Journal. 2002; 324: 545.
- Handoll H, Madhok R. Another study found that most Cochrane reviews are of a good standard. British Medical Journal. 2002; 324: 545.
- Smith G, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. British Medical Journal. 2003; 327: 1459-1461.
- Currie C. Clinical arithmetic. British Medical Journal. 2003; 327: 1418-1419.
- Angell M. Is academic medicine for sale? New England Journal of Medicine. 2000; 342(20): 1516-1518.
- Bodenheimer T. Uneasy Alliance â€“ Clinical Investigators and the Pharmaceutical Industry. New England Journal of Medicine. 2000; 342(20): 1539-1544.
- Ross L, Norton J, Young S, et al. Letters: Is academic medicine for sale? New England Journal of Medicine. 2000; 343(7): 508-510
- Dorman PJ, Counsell C, Sandercock P. Reports of Randomized Trials in Acute Stroke, 1955 to 1995 - What Proportions Were Commercially Sponsored? Stroke. 1999 Oct; 30(10): 1995-8.
- The Cochrane Collaboration. Commercial sponsorship and the Cochrane Collaboration. http://www.cochrane.org/docs/commercialsponsorship.htm (accessed 25 January 2004)
- Chalmers I, Rounding C, Lock K. Descriptive survey of non-commercial randomised controlled trials in the United Kingdom, 1980-2002. British Medical Journal. 2003; 327: 1017
- Delamothe T. Reinvention starts here. British Medical Journal. 2003; 327
- The Cochrane Collaboration: Evaluation of Health Care and Services Using Systematic Reviews of the Results of Randomized Controlled Trials
- Commercial Sponsorship and the Cochrane Collaboration
- Reports of Randomized Trials in Acute Stroke, 1955 to 1995 - What Proportions Were Commercially Sponsored?
- Quality of Cochrane reviews: assessment of sample from 1998
- letter: Quality of Cochrane reviews is better than that of non-Cochrane reviews
- Editorial: Revisiting the Cochrane Collaboration
- Cochrane launches global consultation on drug company sponsorship
- Cochrane plans to allay fears over industry influence
- Cochrane at crossroads over drug company sponsorship
- Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
- Pharmaceutical industry sponsorship and research outcome and quality: systematic review
- Who'd be a Reformer?
- Reinvention starts here
- Clinical Arithmetic
- Descriptive survey of non-commercial randomised controlled trials in the United Kingdom, 1980-2002
- Uneasy Alliance â€” Clinical Investigators and the Pharmaceutical Industry
- Is academic medicine for sale?