science can be so depressing

Excerpts from an interesting piece in the WSJ from David Armstrong and Keith Winstein on anti-depressants in particular-- and more broadly, the bias toward "positive" results in science and the limited ability of the FDA to be effective...

The effectiveness of a dozen popular antidepressants has been exaggerated by selective publication of favorable results, according to a review of unpublished data submitted to the Food and Drug Administration.

As a result, doctors and patients are getting a distorted view of how well blockbuster antidepressants like Wyeth's Effexor and Pfizer Inc.'s Zoloft really work, researchers asserted in this week's New England Journal of Medicine.

Since the overwhelming amount of published data on the drugs show they are effective, doctors unaware of the unpublished data are making inappropriate prescribing decisions that aren't in the best interest of their patients, according to researchers led by Erick Turner, a psychiatrist at Oregon Health & Science University. Sales of antidepressants total about $21 billion a year, according to IMS Health.

Here's what the drug companies say about what they do...

Wyeth and Pfizer declined to comment on the study results. Both companies said they had committed to disclose all study results, although not necessarily in medical journals. GlaxoSmithKline PLC, maker of Wellbutrin and Paxil, said it has posted the results of more than 3,000 trials involving 82 medications on its Web site, and also has filed information on 1,060 continuing trials at a federal government Web site.

Schering-Plough Corp., whose Organon Corp. unit markets Remeron, and Eli Lilly & Co., which makes Prozac, said their study results were indeed published -- not individually, but as part of larger medical articles that combined data from more than one study at a time. The New England Journal study counted a clinical trial as published only if it was the sole subject of an article. "Lilly has a policy that we disclose and publish all the results from our clinical trials, regardless of the outcomes from them," a Lilly spokeswoman said.

Pharmaceutical companies are under no obligation to publish the studies they sponsor and submit to the FDA, nor are the researchers they hire to do the work. The researchers publishing in the New England Journal were able to identify unpublished studies by obtaining and comparing documents filed by the companies with the FDA against databases of medical publications.

And here's what the FDA says about this...

"There is no effort on the part of the FDA to withhold or to not post drug review documents," an FDA representative said. For newer drugs, information is posted online "as soon as possible." Older documents aren't always available online and efforts to add those files to the Web are slowed by "a lack of resources," the agency said, acknowledging that there is a backlog in complying with records requests.

Here's how all of this biases what we think we know...

A total of 74 studies involving a dozen antidepressants and 12,564 patients were registered with the FDA from 1987 through 2004. The FDA considered 38 of the studies to be positive. All but one of those studies was published, the researchers said.

The other 36 were found to have negative or questionable results by the FDA. Most of those studies -- 22 out of 36 -- weren't published, the researchers found. Of the 14 that were published, the researchers said at least 11 of those studies mis-characterized the results and presented a negative study as positive....

In this week's study, the researchers found that failing to publish negative findings inflated the reported effectiveness of all 12 of the antidepressants studied, which were approved between 1987 and 2004. The researchers used a measurement called effect size. The larger the effect size, the greater the impact of a treatment.

[For example], The average effect size of the antidepressant Zoloft rose 64% by the failure to publish negative or questionable data on the drug, the researchers found.