Worst Pills, Best Pills

An expert, independent second opinion on more than 1,800 prescription drugs, over-the-counter medications, and supplements


November 18, 2004

There cannot be two kinds of medicine—conventional and alternative. There is only medicine that has been adequately tested and medicine that has not, medicine that works and medicine that may or may not work.[1]
—New England Journal of Medicine editors Marcia Angell and Jerome Kassirer

Complementary and Alternative Medicine (CAM) is a catch-all term for those healing practices that fall outside of conventional medical practice. CAM includes homeopathy, acupuncture, massage,...

There cannot be two kinds of medicine—conventional and alternative. There is only medicine that has been adequately tested and medicine that has not, medicine that works and medicine that may or may not work.[1]
—New England Journal of Medicine editors Marcia Angell and Jerome Kassirer

Complementary and Alternative Medicine (CAM) is a catch-all term for those healing practices that fall outside of conventional medical practice. CAM includes homeopathy, acupuncture, massage, chiropractic, dietary supplements (including botanicals), meditation, and prayer.[2] Of this prodigious list, this web site concerns only dietary supplements and is limited to the 13 with the largest sales in 2002. Dietary supplements are defined by law as products intended to supplement the diet that contain a vitamin, a mineral, an herb or other botanical, an amino acid, or “a dietary substance for use by man to supplement the diet by increasing the total dietary intake.”[3] Vitamins and Minerals are considered separately.

Plant-based medicines are not inherently dangerous, inherently safe, nor inherently effective. Digitalis lanata, or foxglove, provides digoxin (LANOXIN) for treating heart conditions, and Papaver somniferum, the opium poppy, is the source of morphine, still one of the best pain relievers known. On the other hand, Atropa belladonna, or deadly nightshade, and Strychnos nuxvomica, the source of strychnine, are two of history’s more famous poisons. A long history of use is proof of neither safety nor effectiveness.

Growing Use of Dietary Supplements

The use of dietary supplements and other forms of CAM is widespread and growing.[4],[5] One national study, based on data from 1997, estimated that two-thirds of respondents had used at least one CAM therapy in their lifetimes and that younger people were more likely to do so.[5] A very large national survey estimated that in 2002, 62% of adults used at least one CAM therapy; this number fell to 36% if prayer for health reasons was excluded.[6] The use of “nonvitamin, nonmineral, natural products” in 2002 was reported by 19% of respondents, equivalent to 38 million U.S. residents. The leading products were echinacea (40% of those who used these products in 2002), ginseng (24%), ginkgo biloba (21%), and garlic (20%). All of these products are reviewed on this web site.

Among often-desperate cancer patients, the prevalence of dietary supplement use may be higher than in the general population, perhaps in the range of 21 to 54%.[7],[8] In another study, 63% of cancer patients were believed to have used either vitamins or herbs.[8] According to the National Academy of Sciences, the sale of dietary supplements alone is now an $18 billion-a-year industry.[9]

A large amount of this use occurs below the sightlines of the medical profession. As many as one-third of cancer patients do not disclose their use of vitamins or herbs to their physicians.[7],[10] We encourage all CAM users to be open about their use with their health care providers. Without this information, providers cannot interpret any benefits or adverse effects you may be experiencing.

Regulatory Status of Dietary Supplements

The enormous increase in the sales of dietary supplements did not simply happen by accident. Although there are societal trends favoring products that call themselves “natural” and growing suspicion of the medical establishment by some, the growth in dietary supplement sales is in significant part the result of a single act of Congress: the 1994 Dietary Supplement Health and Education Act (DSHEA). The history of the act is instructive. In the late 1980s, an outbreak of a mysterious condition called eosinophilia-myalgia syndrome occurred among users of a supplement called L-tryptophan, an unproven insomnia remedy. The exact cause of the syndrome was never determined but is assumed to be a contaminant in batches of the supplement produced by a Japanese manufacturer. All told, over 1,500 people fell ill, with at least 37 deaths.[11]

These events strengthened the FDA’s desire to regulate dietary supplements more strictly. In particular, the agency hoped to reduce misleading promotional claims being made for these products and to develop standards for Good Manufacturing Practice (GMP). The effort had the opposite effect. Dietary supplement manufacturers enlisted the aid of Senator Orrin Hatch of Utah, a state with a strong supplement industry, and solicited some 200,000 letters condemning the FDA’s alleged effort to take away the citizenry’s vitamins. The result was DSHEA, which left supplements less regulated than before. Supplement manufacturers have been quick to exploit the opportunity and, by judicious campaign contributions,[12] have reduced the likelihood of the statute’s repeal.

DSHEA clarified that supplements were to be regulated essentially as foods, not drugs, and thus were exempt from the tougher regulation accorded to drugs. In particular, there is no requirement for companies to register either their names or their products with the government. The FDA was charged with promulgating regulations describing GMPs for dietary supplements, but a decade later these have still not been finalized. There is thus no regulation ensuring that the amount of a substance claimed to be in a supplement is actually present, or that contaminants or active drugs are not present. For each dietary supplement we discuss, we include any available published reports of contamination or instances where supplement content varied significantly from the labeled amount. In one study of products not directly evaluated on this web site, 32% of Asian patent medicines examined were found to be contaminated with heavy metals (e.g., lead, arsenic, and mercury) or undeclared pharmaceuticals (e.g., chlorpheniramine [CHLOR-TRIMETON], methyltestosterone, and phenacetin, a banned painkiller).[13] In addition, the quantity of active ingredient can vary according to the particular species, what part of the plant is used, what time of year the plant is harvested, and how the ingredient is extracted from the plant. We are also particularly concerned about a subset of dietary supplements called glandulars (literally crushed organs from other animals); the possibility that supplements containing cow brain might contain the agent that causes mad cow disease has been raised.[14],[15]

Supplement manufacturers wish to distinguish their products from drugs because prior to marketing, all drugs have to demonstrate their safety and effectiveness. By having their products classified as foods, supplement manufacturers evade this critical step, which can take time and money and prove unsuccessful. Thus, no supplement has been demonstrated to be safe and effective under the standards the FDA applies to drugs. Furthermore, while drug companies have to report any adverse events they learn about to the FDA, there is no requirement for the manufacturers of dietary supplements to do so. The Department of Health and Human Services inspector general described the system as “an inadequate safety valve” and noted that under 1% of adverse events associated with dietary supplements are reported to the FDA.[16] A classic example of this was Metabolife, a leading manufacturer of ephedra, which had 18,502 adverse reaction reports in its files, some on scraps of paper, none of which it initially reported to the FDA.[17]

If these products have never been proved safe and effective, how can they continue to be promoted so aggressively? The FDA makes the kind of distinction only a lawyer could love: the regulatory structure for promotions regarding dietary supplements draws on the “distinction” between health claims (not permitted) and structure/function claims (permitted). For example, a supplement manufacturer, without any supporting evidence whatsoever, can assert that its product “promotes prostate health” (a structure/function claim) but is precluded from claiming that it “treats the symptoms of an enlarged prostate” (a health claim). Of course, an older man with difficulty urinating, inability to empty the bladder, and the need to urinate at night understands that a product that “promotes prostate health” is intended to treat prostate enlargement, but Congress permits and even encourages this charade. Occasionally, a company may step over the line and make a health claim so outlandish that even the sensibilities of the Federal Trade Commission (FTC), which regulates the advertising of dietary supplements, are offended and the agency takes action, but these actions are far outweighed by misleading promotions that pass by undisturbed. For example, an FTC report on weight-loss supplements estimated that close to 40% of weight-loss advertisements in 2001, 66% of which were for dietary supplements, made at least one claim that was almost certainly false.[18]

Although the companies are not required to prove safety prior to marketing, Congress went still further by raising the bar for removing a supplement from the market due to lack of safety. The FDA can act only if it can prove a “significant or unreasonable risk of illness or injury under conditions of use suggested or recommended in the labeling.”[19] This authority has been used only once, for the dangerous supplement ephedra, and the agency’s authority to take such action has been challenged, so far unsuccessfully, in court. With the standard for removal from the market so high, the FDA has been forced to protect the public by releasing a series of warnings for certain supplements, including the following:[20]

  • androstenedione, an anabolic steroid
  • aristolochic acid, associated with kidney failure and urinary tract cancer
  • comfrey, associated with liver toxicity
  • kava kava, associated with liver toxicity

Assessing the Effectiveness of Dietary Supplements

For centuries, the basis for establishing the effectiveness of medicines had been the health care provider’s personal experience combined with an oral and written tradition whereby interventions of purported effectiveness were taught to more junior physicians and students. That was all to change. In October 1948, the British Medical Journal published what is arguably the first randomized, controlled trial: a study that proved that streptomycin was more effective than placebo in the treatment of tuberculosis of the lung.[21] The publication of that trial ushered in a new era in modern medicine, one in which rigorous empiricism was to supplant the personal experiences of oneself or others. Only through such methodological rigor, it was argued, could the safety and effectiveness of medicines and other interventions be determined with adequate certainty to justify exposing the public to them. Many medications (and even surgeries) that formed the core of medical practice for decades or even centuries have been debunked by these experiments.[22] Others have proved surprisingly effective.

The ideas of this movement, now carrying the soubriquet evidence-based medicine, provide the theoretical basis for this web site. We apply its principles in all sections, no less so in this one, for there is no reason to think that the movement’s principles cannot and should not be applied to CAM.[2] While evidence can come in various forms, the gold standard for evidence is the randomized controlled trial.

Let us consider a hypothetical study in which we are trying to determine if a putative cholesterol-lowering drug can actually lower cholesterol. Imagine, too, that one group gets the actual drug and the other (the control group) a dummy pill that looks and tastes like the actual drug but is actually inactive (a placebo). Both groups get dietary counseling. In our study, participants are assigned by a systematic process called randomization to either the drug or placebo group, in effect by the toss of a coin; each person is as likely as any other to be assigned to drug or placebo. This ensures that the starting average cholesterols in the two groups will be about equal. Equally important, other characteristics, even those that cannot be measured, should also occur about equally in the two groups. Thus, if in the future someone discovers that, unexpectedly, eye color is a factor in determining cholesterol levels, the designer of our study will be able to say with some assurance that the number of blue-eyed people in the drug and placebo groups was about equal, even if he or she never measured eye color.

If the drug is compared to a placebo, as in our hypothetical trial, it is a special case of the randomized, controlled trial called a placebo-controlled trial. Often one is less interested in whether the putative drug is better than nothing and more interested in knowing whether it is as effective as, or even better than, established treatments. In this case, one would compare the drug to a known treatment in another kind of randomized, controlled trial called an active-controlled trial. In some cases, especially when the condition being treated is life-threatening and the existing treatments are particularly effective, randomizing volunteers to placebos would be unethical. (An ongoing concern regarding all dietary supplements, especially those that are totally ineffective, is that they might supplant more effective medications or delay effective therapy.)

But someone might criticize our trial of the cholesterol-lowering drug by arguing that if the volunteers knew they were in the placebo group they might watch their diets more carefully than those who thought they were protected by the new drug. For this reason, researchers generally purposefully do not disclose to the study volunteers which group they are in; this process is called blinding. (The volunteers are told in an informed consent form that they will not be told to which group they were assigned until after the study is completed.) In order to prevent any bias, the doctors also are not told the assignment of their patients; if they were told, perhaps they would encourage placebo patients to be more careful with their diets or to exercise more. Such a study is called a double-blind study; if it is also randomized and has a placebo control, it is called (naturally enough) a randomized, double-blind, placebo-controlled trial.

For the most part, the dietary supplements addressed on this web site are not being advocated for the treatment of life-threatening illnesses and in some instances the condition in question may not even have an effective treatment. Thus, we believe that each supplement considered on this site should have to demonstrate effectiveness (and safety) in the way the FDA requires for many drugs: by proving effectiveness in two randomized, double-blind, placebo-controlled trials.

Reading the profiles of each dietary supplement on this web site is a bit like reliving medical history. In general, the supplements are first put forth as effective based on word of mouth or long histories of use, often by traditional groups. As demands for evidence escalate, studies are often done, though these often are not randomized, blinded, or placebo-controlled. Many of the studies collated in the oft-cited, largely unreferenced German Commission E are of this sort; these poorly conducted studies are often lauded by proponents and promoters of dietary supplements as evidence of effectiveness. By the late 1990s, studies with the elements described above have usually been done, although even these can be poorly executed (e.g., too small, too short, too many volunteers lost to follow-up). Recently, well-conducted randomized, double-blind, placebo-controlled trials have appeared for a minority of supplements on this site and provide a reasonable basis for beginning to assess the safety and effectiveness of these products.

How Dietary Supplement Profiles Were Compiled

For other drugs reviewed on this web site, we have had access to published articles describing randomized, controlled trials in medical journals, medical textbooks, the FDA-approved label, and, importantly, the detailed review of the drug (based on a review of the raw data from the sponsor’s clinical trials) conducted by the FDA medical officer, at least for more recent drugs (see more detail on our methods in the FAQ section of the web site). This evidence base is far from complete for any dietary supplement. By definition, no supplement has passed an FDA safety and efficacy review (otherwise it would be a drug). This means that not only have none of these products been proved safe and effective by conventional standards, but also, in general, there has been no requirement for testing for cancer or birth defects, no studies to establish the optimal dose or dosing schedule, no chemical tests to make sure the active ingredient gets into the body consistently or is consistently present from batch to batch, and no assurance that contaminants are not present.

The dietary supplements and herbs covered on this web site were identified using a report obtained from the Nutrition Business Journal that ranked the 70 top supplements by U.S. consumer sales in 2002.[23] Many of these supplements are vitamins and minerals. The supplements reviewed on this web site include all other supplements (other than enzymes and hormones) ranked in the top 40. Their 2002 retail sales ranged from $59 million for black cohosh to $1.4 billion for ephedra combinations.

The principal source of our information on dietary supplements has been PubMed, the world’s largest computerized database of medical journal articles. Unfortunately, this is far from foolproof. For St. John's wort  and glucosamine, in particular, there is evidence of publication bias—the selective publication of favorable results. There have been allegations of scientific misconduct leveled against some authors of published articles on dietary supplements.[24],[25],[26],[27] Nonetheless, due to the essentially unregulated nature of these products, these published articles represent the majority of the available data.

In PubMed, we searched for articles on the effectiveness of each supplement (including common synonyms), characterized in the database as “controlled trials,” and restricted ourselves to trials in humans and published in English. In contemporary medicine, the vast majority of important studies now appear in the English language literature. For the drugs reviewed on this web site, the most recent, best-conducted studies all were published in English (there were no references in the articles we reviewed to recent pivotal trials in other languages). We then reviewed the search results, excluding small studies (generally less than 20 subjects in each study group), those evaluating products with multiple ingredients (the effect of any one ingredient cannot be determined), topical or intravenous products, and those in which the study volunteers got only one or two doses of the drug to measure acute effects. We considered only randomized studies with clinically relevant, relatively “hard” outcomes (e.g., generally not those measuring vague outcomes or only the results of blood tests) and did not consider studies of products intended to improve general well-being (as opposed to specific medical conditions) or performance in sporting activities. We collected both placebo- and active-controlled trials, but only considered the latter if the former indicated a benefit. The articles themselves had to be available either from the library of a major national medical center (electronically or in hard copy) or in hard copy at the library of the National Institutes of Health. The rare articles we could not secure through these means were considered too obscure as well as, for practical purposes, inaccessible to the general public. A second search using the supplement name and the search term “adverse events” and the same restrictions as above identified studies on interactions and adverse effects of the supplements. We also conducted separate searches using the search terms “mutagenicity” (for damage to the genes) and “carcinogenicity” (for propensity to cause cancer).

In addition to using the published medical literature, we relied on the actions and publications of several drug regulatory authorities around the world. These include the U.S. FDA, Health Canada, the Medicines Control Agency—Committee on Safety of Medicines in the United Kingdom, the Therapeutics Goods Administration in Australia, and New Zealand’s Medicines and Medical Devices Safety Authority. We put particular emphasis on the views of the respected pharmacology publications The Medical Letter, the Drug and Therapeutics Bulletin, and Prescrire International, which do not accept any support from the pharmaceutical industry. We obtained all available meta-analyses (a statistical technique for combining multiple studies) from the Cochrane Library and searched the 2003 edition of Evaluations of Drug Interactions for interactions classified as “highly clinically significant” or “clinically significant.” The brand names listed include all those containing the particular dietary supplement listed in the 2002 Physicians’ Desk Reference for Nonprescription Drugs and Dietary Supplements or mentioned repeatedly in research articles.

Despite the relative paucity of information available on dietary supplements, we attempted to review these products with the same degree of skepticism that we accord all the drugs on this site. In effect, we asked: Given the available evidence, would this product have been approved as a drug by the FDA?