Are Clinical Trials Misleading?

Written by Dr. Steve Chaney on . Posted in clinical trials

Is Most Of What You’ve Been Told About Vitamins Wrong?

Author: Dr. Stephen Chaney

 

man searching with magnifying glassI am a scientist and a professor. I taught medical students for 40 years. I believe in evidence based medicine. Why would I tell you that many of the clinical trials about the impact of individual nutrients on your health are misleading?

Let me start by sharing a story that I used to tell every new graduate student in my lab. The story goes like this: There is this drunk on the sidewalk, on his hands and knees under a lamppost, just groping around. A policeman comes up to him and says, “What are you doing?” The drunk says, “I’m looking for my housekeys.” The policeman gets down on his hands and knees and he looks too, and finally he says, “I can’t find them anywhere. Are you sure you lost them here?” To which the drunk relies, “Nope, I lost them over there, but the light’s better here.”

The point I was trying to make is that we can only do experiments where the light is good. But the questions we sometimes want to ask are over in the corner, where we can’t really shine the light on it directly. It’s often difficult to look in the right place and/or to ask the right questions.

That’s particularly the case with holistic approaches because holistic approaches, by their very nature, are multi-factorial. You have multiple variables that you’re trying to change at one time. For example, you might want to optimize weight, exercise, vitamins, minerals, and essential fatty acids if you’re trying to look at a healthy lifestyle.

But, in the 21st-century, studies generally focus on individual nutrients or individual drugs in an intervention, placebo-controlled trial. This is considered the “Gold Standard” for evidence based medicine. However, it’s very difficult to evaluate holistic approaches with that kind of study.

 

The Whole Is Greater Than The Parts

internationally renowned expert sessionOne of the examples that I love to use, because it really made an impression on me as a young scientist, occurred at an International Cancer Symposium I attended more than 30 years ago.

I attended a session in which an internationally renowned expert was giving his talk on colon cancer. He said, “I can show you, unequivocally, that colon cancer risk is significantly decreased by a lifestyle that includes a high-fiber diet, a low-fat diet, adequate calcium, adequate B-vitamins, exercise and weight control. But I can’t show you that any one of them, by themselves, is effective.”

The question that came to me as I heard him speak was: “What’s the message that a responsible scientist or responsible health professional should be giving to their patients or the people that they’re advising?” You’ve heard experts saying: “Don’t worry about the fat” “Don’t worry about calcium.” “Don’t worry about B-vitamins.” “Don’t worry about fiber.” “None of them can be shown to decrease the risk of colon cancer.”

Is that the message that we should be giving people? Or should we really be saying what that doctor said many years ago – that a lifestyle that includes all those things significantly decreases the risk of colon cancer?

 

Are Clinical Trials Misleading?

 

clinical trialsA recent paper about how to best evaluate the relationships between nutrition and disease (Shao et al, European Journal of Nutrition, DOI: 10.1007/s00394-017-1460-9) caught my attention. This paper, written by a team of 10 international experts, was a summary of key findings from a recent international meeting of the Council for Responsible Nutrition.

The paper started out by reviewing the strengths of clinical studies in which the effect of a single intervention on a health outcome is evaluated in a double-blind, placebo controlled clinical study; something they referred to as a reductionist approach.

  • A reductionist approach is ideal for evaluating the effect of drug candidates on disease outcomes. That is because:
    • Everyone in the study already has the disease.
    • The drug is meant to be used by itself.
    • It is easy to measure outcomes. The drug either has an effect on the disease, or it doesn’t.
  • A reductionist approach has also been valuable in defining the role of nutrients in preventing deficiency diseases. That is because, in the words of the authors:
    • “A simple cause-effect relationship exists between a particular nutrient and a specific deficiency disease.
    • Symptoms of a specific nutrient deficiency can be explained in terms of the role played by the respective nutrient.
    • Providing the nutrient in the diet can prevent, and in many cases, reverse, the deficiency disease.”

However, the authors went on to say that the use of the reductionist approach to study effect of nutrients on optimal health or holistic approaches to health often has led to misleading results. They characterized these studies as often “leading down a rabbit hole.”

For example, the authors said: “In an effort to uncover the magic bullet, scientists inappropriately studied nutrients in a drug-like context. Unlike drugs, nutrients do not function in isolation and have beneficial effects on multiple tissues and organ systems.”

The authors concluded by saying that if we want to truly understand the role of nutrients on health outcomes, we need to focus on holistic studies in which the effect of multiple nutrients on multiple health outcomes are evaluated.

 

Clinical Trials That Have Mislead Us

 

I realize that the report I just described is conceptual. It’s difficult to wrap your mind around. To better understand how clinical trials employing a reductionist approach can often mislead us, let’s look at some specific examples comparing holistic studies to reductionist studies.

dash dietHealthy diets: Healthy diets have a significant impact on health, but it is not possible to show that individual components of those diets are beneficial: In previous issues of “Health Tips From the Professor,” I have discussed the Mediterranean and DASH diets. I have shared studies showing that the Mediterranean diet dramatically reduces the risk of heart disease, diabetes, cognitive decline, and some forms of cancer. However, you would be hard pressed to show that individual components of the Mediterranean diet have a significant impact on these health outcomes.

Similarly, the DASH diet is as effective as drugs at controlling blood pressure (Moore et al, Hypertension, 38: 155-158, 2001 ). Other than sodium restriction, you would also be hard pressed to show that the individual components of the DASH diet exert a significant effect on blood pressure.

Supplements That Are Going to Kill You: Individual nutrients can sometimes have adverse effects on your health. Those reports generate a lot of negative press, but the adverse effects usually disappear when those nutrients are consumed along with nutrients that complement their effect on whole body metabolism.

Here are two examples of the negative press that you may have heard about the dangers of supplementation, but what the studies actually showed is that a holistic approach to supplementation was superior to supplementation with individual supplements.

For example, there was something called the Iowa Women’s Health Study that got some negative press in 2011 (Mursu et al, Archives of Internal Medicine, 171:1625-1633, 2011). This is one of those studies that led to headlines saying: “Vitamins can kill you.”

The study did show a slight increase in mortality in people who consumed high-dose vitamin B6 or high-dose folic acid by themselves. But in that same study, people who were taking high-dose B complex containing both B6 and folic acid in balance had no increase in mortality.

Another example is vitamin E and prostate cancer. You probably saw the headlines, which said: “Vitamin E increases the risk of prostate cancer.” Those headlines were based on a study published in the Journal of American Medical Association in 2011 (J Klein et al, Journal of the American Medical Association, 306: 1549-1556, 2011). However, in that same study the people who were taking vitamin E and selenium (two nutrients that work together synergistically) had no increase in cancer risk.

There is a good biochemical rationale for those results. Vitamin E converts some reactive oxygen species to peroxides, which are quite dangerous themselves. Selenium is part of an enzyme that converts peroxides to water. Together, vitamin E and selenium convert reactive oxygen species (free radicals) to something that is completely harmless. By itself, vitamin E does only half the job.

Holistic Approaches to Supplementation: The same appears to be true if you look at holistic approaches to supplementation rather than holistic approach to supplementationsupplementing with individual nutrients. A study done by Dr. Gladys Block and published in Nutrition Journal in 2007 (Block et al, Nutrition Journal 2007,6:30 doi: 10.1186/1475-2891-6-30) looked at a holistic approach to supplementation for the very first time.

She compared people who were taking multiple supplements, typically a multivitamin, extra antioxidants, extra B vitamins, carotenoids, fish oil and probiotics; people who were taking only a multivitamin; and people who were using no supplements whatsoever over a 20-year period.

The results were dramatic. The holistic supplement users had one-third the prevalence of angina, heart attacks and congestive heart failure and one-quarter the prevalence of diabetes compared to the other two groups. In contrast, reductionist studies looking at the effect of those nutrients individually have generally been inconclusive.

So just like a holistic approach to health, a holistic approach to supplementation appears to be superior to using individual supplements. This is a small study, but it is an example of the kinds of studies that need to be done in the future, if we are to truly understand the role of holistic approaches for optimizing our health.

 

The Bottom Line

Studies in which the effect of a single intervention on health outcomes is evaluated in a double-blind, placebo-controlled clinical study is considered the “Gold Standard” for evidence based medicine. A recent report has questioned the value of this kind of study in defining the impact of holistic approaches on health outcomes.

  • The authors concluded that the “Gold Standard” of clinical studies, which they referred to as a reductionist approach:
    • Was ideal for evaluating the effect of drugs on preventing or treating diseases.
    • Has been well suited for evaluating the role of individual nutrients in preventing deficiency diseases.
    • Was not well suited for evaluating the role of holistic approaches on health outcomes.
    • Was not well suited for evaluating the role of nutrients for promoting optimal health.
  • The authors concluded by saying that if we want to truly understand the role of nutrients on health outcomes, we need to focus on holistic studies in which the effect of multiple nutrients on multiple health outcomes are evaluated.
  • I shared three examples illustrating cases in which holistic approaches were more accurate than reductionist studies:
    • Healthy diets have a significant impact on health, but it is not possible to show that that individual components of those diets are beneficial.
    • Individual nutrients can sometimes have adverse effects on your health, but the adverse effects disappear when those nutrients are consumed along with nutrients that complement their effect(s) on whole body metabolism.
    • A holistic approach to supplementation can have a significant, beneficial effect on health outcomes, but it is difficult to show any benefit from individual nutrients included in that holistic approach to supplementation.
  • For more details, read the article above.

 

These statements have not been evaluated by the Food and Drug Administration. This information is not intended to diagnose, treat, cure or prevent any disease.

Trackback from your site.

Comments (2)

  • Alayne Campbell

    |

    Very informative, A good yardstick to measure by.

    How do I get a copy of your suggestions for those on kemotherapy?

    Reply

    • Dr. Steve Chaney

      |

      Dear Alayne,
      I am not a medical doctor, so I do not comment on specific medical cases. If you are talking about nutritional supplements, a general rule of thumb is to space supplements and chemotherapy drugs so that they are not present in the bloodstream at the same time. This usually involves a day or two window on either side of chemotherapy during which supplements are not taken. For more specific recommendations, consult your doctor and/or your pharmacist.
      Dr. Chaney

      Reply

Leave a comment

Recent Videos From Dr. Steve Chaney

READ THE ARTICLE
READ THE ARTICLE

Latest Article

Does Protein Supplement Timing Matter?

Posted May 15, 2018 by Dr. Steve Chaney

How Do You Gain Muscle Mass & Lose Fat Mass?

Author: Dr. Stephen Chaney

 

protein supplement timingMost of what you read about protein supplements on the internet is wrong. That is because most published studies on protein supplements:

  • Are very small
  • Are not double blinded.
    • Both the subjects and the investigators knew who got the protein supplement.
  • Are done by individual companies with their product.
    • You have no idea which ingredients are in their product are responsible for the effects they report.
    • You have no idea how their product compares with other protein products.
    • There is no standardization with respect to the amount or type of protein or the addition of non-protein ingredients.

Because of these limitations there is a lot of misleading information on the benefits of protein supplements timing and maximal benefit. Let’s start by looking at why people use protein supplements. Let’s also look at what is generally accepted as true with respect to the best supplement timing.

There are 4 major reasons people consume protein supplements:

  • Enhance the muscle gain associated with resistance training: In this case, protein supplements are customarily consumed concurrently with the workout.
  • Preserve muscle and accelerate fat loss while on a weight loss diet: In this case, protein supplements are customarily consumed with meals or as meal replacements.
  • Provide a healthier protein source. In this case, protein supplements are customarily consumed with meals in place of meat protein.
  • Prevent muscle loss associated with aging or illness. There is no customary pattern associated with this use of protein supplements.

How good are the data supporting the customary timing of protein supplementation? The answer is: Not very good. The timing is based on a collection of weak studies which do not always agree with each other.

The current study  (J.L. Hudson et al, Nutrition Reviews, 76: 461-468, 2018 ) was designed to fill this void in our knowledge. It is a meta-analysis that compares all reasonably good studies that have looked at the effect of protein supplement timing on weight gain or loss, lean muscle mass gain, fat loss, and the ratio of lean muscle mass to fat mass.

How Was The Study Done?

The authors started by doing a literature search of all studies that met the following criteria:

  • The study was a randomized control trial with parallel design. This means that study contained a control group. It does not mean that the investigators or subjects were blinded with respect to which subjects used a protein supplement and which did not.
  • The subjects were engaged in resistance training.
  • The study lasted 6 weeks or longer.
  • Reliable methods were used to measure body composition (lean muscle mass and fat mass).
  • The subjects were healthy and at least 19 years old.
  • There was no restriction on the food the subjects consumed.

The authors started with 2074 published studies and ended up with 34 that met all their criteria. They then separated the studies into two groups – those in which the protein supplements were used with meals and those in which the protein supplements were used between meals.

Both groups were diverse.

  • Group 1 included subjects who consumed their protein supplement with their meal and those who consumed their protein supplement as a meal replacement.
  • Group 2 included subjects who consumed their protein supplement concurrent with exercise (usually immediately after exercise) and those who consumed their protein supplement at a fixed time of day not associated with exercise.

Does Protein Supplement Timing Matter?

 

protein supplement timing workoutsBecause the individual studies were very diverse in the way they were designed, the authors could not calculate a reliable estimate of how much lean muscle mass was increased or fat mass was decreased. Instead, they calculated the percentage of studies showing an increase in lean muscle mass or a decrease in fat mass.

When the authors compared protein supplements consumed with meals versus protein supplements consumed between meals:

  • Weight gain was observed in 56% of the studies of protein supplementation with meals compared to 72% of the studies of protein supplementation between meals. In other words, protein supplements consumed with meals were less likely to lead to weight gain than protein supplements consumed between meals.
  • An increase in lean muscle mass was observed in 94% of the studies of protein supplementation with meals compared to 90% of the studies of protein supplementation between meals. In other words, timing of protein supplementation did not matter with respect to increase in muscle mass.
  • A loss of fat mass was observed in 87% of the studies of protein supplementation with meals compared to 59% of the studies of protein supplementation between meals. In other words, protein supplements consumed with meals were more likely to lead to loss of fat mass.
  • An increase in the ratio of lean muscle mass to fat mass was observed in 100% of the studies of protein supplementation with meals compared to 87% of the studies of protein supplementation between meals. In short, protein supplements consumed with meals were slightly more likely to lead to an increase in the ratio of lean muscle mass to fat mass.

The following seem to suggest protein supplement timing matters:

The authors pointed out that their findings were consistent with previous studies showing that when protein supplements are consumed with a meal they displace some of the calories that otherwise would have been consumed. Simply put, people naturally compensate by eating less of other foods.

In contrast, the authors stated that previous studies have shown that when foods, especially liquid foods, are consumed as snacks (between meals), people are less likely to compensate by reducing the calories consumed in the next meal.

The others concluded: “Concurrently with resistance training, consuming protein supplements with meals, rather than between meals, may more effectively promote weight control and reduce fat mass without influencing improvements in lean [muscle] mass.”

What Are The Limitations Of The Study?

Meta-analyses such as this one, are only as good as the studies included in the meta-analysis. Unfortunately, most sports nutrition studies are very weak studies. Thus, this meta-analysis is a perfect example of the “Garbage In: Garbage Out (GI:GO)” phenomenon.

For example, let’s start by looking at what the term “protein supplement” meant.

  • Because the studies were done by individual companies with their product, the protein supplements in this meta-analysis:
    • Included whey, casein, soy, bovine colostrum, rice or combinations of protein sources.
    • Were isolates, concentrates, or hydrolysates.
    • Contained various additions like creatine, amino acids, and carbohydrate.
  • As I discuss in my book, Slaying the Food Myths, previous studies have shown that optimal protein and leucine levels are needed to maximize the increase in muscle mass and decrease in fat mass associated with resistance exercise. However, neither protein nor leucine levels were standardized in the protein supplements included in this meta-analysis.
  • Previous studies have shown that protein supplements that have little effect on blood sugar levels (have a low glycemic index) are more likely to curb appetite. However, glycemic index was not standardized for the protein supplements included in this meta-analysis.

protein supplement timing workout peopleIn short, the conclusions of this study might be true for some protein supplements, but not for others. We have no way of knowing.

We also need to consider the composition of the two groups.

  • Protein supplements used as meal replacements are more likely to decrease weight and fat mass than protein supplements consumed with meals. Yet, both were included in group 1.
  • Some studies suggest that protein supplements consumed concurrent with resistance exercise are more likely to increase muscle mass than protein supplements consumed another time of day. Yet, both are included in group 2. We also have no idea whether the meals with protein supplements in group 1 were consumed shortly after exercise or at an entirely different time of day.

This was the most glaring weakness of the study because it was completely avoidable. The authors could have grouped the studies into categories that made more sense.

In other words, there are multiple weaknesses that limit the predictive power of this study.

What Can We Learn From This Study?

Despite its many limitations, this study does remind us that protein supplements do have calories. This is of relatively little importance for people whose primary goal is to increase lean muscle mass.

However, most of us are using protein supplements to lose weight or to increase our lean mass to fat mass ratio. Simply put, we are either trying to lean out (shape up) or lose weight. And, we want to lose that weight primarily by getting rid of excess fat. For us, calories do matter. With that in mind:

  • If we are consuming a protein supplement immediately after exercise or between meals we probably should make a conscious effort to reduce our daily caloric intake elsewhere in our diet.
  • Alternatively, we could consume the protein supplement with a meal, but time the meal so it occurs shortly after exercise.

 

The Bottom Line:

 

A recent study looked at the optimal timing of protein supplements consumed by subjects who were engaged in resistance exercise. Specifically, the study compared protein supplements consumed with meals versus protein supplements consumed between meals on weight, lean muscle mass, fat mass, and the ratio of lean muscle mass to fat mass. The study reported:

  • Protein supplements consumed with meals were less likely to lead to weight gain than protein supplements consumed between meals.
  • Timing of protein supplementation did not matter with respect to increase in muscle mass.
  • Protein supplements consumed with meals were more likely to lead to loss of fat mass.
  • Protein supplements consumed with meals were slightly more likely to lead to an increase in the ratio of lean mass to fat mass.

The authors pointed out that their findings were consistent with previous studies showing that when a protein supplement was consumed with a meal it displaces some of the calories that would have been otherwise consumed. Simply put, people naturally compensate by eating less of other foods.

In contrast, the authors said that previous studies have shown that when foods, especially liquid foods, are consumed as snacks (between meals), people are less likely to compensate by reducing the calories consumed in the next meal.

As discussed in the article above, the study has major weaknesses. However, despite its many weaknesses, this study does remind us that protein supplements do have calories. This is of relatively little importance for people whose primary goal is to increase lean muscle mass.

However, for those of us who are using protein supplements to lose weight or to increase our lean mass to fat mass ratio, calories do matter.  With that in mind:

  • If we are consuming a protein supplement immediately after exercise or between meals we probably should make a conscious effort to reduce our daily caloric intake elsewhere in our diet.
  • Alternatively, we could consume the protein supplement with a meal, but time the meal so it occurs shortly after exercise.

For more details, read the article above:

 

These statements have not been evaluated by the Food and Drug Administration. This information is not intended to diagnose, treat, cure or prevent any disease.

UA-43257393-1