Proving Health-Advertising Claims with Science: A 2023 Legal Guide

By Jesse Langel, Esq., LL.M.

| February 27, 2023

This blog explores when and how companies must support (substantiate") their health-related advertising claims.

Health Claims analyzed in false advertising cases by Jesse Langel

How Scientific Studies are Used to Prove Health-Advertising Claims

Science is a systematic and logical approach to discovering how the natural world works. It involves making observations, formulating hypotheses, testing them through experiments, and then analyzing the data to draw conclusions. The ultimate goal of science is to develop theories that explain natural phenomena and predict future observations.

Regarding health advertising claims, science is used to provide evidence that supports or refutes the validity of the claims. For example, if a company claims a particular product can help people lose weight, it may commission a study to test its effectiveness. The study would randomly assign participants to use the product or a placebo and then measure the outcomes of interest, such as weight loss. The study results would be analyzed using statistical methods to determine whether the product was significantly more effective than the placebo.

For health advertising claims to be supported by science, the studies must meet specific standards. The studies are explained below. But in essence, the studies should be well-designed and executed, with appropriate sample sizes and controls, and the results should be reproducible by independent researchers. Additionally, the studies should be published in reputable scientific journals, and the claims made by the company should be consistent with the results of the studies.

**Must Marketers Back Up Advertising Claims Before Products are sold?**

Yes. The strength of support for an advertising claim will depend on the messages being conveyed by the ad. But the threshold level of proof for an advertising claim is a “reasonable basis.”

What constitutes a “reasonable basis” depends on two primary factors:

The actual claim being made; and
how the ad claim is being presented in the context of the entire “mosaic” of the product, including any qualifying disclaimers.

This blog is Part II of How to Avoid Trouble with the FTC: Advertising Guide (2023).

What Standard of Proof Is Needed for Advertising Claims?

Under FTC law, advertising claims must be

truthful, not deceptive or misleading, and
evidence-based.

FTC standards in health claims are rigorous. Consumers need to have confidence in the accuracy of the information being presented in health-related advertising.

States like New York (NYGBL § 349) often enact “Little FTC Acts” that mirror FTC standards.

What Factors Determine the Amount and Type of Substantiation Required for Ad Claims?

Known as the “Pfizer Factors” that were developed in a 1972 FTC case, marketers should consider the following:

The type of product: Products that make claims as to health or safety require a higher level of scientific substantiation.
The type of claim: Sensibly, any claim that consumers cannot assess on their own needs more competent and reliable substantiation. Health claims in dietary supplements or drugs need heightened support to account for human health and wellness variations.
Benefits v. feasibility of substantiating truthful claims: This is a practical weighing of the evidence versus outcomes. But the general rule is that it should not be used if a claim cannot be substantiated.
The consequences of a false claim: The worst-case scenario is a physical injury because a consumer foregoes needed medical treatment based on the false assertions of a “therapeutic” supplement. The best-case scenario is that the consumer wastes money buying the product. Paying a premium for false advertising claims—over less-expensive alternatives—is generally considered an injury in false-advertising law.
What experts would believe is reasonable substantiation: Accepted norms in the relevant fields of research will be considered. The FTC will rely on experts in particular fields, products, or ingredients in health-related products. For example, cardiologists would be consulted for alleged health benefits to heart health. Government agencies may already have standards of substantiation for particular types of claims. The FDA and the National Institute of Health are two such agencies. Other “authoritative bodies,” such as the National Academy of Sciences, will be given great deference on scientific standards if they have weighed in.

Substantiation of health-advertising claims

Evidentiary Standard for Ads that Make Safety Claims?

“Competent and reliable scientific evidence,” as set forth in the following cases:

Roca Labs, Inc., 345 F. Supp. 3d at 1387 (competent and reliable scientific evidence required to support health-related claims, including weight-loss claims).
POM Wonderful LLC v. FTC,777 F.3d 478, 495-97 (D.C. Cir. 2015) (affirming Commission’s competent and reliable scientific evidence standard for disease-related claims about food products).
Direct Mktg. Concepts, Inc.,569 F. Supp. 2d at 300 (“because those are non-establishment health-related efficacy claims, the defendants must be able to point to ‘competent and reliable scientific evidence’ as substantiation”) (citing Removatron Int’l Corp.,884 F.2d at 1498 (1st Cir. 1989)); Nat’l Urological Grp., Inc., 645 F. Supp. 2d at 1190 (applying the same standard to weight loss and erectile dysfunction claims for dietary supplements).
FTC v. QT, Inc., 448 F. Supp. 2d 908, 961 (N.D. Ill. 2006) (applying the same standard to pain-related claims for “ionized” bracelet), aff’d, 512 F.3d 858 (7th Cir. 2008).

What is “Competent and Reliable Scientific Evidence” under FTC Advertising Standards?

Under FTC v. Roca Labs, Inc., (2019), the following definition applies

Test, analyses, research, or studies that:

have been conducted and evaluated in an objective manner by experts in the relevant disease, condition, or function to which the representation relates; and
are generally accepted in the profession to yield accurate and reliable results.

Also, research that substantiates a health claim must be

sufficient in quality and quantity;
based on standards generally accepted in the relevant scientific fields; and
considered in light of the entire body of relevant and reliable scientific evidence.

Here is a stipulated judgment in FTC v. Sunrise Nutraceuticals, (2016) in which Sunrise agreed not to make claims purporting to eliminate symptoms of opiate withdrawal and addiction. In order to make such claims, Sunrise would need to possess competent and reliable evidence, including the following:

human clinical testing
randomized and double-blind testing
placebo-controlled
conducted by researchers qualified by training and experience to conduct such testing
of sufficient quality and quantity based on general standards
accepted by experts in the relevant disease, condition, or function

Similarly, here is the stipulated judgment in Lumos Labs, Inc., (2016) relating to the Lumosity brain-training website. In this agreed-upon Injunction Order, Lumosity agreed to stop its marketing claims of “delaying age-related decline in memory, cognitive impairment, dementia, or Alzheimer’s disease” unless it possessed competent and reliable scientific evidence including:

human clinical testing
blinded to the maximum extent practicable
conducted by researchers qualified by training and experience to conduct such testing
of sufficient quality and quantity based on general standards
accepted by experts in the relevant disease, condition, or function

Lumosity agreed to a money judgment of $50 million in monetary relief. Lumosity also had to turn over its customer subscription list from 2009-2014, and notify all auto-renew subscribers of the settlement.

Here is a third stipulated judgment relating to marketing promises of improving memory and preventing cognitive decline in FTC v. Keyview Labs, Inc., (2015). Keyview labs was ordered to a) cease making claims about tests and studies b) preserve all human clinical tests and studies, and c) pay $91 million dollars in damages.

What is Competent and Reliable Evidence?

For health-related claims, “randomized controlled human clinical testing” (RCT) is usually required to meet the competent-and-reliable-scientific standard.

What is Randomized, Controlled Human Clinical Testing (“RCT”)?

Randomized controlled human clinical testing is a medical research study designed to evaluate the safety and effectiveness of a new treatment or intervention in humans. This type of study involves randomly assigning participants to one of two or more groups, with each group receiving a different treatment or intervention.

In a randomized controlled trial, the participants are assigned to different groups in a random manner, which helps to minimize bias and ensures that the results are statistically valid. One group, known as the treatment group, receives the new treatment or intervention, while the other group, known as the control group, receives a placebo or standard treatment.

The study is designed to be controlled, meaning that the conditions under which the participants are evaluated are carefully controlled and monitored to ensure that the results are accurate and reliable. The study is also designed to be blinded, meaning that the participants and the researchers evaluating the results do not know which group the participants are in, to minimize bias.

The primary objective of a randomized controlled trial is to determine whether the new treatment or intervention is safe and effective. By carefully measuring and comparing the outcomes of the treatment group and control groups' outcomes, researchers can determine whether the new treatment or intervention is more effective than the standard treatment or placebo.

Randomized Controlled Clinical Testing in Health Advertising by Jesse Langel

Examples of Randomized Controlled Human Clinical Testing (RCT)

COVID-19 vaccine trials: These trials involved thousands of people and were conducted to determine the safety and effectiveness of COVID-19 vaccines.
Cancer treatment trials: These trials evaluate new treatments for cancer, such as chemotherapy drugs, immunotherapy, and targeted therapies.
Drug trials for chronic conditions: These trials evaluate new drugs for conditions such as diabetes, hypertension, and heart disease.
Behavioral therapy trials: These trials evaluate the effectiveness of different types of behavioral therapy for conditions such as depression, anxiety, and addiction.

Some FTC Cases Establishing RCTs as the prevailing standard for health claims:

Roca Labs, Inc., 345 F. Supp. 3d at 1387 (requiring a randomized controlled human clinical trial (“RCT”) to substantiate weight-loss claims)
FTC v. Nat’l Urological Grp., Inc., No. 1:04-CV-3294-CAgP (applying an RCT substantiation standard to weight-loss claims)
POM Wonderful LLC, 777 F.3d at 504-05(affirming Commission holding that competent and reliable scientific evidence consisting of RCTs is needed for disease-related claims)
Nat’l Urological Grp., Inc., 645 F. Supp. 2d at 1202-03 (accepting undisputed expert testimony that erectile dysfunction claims require well-designed, placebo-controlled, randomized, double-blind clinical trials for substantiation)
FTC v. Coorga Nutraceuticals Corp., 201 F. Supp. 3d 1300 (D. Wyo. 2016) (Human clinical testing for claims that product reverses or prevents the formation of gray hair)
Direct Mktg. Concepts, Inc., 569 F. Supp. 2d at 303 (“Double-blind, placebo-controlled studies are necessary to substantiate health-related efficacy claims.”)
Removatron Int’l Corp., 111 F.T.C. 206 (1988), aff’d, 884 F.2d 1489, 1498 (1st Cir. 1989) (requiring “adequate and well-controlled clinical testing” to substantiate claims for hair removal product)
Thompson Med. Co., 104 F.T.C. at 826 (requiring well-controlled clinical studies to substantiate certain analgesic drug claims)

As to the reliability of RCT (randomized, controlled human clinical testing), the FTC considers the following study parameters:

sample size
duration
hypotheses
variabilities

What are "Phase Trials"?

Phase I trials: The first tests of a new treatment in humans and are designed to evaluate its safety, dosage, and potential side effects.
Phase II trials: These trials evaluate the effectiveness of a new treatment in a small group of people with a specific condition or disease.
Phase III trials: These trials involve larger groups of people and are designed to determine the safety and effectiveness of a new treatment compared to the standard treatment or a placebo.
Phase IV trials: These trials are conducted after a new treatment has been approved by regulatory agencies and are designed to monitor its long-term safety and effectiveness.

Should Advertisers Use Independent Experts?

The FTC suggests using independent experts in the relevant fields of research. Conceiving, implementing, and interpreting well-designed tests and trials should be undertaken by professionals.

Among other things, “independent experts can provide unbiased assessments of the validity of studies, how they fit within the relevant scientific literature, and what conclusions can be legitimately drawn from the results.”

Ad Claims Touting Supporting Proof Must be Real (Examples)

The below examples examine needed substantiation for assertions about the amount, type, and strength of evidence.

The phrase “scientists now agree!” would require a credible display of scientific consensus.
“Nobel Prize Winning” research must relate to the product’s claimed benefits, not unrelated use of the product’s active ingredients.
The claim “$5 Million in NIH Research” must have examined both safety and efficacy.
The claim “clinically tested ingredient” on a drink that claims to improve blow flow and endurance implies that the results of the drink (not only the ingredient) improve blood flow and endurance.

What Type of Evidence Supports a Health Product’s Ad Claims?

Randomized, Controlled Human Clinical Trials (RCTs) are the most reliable form of evidence and are generally the type of substantiation that experts would require for health benefit claims. See POM Wonderful LLC, 777 F.3d at 504-05(affirming Commission holding that competent and reliable scientific evidence consisting of RCTs is needed for disease-related claims).

The number of clinical trials (RCT’s) will vary. More research and more independently-conducted studies add weight to the study evidence.

Duplicated testing trials reduce the risk of biases and errors. The quality of the studies matters most. If the studies are flawed in design or execution, they will not amount meet the standard of competent and scientific. For example, in QT, Inc., significant flaws in multiple studies failed to substantiate pain relief claims for “Q-ray” bracelet).

What is Epidemiological Evidence in Health Advertising?

Epidemiological evidence in health-product advertising refers to the use of research studies that investigate the patterns, causes, and effects of health and disease conditions in populations to support claims made about the health benefits of a particular product.

Epidemiological studies often mean observational studies. These studies may be valuable to show an association with a product or an ingredient, but they don’t prove a causal link. For example, according to expert testimony in POM Wonderful LLC, 777 F.3d at 495, observational research was insufficient to establish a causal link between a food or nutrient and a reduction-in-disease risk.

But human clinical testing is not always available. For example, in the field of nutrition, it can take decades to establish a link between a nutrient and the risk of developing a disease. In this case, the FTC may accept high-quality epidemiological evidence to substantiate a claim in those limited cases where:

An acceptable substitute for RCTs are replaced epidemiological evidence provided by experts; and
RCTs aren’t otherwise feasible.

Permissible claims based on epidemiological evidence in health-product labeling and advertising include:

Health claims: Claims that describe the relationship between a product and a specific health condition. For example, "This product may reduce the risk of heart disease."
Qualified health claims: Claims that describe a relationship between a product and a specific health condition, but the relationship is not well-established. For example, "Some scientific evidence suggests that consumption of this product may reduce the risk of certain forms of cancer. However, FDA has determined that this evidence is limited and not conclusive."

Example of epidemiological studies in health-supplement advertising:

A company that makes a vision supplement claims that two large epidemiologic studies connect the long-term use of the supplement with better vision after 70 years of age. Although some experts opine that natural human biology accounts for the pattern, a “clinical intervention” to prove the claim through RCTs would take ten years and be costly. The company may use epidemiological studies if experts agree on its potential efficacy and no available evidence contradicts it. But since “long-term” use supported the studies, the company may not express or imply that short-term use can produce the effect.

What is a Clinical Intervention Trial for Drug Studies?

A clinical intervention trial is a type of clinical research study in which researchers investigate the effects of an intervention, such as a new drug, medical device, or treatment, on a group of participants. These trials aim to evaluate the safety and effectiveness of the intervention and are typically conducted in phases, each designed to answer different research questions.

Clinical intervention trials typically involve randomly assigning participants to either an intervention group that receives the treatment or a control group that does not. The results from the two groups FDA approves for widespread use.

Clinical intervention trials are an essential part of the drug and medical device development process. They provide critical information about the safety and effectiveness of new interventions before the FDA approves them for widespread use. They are also used to evaluate the effectiveness of established treatments in different populations and settings.

Can Animal Studies Substantiate Human Health Claims?

No. Although potentially useful for background, animal and in vitro studies must be confirmed by RCTs. In Nat’l Urological Grp., Inc.,(2017), the court held that only human studies—not data extrapolated from animal studies or in vitro studies—can confirm the cause-and-effect relationship in humans. In that case, a $40 million judgment was entered against the defendants who misled consumers with its “nutraceutical” supplement that claimed to cause weight loss rapidly.

Similarly, in FTC v. SlimAmerica, Inc.,(1999), the court held that animal and in vitro studies “cannot be characterized as serious scientific research” without medical proof that effects would be the same in humans.” SlimAmerica involved another weight-loss supplement claiming to “blast away up to 50% of your body fat in record time.” The company also used a fraudulent medical endorsement.

Animal studies may have some predictive value but must be confirmed by human clinical testing. In vitro studies involve isolated cells or tissues; They may shed light on possible mechanisms of action but cannot reliably substantiate effects in humans.

Examples of animal and in vitro studies:

Company X uses animal and in vitro studies to support its claim that humans can easily absorb a vitamin into the human bloodstream. But the animals used are different biologically because they can more easily synthesize the vitamin. During the in vitro studies, Company X used a higher compound concentration than used in the product. Using humans for studies was not only feasible and accepted but necessary, experts agree by consensus, to demonstrate human vitamin absorption.

What’s the Difference Between Individual Studies and Scientific Research?

Individual studies typically refer to studies conducted by individuals or small groups of researchers, often with limited resources and funding. These studies may be exploratory or preliminary in nature, and their results are often based on small sample sizes. Individual studies may provide useful insights or generate hypotheses for further investigation, but they are generally considered less reliable than scientific research.

Scientific research, on the other hand, refers to a systematic and rigorous approach to studying a particular phenomenon or question. Scientific research typically involves the use of established research methods and protocols, and the results are based on large sample sizes and statistical analysis. Scientific research is often conducted by teams of researchers, often with significant funding and resources, and the findings are subject to peer review and scrutiny by the scientific community.

Do Public Health Recommendations Qualify as Substantiation of Health Claims?

Public-health recommendations by medical organizations do not constitute substantiation because they do not prove a health benefit linked to a specific product. Although such recommendations may reflect sound opinions, they are not based on human clinical trials. In POM Wonderful LLC, 777 F.3d at 495, the court found that public-health recommendations not based on RCTs fail to establish a causal link.

Public-health recommendations are guidelines, advice, or instructions issued by public health agencies, organizations, or experts to promote healthy behavior, prevent illness, or mitigate health risks. Examples of public-health recommendations include vaccination recommendations, dietary guidelines, and physical activity guidelines.

Public-health recommendations may be considered as a factor in determining whether an advertiser had a reasonable basis for their claims, particularly if the recommendations are widely recognized and accepted within the relevant industry or scientific community.

Example:

An online brain-training program advertises on the radio that NYU Medical school recommends that adults engage in regular mental stimulation to stop the effects of dementia. This recommendation does not equate to substantiation because it needs studies linking the program to memory benefits. Reliable clinical testing would be required to make that claim to avoid deceptive implications.

Factors That Make Human Clinical Studies Reliable

Design, implementation, and results analysis
Human clinical testing
Treatment group and control group (placebo); comparing the results of each
Variable tracking
Randomize the selection of humans (age, gender, health) to avoid selection bias.
Double blinding: participants and researchers do not know the roles
Statistically significant results: The different outcomes between the treatment and control groups should be statistically significant. We must know if the measured effect results from the placebo effect, an unrelated improvement over time, or just chance. The testers should report all outcomes—not only the positive ones. Statistical adjustments for false positives may be needed when using multiple tests on the same data.
Lack of “p-hacking” or data mining after a study is completed. P-hacking is manipulating or analyzing data post hoc or after the study. It’s done to achieve statistical significance, or a positive result, that may not be genuine or meaningful. It often involves trying out multiple statistical tests or models on the same data until a desired result is found.
Clinically meaningful results: Aside from being statistically significant, the results must be clinically significant, which is meaningful to consumers.

Studies that lack the above elements are more vulnerable to bias and inaccuracies running afoul of the FTC’s competent and reliable scientific evidence standard for substantiating health-relating claims.

Other attributes of competent and reliable scientific studies:

Clear and detailed “protocol” for research. In research, a "protocol" refers to a detailed and structured plan that outlines a study or experiment's objectives, design, methodology, and organization. It includes specific information about the research question or hypothesis, the target population, the selection criteria for participants, the data collection and analysis procedures, and any ethical considerations that need to be addressed. By following a well-defined protocol, researchers can minimize the risk of errors, biases, and inconsistencies, ultimately improving the reliability and validity of the study's results.
Primary and secondary outcome measures should be well-defined and specified in advance.
- Primary outcome measures are the main results researchers aim to measure and evaluate to determine the effectiveness of the intervention or treatment being studied. These outcomes directly address the primary research question or hypothesis and are the basis for drawing conclusions about the intervention's efficacy or safety. Primary outcome measures should be clearly defined, objective, and relevant to the study's goals.
- Examples of primary outcome measures include:
  - Reduction in disease symptoms or severity
  - Improvement in patient survival rates
  - Decrease in the frequency of specific adverse events
- Secondary outcome measures are additional results that researchers may examine to provide further insights into the intervention's effects, though they are not the primary focus of the study. Secondary outcome measures can help researchers evaluate other aspects of the intervention, such as cost-effectiveness, quality of life, or safety in specific subpopulations. Like primary outcome measures, secondary outcome measures should also be well-defined and predetermined before the study begins.
- Examples of secondary outcome measures include:
  - Changes in biomarkers or physiological indicators
  - Patient-reported outcomes, such as quality of life or pain levels
  - Healthcare utilization or cost-related outcomes
Furnish the research protocol to the Institutional Review Board (IRB) to ensure the research is ethical and safe. The scientific community considers this step to be generally accepted.
Register the clinical trial in a public database to improve transparency and reduce "publication bias," such as failing to disclose negative results or publishing only a subset of favorable outcomes.
Disclose criteria included and excluded relevant to the population studied.
Disclose people who drop out, don't follow instructions, or change their habits, as these factors can affect the results. One way to handle this is to analyze the data of all participants, including those who left or didn't follow the plan, which is called an "intent-to-treat" analysis.
Use studies long enough in duration and follow-up where necessary to reliably convey the length of time consumers can expect for effects.
Use studies large enough to detect potential side effects.
Strengthen the findings by disclosing a) dose-response relationships (i.e., the larger the dose, the greater the effect) and b) other biological or chemical mechanisms that could explain the effects observed.
A well-crafted report should follow the well-crafted studies. The FTC should not be expected to evaluate abstracts or informal summaries. The report needs sufficient detail, a section on deviations, and a close adherence to the research protocol.
A rigorous, unbiased peer-review process is preferred. The same peer review process of scientific journals is ideal, but if needed, the FTC can examine the underlying documents and the raw data.

Can advertisers use summaries or abstracts of studies as substantiation of health claims?

Maybe, but only if they verify the study design, implementation, and results. The advertiser must ensure that the research uses a comparable ingredient formulation, and they must check whether the study was adequately controlled and yielded statistically significant between-group results.

Can weak studies qualify as substantiation of health claims under FTC advertising law?

Generally, no. Here is an example. A supplement provider makes unqualified claims about anti-clotting due to a compound extracted from fruit. Only two clinical studies exist to support this health claim. One involved a one-week test period with no control group. Although well-controlled and longer in duration, the second study was not statistically significant. The studies are probably too weak to substantiate the ant-clotting claim.

Another example would be an herbal supplement that claims to promote healthy vision and is approved in Germany for this purpose. The approval in Germany is based on its monograph on two abstracts of studies showing that the herbal supplement significantly improved the vision of healthy individuals in the treatment group over the placebo group. But, according to the FTC Health Products Compliance Guidance, "approval of the supplement under the German monograph doesn't constitute substantiation that the supplement is effective." The advertiser should confirm the applicability of the research and verify that dosages and formulations are comparable.

What is a monograph in federal food law?

A monograph in federal food law refers to a detailed document that provides guidance or regulation on using a particular ingredient, drug, or substance in food products. Monographs are usually published by the U.S. Food and Drug Administration (FDA) and outline acceptable levels of use, manufacturing standards, labeling requirements, and other important information related to the specific substance.

Monographs are essential to federal food law as they provide clear guidelines for manufacturers and ensure that food products are safe for consumption. Monographs may prevent misleading claims about a product and inform consumers about their food's ingredients.

Some examples of monographs published by the FDA include those for food additives, color additives, and generally recognized as safe (GRAS) substances. These monographs provide clear and consistent guidance to manufacturers to ensure that their products are safe and meet all regulatory requirements.

Case Study: Does a vitamin shake help alleviate symptoms of osteoarthritis?

A company promoting a liquid protein and vitamin supplement conducted a study to determine its effectiveness in alleviating symptoms of osteoarthritis. Below is the design of the study followed by the conclusion.

200 individuals with varying degrees of osteoarthritis participated in the study.
The research was randomized, placebo-controlled, double-blind, and utilized a reliable measure of osteoarthritis symptoms, evaluating participants at regular intervals over a 90-day duration.
The study's author reported that those using the supplement experienced a statistically significant improvement in symptoms from the beginning to the end of the study.
Both the treatment and control groups witnessed some improvement over time, but the treatment group's improvement was not statistically greater than the control group's.
The study did not compare the treatment group's improvement to that of the control group.
The company then examined the study's data for statistically significant outcomes, focusing on comparisons that were not part of the initial protocol.
This subsequent analysis (post hoc) of the results revealed that, within a small group of individuals with the mildest form of osteoarthritis, there was a statistically greater improvement.

Conclusion: The follow-up analysis of the data does not provide reliable evidence of the supplement's benefits for people with mild osteoarthritis. Additional research focusing on individuals with mild osteoarthritis should be conducted to confirm any potential benefits for this population.

Five examples of randomized, controlled, double-blind studies, conducted across various fields of research:

Clinical trial for a new drug: A study testing the efficacy of a new drug for treating high blood pressure could involve randomly assigning participants to either the treatment group (receiving the new drug) or the control group (receiving a placebo). Both participants and researchers would be unaware of which group each participant belongs to, thus maintaining the double-blind aspect of the study.
Nutritional supplement study: A study examining the effects of a specific vitamin supplement on energy levels could involve randomly assigning participants to either the treatment group (receiving the vitamin supplement) or the control group (receiving a placebo). Again, both participants and researchers would be unaware of each participant's group assignment, maintaining the double-blind aspect.
Psychological intervention study: A study evaluating the effectiveness of a cognitive-behavioral therapy (CBT) intervention for treating anxiety could involve randomly assigning participants to either the treatment group (receiving CBT) or the control group (receiving a sham intervention). To maintain the double-blind aspect, participants would be unaware of which intervention they are receiving, and evaluators assessing the outcomes would also be blinded to the group assignments.
Exercise intervention study: A study investigating the effects of a specific exercise program on muscle strength could involve randomly assigning participants to either the treatment group (receiving the exercise program) or the control group (performing a different, non-strength focused exercise). To maintain the double-blind aspect, participants would not be informed of the specific aims of each exercise program, and evaluators assessing muscle strength outcomes would be blinded to group assignments.
Vaccination study: A study testing the efficacy and safety of a new vaccine could involve randomly assigning participants to either the treatment group (receiving the new vaccine) or the control group (receiving a placebo or an existing vaccine). Both participants and researchers would be unaware of which group each participant belongs to, maintaining the double-blind aspect of the study.

Case study: Does a brain stimulation device clinically help depression?

Design of Study:

A manufacturer tests its at-home brain stimulation device for its effects on depression.
It's a randomized, controlled, double-blind study.
Eight "validated measures" were used to assess the impact of the device on symptoms of depression.
In only one of the eight measures, the device showed better improvement than the group that didn't. The other seven methods showed no difference between the two groups.
The study didn't account for the possibility of getting a false positive result due to testing multiple outcomes.

Conclusion: Since only one out of eight tests showed significant improvement, it could be just a random occurrence. So, the seller can't use this single positive result to claim that their device effectively treats depression.

What are "validated measures" in scientific studies in health studies?

"Validated measures" are assessment tools or instruments that have been scientifically tested and proven to accurately and reliably measure the intended outcomes or variables.

Using validated measures is essential in research, ensuring the data collected is consistent, reliable, and accurately reflects the phenomenon being studied. These measures typically undergo a rigorous development, testing, and refinement process to confirm that they effectively capture the desired information. For example, a blood test to measure a specific biomarker associated with the supplement's effect would be considered a validated objective measure.

In assessing symptoms of depression, validated measures may include questionnaires, self-report scales, or clinician-administered instruments that have been demonstrated to accurately and consistently measure depressive symptoms and their severity. Multiple validated measures can help increase the robustness of the study's findings and provide a more comprehensive understanding of the device's impact on depression.

Case study: Does a dietary supplement treat erectile dysfunction?