Breaking Out of the 'Seems Good for My Health' Illusion. A Framework and Check Template to Instantly Spot the 'Evidence Strength' of Medical Research

The internet is overflowing with health information like “Just chewing well makes you lose weight!” or “Eating ___ extends your lifespan!” Even if you are enticed by claims like “proven by the latest research,” the quality of those papers varies widely, and in reality, it’s not uncommon for them to be nothing more than a simple questionnaire survey.

Experts at medical schools and research institutions grade the countless papers by fitting them into a pyramid called “strength of evidence (scientific basis).” In this article, we explain a powerful evaluation template that allows anyone to easily judge “how credible this research is.” If you master this, you should no longer be swayed by the truth or falsehood of health information.

The Evidence Pyramid: Differences in Strength Depending on Study Design

When evaluating the quality of a paper, what is emphasized most is “what kind of design was used to collect the data (study design).” Let’s look at them in order from lowest strength to highest.

1. Animal/Cell Experiments (Weakest Evidence): Research like “we fed mice hard food and their brain cells increased.” Important for elucidating mechanisms, but there is no guarantee it applies directly to humans.
2. Expert Opinions / Case Reports: Personal rules of thumb like, “I have seen patients who chew well for many years, and they are healthy.” This falls into a very weak category as scientific evidence.
3. Cross-Sectional Studies (Observational Studies): Questionnaire surveys at “a certain point in time” (some of E05, etc.). Even if the result “people who answered they chew well had less obesity” comes out, there’s no guarantee whether that’s the cause or the result (reverse causality).
4. Cohort Studies (Longitudinal Studies): Studies tracking thousands of people over several years to decades (related to E06, etc.). The strength is quite high because you can follow the flow of time: “People who were initially healthy but had a low chewing count became obese a few years later.”
5. Randomized Controlled Trials (RCT): An experiment where subjects are divided by lottery into a “gum-chewing group” and a “non-chewing group,” and changes are observed after a certain period. Because other factors (confounding factors) can be eliminated, it is considered strong evidence (clinical trials in E12, etc.).
6. Systematic Reviews and Meta-Analyses (Strongest Evidence): Gathering dozens of past high-quality RCT papers, comprehensively recalculating them, and drawing a “conclusion.” This is the ultimate deciding factor in the current medical world.

”Three Checkpoints” to Spot Chewing Evidence

In addition to the pyramid, here is a dedicated evaluation template with three points that should be applied specifically when reading papers related to “chewing.”

1. Measurement Bias: Self-Reported or Objective Indicator?

How was the data “chewing well” obtained?

• Weak: Questionnaire to subjects: “Do you usually chew well? (Yes/No)”
• Strong: Measured the degree of color change in color-changing gum (color tester), or accurately counted the number of masseter muscle activities with a wearable electromyography sensor.

2. Adjustment for Confounding Factors: Is it Really Thanks to “Chewing”?

If the result “people who chew well live longer” comes out, confirm if there are any overlooked factors.

• Weak: Simple comparison of average values.
• Strong: Is the figure the result of statistically adjusting (eliminating) all other elements related to lifespan, such as the subjects’ “age,” “income level,” “exercise habits,” and “smoking history”?

3. Confirmation of Sponsor Bias (COI)

• Is the funder of an RCT touting the health benefits of gum a major gum manufacturer? This in itself is not evil, but it is necessary to hold a certain reservation in interpretation (confirmation of “Conflict of Interest: COI”).

Summary: Upgrading from an Information Consumer to an “Evaluator”

Next time you see news like “The Latest Chewing Diet Method!”, instead of jumping on it immediately, try asking, “What tier of the pyramid is the paper acting as the source for that?” and “Is the frequency self-reported?” By possessing this evidence evaluation template (grading perspective), you will not be misled by media hype, and it becomes possible to extract and practice only truly valuable health habits.