Critical Thinking: Understanding the Hierarchy of Evidence

The emergence of social media has given the voiceless a platform to be heard, unfortunately, it has also built a home for uneducated partial statements and lack of context. This is often the case with celebrities and their diets. In fact, the average layperson’s knowledge of nutrition and exercise typically consists of news and media, commercial books, product advertisements, advice from the local gym bro, advice from celebrities, fitness celebrities (rolls eyes) and personal experience. Most people don’t value research and review, they value advice from their friends (qualified or not) and celebrity (qualified or not).

Unfortunately we live in a society where there is a massive supply of information, but no guarantee as to the quality of that information.

In others words, some information that gets spewed on social media is just rubbish.

Fortunately, we can get past this hurdle by accepting responsibility for our own nutrition and training knowledge and being accountable to ourselves for our learning and development. Once we’ve done that, we can start to look towards more reliable sources for our information. This is where the Hierarchy of Evidence comes in. This allows you to take a top down approach to finding and rating the best evidence available to answer your question. Most experts agree that the higher up the study is in the hierarchy, the more rigorous the methodology would be and therefore, it’s more likely that the study design can minimize the effect of bias on the results of the study. Let’s take a look at each category from top down.

Filtered Sources:

Systematic Reviews and Meta Analyses: A systematic review is a type of literature review that collects and critically analyses multiple research studies or papers. A review of existing studies is often quicker and cheaper than embarking on a new study.

Critically Appraised Topics: A critically appraised topic (or CAT) is a short summary of evidence on a topic of interest, usually focused around a clinical question. A CAT is like a shorter and less rigorous version of a systematic review, summarizing the best available research evidence on a topic.

Critically Appraised Individual Articles: Critically-Appraised Individual Articles evaluate and synopsis individual research studies.

Unfiltered Sources:

Randomised Controlled Trial: A study design that randomly assigns participants into an experimental group or a control group. As the study is conducted, the only expected difference between the control and experimental groups in a randomized controlled trial (RCT) is the outcome variable which is being studied.

Cohort Studies: A type of medical research used to investigate the causes of disease, establishing links between risk factors and health outcomes. Cohort studies are usually forward-looking – that is, they are “prospective” studies, or planned in advance and carried out over a future period of time.

Case control studies and reports: Observational studies where no intervention is attempted. The goal is to retrospectively determine the exposure to the risk factor of interest from each of the two groups of individuals: cases and controls. These studies are designed to estimate odds. Case control studies are also known as “retrospective studies” and “case-referent studies.”

Background information and Expert Opinion: Admissible testimony relating to a professional, scientific, or technical subject. Expert evidence is based on formal and/or special study, training, or experience that imparts the competency to form an opinion upon matters associated with that subject.

If you’re trying to evaluate fitness industry research claims, you should ask the following questions as a baseline before coming to your conclusion:

  • What type of study design was it?
  • What were the subject profiles?
  • Were there short or long term effects examined or implicated?
  • What methods were used, and how tight were the controls?
  • Was the experimental protocol relevant to the conditions?
  • Did the results offer any real world relevance?

Hopefully one day we learn to value evidence over celebrity, until then, it’s important to remain sceptical when processing information, but not cynical.

Being a sceptical thinker means you don’t lean towards your own or someone else’s bias. Being sceptical means you’re neutral in your approach and not looking to confirm your pre-existing beliefs or theories.

Instead you’re looking to interpret data in a logical and practical way that is supported by existing evidence in a way that ensures causation is not mistaken for correlation.

Leave a Comment

Your email address will not be published. Required fields are marked *