JUMP TO

General Overview
Research Question & Hypothesis
- Is the research question addressing a focused issue?
Study Design
- Systematic Review
- Randomized Controlled Trials
- Cohort Studies
- Is the study design appropriate to be used for the research question(s)?
- Where does it lie in the hierarchy of evidence in research?
Note: Please refer to Specific Study Designs for a breakdown regarding each study design.
PICO elements
- Population
- Intervention
- Comparison
- Outcome
- What are the PICO elements based on your study/ clinical scenario?
Study population
- Are the groups defined precisely?
- What are the inclusion/ exclusion criteria from the study?
- Were the participants representative of a defined population (geographically and/or temporally)?
- Was there an established reliable system for selecting the participants? (e.g random/stratified sampling)
- Was the sample size adequate for significant results to be obtained?
- Were there any differences between the control & intervention groups apart from the exposure?
- Were there any differences in the baseline characteristics of the study participants?
- Was selection bias present?
Outcomes/Endpoints
- Primary & Secondary
- Outcome Measures
- What were the primary and secondary outcomes of the study?
- How were the outcomes measured?
- Is the time frame of the study relevant to disease/exposure?
Statistical Analysis
- What were the analysis methods used?
- Were all participants analyzed? (including those lost to followup)
- Were confounding factors outlined and accounted for? If so, what were the methods?
Results
- What are the results of the study?
- Were the results statistically significant? (consider the confidence interval)
- Were the results precise? (consider the p-value)
- Were all variables assessed?
- Consider the Number Needed to Treat/Harm (NNT/NNH)
Evaluation of Study
Strengths & Limitations
- Were there any limitations in the study? If so, what were the limitations?
- Was followup of patients sufficiently long and complete?
Applied Clinical Significance
- Reproducibility of trial (if beneficial)
- Significance in local population
- Can the results obtained from this study be subsequently reproduced in a separate study? (ie. results are not due to chance and are consistent with evidence from other studies)
- Is the treatment/intervention beneficial to the local population of patients?
Conclusions
- Is the treatment/intervention feasible in the local setting?
- Were the results of the study reliable?
- Were all clinical outcomes considered?
- Do the benefits of treatment/intervention outweigh the potential risks of harm and costs?

Specific Study Designs
Randomized Controlled Trials (RCTs)
A. Are the results of the trial valid?
- Was the issue addressed by the trial clearly focused?
- Was the assignment of patients to treatments/ interventions randomized?
- Were the groups similar at the start of the trial?
- Were patients/ participants, healthcare workers and study personnel ‘blinded’ to the treatment? ( e.g single/double blinded)
- Were all patients analyzed in the groups to which they were randomized at the study conclusion?
- Apart from the intervention, were the groups treated equally?
B. What were the results?
- How large was the treatment effect?
- How precise was the estimate?
C. Are the results applicable and helpful in practice?
- Can the results be applied in context (or to a local population)?
- Are the benefits from the treatment/ intervention worth the potential harms and costs?
Cohort Studies
A. Are the results of the trial valid?
- Did the study address a ‘focused’ issue?
- Was the cohort recruited in an acceptable way?
- Were there clearly defined groups of participants, similar in other features, apart from the exposure of interest?
- Was the detection of outcomes accurate and similar in both groups to minimize bias?
- Have the authors identified possible confounding factors? (If so, were statistical adjustments made?)
- Was the follow-up of subjects complete? (Were any drop-outs accounted for?)
B. What were the results?
- What is the relative risk/ risk ratio?
- How strong is the association between the exposure and the outcome?
- How precise is the estimate of the risk? (i.e, confidence interval of the relative risk)
- Reliability of results
- Any bias, chance or confounding present?
- Are the design or method flawed, making the results unreliable?
C. Are the results applicable and helpful locally?
- Can the results be applied to the local population?
- Do the results of this study fit with other available evidence?
- What are the implications of this study to clinical practice?
Systematic Reviews
A. Are the results of the trial valid?
- Did the review address a focused question?
- Yes/no and why?
- Did the studies possess an appropriate study design to the review question?
- Selection criteria of papers included
- Relevancy and adequacy of papers reviewed
- Bibliographic databases used
- Follow up from reference lists
- Personal contact with experts
- Search for unpublished as well as published studies
- Search for non-English language studies
- Methods of assessing paper’s quality
- Were the results of studies combined?
- If so, was it appropriate to do so?
- Were the results similar from study to study?
- Were results of included studies clearly stated?
- Were variations in results justified/explained?
B. What were the results?
- Overall results
- How were they reported? ((NNT/NNH, RR, OR etc.)
- Interpretation of results
- Precision of results
- Confidence Interval & Interpretation
C. Are the results applicable and helpful locally?
- Can the results be applied in context (or to a local population)?
- How similar are the review’s population to the local population
- Were all important outcomes considered?
- Are there any pertinent outcomes that were overlooked in this study ?
- Are the benefits worth the harm and cost?
Authors: Alaa Hamdi, Mikhirish Kumaran (SIGMUM 2021/2022)
References
- Individual Summative Assessment Marking Grid – Critical Appraisal of a Study of Therapy 2021 [assignment notes on Internet]. Melbourne: Monash University, Faculty of Medicine, Nursing and Health Sciences; 2021 [cited 20 September 2021].
- The University of British Columbia – Department of Emergency Medicine [Internet].CASP COHORT STUDY CHECKLIST; 2021 [cited 20 September 2021]; [2,4-6] Available from: https://med-fom-emerg.sites.olt.ubc.ca/files/2014/02/CASP-Cohort-Study-Checklist.pdf
- The University of British Columbia. Critical Appraisal Checklist [Internet]. Vancouver, BC Canada : Department of Emergency Medicine. CASP RCT STUDY CHECKLIST; 2021 [cited 20 September 2021]; [2-5]. Available from: https://med-fom-emerg.sites.olt.ubc.ca/files/2014/02/CASP-RCT-Checklist.pdf
- The University of British Columbia. Critical Appraisal Checklist [Internet]. Vancouver, BC Canada: Department of Emergency Medicine; 2013[cited 2021Sep20]. Available from:https://med-fom-emerg.sites.olt.ubc.ca/files/2014/02/CASP-Cohort-Study-Checklist.pdf
- Summative Assessment Task. CAT Harm Worksheet (RCT COHORT)2021 [assignment notes on Internet]. Melbourne: Monash University, Faculty of Medicine, Nursing and Health Sciences; 2021 [cited 20 September 2021].
- Individual Summative Assessment Marking Grid – Critical Appraisal of a Study of Harm 2021 [assignment notes on Internet]. Melbourne: Monash University, Faculty of Medicine, Nursing and Health Sciences; 2021 [cited 20 September 2021].
- The University of British Columbia. Critical Appraisal Checklist [Internet]. Vancouver, BC Canada: Department of Emergency Medicine; 2013[cited 2021Sep20]. Available from:https://med-fom-emerg.sites.olt.ubc.ca/files/2014/02/CASP-Systematic-Review-Checklist.pdf
Other guides
A Guide on Presenting at a Journal Club
Know more about what to include in your presentation to convey your findings with your peers, and a lot of practical tips to help you succeed in a journal club!
📚 Journal Club
A platform for staying up-to-date with the latest surgical research.