Reliability & Validity of Clinical Prediction Rules for Screening Low-Risk Cervical Blunt Trauma – CADRE Systematic Review

Study Title: Validity and reliability of clinical prediction rules used to screen for cervical spine injury in alert low-risk patients with blunt trauma to the neck: part 2. A systematic review from the Cervical Assessment and Diagnosis Research Evaluation (CADRE) Collaboration

Authors: Moser N, Lemeunier N, Southerst D, Shearer H et al.

Publication Information: European Spine Journal 2018; 27(6): 1219-1233. doi: 10.1007/s00586-017-5301-6.

Comment from Dr. Shawn Thistle:

Although not common in most offices, you may at some point be confronted with a cervical trauma patient, whether directly in your clinic, or via some secondary responsibility such as athletic event supervision. As such, we as healthcare providers require screening tools that are both highly sensitive and reliable for deciding which patients require imaging.  This paper updates us on the literature pertaining to the best-available tools…enjoy!

Introduction:

Though not common, more than one million Americans are assessed for potentially serious cervical spine injuries (such as fracture, dislocation or ligamentous instability) annually (1). It is critical that these injuries are diagnosed in a timely manner, as consequences of misdiagnosis or mismanagement could lead to spinal cord injury, or even death (2). As such, clinicians require screening tools that are both highly sensitive and reliable.

Although imaging is the primary modality for assessing the anatomical integrity of the cervical spine following blunt trauma, more than 98% of radiographs ordered in these cases are negative for fracture (3-6). This exposes patients to radiation in cases when imaging may not be indicated (7) and is associated with significant costs and strain on the health care system (8-10). As such, clinical practice guidelines recommend using clinical prediction rules as a decision matrix, by incorporating three or more variables from the history, physical examination or simple diagnostic tests to guide diagnostic (or treatment) decisions (11, 12).

The 2000-2010 Bone and Joint Decade Task Force on Neck Pain and Its Associated Disorders (known as the Neck Pain Task Force, or NPTF) concluded that The Nexus Low-Risk Criteria (NLC) and The Canadian C-Spine Rule (CCR) demonstrated high sensitivity and negative predictive values (13), and the high sensitivity of these rules has been recently confirmed in a systematic review by Michaleff et al. (14).

Given the emergence of new evidence regarding the reliability and validity of the NLC and CCR, the aim of this systematic review is to update the findings of the NPTF on the validity and reliability of clinical prediction rules used to screen for cervical spine injury in alert, low-risk patients aged 16 and older with blunt trauma to the neck. Essentially, the question is: when do we need to x-ray/image cervical trauma patients?

Pertinent Results:

  • A total of 397 titles and abstracts were screened for eligibility and 41 potentially relevant articles were identified for full-text screening, and six were eligible for critical appraisal.
  • Five studies were deemed to be of high quality (15-19).
  • Three studies assessed the validity of the CCR (15-17). It was studied in patients presenting to emergency departments (ED) and the sensitivities ranged from 0.9-1.0. Two studies reported negative predictive values and both reported 100% (15, 17). These studies included the application of these rules by emergency nurses, physicians, paramedics and physicians using paramedics’ field notes (15-17).
  • Three studies assessed the inter-rater reliability of the Canadian C-Spine Rule (CCR) (16-18). One found an inter-rater reliability of κ = 0.60 (representing moderate agreement – 95% CI: 0.50-0.62) between physicians and nurses (16), while another found it to be κ = 0.75 between these two groups (moderate agreement – 95% CI: 0.67-0.84) and κ = 0.78 (also moderate agreement – 95% CI: 0.72-0.84) for nurses only (17). Finally, the third study reported almost perfect inter-rater reliability among paramedics (κ = 0.93; 95% CI: 0.87-0.99) (18).
  • One study assessed the inter-rater reliability of the Nexus Low-Risk Criteria (NLC) and reported weak inter-rater reliability (κ = 0.53; 95% CI: 0.35-0.72) (19). The items with the lower agreement were altered mental status, focal neurological deficit and distracting injury (19)

Clinical Application & Conclusions:

This review examined the validity and reliability of the Canadian C-Spine Rule (CCR) and Nexus Low-Risk Criteria (NLC) in alert, low-risk adult patients with blunt trauma to the neck. The CCR was found to be highly sensitive, indicating that it is extremely helpful for clinicians to exclude a significant cervical spine injury (see outline below). It is important to recognize that the specificity is (expectedly) lower than the sensitivity, and thus an important number of false positives will occur (better to have a false positive than to miss a fracture!).

While only one study examined the NLC, the levels of agreement were unsatisfactory. The findings suggest there may be subjectivity in how the NLC criteria are interpreted, and interpretation may be influenced by practitioners’ expertise, background and level of experience.

When evaluating these results in the context of the previous findings of the NPTF, the CCR was reported superior to the NLC with respect to sensitivity (99.4% [95% CI: 96–100] vs. 90.7% [95% CI: 85–94], respectively) and specificity (45.1% [95% CI: 44–46] versus 36.8% [95% CI: 36–38], respectively) (9). The NPTF concluded that use of the CCR would inform clinicians on the necessity of further diagnostic imaging and reduce rates of radiography (9).

Importantly, the reviewers noted the CCR appears to improve the efficiency of health care resource use. Importantly, the professionals studied in this review reported feeling “comfortable” or “very comfortable” using the CCR (15-18).

What is the Canadian C-Spine Rule?

Study Methods:

  • A systematic search strategy was developed in consultation with a health sciences librarian and reviewed by a second librarian.
  • Four databases were searched from January, 2005 to November, 2015 using appropriate search terms for each database. Reference lists of included studies and related systematic reviews were screened for additional resources.
  • Pairs of independent authors screened titles and abstracts for relevant and possibly relevant citations, and possibly relevant citations were reviewed using the full text.
  • Only studies published in English or French in a peer-reviewed journal which included samples of 20 or more participants (per group) who were alert and low-risk, 16 years and older and had grades I-IV neck pain (NP) were included in this review. Included studies must have assessed the validity or reliability of clinical prediction rules used to screen alert low-risk patients.
  • Pairs of independent authors appraised all relevant studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) (20) and the modified Quality Appraisal Tool for Studies of Diagnostic Reliability (QAREL) (21) for diagnostic reliability studies. Studies with low risk of bias were included in the best evidence synthesis.
  • One reviewer extracted data from low risk of bias studies and built evidence tables. A second reviewer confirmed the data.
  • A qualitative synthesis (best evidence synthesis) was performed and the findings were interpreted in the context of the findings of the NPTF (13).
  • Included studies were classified based on the system of Sackett and Haynes based on the type of research conducted (22). Phase I and II studies are exploratory and require further evaluation, phase III studies assess the ability of test to determine between those with and without the target condition. Phase IV studies measure the utility of tests by assessing whether patients who undergo a test have better health outcomes than those who do not. Phase III and IV studies may inform recommendations and form the basis for widespread adoption of clinical testing.

Study Strengths / Weaknesses

Strengths:

  • A clearly defined researched question with a thorough and systematic search.
  • Independent screening of titles and abstracts, and full texts.
  • Only those trials assessed as being of high quality were included.
  • Assessment of risk of biased was performed with a validated set of criteria.
  • Two authors independently extracted the data from the included articles.
  • This study used the principles of best evidence synthesis to inform scientific judgement. 

Weaknesses:

  • The primary limitation of this study relates more to the quality of the body of evidence than the methodology of the review itself, particularly with respect to the NLC.
  • The included studies did not adequately describe the participants or the clinical setting which limits the external validity of the review, particularly with respect to non-emergency practice.

Research Review from RRS Education (www.rrseducation.com )

Additional References:

  1. National Center for Health Statistics (2012) Health, United States, 2011: With Special Feature on Socioeconomic Status and Health. Hyattsville, MD.
  2. Davis JW, Phreaner DL, Hoyt DB, et al. The etiology of missed cervical spine injuries. J Trauma 1993; 34(3): 342–346.
  3. Vandemark RM. Radiology of the cervical spine in trauma patients: practice pitfalls and recommendations for improving efficiency and communication. AJR Am J Roentgenol 1990; 155(3): 465–472.
  4. Bayless P, Ray VG. Incidence of cervical spine injuries in association with blunt head trauma. Am J Emerg Med 1989; 7(2): 139–142.
  5. Jacobs LM, Schwartz R (1986) Prospective analysis of acute cervical spine injury: a methodology to predict injury. Ann Emerg Med 1986; 15(1): 44–49.
  6. Daffner RH. Cervical radiography in the emergency department: who, when, how extensive? J Emerg Med 1993; 11(5): 619–620.
  7. Stiell IG, Wells GA, Hoag RH et al. Implementation of the Ottawa knee rule for the use of radiography in acute knee injuries. JAMA 1997; 278(23): 2075–2079.
  8. Griffith B, Bolton C, Goyal N et al. Screening cervical spine CT in a level I trauma center: overutilization? AJR Am J Roentgenol 2011; 197(2): 463–467. doi:10.2214/AJR.10.5731.
  9. Stiell IG, Clement CM, McKnight RD et al. The Canadian C-spine rule versus the NEXUS low-risk criteria in patients with trauma. N Engl J Med 2003; 349(26): 2510–2518.
  10. Moloney TW, Rogers DE. Medical technology – a different view of the contentious debate over costs. N Engl J Med 1979; 301(26): 1413–1419.
  11. Como JJ, Diaz JJ, Dunham CM et al. (2009) Practice management guidelines for identification of cervical spine injuries following trauma: update from the eastern association for the surgery of trauma practice management guidelines committee. J Trauma 2009; 67(3): 651–659. doi:10.1097/ TA.0b013e3181ae583b.
  12. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999; 33(4): 437–447.
  13. Nordin M, Carragee EJ, Hogg-Johnson S et al. Bone and joint decade 2000–2010 task force on neck pain and its associated disorders: assessment of neck pain and its associated disorders: results of the Bone and Joint Decade 2000–2010 Task Force on neck pain and its associated disorders. Spine 2008; 33(4 Suppl): S101–S122.
  14. Michaleff ZA, Maher CG, Verhagen AP et al. Accuracy of the Canadian C-spine rule and NEXUS to screen for clinically important cervical spine injury in patients following blunt trauma: a systematic review. CMAJ 2012; 184(16): E867– E876. doi:10.1503/cmaj.120675.
  15. Coffey F, Hewitt S, Stiell I et al. Validation of the Canadian C-spine Rule in the UK emergency department setting. Emerg Med J 2011; 28(10): 873–876.
  16. Miller P, Coffey F, Reid A et al. Can emergency nurses use the Canadian cervical spine rule to reduce unnecessary patient immobilisation? Accid Emerg Nurs 2006; 14(3): 133–140.
  17. Stiell IG, CeM Clement, O’Connor A et al. Multicentre prospective validation of use of the Canadian C-spine rule by triage nurses in the emergency department. CMAJ 2010; 182(11): 1173–1179.
  18. Vaillancourt C, Stiell IG, Beaudoin T et al. The out-of-hospital validation of the Canadian C-spine Rule by paramedics. Ann Emerg Med 2009; 54(5): 663-71.
  19. Matteucci MJ, Moszyk D, Migliore SA Agreement between resident and faculty emergency physicians in the application of NEXUS criteria for suspected cervical spine injuries. J Emerg Med 2015; 48(4): 445–449.
  20. Whiting PF, Rutjes AW, Westwood ME et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155(8): 529–536.
  21. Lucas N, Macaskill P, Irwig L et al. The reliability of a quality appraisal tool for studies of diagnostic reliability (QAREL). BMC Med Res Methodol 2013; 13: 11.
  22. Sackett DL, Haynes RB. The architecture of diagnostic research. BMJ 2002; 324: 539–541.

Dr. Shawn Thistle – RRS Education’s CEO

I am a practicing chiropractor, educator, international speaker, knowledge-transfer leader, entrepreneur & medicolegal consultant. I am the Founder & CEO of RRS Education, a continuing education company providing weekly Research Reviews, informative Seminars & convenient Online Courses for chiropractors, physiotherapists & osteopaths around the world. I have lectured as a part-time faculty member at the Canadian Memorial Chiropractic College in the Orthopedics Department for 13 years. My skillset as an educator and consultant is further strengthened by my experience in expert medicolegal reporting in chiropractic malpractice cases.

If you have questions for Dr. Thistle, please email Dr. John Pecora @ JPecoradc@wbcgp.com


RRS Education has been a trusted source of continuing education solutions since 2006. RRS Education provides weekly Research Reviews like this one (subscription required) as well as evidence-based online courses to help busy clinicians review and integrate current research into their patient care. For more information, visit: www.rrseducation.com.

RRS Education Discount offer: SAVE 20% on all professional subscriptions!

RRS Education posts a new Research Review like this one each week and we cover a variety of interesting clinical topics! Our Reviews can be read online, and audio versions are also available for streaming or download.

MeyerDC members can save 20% on a subscription to RRS Education’s weekly Research Reviews (including access to the database, containing over 830 Reviews!) using this coupon code: MEYERDC2018 (case sensitive)

SUBSCRIBE NOW: https://www.rrseducation.com/research-reviews


For more info on products mentioned in this article or for info on current product specials, please contact your personal MeyerDC Account Manager.

If you don’t know who your Account Manager is, please call 800.472.4221 to find out.

Recommended Products from DC Aligned