Knowledge

POCUS Competency

Competency Metrics: Acquisition

While POCUS competency assessment has so far been largely left up to individual educational institutions, various organizations have provided guidelines or recommended scales to help inform competency assessment. Notably, consistency can be an important factor in determining learner competency, suggesting a need for longitudinal monitoring by experts.

 

Delete

Image Quality Metrics

 

Publication

Keith Boniface MD, Kat Ogle MD, Ahmad Aalam MBBS, Maxine LeSaux, Matt Pyle MD, Sohaib Mandoorah MBBS, Hamid Shokoohi MD, MPH. (2019). Direct Observation Assessment of Ultrasound Competency Using a Mobile Standardized Direct Observation Tool Application With Comparison to Asynchronous Quality Assurance Evaluation. AEM Education and Training, 3(2), 172-178.

Key Findings

  • In line with Council of Emergency Medicine Residency–Academy of Emergency Ultrasound guidelines and image review sessions, a scale of 1 to 5 for image quality was developed and presented in Figure 1 on page 175 as follows: 
    • 1 - No recognizable structures, no objective data can be gathered 
    • 2 - Minimally recognizable structures but insufficient data for diagnosis
    • 3 - Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws
    • 4 - Minimal criteria met for diagnosis, all structures imaged well and diagnosis easily supported
    • 5 - Minimal criteria met for diagnosis, all structures imaged with excellent image quality and diagnosis completely supported and suitable for educational purposes
  • A score of 3 or above is required for a diagnosis

Paper Implications

  • Ultrasound image quality can be assessed by examiners or image reviewers on a scale of one to five

Publication

David Bahner, MD, RDMS, Eric J. Adkins, MD, MSc, Rollin Nagel, PhD, David Way, MEd,Howard A. Werman, MD, Nelson A. Royall, MD. (2011). Brightness Mode Quality UltrasoundImaging Examination Technique (B-QUIET)Quantifying Quality in Ultrasound Imaging. J Ultrasound Med, 30,1649–1655.

Key Findings

  • A developed B-QUIET scale on page 1651 for ultrasound image quality assessment includes "Image Anatomy" with scores of 1 to 4 covering "Unacceptable," "Needs Improvement," "Acceptable," and "Optimal", respectively, for:
    • Near Field (Top of Screen)
    • Receding Edge (Right Side)
    • Far Field (Bottom of Screen)
    • Leading Edge (Left Side)
  • A score of 1 indicates "Unclear representation", 2 means "Image partially distorted", 3 means "Adequate visualized anatomy", and 4 means "Optimal anatomy representation"

Paper Implications

  • Ultrasound image quality assessment scales still require expert review in order to score image quality

Publication

Scott J. Millington, Robert T. Arntfeld, Robert Jie Guo, Seth Koenig, Pierre Kory, Vicki Noble, Haney Mallemat and Jordan R. Schoenherr. (2017). The Assessment of Competency in Thoracic Sonography (ACTS) scale: validation of a tool for point-of-care ultrasound. Crit Ultrasound J, 9,25.

Key Findings

  • Using the Assessment of Competency in Thoracic Sonography (ACTS) scale, image quality can be assessed using a scale of 1 to 5, with a score of 0 reserved for "not obtained"
    • Score of 1 indicates "Image quality too poor to permit meaningful interpretation"
    • Score of 3 indicates "Suboptimal image quality, but basic image interpretation possible"
    • Score of 5 indicates "Good image quality, meaningful image interpretation easy"
  • Most learner improvement happens in the first 25-30 practice studies

Paper Implications

  • Quantitative scoring can help in learner assessment

 

Delete

Image Acquisition Skill Metrics

 

Publication

David P. Bahner, MD, RDMS, Eric J. Adkins, MD, MSc, Rollin Nagel, PhD, David Way, MEd, Howard A. Werman, MD, Nelson A. Royall, MD. (2011). Brightness Mode Quality Ultrasound Imaging Examination Technique (B-QUIET) Quantifying Quality in Ultrasound Imaging. J Ultrasound Med, 30,1649–1655.

Key Findings

  • According to Figure 1 on page 1651, technical skills for ultrasound image acquisition that can be assessed on a scale of 1 to 4 include:
    • Resolution - Application/Focal zone, where 1 = wrong application, wrong focal zone, 2 = correct application, wrong focal zone, 3 = wrong application, correct focal zone, 4 = Correct application, correct focal zone
    • Depth - Field of view, where 1 = Image too small (<half of screen), 2 = Image cut off or not maximized at present depth (>4 cm from ideal depth), 3 = Adequate depth (within 1-3 cm), 4 = optimal depth (intended image fills the screen)
    • Gain (Segmental/Overall), where 1 = Inappropriate (too white, too dark throughout image), 2 = Poor gain setting in near or far field, 3 = Adequate gain settings (minimal echoes in fluid filled structures, all structures contrasted), 4 = Optimal gain settings and greyscale (No internal echoes in vascular or fluid filled structures, appropriate tones to soft tissue)

Paper Implications

  • Technical skills of interest include resolution, depth and gain

Publication

Andre Kumar, MD, John Kugler, MD, and Trevor Jensen, MD. (2019). Evaluation of Trainee Competency with Point-of-Care Ultrasonography (POCUS): a Conceptual Framework and Review of Existing Assessments. J Gen Intern Med, 34(6),1025–1031.

Key Findings

  • According to Table 2 on page 1027, potential technical competency components for assessment include:
    • Probe selection
    • Image mode selection (e.g., cardiac, abdominal)
    • Proper image orientation marker (e.g., “dot”) selection
    • Probe positioning
    • Image depth
    • Image gain
    • Centering of target structure
    • Troubleshooting difficult windows
    • Demonstrates advanced functions (M-mode, Doppler, image capture) 

Paper Implications

  • While the paper does not recommend specific metrics for each competency component, this list provides a guideline for institutions to develop their own metrics

Publication

Martin G. Tolsgaard ,Tobias Todsen,Jette L. Sorensen,Charlotte Ringsted,Torben Lorentzen,Bent Ottesen,Ann Tabor. (2013). International Multispecialty Consensus on How to Evaluate Ultrasound Competence: A Delphi Consensus Survey. PLoS ONE, 8(2), e57687.

Key Findings

  • In Table 3, the publication lists an Objective Structured Assessment of Ultrasound Skills (OSAUS). Among the components includes “Image optimization”
    • Image optimization means “Consistently ensuring optimal image quality by adjusting gain, depth, focus, frequency, etc.” on a scale of 1 to 5 where 1 = fails to optimize images, 3 = competent image optimization but not done consistently, and 5 = consistent optimization of images

Paper Implications

  • Consistency can be a key part of assessing competency of image acquisition, assessing a singular instance of image acquisition is most likely insufficient

Publication

Søren Helbo Skaarup, Christian B. Laursen, Anne Sofie Bjerrum, and Ole Hilberg. (2017). Objective and Structured Assessment of Lung Ultrasound Competence A Multispecialty Delphi Consensus and Construct Validity Study. Ann Am Thorac Soc, 14(4), 555–560.

Key Findings

  • In Table 2 on page 557, containing a list of competency components in a lung ultrasound objective structured assessment of technical skills tool, a section titled “Technical Skills” on scales of 1 to 5 includes:
    • Correct placement of patient (e.g., supine when scanning for pneumothorax), where 1 = Wrong placement (e.g., evaluating pleural effusion with patient in Trendelenburg position) and 5 = Optimal placement (e.g., evaluating pleural effusion with patient in seated position)
    • Correct choice of transducer, where 1 = Wrong choice of transducer and 5 = Optimal choice of transducer
    • Correct depth, where 1 = Wrong depth setting and 5 = Optimal depth setting
    • Correct gain, where 1 = Wrong setting of gain and 5 = Optimal setting of gain
    • Correct handling of transducer, where 1 = Poor transducer handling and 5 = Optimal transducer handling

Paper Implications

  • Notably, the tool uses a numerical scale, but scoring learners within the scale requires subjective judgement - likely, experienced POCUS experts are needed for learner assessment using this tool