Knowledge

POCUS Competency

B-Lines: Interrater & Intrarater Agreement

Delete

In a recent study on interrater agreement (ICC) and intrarater agreement for B-line counts:

  • ICC between experts = 0.603
  • ICC between expert and novice was 0.593,
  • intrarater agreement between expert and self was 0.586-0.676. 

Another study reported ICC of 0.82 between experts for B-line counts.

Efforts have been made to develop frameworks for assessing competency in point-of-care ultrasound (POCUS), but more can be done to ensure technical skill and clinical utilization. However, additional research could be done to obtain evidence for POCUS training effectiveness and longitudinal trainee competency in support of policy guideline development.


Delete

Expert vs Expert Agreement


Publication

Gullett, J., Donnelly, J.P., Sinert, R., Hosek, B., Fuller, D., Hill, H., Feldman, I., Galetto, G., Auster, M., Hoffmann, B. Interobserver agreement in the evaluation of B-lines using bedside ultrasound. J Crit Care. 2015 Dec;30(6):1395-9.

Key Findings

  • The average agreement (ICC) across all lung zones for B-line counts between emergency medicine expert sonographers in patients presenting with undifferentiated dyspnea was 0.603
  • Agreement for B-line counts between experts varied based on zone of the lung, with the anterior/superior left lung zone showing highest agreement (ICC 0.891)
  • Intrarater agreement (Expert vs self) for B-line counts was ICC 0.586-0.676

Paper Implications

  • Variance in B-line counts in POCUS may strongly depend on the operator and pathology




Publication

Moore CL, Wang J, Battisti AJ, Chen A, Fincke J, Wang A, Wagner M, Raju B, Baloescu C. Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device. J Ultrasound Med. 2021 Dec 28. doi: 10.1002/jum.15935. Epub ahead of print.

Key Findings

  • ICC between experts for B-line counts was 0.82 on lung ultrasound clips obtained by novice POCUS sonographers
  • ICC between expert and artificial intelligence was 0.56 on lung ultrasound clips obtained by novice POCUS sonographers

Paper Implications

  • More research is required to develop AI to be comparable to expert review


Delete

Novice vs Expert Agreement


Publication

Gullett, J., Donnelly, J.P., Sinert, R., Hosek, B., Fuller, D., Hill, H., Feldman, I., Galetto, G., Auster, M., Hoffmann, B. Interobserver agreement in the evaluation of B-lines using bedside ultrasound. J Crit Care. 2015 Dec;30(6):1395-9.

Key Findings

  • The average agreement (ICC) across all lung zones for B-line counts between expert and novice in patients presenting with undifferentiated dyspnea was 0.593
  • Agreement for B-line counts varied between expert and novice based on zone of the lung, with the highest agreement in the anterior/superior right lung zone (ICC 0.756), with the lowest agreement in the lateral/inferior lung zones

Paper Implications

  • Additional attention could be focused on the inferior, lateral lung zones when training for POCUS in B-line counts for undifferentiated dyspnea