- ICC between experts = 0.603
- ICC between expert and novice was 0.593,
- intrarater agreement between expert and self was 0.586-0.676.
Another study reported ICC of 0.82 between experts for B-line counts.
Efforts have been made to develop frameworks for assessing competency in point-of-care ultrasound (POCUS), but more can be done to ensure technical skill and clinical utilization. However, additional research could be done to obtain evidence for POCUS training effectiveness and longitudinal trainee competency in support of policy guideline development.
Gullett, J., Donnelly, J.P., Sinert, R., Hosek, B., Fuller, D., Hill, H., Feldman, I., Galetto, G., Auster, M., Hoffmann, B. Interobserver agreement in the evaluation of B-lines using bedside ultrasound. J Crit Care. 2015 Dec;30(6):1395-9.
- The average agreement (ICC) across all lung zones for B-line counts between emergency medicine expert sonographers in patients presenting with undifferentiated dyspnea was 0.603
- Agreement for B-line counts between experts varied based on zone of the lung, with the anterior/superior left lung zone showing highest agreement (ICC 0.891)
- Intrarater agreement (Expert vs self) for B-line counts was ICC 0.586-0.676
- Variance in B-line counts in POCUS may strongly depend on the operator and pathology
Moore CL, Wang J, Battisti AJ, Chen A, Fincke J, Wang A, Wagner M, Raju B, Baloescu C. Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device. J Ultrasound Med. 2021 Dec 28. doi: 10.1002/jum.15935. Epub ahead of print.
- ICC between experts for B-line counts was 0.82 on lung ultrasound clips obtained by novice POCUS sonographers
- ICC between expert and artificial intelligence was 0.56 on lung ultrasound clips obtained by novice POCUS sonographers
- More research is required to develop AI to be comparable to expert review
- Additional attention could be focused on the inferior, lateral lung zones when training for POCUS in B-line counts for undifferentiated dyspnea