Interobserver Agreement and Correlation of an Automated Algorithm for B‐Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device
Objectives B‐lines are ultrasound artifacts that can be used to detect a variety of pathologic lung conditions. Computer‐aided methods to detect and quantify B‐lines may standardize quantification and improve diagnosis by novice users. We sought to test the performance of an automated algorithm for...
Saved in:
Published in | Journal of ultrasound in medicine Vol. 41; no. 10; pp. 2487 - 2495 |
---|---|
Main Authors | , , , , , , , , |
Format | Journal Article |
Language | English |
Published |
Hoboken, USA
John Wiley & Sons, Inc
01.10.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Objectives
B‐lines are ultrasound artifacts that can be used to detect a variety of pathologic lung conditions. Computer‐aided methods to detect and quantify B‐lines may standardize quantification and improve diagnosis by novice users. We sought to test the performance of an automated algorithm for the detection and quantification of B‐lines in a handheld ultrasound device (HHUD).
Methods
Ultrasound images were prospectively collected on adult emergency department patients with dyspnea. Images from the first 124 patients were used for algorithm development. Clips from 80 unique subjects for testing were randomly selected in a predefined proportion of B‐lines (0 B‐lines, 1–2 B‐lines, 3 or more B‐lines) and blindly reviewed by five experts using both a manual and reviewer‐adjusted process. Intraclass correlation coefficient (ICC) and weighted kappa were used to measure agreement, while an a priori threshold of an ICC (3,k) of 0.75 and precision of 0.3 were used to define adequate performance.
Results
ICC between the algorithm and manual count was 0.84 (95% confidence interval [CI] 0.75–0.90), with a precision of 0.15. ICC between the reviewer‐adjusted count and the algorithm count was 0.94 (95% CI 0.90–0.96), and the ICC between the manual and reviewer‐adjusted counts was 0.94 (95% CI 0.90–0.96). Weighted kappa was 0.72 (95% CI 0.49–0.95), 0.88 (95% CI 0.74–1), and 0.85 (95% CI 0.89–0.96), respectively.
Conclusions
This study demonstrates a high correlation between point‐of‐care ultrasound experts and an automated algorithm to identify and quantify B‐lines using an HHUD. Future research may incorporate this HHUD in clinical studies in multiple settings and users of varying experience levels. |
---|---|
Bibliography: | This work was accomplished with funding from Philips Healthcare and a grant from the Department of Health and Human Services; Office of the Assistant Secretary for Preparedness and Response; Biomedical Advanced Research and Development Authority, under Contract No. 75A50120C00097. Interobserver Agreement and Correlation of an Automated Algorithm for B‐line Quantification with Expert Sonologist Review in a Handheld Ultrasound Device Dr Moore reports research funding from Philips Healthcare; Jin Wang is an employee of Philips Healthcare; Dr Battisti was an employee of Philips Healthcare at the time this work was performed; Dr Chen is an employee of Philips Healthcare; Dr Fincke is an employee of Philips Healthcare; Dr Wang reports support from Philips Healthcare for participation in the project; Dr Wagner reports that his employer (Prisma Health Department of Internal Medicine) received funds from Philips Healthcare for participation in the project; Dr Raju is an employee of Philips Healthcare; Dr Baloescu reports research funding from Philips Healthcare. ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
ISSN: | 0278-4297 1550-9613 |
DOI: | 10.1002/jum.15935 |