A Depth-Guided Annotation Tool for B-Line Quantification in Lung Ultrasound

Abstract

Pulmonary congestion is a critical and common complication of congestive heart failure, requiring timely and accurate monitoring to guide clinical decision-making. Lung ultrasound (LUS) has emerged as a promising point-of-care tool for assessing pulmonary fluid status due to its portability, safety, and sensitivity. However, current LUS interpretation methods, particularly manual B-line counting, are highly subjective and suffer from substantial inter- and intra-observer variability. This variability limits reproducibility, hampers clinical integration, and challenges the development of robust AI models for LUS analysis. This thesis presents the design, implementation, and evaluation of AnnotateUltrasound, a novel open-source module for structured LUS annotation within the 3D Slicer platform. The tool introduces a standardized sector-based annotation schema and a visual depth guide to reduce subjectivity in pleural B-line coverage estimation. A human-centered design process, informed by iterative clinical feedback, shaped a user-friendly interface with structured annotation, efficient navigation, and support for multi-rater workflows. Empirical evaluation involved a user study with 18 participants from clinical and non-clinical backgrounds. Results show that the depth guide reduced inter-rater variability (mean MAD: 0.063 to 0.034) and improved overall inter-rater agreement. Intra-rater consistency also improved with the guide (correlation r = 0.85 to 0.92), supporting the guide’s role in enhancing reproducibility. Participants reported high usability (mean SUS score: 83.2) and reduced cognitive workload (NASA-TLX). Qualitative feedback further highlighted the tool’s utility as both a reproducible annotation platform and an effective educational aid. The AnnotateUltrasound module is already in use by clinicians, including researchers at Harvard-affiliated institutions, to support large-scale dataset curation, gold-standard adjudication, and AI model development. This tool addresses a critical gap in structured LUS annotation workflows by enabling reproducible, sector-based quantification of B-lines and pleural features. Its AI-ready design lays the groundwork for integrating automated models into diagnostic and annotation pipelines, ultimately supporting reproducible lung ultrasound analysis in heart failure care and beyond.