close
Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Feb:60:101602.
doi: 10.1016/j.media.2019.101602. Epub 2019 Nov 8.

Automatic kidney segmentation in ultrasound images using subsequent boundary distance regression and pixelwise classification networks

Shi Yin  1 , Qinmu Peng  2 , Hongming Li  3 , Zhengqiang Zhang  4 , Xinge You  5 , Katherine Fischer  6 , Susan L Furth  7 , Gregory E Tasian  8 , Yong Fan  9
Affiliations

Automatic kidney segmentation in ultrasound images using subsequent boundary distance regression and pixelwise classification networks

Shi Yin et al. Med Image Anal. 2020 Feb.

Abstract

It remains challenging to automatically segment kidneys in clinical ultrasound (US) images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we propose subsequent boundary distance regression and pixel classification networks to segment the kidneys automatically. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from US images. These features are used as input to learn kidney boundary distance maps using a boundary distance regression network and the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixelwise classification network in an end-to-end learning fashion. We also adopted a data-augmentation method based on kidney shape registration to generate enriched training data from a small number of US images with manually segmented kidney labels. Experimental results have demonstrated that our method could automatically segment the kidney with promising performance, significantly better than deep learning-based pixel classification networks.

Keywords: Boundary detection; Boundary distance regression; Pixelwise classification; Ultrasound images.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.
Kidneys in US images may have varied shapes and the kidney pixels typically have heterogeneous intensities and textures.
Fig. 2.
Fig. 2.
Transfer learning network, and subsequent boundary distance regression and pixel classification networks for fully automatic kidney segmentation in US images. The boundary detection network (Bnet) is trained using a distance loss function and the end-to-end subsequent segmentation network is trained by combining the distance loss function and a softmax loss function.
Fig. 3.
Fig. 3.
The architecture of theVGG16 model.
Fig. 4.
Fig. 4.
The transfer learning architecture of the Deeplab model (Chen et al., 2018b). We extracted the pretrained feature maps from the exiting Deeplab model.
Fig. 5.
Fig. 5.
Network architecture of the boundary distance regression network.
Fig. 6.
Fig. 6.
An example kidney US image and kidney boundary (a), its boundary distance map (b), and its normalized potential distance map with λ = 1 (c). The colorbar of (c) is in log scale.
Fig. 7.
Fig. 7.
Architecture of the kidney pixel classification network.
Fig. 8.
Fig. 8.
Data-augmentation based on TPS transformation and flipping. (a) is a moving image, (b) is a fixed image, (c) is the registered image and (d) is the flipped registered image. The kidney shape denoted by the red curve is approximately modeled as an ellipse denoted by the yellow curves. The yellow stars denote the landmark points of the TPS transformation.
Fig. 9.
Fig. 9.
A multi-task learning based segmentation networks (top) under comparison with the proposed subsequent segmentation network. In both the networks, the boundary detection (boundary distance regression) network and the pixelwise classification network shared the same transfer learning network.
Fig. 10.
Fig. 10.
Traces of the training loss (left) and validation accuracy (right) associated with 3 different training strategies.
Fig. 11.
Fig. 11.
Example segmentation results obtained with 3 training strategies. (a) input image and ground truth boundary, (b) results of the training from scratch without data augmentation, (c) results of the transfer learning without data augmentation, and (d) results of the transfer-learning with data augmentation.
Fig. 12.
Fig. 12.
Results of the kidney boundary detection networks trained using different loss functions. The 1st and 4rd rows show predicted boundary distance maps, the 2st and 5rd rows show boundary binary maps, and the 3nd and 6th rows show kidney masks obtained by the morphology operation and minimum spanning tree based post-processing method.
Fig. 13.
Fig. 13.
Representative segmentation results obtained by different deep learning networks trained without data augmentation (the 1st, 3nd, and 5rd rows) and with data augmentation (the 2th, 4th, and 6th rows).
Fig. 14.
Fig. 14.
Results for the boundary detection network and the end-to-end learning networks. (a) input kidney US images, (b) binary skeleton maps of the predicted distance maps, (c) kidney masks obtained with the minimum spanning tree based post-processing, (d) predicted distance maps obtained by the end-to-end subsequent segmentation networks, (e) kidney masks obtained by the end-to-end subsequent segmentation network, and (f) kidney masks obtained by manual labels.
Fig 15.
Fig 15.
Results for the multi-task learning based segmentation network and the end-to-end learning networks. (a) input kidney US images, (b) manual label, (c) results obtained by Deeplab, (d) results obtained by task 1 of the multi-task learning (MTL) network, (e) results obtained by task 2 of the MTL network, and (f) results obtained by the subsequent segmentation network.
Fig 16.
Fig 16.
Example segmentation results of kidney images with boundary cut. The boundary cut is indicated by the yellow arrows.
See this image and copyright information in PMC

References

    1. Ardon R, Cuingnet R, Bacchuwar K, Auvray V, 2015. Fast kidney detection and segmentation with learned kernel convolution and model deformation in 3D ultrasound images, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 268–271.
    1. Badrinarayanan V, Kendall A, Cipolla R, 2017. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 2481–2495. - PubMed
    1. Bertasius G, Shi J, Torresani L, 2015. High-for-low and low-for-high: Efficient boundary detection from deep object features and its applications to high-level vision, Proceedings of the IEEE International Conference on Computer Vision, pp. 504–512.
    1. Bookstein FL, 1989. Principal warps: thin-plate splines and the decomposition of deformations. IEEE transactions on pattern analysis and machine intelligence 11, 567–585.
    1. Boyd S, Vandenberghe L, 2004. Convex optimization. Cambridge university press.

Publication types