-
摘要: 針對目前視網膜血管分割中存在的細小血管提取不完整、分割不準確的問題，從血管形狀拓撲關系利用的角度出發，探索多任務卷積神經網絡設計，提出骨架圖引導的級聯視網膜血管分割網絡框架。該框架包含血管骨架圖提取網絡模塊、血管分割網絡模塊和若干自適應特征融合結構體。骨架提取輔助任務用于提取血管中心線，能夠最大限度地保留血管拓撲結構特征；自適應特征融合結構體嵌入在兩個模塊的特征層間。該結構體通過學習像素級的融合權重，有效地將血管拓撲結構特征與血管局部特征相融合，加強血管特征的結構信息響應。為了獲得更完整的骨架圖，骨架圖提取網絡還引入了基于圖的正則化損失函數用于訓練。與最新的血管分割方法相比，該方法在3個公共視網膜圖像數據集上均獲得第一名，在DRIVE，STARE和CHASEDB1中其F1值分別為83.1%，85.8%和82.0%。消融實驗表明骨架圖引導的視網膜血管分割效果更好，并且，基于圖的正則化損失也能進一步提高血管分割準確性。通過將骨架提取模塊和血管分割模塊替換成不同的卷積網絡驗證了框架的普適性。Abstract: Accurate identification of retinal vessels is essential for assisting doctors in screening early fundus diseases. Diabetes, hypertension, and cardiovascular disease can cause abnormalities of the retinal vascular structure. Retinal vessel segmentation maps can be quickly obtained using the automated retinal vessel segmentation technology, which saves time and cost of manually identifying retinal vessels. Aiming at the problem of incomplete and inaccurate extraction of fine retinal vessels, this paper explored the design of a multitask convolutional neural network and the topological relationship of retinal vessels. A cascaded retinal vessel segmentation network framework guided by a skeleton map was proposed. The auxiliary task of skeleton extraction was used to extract vessel centerlines, which could maximally preserve topological structure information. SAFF cascaded the two modules by remaining embedded between their feature layers. This process could effectively fuse the structural features with the vessel local features by learning pixel-wise fusion weight and thus enhancing the structural response of features in the vessel segmentation module. To obtain a complete skeleton map, the skeleton map extraction module introduced a graph-based regularization loss function for training. Compared with the latest vessel segmentation methods, the proposed approach wins the first place among the three public retinal image datasets. F1 metrics of the proposed method achieved 83.1%, 85.8%, and 82.0% on the DRIVE, STARE, and CHASEDB1 datasets, respectively. Ablation studies have shown that skeleton map-guided vessel segmentation is more effective, and graph-based regularization loss further improves accuracy of the retinal vessel segmentation compared to the vanilla network. Moreover, the framework generality is verified by replacing the skeleton map extraction and vessel segmentation modules with various convolutional networks.
-
圖 1 骨架圖引導的視網膜血管分割網絡框架
Figure 1. Skeleton map-guided retinal vessel segmentation framework
圖 4 自適應特征融合模塊和注意力門控的對比。（a）深層特征
$ {{\boldsymbol{f}}_{\rm{g}}} $ 過濾淺層特征$ {{\boldsymbol{f}}_{\rm{l}}} $ ；（b）包含結構信息的骨架特征$ {{\boldsymbol{f}}_{\rm{s}}} $ 和血管特征$ {{\boldsymbol{f}}_{{\rm{ves}}}} $ 融合Figure 4. Comparison of the self-adaptive feature fusion block and attention gating: (a) deeper features
$ {{\boldsymbol{f}}_{\rm{g}}} $ filter shallower features$ {{\boldsymbol{f}}_{\rm{l}}} $ ; (b) vessel features$ {{\boldsymbol{f}}_{{\rm{ves}}}} $ fuse skeleton features$ {{\boldsymbol{f}}_{\rm{s}}} $ containing structural information
圖 5 消融實驗中每輪訓練后在不同驗證集上的F1值。（a）DRIVE；（b）CHASEDB1
Figure 5. F1 on the validation set after each training iteration in ablation experiments: (a) DRIVE; (b) CHASEDB1
圖 6 框架在DRIVE（a）和CHASEDB1（b）數據集上的訓練損失
Figure 6. Training loss of the framework on the DRIVE (a) and CHASEDB1 (b) datasets
表 1 本文提出的方法和近期的先進方法在F1值、敏感性Se、準確率Acc、AUC的比較結果
Table 1. Comparison results between our proposed method and the recent advanced methods of the F1 score, Sensitivity, Accuracy, and AUC
Method DRIVE STARE CHASEDB1 F1/% Se/% Acc/% AUC/% F1/% Se/% Acc/% AUC/% F1/% Se/% Acc/% AUC/% Segment[37] — 76.5 95.4 97.5 — 75.8 96.1 98.0 — 76.3 96.1 97.8 DS[11] — 87.3 95.0 98.0 — 76.7 97.1 98.8 — 76.7 97.7 99.0 DUNet[47] — 78.9 97.0 98.6 — 74.3 97.3 98.7 — 82.3 97.2 98.6 Cascade[49] 80.9 76.5 95.4 — 81.3 75.2 96.4 — 78.1 77.3 96.0 — DualUNet[48] 82.7 79.4 95.7 97.7 — — — — 80.4 80.7 96.6 98.1 CE-Net[34] — 83.1 95.5 97.8 — 78.4 95.8 97.9 — — — — STD[40] — 81.5 97.0 98.6 — — — — — — — — IterNet[46] 82.2 77.9 95.7 98.1 81.5 77.2 97.0 98.8 80.7 79.7 96.6 98.5 Our method 83.1 83.7 97.1 98.8 85.8 86.4 97.1 99.1 82.0 84.5 97.7 99.1 
下載: 導出CSV
表 2 第一組消融實驗結果
Table 2. Results of the first ablation experiments
Control group DRIVE CHASEDB1 F1/% Se/% Acc/% AUC/% F1/% Se/% Acc/% AUC/% Single task
network84.2 84.6 96.5 98.8 81.3 82.9 97.1 98.8 +Skeleton
extraction85.0 85.5 97.7 99.2 81.8 83.6 97.6 99.0 +Structure loss 85.1 86.3 97.7 99.3 82.0 84.5 97.7 99.1
下載: 導出CSV
中文字幕在线观看表 3 第二組消融實驗結果
Table 3. Results of the second ablation experiments
Network DRIVE CHASEDB1 F1/% Se/% Acc/% AUC/% F1/% Se/% Acc/% AUC/% ResNet34 83.1 83.7 97.1 98.8 82.0 84.5 97.1 99.1 ResNet18 82.9 83.5 97.0 98.7 81.7 83.8 97.6 99.0 VGG16 82.8 83.2 97.0 98.7 81.6 83.7 97.6 99.0
下載: 導出CSV
-
參考文獻
[1] Cong M, Wu T, Liu D, et al. Prostate MR/TRUS image segmentation and registration methods based on supervised learning. Chin J Eng, 2020, 42(10): 1362叢明, 吳童, 劉冬, 等. 基于監督學習的前列腺MR/TRUS圖像分割和配準方法. 工程科學學報, 2020, 42(10):1362 [2] Ma B Y, Jiang S F, Yin D, et al. Image segmentation metric and its application in the analysis of microscopic image. Chin J Eng, 2021, 43(1): 137馬博淵, 姜淑芳, 尹豆, 等. 圖像分割評估方法在顯微圖像分析中的應用. 工程科學學報, 2021, 43(1):137 [3] Tso M O M, Jampol L M. Pathophysiology of hypertensive retinopathy. Ophthalmology, 1982, 89(10): 1132 doi: 10.1016/S0161-6420(82)34663-1 [4] Yu S, Xiao D, Kanagasingam Y. Machine learning based automatic neovascularization detection on optic disc region. IEEE J Biomed Heal Inform, 2018, 22(3): 886 doi: 10.1109/JBHI.2017.2710201 [5] Becker C, Rigamonti R, Lepetit V, et al. Supervised feature learning for curvilinear structure segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, 2013: 526 [6] Tolias Y A, Panas S M. A fuzzy vessel tracking algorithm for retinal images based on fuzzy clustering. IEEE Trans Med Imaging, 1998, 17(2): 263 doi: 10.1109/42.700738 [7] Soares J V B, Leandro J J G, Cesar R M, et al. Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging, 2006, 25(9): 1214 doi: 10.1109/TMI.2006.879967 [8] Sebbe R, Gosselin B, Coche E, et al. Segmentation of opacified thorax vessels using model-driven active contour // 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. Shanghai, 2006: 2535 [9] Pal S, Chatterjee S, Dey D, et al. Morphological operations with iterative rotation of structuring elements for segmentation of retinal vessel structures. Multidimens Syst Signal Process, 2019, 30(1): 373 doi: 10.1007/s11045-018-0561-9 [10] Chang C C, Lin C C, Pai P Y, et al. A novel retinal blood vessel segmentation method based on line operator and edge detector // 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Kyoto, 2009: 299 [11] Zhang Y S, Chung A C S. Deep supervision with additional labels for retinal vessel segmentation task // International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, 2018: 83 [12] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, 2015: 234 [13] Guo C L, Szemenyei M, Hu H, et al. Channel attention residual U-Net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03702 [14] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, 2016: 770 [15] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Proceedings of the 31st Conference on neural information processing systems (NIPS 2017). Long Beach, 2017: 5998 [16] Hu J, Shen L, Sun G. Squeeze-and-excitation networks // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018: 7132 [17] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions //Proceedings of the 4th International Conference on Learning Representations. San Juan, 2016 [18] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4): 834 doi: 10.1109/TPAMI.2017.2699184 [19] Chen L C, Szemenyei M, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[J/OL]. arXiv preprint (2017-12-5) [2021-6-10].https://arxiv.org/abs/1706.05587 [20] Chen L C, Papandreou G, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation // Computer Vision – ECCV 2018. Munich, 2018: 833 [21] Song H M, Wang W G, Zhao S Y, et al. Pyramid dilated deeper ConvLSTM for video salient object detection // Computer Vision – ECCV 2018. Munich, 2018: 744 [22] Xie S N, Tu Z W. Holistically-nested edge detection // 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, 2015: 1395 [23] Orlando J I, Blaschko M. Learning fully-connected CRFs for blood vessel segmentation in retinal images // International Conference on Medical Image Computing and Computer-Assisted Intervention. Boston, 2014: 634 [24] Ricci E, Perfetti R. Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging, 2007, 26(10): 1357 doi: 10.1109/TMI.2007.898551 [25] Ganin Y, Lempitsky V. N4-fields: Neural network nearest neighbor fields for image transforms // Asian Conference on Computer Vision. Singapore, 2015: 536 [26] Dollár P, Zitnick C L. Structured forests for fast edge detection // 2013 IEEE International Conference on Computer Vision. Sydney, 2013: 1841 [27] Maninis K K, Pont-Tuset J, Arbeláez P, et al. Deep retinal image understanding // Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Athens, 2016: 140 [28] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition //Proceedings of the 3th International Conference on Learning Representations. San Diego, 2015 [29] Zhang S H, Fu H Z, Yan Y G, et al. Attention guided network for retinal image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 797 [30] Guo C L, Szemenyei M, Yi Y G, et al. SA-UNet: spatial attention U-net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03696 [31] Mou L, Zhao Y T, Chen L, et al. CS-net: Channel and spatial attention network for curvilinear structure segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 721 [32] Jiang Y, Tan N, Peng T T, et al. Retinal vessels segmentation based on dilated multi-scale convolutional neural network. IEEE Access, 2019, 7: 76342 doi: 10.1109/ACCESS.2019.2922365 [33] Hatamizadeh A, Hosseini H, Liu Z Y, et al. Deep dilated convolutional nets for the automatic segmentation of retinal vessels[J/OL]. arXiv preprint (2019-7-21) [2021-6-10]. https://arxiv.org/abs/1905.12120 [34] Gu Z, Cheng J, Fu H, et al. CE-net: Context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging, 2019, 38(10): 2281 doi: 10.1109/TMI.2019.2903562 [35] Mo J, Zhang L. Multi-level deep supervised networks for retinal vessel segmentation. Int J Comput Assist Radiol Surg, 2017, 12(12): 2181 doi: 10.1007/s11548-017-1619-0 [36] Hu K, Zhang Z Z, Niu X R, et al. Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing, 2018, 309: 179 doi: 10.1016/j.neucom.2018.05.011 [37] Yan Z Q, Yang X, Cheng K T. Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans Biomed Eng, 2018, 65(9): 1912 doi: 10.1109/TBME.2018.2828137 [38] Zhang Z J, Fu H Z, Dai H, et al. ET-net: A generic edge-aTtention guidance network for medical image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 442 [39] Kang H, Gao Y Q, Guo S, et al. AVNet: A retinal artery/vein classification network with category-attention weighted fusion. Comput Methods Programs Biomed, 2020, 195: 105629 doi: 10.1016/j.cmpb.2020.105629 [40] Zhang S H, Fu H Z, Xu Y W, et al. Retinal image segmentation with a structure-texture demixing network // Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Lima, 2020: 765 [41] Zheng S M, Zhang T Y, Zhuang J W, et al. A two-stream meticulous processing network for retinal vessel segmentation[J/OL]. arXiv preprint (2020-1-15) [2021-6-10]. https://arxiv.org/abs/2001.05829 [42] Zou B J, Dai Y L, He Q, et al. Multi-label classification scheme based on local regression for retinal vessel segmentation. IEEE/ACM Trans Comput Biol Bioinform, 2020, PP(99): 1 [43] Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Commun ACM, 1984, 27(3): 236 doi: 10.1145/357994.358023 [44] Hakim L, Yudistira N, Kavitha M, et al. U-net with graph based smoothing regularizer for small vessel segmentation on fundus image // Proceedings of the 26th International Conference on Neural Information Processing. Sydney, 2019: 515 [45] Oktay O, Schlemper J, Folgoc L L, et al. Attention U-net: Learning where to look for the pancreas. 2018[J/OL]. arXiv preprint (2018-5-20) [2021-6-10]. https://arxiv.org/abs/1804.03999 [46] Li L Z, Verma M, Nakashima Y, et al. IterNet: retinal image segmentation utilizing structural redundancy in vessel networks // 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass, 2020: 3645 [47] Jin Q G, Meng Z P, Pham T D, et al. DUNet: A deformable network for retinal vessel segmentation. Knowl Based Syst, 2019, 178: 149 doi: 10.1016/j.knosys.2019.04.025 [48] Wang B, Qiu S, He H G. Dual encoding U-net for retinal vessel segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 84 [49] Wang X H, Jiang X D, Ren J F. Blood vessel segmentation from fundus image by a cascade classification framework. Pattern Recognit, 2019, 88: 331 doi: 10.1016/j.patcog.2018.11.030 [50] Niemeijer M, Staal J, van Ginneken B, et al. Comparative study of retinal vessel segmentation methods on a new publicly available database // Proceedings of SPIE‒The International Society for Optical Engineering, 2004, 5370 I: 648 [51] Hoover A D, Kouznetsova V, Goldbaum M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging, 2000, 19(3): 203 doi: 10.1109/42.845178 [52] Owen C G, Rudnicka A R, Mullen R, et al. Measuring retinal vessel tortuosity in 10-year-old children: Validation of the computer-assisted image analysis of the retina (CAIAR) program. Invest Ophthalmol Vis Sci, 2009, 50(5): 2004 doi: 10.1167/iovs.08-3018 -
點擊查看大圖
圖(6) / 表(3)
計量
- 文章訪問數: 735
- HTML全文瀏覽量: 376
- PDF下載量: 80
- 被引次數: 0
