TY - GEN
T1 - Learning metric graphs for neuron segmentation in electron microscopy images
AU - Luther, Kyle
AU - Seung, H. Sebastian
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/4
Y1 - 2019/4
N2 - In the deep metric learning approach to image segmentation, a convolutional net densely generates feature vectors at the pixels of an image. Pairs of feature vectors are trained to be similar or different, depending on whether the corresponding pixels belong to same or different ground truth segments. To segment a new image, the feature vectors are computed and clustered. Both empirically and theoretically, it is unclear whether or when deep metric learning is superior to the more conventional approach of directly predicting an affinity graph with a convolutional net. We compare the two approaches using brain images from serial section electron microscopy images, which constitute an especially challenging example of instance segmentation. We first show that seed-based postprocessing of the feature vectors, as originally proposed, produces inferior accuracy because it is difficult for the convolutional net to predict feature vectors that remain uniform across large objects. Then we consider postprocessing by thresholding a nearest neighbor graph followed by connected components. In this case, segmentations from a 'metric graph' turn out to be competitive or even superior to segmentations from a directly predicted affinity graph. To explain these findings theoretically, we invoke the property that the metric function satisfies the triangle inequality. Then we show with an example where this constraint suppresses noise, causing connected components to more robustly segment a metric graph than an unconstrained affinity graph.
AB - In the deep metric learning approach to image segmentation, a convolutional net densely generates feature vectors at the pixels of an image. Pairs of feature vectors are trained to be similar or different, depending on whether the corresponding pixels belong to same or different ground truth segments. To segment a new image, the feature vectors are computed and clustered. Both empirically and theoretically, it is unclear whether or when deep metric learning is superior to the more conventional approach of directly predicting an affinity graph with a convolutional net. We compare the two approaches using brain images from serial section electron microscopy images, which constitute an especially challenging example of instance segmentation. We first show that seed-based postprocessing of the feature vectors, as originally proposed, produces inferior accuracy because it is difficult for the convolutional net to predict feature vectors that remain uniform across large objects. Then we consider postprocessing by thresholding a nearest neighbor graph followed by connected components. In this case, segmentations from a 'metric graph' turn out to be competitive or even superior to segmentations from a directly predicted affinity graph. To explain these findings theoretically, we invoke the property that the metric function satisfies the triangle inequality. Then we show with an example where this constraint suppresses noise, causing connected components to more robustly segment a metric graph than an unconstrained affinity graph.
KW - Image segmentation
KW - Machine learning
KW - Microscopy - electron
UR - http://www.scopus.com/inward/record.url?scp=85073897750&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85073897750&partnerID=8YFLogxK
U2 - 10.1109/ISBI.2019.8759576
DO - 10.1109/ISBI.2019.8759576
M3 - Conference contribution
AN - SCOPUS:85073897750
T3 - Proceedings - International Symposium on Biomedical Imaging
SP - 244
EP - 248
BT - ISBI 2019 - 2019 IEEE International Symposium on Biomedical Imaging
PB - IEEE Computer Society
T2 - 16th IEEE International Symposium on Biomedical Imaging, ISBI 2019
Y2 - 8 April 2019 through 11 April 2019
ER -