Automatic segmentation of neural tissue in 3D electron microscopy images using deep learning. The la
Bellevue (WA), 8th of November 2017 - DRVISION released the first commercially available deep learning solution for automated neuron segmentation in 3D electron microscopy images, in Aivia 6. DRVISION has trained and deployed a state-of-the-art fully convolutional neural network architecture, known as U-net with densely connected blocks (1 and 2), that has been shown to outperform other modern deep learning models for semantic image segmentation. The DRVISION-trained deep learning model was benchmarked on a 1kx1kx100 pixel image resulting in the detection of every cell automatically in less than 25 mins running on a consumer-grade gaming laptop. This task would normally take a human 2 to 4 weeks to accomplish, depending on user experience and cell density in the sample. Aivia 6 users will have the option to optimize the tool to their own data sets by providing a small amount of additional training data with ground truths. DRVISION will use this data to augment the deep learning model using transfer learning (3). The updated deep learning model is then deployed to the user. DRVISION will continue to test and validate the applicability of newly published deep learning architectures and will deploy the top performing ones in future Aivia versions both for electron microscopy image analysis and other applications.
Figure 1 – (left) Unprocessed electron micrograph showing the nanostructure of neurites in the mouse neocortex. (center) Confidence map output of the DRVISION-trained deep learning model. The model identifies the cell membranes that define individual neurite segments. (right) Example of 2D segmentation result showing each cell with a unique color. Raw (public domain) data set from (4) which was originally published in (5).
Figure 2 – 3D representation of subset of neurite segments automatically detected using the DRVISION-trained deep learning model for 3D electron microscopy images. The image shows neurites segments in very close proximity to each other or even touching. 90% of the segmented neurites have been made transparent (not visible) so that the morphology of the depicted subset can be seen more clearly.
Aivia 6 also features an innovative volume rendering engine capable of interactively displaying multi-terabyte 3D/4D microscopy images in real-time. The growing popularity of light sheet microscopes and tissue clearing techniques enable the scientific community to image large samples (on the scale of whole organs or even organisms) at high temporal and spatial resolution. As a result, researchers routinely generate 3D/4D images of 10s of gigabytes (GB) up to multi-terabytes (TB) which fail to render in nearly all microscopy image visualization tools currently available. With Aivia 6, a 37 GB 3D image is displayed in less than 3 seconds. While a 240 GB 4D image (2 GB per 3D time point) renders in 1.5 seconds.
Figure 3 – (left) 3D fluorescence microscopy image of a whole mouse brain. Image dimensions are 2160x2560x1565, 16 bit. Such an image loads and renders in Aivia 6 in less than 3 seconds. (right) As the user zooms in on a given region of interest the finer details of the sample become visible. In this case individual pyramidal neurons are visible. Image courtesy of Daniel Bloodgood, Kash Lab (University of North Carolina). Aivia 6 users can freely pan and zoom in/out of any ROI and they will always be presented with the best 3D/4D rendering.
“Machine and deep learning approaches are poised to revolutionize the way the microscopy community conducts image analysis. Examples from other communities and industries clearly show the benefits and the biomedical scientific community is starting to adopt machine learning aided image analysis. Aivia 6 introduces the first commercially available deep learning model for 3D electron microscopy image segmentation.” Commented Luciano Lucas, PhD (Executive VP at DRVISION). “We are also excited about the high-performance volume rendering engine we are launching today as it is capable of interactively rendering multi-terabyte 3D/4D images. For the first time, many of the researchers using light sheet microscopes will be able to visualize their 3D/4D data sets in real-time.”
More info: Aivia is a pioneering software for image visualization, morphometric analysis and phenotype classification / discovery. For visualization it can display images both on a regular computer screen and in virtual reality using the HTC Vive or the Oculus Rift headsets. Aivia’s analysis portfolio includes, in addition to the new deep learning model in Aivia 6, 15 analysis pipelines for 2 to 4D fluorescence microscopy images as well as 2D and 2D plus time phase contrast images. The most popular tools are 3D object analysis, 3D neuron and spine analysis, cell detection and tracking, colocalization, exocytosis and calcium oscillation analysis. Downstream of the analysis Aivia enables users to leverage the power of supervised machine learning to identify and discover novel phenotypes.
Aivia 6 trial license, webpage, release notes:
Jegou S., et al., arXiv:1611.09326v3 [cs.CV]
Huang G., et al., arXiv:1608.06993v4 [cs.CV]
Donahue J., et al., arXiv:1310.1531v1 [cs.CV]
Kasthuri N, http://neurodata.io/Kasthurietal2014.
Kasthuri N, et al., Cell, Volume 162, Issue 3, p648–661, 30 July 2015