Plant Cell Quantification

Welcome to the Inside of Plant Cells

This work focuses on quantification analysis of plant cells in microscopic images
The goal of this work is to find the dynamic interection between different structures in plant cells
Current work includes segmenting filamentous structures at instance level and quantification analysis on filamentous structures

Microtubules Quantification: Github Link

Actin Quantification: Github Link

Microtubules dataset with instance labels: Google Drive Link

Author: Yi Liu Homepage

Our most recent work on actin filaments quantification! Oral presentation at ICIP 2020

Presenter: Yi Liu

---------------------------------------------------------------------------------------------------------------------------------

Demo of stromule tracking on actin/microtubule filaments

tracking start from 45s

---------------------------------------------------------------------------------------------------------------------------------

Demo of Our Instance Segmentation Result

Since there are too many filaments in one image, we present indiviudal filaments one by one

---------------------------------------------------------------------------------------------------------------------------------

Part1: Binary Segmentation Microtubules and Actin filaments

In this part, we introduce a novel deep network architecture for filament segmentation in confocal microscopy images that improves upon the state-of-the-art U-net and SOAX methods. We also propose a strategy for data annotation, and create datasets for microtubule and actin filaments.

Liu Y, Treible W, Kolagunda A, Nedo A, Saponaro P, Caplan J, Kambhamettu C. Densely connected stacked u-network for filament segmentation in microscopy images. InProceedings of the European Conference on Computer Vision (ECCV) 2018 (pp. 0-0).
}

Part2: Instance Segmentation of Microtubules

In this Part, we introduce a orientation-aware neural network, which contains six orientation-associated branches. Each branch detects filaments with specific range of orientations, thus separating them at junctions, and turning intersections to overpasses. A terminus pairing algorithm is also proposed to regroup filaments from different branches, and achieve individual filaments extraction.

Liu, Y., Kolagunda, A., Treible, W., Nedo, A., Caplan, J., & Kambhamettu, C. (2019). Intersection to Overpass: Instance Segmentation on Filamentous Structures With an Orientation-Aware Neural Network and Terminus Pairing Algorithm. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 0-0).

Part3: Instance Segmentation of Actins

In this Part, we adopt a Convolutional Neural Network (CNN)to segment actin filaments first, and then we utilize a mod-ified Resnet to detect junctions and endpoints of filaments.With binary segmentation and detected keypoints, we applya fast marching algorithm to obtain the number and length ofeach actin filament in microscopic images.

Y. Liu, A. Nedo, K. Seward, J. Caplan and C. Kambhamettu, "Quantifying Actin Filaments in Microscopic Images using Keypoint Detection Techniques and A Fast Marching Algorithm," 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 2020, pp. 2506-2510, doi: 10.1109/ICIP40778.2020.9191337.

Details of Part 1: Binary Segmentation Microtubules and Actin filaments

Annotation process. (a) Original image of mircrotubules. (b) SOAX Segmentation result

(d) Manually labeled ground truth based on U-net result

An illustration of our proposed network

Segmentation of microtubules.

From left to right: original image, ground truth,SOAX, U-net module, proposed network with cross connection

Details of Part 2: Instance Segmentation of Microtubules

Intuition of our proposed method:

Regrouping the fragments at intersections is the bottle neck to group individual filaments.

We propose an orientation-aware neural network which can sort filaments into six bins according to their orientation angle

Each filament will automatically classiffied into certain layer, the intuition is similar like turning an intersection to overpass

An illustration of our proposed pipeline:

Step 1: The network take a segmentation as input, and six orientation-associated outputs willbe generated.

Step 2: The proposed algorithm is applied on these six outputs and regroup fragments into individual filaments.

An example of recursively grouping the most eligiblecandidate from different layers

Reconnection by their geometric properties

Comparison of different methods on extracting individ-ual filament. (a) Original image (b) Segmentation

(f)SOAX without filtering out small fragments. Different colors in-dicate different labels, and some color might look similar, but theyare actually different labels.

Details of Part 3: Instance Segmentation of Actins

The pipeline of our proposed approach. (a) Microscopic image of actin filaments. (b) Binary segmentation.

(c) Heatmapsof junctions and endpoints. (d) Offset maps of junctions and endpoints. (e) Prediction of junctions and endpoints.

(f) Geodesicdistance map from junctions and endpoints, which are color of cyan.

(g) The local maximum value of the geodesic distancemap, which is represented in blue with a pink circle.

(a) to (b): We use a neural network to obtain binary segmentation.

(b)to (c),(d) and (e): Our modified Resnet takes binary segmentation results as inputs and outputs offset maps and heatmaps ofjunctions and endpoints.

Then we integrate (c) and (d) to obtain refined locations of these points.

(e) to (f) and (g): We setpredicted keypoints as start points and utilize a fast marching algorithm to calculate the geodesic distance map. The local peakvalues of the geodesic distance map can represent the half-length value of the actin filaments.