Combining Multiplexed Ion Beam Imaging (MIBI) with Convolutional Neural Networks to accurately segment cells in human tissue

Abstract

Background: Multiplexed imaging is a rapidly growing field that promises to substantially increase the number of proteins that can be imaged simultaneously. We have developed Multiplexed Ion Beam Imaging by Time of Flight (MIBI-TOF), which uses elemental reporters conjugated to primary antibodies that are then quantified using a time of flight mass-spectrometer. This technique allows for more than 40 distinct proteins to visualized at once in the same clinical samples. This has already yielded significant insights into the interactions and relationships between the many different immune cell populations present in the tumor microenvironment. However, one of the remaining challenges in analyzing such data is accurately determining target protein expression values for each cell in the image. This requires the precise delineation of boundaries between cells that are often tightly packed next to one another. Current methods to address this challenge largely rely on DNA intensity to make these splits, and are thus mostly limited to nuclear segmentation. Methods: We have developed a novel convolutional neural network to perform whole-cell segmentation from multiplexed imaging data. Rather than relying only on DNA signal, we use a panel of morphological markers. Our method integrates the information from these distinct proteins, allowing it to segment large cancer cells, small lymphocytes, and normal epithelium at the same time without requiring fine-tuning or manual adjustment. Results: By combining our novel imaging platform with new computational tools, we are able to achieve extremely accurate segmentation of whole cells in tissue. Our approach compares favorably with many of the currently used tools for segmentation. We show that our improvements in accuracy come both from our novel imaging approach as well as algorithmic advances. We perform significantly better than traditional machine learning algorithms trained on the same dataset. Additionally, we show that our algorithm can be trained to identify cells across a range of cancer histologies and disease grades. Conclusions: We have developed a robust and accurate approach to whole-cell segmentation in human tissues. We show the superiority over this method over current state of the art algorithms. The accurate segmentation generated by our approach will enable the analysis of complex tissue architectures with highly overlapping cell types, and will help to advance our understanding of the interactions between cell types in the diseased state.

Files

Additional details