Published January 2021
| Supplemental Material + Submitted
Journal Article
Open
DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes
Chicago
An error occurred while generating the citation.
Abstract
Deep learning is transforming the analysis of biological images, but applying these models to large datasets remains challenging. Here we describe the DeepCell Kiosk, cloud-native software that dynamically scales deep learning workflows to accommodate large imaging datasets. To demonstrate the scalability and affordability of this software, we identified cell nuclei in 10⁶ 1-megapixel images in ~5.5 h for ~US$250, with a cost below US$100 achievable depending on cluster configuration. The DeepCell Kiosk can be downloaded at https://github.com/vanvalenlab/kiosk-console; a persistent deployment is available at https://deepcell.org/.
Additional Information
© The Author(s), under exclusive licence to Springer Nature America, Inc. 2021. Received 15 October 2019; Accepted 23 November 2020; Published 04 January 2021. We thank numerous colleagues including A. Anandkumar, M. Angelo, J. Bois, I. Brown, A. Butkovic, L. Cai, I. Camplisson, M. Covert, M. Elowitz, J. Freeman, C. Frick, L. Geontoro, A. Ho, K. Huang, K. C. Huang, G. Johnson, L. Keren, D. Litovitz, D. Macklin, U. Manor, S. Patel, A. Raj, N. Pelaez Restrepo, C. Pavelchek, S. Shah and M. Thomson for helpful discussions and contributing data. We gratefully acknowledge support from the Shurl and Kay Curci Foundation, the Rita Allen Foundation, the Paul Allen Family Foundation through the Allen Discovery Center at Stanford University, the Rosen Center for Bioengineering at Caltech, Google Research Cloud, Figure 8's AI For Everyone award, and a subaward from NIH U24-CA224309-01. Data availability: All data that were used to generate the figures in this paper are available at https://deepcell.org/data and at https://github.com/vanvalenlab/deepcell-tf under the deepcell.datasets module. Code availability: We used Kubernetes and TensorFlow, along with the scientific computing stack for Python. A persistent deployment of the software described can be accessed at https://deepcell.org/. All source code, including version requirements and explicit usage, is under a modified Apache license and is available at https://github.com/vanvalenlab. Detailed instructions are available at https://deepcell-kiosk.readthedocs.io. Author Contributions: D.B., W.G. and D.V.V. conceived the project; D.B., W.G., E.O. and D.V.V. designed the software architecture; D.B., E.O. and W.G. wrote the core components of the software; D.B., E.M., M.S., E.B., V.V., B.C., E.O., W.G. and D.V.V. contributed to the code base; T.K. and E.P. collected data for annotation; E.M., M.S., N.G., D.B., W.G. and D.V.V. wrote documentation; D.B., E.M., W.G. and D.V.V. wrote the paper; D.V.V. supervised the project. Competing interests: The authors have filed a provisional patent for the described work; the software described here is available under a modified Apache license and is free for non-commercial uses. Peer review information: Nature Methods thanks Ola Spjuth and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.Errata
Bannon, D., Moen, E., Schwartz, M. et al. Publisher Correction: DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods (2021). https://doi.org/10.1038/s41592-021-01059-wAttached Files
Submitted - 505032v4.full.pdf
Supplemental Material - 41592_2020_1023_MOESM1_ESM.pdf
Supplemental Material - 41592_2020_1023_MOESM2_ESM.pdf
Files
505032v4.full.pdf
Additional details
- Alternative title
- Dynamic allocation of computational resources for deep learning-enabled cellular image analysis with Kubernetes
- Eprint ID
- 98655
- Resolver ID
- CaltechAUTHORS:20190916-101510993
- Shurl and Kay Curci Foundation
- Rita Allen Foundation
- Paul Allen Family Foundation
- Donna and Benjamin M. Rosen Bioengineering Center
- Google Research Cloud
- Figure 8's AI for Everyone
- NIH
- U24-CA224309-01
- Created
-
2019-09-16Created from EPrint's datestamp field
- Updated
-
2023-06-01Created from EPrint's last_modified field
- Caltech groups
- Rosen Bioengineering Center, Division of Biology and Biological Engineering (BBE)