Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published February 5, 2021 | Submitted
Report Open

Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC)

Abstract

Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of robust classifiers under LSST-like conditions of a non-representative training set for a large photometric test set of imbalanced classes. Over 1,000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between Sep 28, 2018 and Dec 17, 2018, ultimately identifying three winners in February 2019. Participants produced classifiers employing a diverse set of machine learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multi-layer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state-of-the-art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next generation PLAsTiCC data set.

Additional Information

Attribution 4.0 International (CC BY 4.0) This paper has undergone internal review in the LSST Dark Energy Science Collaboration. Author contributions are listed below. R. Hložek: data curation, formal analysis, funding acquisition, investigation, project administration, software, supervision, validation, visualization, writing - editing, lead writing - original draft K. Ponder: software development and implementation for validation, testing, visualization and analysis of results, writing - editing, writing - original draft A.I. Malz: conceptualization, formal analysis, investigation, methodology, visualization, writing - editing, writing - original draft M. Dai: software development and implementation for validation, testing, analysis of results, writing - editing G. Narayan: simulation, software, example kit, validation, visualization, writing - editing, writing - original draft E.E.O. Ishida: software development and implementation for validation, testing, Kaggle liason, analysis of results, writing - editing T. Allam Jr: investigation, development of software for simulation validation A. Bahmanyar: investigation, implementation of metric methodology and software R. Biswas: conceptualization, methodology, software for validation and testing, writing - editing L. Galbany: implementation of validation software, writing - editing S.W. Jha: conceptualization, simulation validation D. Jones: conceptualization, simulation validation, writing - editing R. Kessler: simulation development and validation testing, data generation, writing - editing M. Lochner: conceptualization, simulation validation, writing - editing A.A. Mahabal: conceptualization, simulation validation K.S. Mandel: conceptualization, simulation validation, writing - editing J.R. Martínez-Galarza: conceptualization, simulation validation, writing - editing J.D. McEwen: conceptualization, simulation validation D. Muthukrishna: conceptualization, simulation validation, results analysis and testing, writing - editing H.V. Peiris: conceptualization, supervision, funding acquisition, writing - editing C.M. Peters: conceptualization, simulation validation C.N. Setzer: conceptualization, simulation validation The Photometric LSST Astronomical Time Series Classification Challenge data set generation relied on numerous members of the astronomical community to provide models of astronomical transients and variables. Photometric LSST Astronomical Time Series Classification Challenge is made possible through an intercollaboration agreement between LSST's Dark Energy Science Collaboration and the Transient and Variable Stars Science Collaboration. These models are described in Kessler et al. (2019a). We are grateful to Federica Bianco and staff at NYU for hosting the PLAsTiCC team meeting. The awards for PLAsTiCC contest were generously supported by Kaggle. We thank staff� at Kaggle, particularly Elizabeth Park, Maggie Demkin and Sohier Dane, for their help making the contest a success. We thank Federica Bianco, Francisco F�örster Burón, Chad Schafer and Melissa Graham for providing internal review of this document for the DESC and TVS collaborations, and thank Seth Digel for comments on this draft. This work was supported by an LSST Corporation Enabling Science grant. The financial assistance of the National Research Foundation (NRF) towards this research is hereby acknowledged. Opinions expressed and conclusions arrived at, are those of the authors and are not necessarily to be attributed to the NRF. UK authors acknowledge funding from the Science and Technology Funding Council. Canadian co-authors acknowledge support from the Natural Sciences and Engineering Research Council of Canada. The Dunlap Institute is funded through an endowment established by the David Dunlap family and the University of Toronto. The authors at the University of Toronto acknowledge that the land on which the University of Toronto is built is the traditional territory of the Haudenosaunee, and most recently, the territory of the Mississaugas of the New Credit First Nation. They are grateful to have the opportunity to work in the community, on this territory. R. H. acknowledges support from CIFAR, and the Azrieli and Alfred. P. Sloan foundations. K. A .P. acknowledges support from the Berkeley Center for Cosmological Physics; the Director, Office of Science, Office of High Energy Physics of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231; and U.S. Department of Energy Office of Science under Contract No.DE-AC02-76SF00515. E. E. O. I. acknowledges support from CNRS 2017 MOMENTUM grant. A. I. M acknowledges support from the Max Planck Society and the Alexander von Humboldt Foundation in the framework of the Max Planck-Humboldt Research Award endowed by the Federal Ministry of Education and Research. L. G. was funded by the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 839090. D. O. J acknowledges support provided by NASA Hubble Fellowship grant HST-HF2-51462.001 awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., for NASA, under contract NAS5-26555. D.O.J. also acknowledges support from the Gordon & Betty Moore Foundation. This work has been partially supported by the Spanish grant PGC2018-095317-B-C21 within the European Funds for Regional Development (FEDER). C. N. S., R. B. and H. V. P. were partially supported by the research environment grant "Gravitational Radiation and Electromagnetic Astrophysical Transients (GREAT)" funded by the Swedish Research Council (VR) under Dnr 2016-06012. RB and HVP were additionally supported by the research project grant "Understanding the Dynamic Universe" funded by the Knut and Alice Wallenberg Foundation under Dnr KAW 2018.0067. M. D. acknowledges support from the Horizon Fellowship at the Johns Hopkins University. At Rutgers University (M. D., S. W. J.), this research was supported by NSF award AST-1615455 and DOE award DE-SC0011636. The DESC acknowledges ongoing support from the Institut National de Physique Nucléaire et de Physique des Particules in France; the Science & Technology Facilities Council in the United Kingdom; and the Department of Energy, the National Science Foundation, and the LSST Corporation in the United States. DESC uses resources of the IN2P3 Computing Center (CC-IN2P3-Lyon/Villeurbanne - France) funded by the Centre National de la Recherche Scienti�que; the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231; STFC DiRAC HPC Facilities, funded by UK BIS National E-infrastructure capital grants; and the UK particle physics grid, supported by the GridPP Collaboration. This work was performed in part under DOE Contract DE-AC02-76SF00515.

Attached Files

Submitted - 2012.12392.pdf

Files

2012.12392.pdf
Files (4.0 MB)
Name Size Download all
md5:31ee24aa70e79f46589238b934257c00
4.0 MB Preview Download

Additional details

Created:
August 20, 2023
Modified:
October 23, 2023