Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published July 2022 | Published + Submitted
Book Section - Chapter Open

Screening p-Hackers: Dissemination Noise as Bait

Abstract

We show that adding noise to data before making data public is effective at screening p-hacked findings: spurious explanations of the outcome variable produced by attempting multiple econometric specifications. Noise creates "baits'' that affect two types of researchers differently. Uninformed p-hackers who engage in data mining with no prior information about the true causal mechanism often fall for baits and report verifiably wrong results when evaluated with the original data. But informed researchers who start with an ex-ante hypothesis about the causal mechanism before seeing any data are minimally affected by noise. We characterize the optimal level of dissemination noise and highlight the relevant trade-offs in a simple theoretical model. Dissemination noise is a tool that statistical agencies (e.g., the US Census Bureau) currently use to protect privacy, and we show this existing practice can be repurposed to improve research credibility.

Additional Information

© 2022 Copyright held by the owner/author(s). This research was made possible through the support of the Linde Institute. Echenique also thanks the NSF's support through the grants SES-1558757 and CNS-1518941. We are grateful for comments from Sylvain Chassang, Eva Jin, Albert Ma, Pascal Michaillat, Marco Ottaviani, Nathan Yoder, and the audiences at the University of Pennsylvania, Caltech, Universidad de la República, Boston University, and ASSA 2022.

Attached Files

Published - 3490486.3538358.pdf

Submitted - 2103.09164.pdf

Files

3490486.3538358.pdf
Files (1.6 MB)
Name Size Download all
md5:5b7dfb124a98bcfbbb6350cdabf5c724
755.6 kB Preview Download
md5:422050d4150e41eb046beed8a450cc4a
797.5 kB Preview Download

Additional details

Created:
August 20, 2023
Modified:
December 21, 2023