Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published June 2013 | Submitted
Book Section - Chapter Open

Boundary Detection Benchmarking: Beyond F-Measures

Abstract

For an ill-posed problem like boundary detection, human labeled datasets play a critical role. Compared with the active research on finding a better boundary detector to refresh the performance record, there is surprisingly little discussion on the boundary detection benchmark itself. The goal of this paper is to identify the potential pitfalls of today's most popular boundary benchmark, BSDS 300. In the paper, we first introduce a psychophysical experiment to show that many of the "weak" boundary labels are unreliable and may contaminate the benchmark. Then we analyze the computation of f-measure and point out that the current benchmarking protocol encourages an algorithm to bias towards those problematic "weak" boundary labels. With this evidence, we focus on a new problem of detecting strong boundaries as one alternative. Finally, we assess the performances of 9 major algorithms on different ways of utilizing the dataset, suggesting new directions for improvements.

Additional Information

© 2013 IEEE. Date of Conference: 23-28 June 2013. The first author would like to thank Zhuowen Tu, Yin Li and Liwei Wang for their thoughtful discussions. The research was supported by the ONR via an award made through Johns Hopkins University, by the G. Harold & Leila Y. Mathers Charitable Foundation, by Army Research Lab with 62250-CS and the Office of Naval Research N00014-12-10883. INSPEC Accession Number: 13824352.

Attached Files

Submitted - cvpr13.pdf

Files

cvpr13.pdf
Files (8.2 MB)
Name Size Download all
md5:bfc0bff5dba2e6ecd6e958fa399f3173
8.2 MB Preview Download

Additional details

Created:
September 22, 2023
Modified:
October 23, 2023