Adaptive data analysis: theory and applications

Abstract

In many applications in science, engineering and mathematics, it is useful to understand functions depending on time and/or space from many different points of view. Accordingly, a wide range of transformations and analysis tools have been developed over time. Fourier series and the Fourier transform, first proposed almost 200 years ago, provided one of the first mechanisms to write a complex waveform as the linear combination of elementary wave functions; many more would follow. Nearly 100 years ago, it became clear that for some applications it is especially useful that the elementary 'building block functions', into which more complex signals are decomposed, have a limited spread in both time and frequency—transformations or representations that used such simultaneous time–frequency (or space/spatial frequency) localization have been important tools in micro-local arguments in mathematics, quantum mechanics and semi-classical approximations, and many types of signal and data analysis. Typically, the tools used to compute such transforms or representations are linear in the input—making them (fairly) easy to implement, and versatile instruments in the data analyst's toolbox. Yet, in some cases, the very versatility of these linear tools makes them come up short, and to obtain a more detailed, precise analysis, it becomes necessary to adapt parameters and procedures to (often local) behaviour changes of the data or signal. Examples of this abound. With the advances of sensor technology, we are dealing with vast increases in not only the volume of data to be analysed, but also in their quality—leading to the ubiquitous discussions of what to do with all these 'big data'. Big data provide not only a challenge, but also an opportunity, especially because computation and storage have likewise become much more powerful. In response to these needs and opportunities, adaptive data analysis methods are being developed and explored for many different scientific and engineering frameworks.

Additional details