Astrostatistics: Opening the Black Box

The large datasets being generated by current and future astronomical surveys give us the ability to answer questions at a breadth and depth that was previously unimaginable. Yet datasets which strive to be generally useful are rarely ideal for any particular science case: measurements are often sparser, noisier, or more heterogeneous than one might hope. To adapt tried-and-true statistical methods to this new milieu of large-scale, noisy, heterogeneous data often requires us to re-examine these methods: to pry off the lid of the black box and consider the assumptions they are built on, and how these assumptions can be relaxed for use in this new context. In this talk I’ll explore a case study of such an exercise: our extension of the Lomb-Scargle Periodogram for use with the sparse, multi-color photometry expected from LSST. For studies involving RR-Lyrae-type variable stars, we expect this multiband algorithm to push the effective depth of LSST two magnitudes deeper than for previously used methods.

Transcript

1. Jake VanderPlas Astrostatistics: Opening the Black Box Jake VanderPlas 11-10-2015
2. Jake VanderPlas Annie Jump Cannon processed 300,000 stellar spectra in her lifetime… by hand! Big Data in Astronomy:
3. Jake VanderPlas Annie Jump Cannon processed 300,000 stellar spectra in her lifetime… by hand! SDSS gathered ~3 million spectra in 10 years ~30,000 GB catalog over a decade Big Data in Astronomy:
4. Jake VanderPlas Annie Jump Cannon processed 300,000 stellar spectra in her lifetime… by hand! SDSS gathered ~3 million spectra in 10 years ~30,000 GB catalog over a decade LSST will do an SDSS-scale photometric survey every night for 10 years! Big Data in Astronomy:
5. Jake VanderPlas Astronomy’s Data Revolution: Orders-of-magnitude growth in data requires many new statistical and algorithmic approaches. We should expect the jump from current data to LSST to be no different.
6. Jake VanderPlas Large Synoptic Survey Telescope (LSST) Exemplar of the new data-intensive astronomy - photometry of the full southern sky every 3-4 nights for 10 years - ugrizy multiband data - 30,000GB per night - Final catalog: 100s of Petabytes - ~1000 observations per field
7. Jake VanderPlas http://www.lsst.org/scientists/scibook LSST Science Book ~600 Pages, 245 authors, nearly every astronomy sub-domain represented. Scope of the dataset will be transformative. But challenges abound: survey data designed to be generally useful is rarely optimal for your science. Your favorite methods may not work anymore. . . . . . enter AstroStatistics
8. Jake VanderPlas Astrostatistics (n.) The application of Statistics to the study and analysis of Astronomical Data — Wiktionary Jake VanderPlas
9. Jake VanderPlas Astrostatistics (n.) The adaptation of standard methods — and development of new ones — for use with modern large, noisy, and/or heterogeneous datasets. — JTV Jake VanderPlas
10. Jake VanderPlas Jake VanderPlas Astrostatistics Case Study: Mapping the Milky Way with RR Lyrae
11. Jake VanderPlas Background: RR Lyrae-type Stars Jake VanderPlas Wikipedia A particular class of variable star: Easily detectable via distinct lightcurve shape: Wikipedia Standard Candles: Direct tracer of distance! M V = (0.23 ± 0.04) [Fe/H] + (0.93 ± 0.12) (Chaboyer et al. 1999)
12. Jake VanderPlas Mapping the MW with RR Lyrae Sesar et al. 2010 SDSS II Stripe 82: - 483 RR Lyrae to r~22 - 300 deg2 - d ~ 100 kpc Analysis supports the idea of an early-forming smooth inner halo, and late-forming accreted outer halo.
13. Jake VanderPlas RR Lyrae in LSST ? ? ? SDSS II LSST 300 deg2 ~20,000 deg2 r ~ 22 mags r ~ 24 mags d ~ 100 kpc d ~ 300 kpc 483 RR Lyrae > 30,000 RR Lyrae?? (nobody knows!)
14. Jake VanderPlas Every MW Satellite with time-series available has ≥ 1 observed RR Lyr Jake VanderPlas (Boettcher et al. 2013, Table 4; See also Baker & Willman 2015) Sesaret al. 2013 A single halo RR-Lyr can indicate structure: Baker & Willman 2015 Two RR-Lyr past ~100kpc almost certainly indicate a MW Satellite! Science with RR-Lyrae
15. Jake VanderPlas In other words: any single distant RR Lyrae detected will almost certainly yield new constraints on MW potential, formation history, etc.
16. Jake VanderPlas 1. Gather time-series observations 2. Detect periodic objects - Lomb-Scargle Periodogram - Supersmoother - AoV Periodogram - CARMA models - etc. 3. Fit Templates at matching periods 4. Do Science!!! Jake VanderPlas How to find RR Lyrae
17. Jake VanderPlas If only it were that straightforward...
18. Jake VanderPlas Jake VanderPlas Oluseyi 2012 Simulations: Detailed simulation of RR Lyrae observations in 10 years of LSST Best available data on RR-Lyr templates, populations, LSST cadence, etc. (Primarily based on Stripe82 sample, Sesar et al. 2010)
19. Jake VanderPlas Jake VanderPlas Oluseyi 2012 Simulations: Faintest RR- Lyrae: Pessimistic period recovery even with 5-10 years of LSST data! solid lines = RRab; dashed lines = RRc Results for Universal Cadence fields
20. Jake VanderPlas Jake VanderPlas Oluseyi 2012 Simulations: solid lines = RRab; dashed lines = RRc Universal Cadence Overlap Regions
21. Jake VanderPlas Jake VanderPlas Oluseyi 2012 Simulations: solid lines = RRab; dashed lines = RRc Universal Cadence Overlap Regions - 50% completeness at g~22 in ~3-4 years - 50% completeness at g~24.5 in ~10 years Do we really have to wait until 2029 to detect faint MW dwarfs with LSST?
22. Jake VanderPlas Jake VanderPlas LSST is not designed for RR Lyr Detection! Sparsity: ~one visit every ~three nights (cf. 0.4-1.0 day period of RR Lyrae) Heterogeneity: only one band (ugrizy) per visit Noise: Interesting objects near the detection limit Data Size: Expensive period searches untenable (~30sec budget per object)
23. Jake VanderPlas Quick Test: Jake VanderPlas - Standard Lomb-Scargle Periodogram - 60 visits over 6 months; 5 bands per visit (SDSS-like data)
24. Jake VanderPlas Jake VanderPlas Quick Test: - Standard Lomb-Scargle Periodogram - 60 visits over 6 months; 5 bands per visit (SDSS-like data)
25. Jake VanderPlas Jake VanderPlas Quick Test: - Standard Lomb-Scargle Periodogram - 60 visits over 6 months; single band each visit (LSST-like data)
26. Jake VanderPlas Standard single-band methods fail for sparse LSST-type data! Jake VanderPlas Quick Test: - Standard Lomb-Scargle Periodogram - 60 visits over 6 months; single band each visit (LSST-like data)
27. Jake VanderPlas Jake VanderPlas Let’s think about a periodogram which can utilize multiple bands simultaneously . . .
28. Jake VanderPlas Jake VanderPlas Period Detection in Multi-band Photometry . . . - Welch & Stetson 1993 – variability index for two simultaneous bands - Sesar 2010 – Supersmoother on g-band primarily; on r & i to evaluate; skip u & z - Suveges 2012 – use PCA to combine info from simultaneous measurements - Oluseyi 2012 – single-band SuperSmoother analysis in g-r-i, look for ⅔ agreement
29. Jake VanderPlas Jake VanderPlas The Lomb-Scargle Periodogram If you’ve ever come across the Lomb-Scargle Periodogram, you’ve probably seen something like this... But this obfuscates the beauty of the algorithm: the classical periodogram is essentially the 2 of a single sinusoidal model-fit to the data:
30. Jake VanderPlas Jake VanderPlas Standard Lomb-Scargle cf. Lomb (1976), Scargle (1982) Figure: VanderPlas & Ivezic 2015 Periodogram peaks are frequencies where a sinusoid fits the data well:
31. Jake VanderPlas Jake VanderPlas Two Naive Multiband Approaches 5 bands/night 1. Ignore band distinction and fit a single periodogram to all bands. (model is highly biased: under-fits the data) 2. Fit an independent periodogram within each band; combine the 2 of all K bands (model is too flexible: over-fits the data)
32. Jake VanderPlas Jake VanderPlas Two Naive Multiband Approaches 1 band/night 1. Ignore band distinction and fit a single periodogram to all bands. (model is highly biased: under-fits the data) 2. Fit an independent periodogram within each band; combine the 2 of all K bands (model is too flexible: over-fits the data)
33. Jake VanderPlas Jake VanderPlas Two Naive Multiband Approaches 1. Ignore band distinction and fit a single periodogram to all bands. (model is highly biased: under-fits the data) 2. Fit an independent periodogram within each band; combine the 2 of all K bands (model is too flexible: over-fits the data) Can we find a middle ground?
34. Jake VanderPlas Jake VanderPlas Connection between Fourier periodogram and least squares allows us begin generalizing the periodogram . . .
35. Jake VanderPlas Jake VanderPlas Floating Mean Model cf. Ferraz-Mello (1981); Cumming et al (1999); Zechmeister & Kurster (2009) Figure: VanderPlas & Ivezic 2015 . . . in which we simultaneously fit the mean
36. Jake VanderPlas Jake VanderPlas Truncated Fourier Model cf. Bretthorst (1988) Figure: VanderPlas & Ivezic 2015 . . . in which we fit for higher-order periodicity
37. Jake VanderPlas Jake VanderPlas Truncated Fourier Model cf. Bretthorst (1988) Figure: VanderPlas & Ivezic 2015 . . . in which we fit for higher-order periodicity
38. Jake VanderPlas Jake VanderPlas Regularized Model . . . in which we penalize regression coefficients to simplify an overly-complex model. The “trick” is adding a strong prior which pushes coefficients to zero: higher terms are only used if actually needed!
39. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram
40. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram - define a truncated Fourier base component which contributes equally to all bands.
41. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram - for each band, add a truncated Fourier band component to describe deviation from base model
42. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram + = Regularize the band component to drive common variation to the base model.
43. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram Regularize the band component to drive common variation to the base model. + = Key Idea: Regularization reduces added model complexity & pushes common variation into the base model.
44. Jake VanderPlas Jake VanderPlas Putting it all together: The Multiband Periodogram Regularize the band component to drive common variation to the base model. + = Key Idea: Regularization reduces added model complexity & pushes common variation into the base model. Key Strength: This is a straightforward linear model that can be solved quickly in closed- form (LSST-scale!)
45. Jake VanderPlas Jake VanderPlas Multiband Periodogram on sparse, LSST-style data . . .
46. Jake VanderPlas Jake VanderPlas Multiband Periodogram on sparse, LSST-style data . . . Detects period with high significance when single-band approaches fail!
47. Jake VanderPlas Jake VanderPlas Comparing Approaches: Stripe-82 Data (5 bands per visit) Template-fit Period Single-Band Period Multi-Band Period
48. Jake VanderPlas Jake VanderPlas Comparing Approaches: Stripe-82 Data (single band per visit) Template-fit Period Single-Band Period Multi-Band Period
49. Jake VanderPlas Jake VanderPlas Prospects for LSST Based on simulated LSST cadence & photometric errors; see VanderPlas & Ivezic (2015) Fraction Recovered 6 months 1 year 2 years 5 years multiband model Oluseyi (2012) approach g-band mag
50. Jake VanderPlas Jake VanderPlas Prospects for LSST Based on simulated LSST cadence & photometric errors; see VanderPlas & Ivezic (2015) Fraction Recovered 6 months 1 year 2 years 5 years multiband model Oluseyi (2012) approach g-band mag e.g. after 2 years: ~0% →~75% completeness at g ~ 24.5!
51. Jake VanderPlas Jake VanderPlas Prospects for LSST Based on simulated LSST cadence & photometric errors; see VanderPlas & Ivezic (2015) Fraction Recovered 6 months 1 year 2 years 5 years multiband model Oluseyi (2012) approach g-band mag ~2 mag improvement in effective depth of LSST!
52. Jake VanderPlas Code to reproduce the study & figures (including all figures in these slides): http://github.com/jakevdp/multiband_LS/ Python multiband implementation: http://github.com/jakevdp/gatspy/ “If it’s not reproducible, it’s not science.”
53. Jake VanderPlas Back to our Motivation: ? ? ? ~100kpc
54. Jake VanderPlas Jake VanderPlas Other Recent Progress: - Long, Chi, & Baraniuk (2015) Multiband extension of Lomb-Scargle — uses a nonlinear regularization on amplitude & phase offset. Better physical motivation, but more computationally intensive. - Mondrik, Long, & Marshall (2015) Multiband extension of Analysis of Variance periodogram — also explores dependence of multiband detections on survey cadence.
55. Jake VanderPlas Jake VanderPlas Interesting Pre-LSST Datasets - Pan-STARRS Natural testing ground for multiband methods, though data is very sparse; currently some RR Lyrae studies underway (B. Sesar; in prep). - SDSS Stripe 82 Reprised LSST’s analysis pipeline is capable of going much deeper via “forced photometry”. Stripe 82 reanalysis is leading to interesting progress in QSO science (Y. AlSayyad; in prep) Could we find deeper RR Lyrae in re-processed SDSS data?
56. Jake VanderPlas Jake VanderPlas - Realize that future surveys will likely not be optimized for your particular science interests - Identify where standard algorithms & statistical methods will fail .. Astrostatistics: Opening the Black Box - Understand the methods you want to apply & the assumptions behind them. - Adapt the methods for use with sparse, heterogeneous, noisy, large datasets.
57. Jake VanderPlas Jake VanderPlas Email: jakevdp@uw.edu Twitter: @jakevdp Github: jakevdp Web: http://vanderplas.com/ Blog: http://jakevdp.github.io/ Thank You!
58. Jake VanderPlas Jake VanderPlas
59. Jake VanderPlas Jake VanderPlas Astrostatistics: Opening the Black Box abstract: The large datasets being generated by current and future astronomical surveys give us the ability to answer questions at a breadth and depth that was previously unimaginable. Yet datasets which strive to be generally useful are rarely ideal for any particular science case: measurements are often sparser, noisier, or more heterogeneous than one might hope. To adapt tried-and-true statistical methods to this new milieu of large-scale, noisy, heterogeneous data often requires us to re-examine these methods: to pry off the lid of the black box and consider the assumptions they are built on, and how these assumptions can be relaxed for use in this new context. In this talk I’ll explore a case study of such an exercise: our extension of the Lomb-Scargle Periodogram for use with the sparse, multi-color photometry expected from LSST. For studies involving RR-Lyrae-type variable stars, we expect this multiband algorithm to push the effective depth of LSST two magnitudes deeper than for previously used methods.