Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Astrostatistics: Opening the Black Box

Astrostatistics: Opening the Black Box

The large datasets being generated by current and future astronomical surveys give us the ability to answer questions at a breadth and depth that was previously unimaginable. Yet datasets which strive to be generally useful are rarely ideal for any particular science case: measurements are often sparser, noisier, or more heterogeneous than one might hope. To adapt tried-and-true statistical methods to this new milieu of large-scale, noisy, heterogeneous data often requires us to re-examine these methods: to pry off the lid of the black box and consider the assumptions they are built on, and how these assumptions can be relaxed for use in this new context. In this talk I’ll explore a case study of such an exercise: our extension of the Lomb-Scargle Periodogram for use with the sparse, multi-color photometry expected from LSST. For studies involving RR-Lyrae-type variable stars, we expect this multiband algorithm to push the effective depth of LSST two magnitudes deeper than for previously used methods.

Jake VanderPlas

November 11, 2015
Tweet

More Decks by Jake VanderPlas

Other Decks in Science

Transcript

  1. Jake VanderPlas
    Astrostatistics:
    Opening the Black Box
    Jake VanderPlas
    11-10-2015

    View full-size slide

  2. Jake VanderPlas
    Annie Jump Cannon processed 300,000
    stellar spectra in her lifetime… by hand!
    Big Data in Astronomy:

    View full-size slide

  3. Jake VanderPlas
    Annie Jump Cannon processed 300,000
    stellar spectra in her lifetime… by hand!
    SDSS gathered ~3
    million spectra in
    10 years
    ~30,000 GB
    catalog over a
    decade
    Big Data in Astronomy:

    View full-size slide

  4. Jake VanderPlas
    Annie Jump Cannon processed 300,000
    stellar spectra in her lifetime… by hand!
    SDSS gathered ~3
    million spectra in
    10 years
    ~30,000 GB
    catalog over a
    decade
    LSST will do an SDSS-scale
    photometric survey every night
    for 10 years!
    Big Data in Astronomy:

    View full-size slide

  5. Jake VanderPlas
    Astronomy’s Data Revolution:
    Orders-of-magnitude growth in
    data requires many new statistical
    and algorithmic approaches.
    We should expect the jump
    from current data to LSST to
    be no different.

    View full-size slide

  6. Jake VanderPlas
    Large Synoptic Survey Telescope (LSST)
    Exemplar of the new data-intensive astronomy
    - photometry of the full southern sky
    every 3-4 nights for 10 years
    - ugrizy multiband data
    - 30,000GB per night
    - Final catalog: 100s of Petabytes
    - ~1000 observations per field

    View full-size slide

  7. Jake VanderPlas
    http://www.lsst.org/scientists/scibook
    LSST Science Book
    ~600 Pages, 245 authors, nearly every
    astronomy sub-domain represented.
    Scope of the dataset will be transformative.
    But challenges abound:
    survey data designed to be
    generally useful is rarely
    optimal for your science.
    Your favorite methods may
    not work anymore. . .
    . . . enter AstroStatistics

    View full-size slide

  8. Jake VanderPlas
    Astrostatistics (n.)
    The application of Statistics to the study and
    analysis of Astronomical Data
    — Wiktionary
    Jake VanderPlas

    View full-size slide

  9. Jake VanderPlas
    Astrostatistics (n.)
    The adaptation of standard methods — and
    development of new ones — for use with
    modern large, noisy, and/or heterogeneous
    datasets.
    — JTV
    Jake VanderPlas

    View full-size slide

  10. Jake VanderPlas
    Jake VanderPlas
    Astrostatistics Case Study:
    Mapping the Milky Way with RR Lyrae

    View full-size slide

  11. Jake VanderPlas
    Background: RR Lyrae-type Stars
    Jake VanderPlas
    Wikipedia
    A particular class of
    variable star:
    Easily detectable via distinct
    lightcurve shape:
    Wikipedia
    Standard Candles:
    Direct tracer of distance!
    M
    V
    = (0.23 ± 0.04) [Fe/H] + (0.93 ± 0.12)
    (Chaboyer et al. 1999)

    View full-size slide

  12. Jake VanderPlas
    Mapping the MW with RR Lyrae
    Sesar et al. 2010
    SDSS II Stripe 82:
    - 483 RR Lyrae to r~22
    - 300 deg2
    - d ~ 100 kpc
    Analysis supports the idea
    of an early-forming
    smooth inner halo,
    and late-forming
    accreted outer halo.

    View full-size slide

  13. Jake VanderPlas
    RR Lyrae in LSST
    ? ? ?
    SDSS II LSST
    300 deg2 ~20,000 deg2
    r ~ 22 mags r ~ 24 mags
    d ~ 100 kpc d ~ 300 kpc
    483 RR Lyrae > 30,000 RR Lyrae??
    (nobody knows!)

    View full-size slide

  14. Jake VanderPlas
    Every MW Satellite with
    time-series available has
    ≥ 1 observed RR Lyr
    Jake VanderPlas
    (Boettcher et al. 2013, Table 4;
    See also Baker & Willman 2015)
    Sesaret al. 2013
    A single halo RR-Lyr can
    indicate structure:
    Baker & Willman 2015
    Two RR-Lyr past ~100kpc almost
    certainly indicate a MW Satellite!
    Science with RR-Lyrae

    View full-size slide

  15. Jake VanderPlas
    In other words: any single
    distant RR Lyrae detected will
    almost certainly yield new
    constraints on MW potential,
    formation history, etc.

    View full-size slide

  16. Jake VanderPlas
    1. Gather time-series observations
    2. Detect periodic objects
    - Lomb-Scargle Periodogram
    - Supersmoother
    - AoV Periodogram
    - CARMA models
    - etc.
    3. Fit Templates at matching periods
    4. Do Science!!!
    Jake VanderPlas
    How to find RR Lyrae

    View full-size slide

  17. Jake VanderPlas
    If only it were that
    straightforward...

    View full-size slide

  18. Jake VanderPlas
    Jake VanderPlas
    Oluseyi 2012 Simulations:
    Detailed simulation of
    RR Lyrae observations
    in 10 years of LSST
    Best available data on RR-Lyr
    templates, populations,
    LSST cadence, etc.
    (Primarily based on
    Stripe82 sample,
    Sesar et al. 2010)

    View full-size slide

  19. Jake VanderPlas
    Jake VanderPlas
    Oluseyi 2012 Simulations:
    Faintest RR-
    Lyrae:
    Pessimistic period
    recovery even
    with 5-10 years of
    LSST data!
    solid lines = RRab; dashed lines = RRc
    Results for Universal Cadence fields

    View full-size slide

  20. Jake VanderPlas
    Jake VanderPlas
    Oluseyi 2012 Simulations:
    solid lines = RRab; dashed lines = RRc
    Universal Cadence Overlap Regions

    View full-size slide

  21. Jake VanderPlas
    Jake VanderPlas
    Oluseyi 2012 Simulations:
    solid lines = RRab; dashed lines = RRc
    Universal Cadence Overlap Regions
    - 50% completeness at g~22 in ~3-4 years
    - 50% completeness at g~24.5 in ~10 years
    Do we really have to wait until 2029 to
    detect faint MW dwarfs with LSST?

    View full-size slide

  22. Jake VanderPlas
    Jake VanderPlas
    LSST is not designed for
    RR Lyr Detection!
    Sparsity: ~one visit every
    ~three nights (cf. 0.4-1.0
    day period of RR Lyrae)
    Heterogeneity: only one
    band (ugrizy) per visit
    Noise: Interesting objects
    near the detection limit
    Data Size: Expensive
    period searches
    untenable (~30sec budget
    per object)

    View full-size slide

  23. Jake VanderPlas
    Quick Test:
    Jake VanderPlas
    - Standard Lomb-Scargle Periodogram
    - 60 visits over 6 months; 5 bands per visit
    (SDSS-like data)

    View full-size slide

  24. Jake VanderPlas
    Jake VanderPlas
    Quick Test:
    - Standard Lomb-Scargle Periodogram
    - 60 visits over 6 months; 5 bands per visit
    (SDSS-like data)

    View full-size slide

  25. Jake VanderPlas
    Jake VanderPlas
    Quick Test:
    - Standard Lomb-Scargle Periodogram
    - 60 visits over 6 months; single band each visit
    (LSST-like data)

    View full-size slide

  26. Jake VanderPlas
    Standard single-band methods
    fail for sparse LSST-type data!
    Jake VanderPlas
    Quick Test:
    - Standard Lomb-Scargle Periodogram
    - 60 visits over 6 months; single band each visit
    (LSST-like data)

    View full-size slide

  27. Jake VanderPlas
    Jake VanderPlas
    Let’s think about a periodogram
    which can utilize multiple bands
    simultaneously . . .

    View full-size slide

  28. Jake VanderPlas
    Jake VanderPlas
    Period Detection in Multi-band
    Photometry . . .
    - Welch & Stetson 1993 – variability index for
    two simultaneous bands
    - Sesar 2010 – Supersmoother on g-band
    primarily; on r & i to evaluate; skip u & z
    - Suveges 2012 – use PCA to combine info
    from simultaneous measurements
    - Oluseyi 2012 – single-band SuperSmoother
    analysis in g-r-i, look for ⅔ agreement

    View full-size slide

  29. Jake VanderPlas
    Jake VanderPlas
    The Lomb-Scargle Periodogram
    If you’ve ever come across the Lomb-Scargle Periodogram,
    you’ve probably seen something like this...
    But this obfuscates the beauty of the algorithm: the
    classical periodogram is essentially the 2 of a single
    sinusoidal model-fit to the data:

    View full-size slide

  30. Jake VanderPlas
    Jake VanderPlas
    Standard Lomb-Scargle
    cf. Lomb (1976), Scargle (1982)
    Figure: VanderPlas & Ivezic 2015
    Periodogram peaks are frequencies where
    a sinusoid fits the data well:

    View full-size slide

  31. Jake VanderPlas
    Jake VanderPlas
    Two Naive Multiband Approaches
    5 bands/night
    1. Ignore band distinction and fit a single periodogram to
    all bands.
    (model is highly biased: under-fits the data)
    2. Fit an independent periodogram within each band;
    combine the 2 of all K bands
    (model is too flexible: over-fits the data)

    View full-size slide

  32. Jake VanderPlas
    Jake VanderPlas
    Two Naive Multiband Approaches
    1 band/night
    1. Ignore band distinction and fit a single periodogram to
    all bands.
    (model is highly biased: under-fits the data)
    2. Fit an independent periodogram within each band;
    combine the 2 of all K bands
    (model is too flexible: over-fits the data)

    View full-size slide

  33. Jake VanderPlas
    Jake VanderPlas
    Two Naive Multiband Approaches
    1. Ignore band distinction and fit a single periodogram to
    all bands.
    (model is highly biased: under-fits the data)
    2. Fit an independent periodogram within each band;
    combine the 2 of all K bands
    (model is too flexible: over-fits the data)
    Can we find a
    middle ground?

    View full-size slide

  34. Jake VanderPlas
    Jake VanderPlas
    Connection between Fourier
    periodogram and least squares
    allows us begin generalizing the
    periodogram . . .

    View full-size slide

  35. Jake VanderPlas
    Jake VanderPlas
    Floating Mean Model
    cf. Ferraz-Mello (1981); Cumming et al (1999);
    Zechmeister & Kurster (2009)
    Figure: VanderPlas & Ivezic 2015
    . . . in which we simultaneously fit the mean

    View full-size slide

  36. Jake VanderPlas
    Jake VanderPlas
    Truncated Fourier Model
    cf. Bretthorst (1988)
    Figure: VanderPlas & Ivezic 2015
    . . . in which we fit for higher-order periodicity

    View full-size slide

  37. Jake VanderPlas
    Jake VanderPlas
    Truncated Fourier Model
    cf. Bretthorst (1988)
    Figure: VanderPlas & Ivezic 2015
    . . . in which we fit for higher-order periodicity

    View full-size slide

  38. Jake VanderPlas
    Jake VanderPlas
    Regularized Model
    . . . in which we penalize regression coefficients
    to simplify an overly-complex model.
    The “trick” is adding a strong prior which pushes coefficients
    to zero: higher terms are only used if actually needed!

    View full-size slide

  39. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram

    View full-size slide

  40. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram
    - define a truncated
    Fourier base
    component which
    contributes equally
    to all bands.

    View full-size slide

  41. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram
    - for each band, add a
    truncated Fourier
    band component to
    describe deviation
    from base model

    View full-size slide

  42. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram
    + =
    Regularize the band component to drive
    common variation to the base model.

    View full-size slide

  43. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram
    Regularize the band component to drive
    common variation to the base model.
    + =
    Key Idea: Regularization reduces
    added model complexity &
    pushes common variation into
    the base model.

    View full-size slide

  44. Jake VanderPlas
    Jake VanderPlas
    Putting it all together:
    The Multiband Periodogram
    Regularize the band component to drive
    common variation to the base model.
    + =
    Key Idea: Regularization reduces
    added model complexity &
    pushes common variation into
    the base model.
    Key Strength: This is a
    straightforward linear model that
    can be solved quickly in closed-
    form (LSST-scale!)

    View full-size slide

  45. Jake VanderPlas
    Jake VanderPlas
    Multiband Periodogram
    on sparse, LSST-style data . . .

    View full-size slide

  46. Jake VanderPlas
    Jake VanderPlas
    Multiband Periodogram
    on sparse, LSST-style data . . .
    Detects period with high significance
    when single-band approaches fail!

    View full-size slide

  47. Jake VanderPlas
    Jake VanderPlas
    Comparing Approaches:
    Stripe-82 Data (5 bands per visit)
    Template-fit Period
    Single-Band Period Multi-Band Period

    View full-size slide

  48. Jake VanderPlas
    Jake VanderPlas
    Comparing Approaches:
    Stripe-82 Data (single band per visit)
    Template-fit Period
    Single-Band Period Multi-Band Period

    View full-size slide

  49. Jake VanderPlas
    Jake VanderPlas
    Prospects for LSST
    Based on simulated LSST cadence & photometric errors;
    see VanderPlas & Ivezic (2015)
    Fraction Recovered
    6 months
    1 year
    2 years
    5 years
    multiband model
    Oluseyi (2012) approach
    g-band mag

    View full-size slide

  50. Jake VanderPlas
    Jake VanderPlas
    Prospects for LSST
    Based on simulated LSST cadence & photometric errors;
    see VanderPlas & Ivezic (2015)
    Fraction Recovered
    6 months
    1 year
    2 years
    5 years
    multiband model
    Oluseyi (2012) approach
    g-band mag
    e.g. after 2 years: ~0% →~75%
    completeness at g ~ 24.5!

    View full-size slide

  51. Jake VanderPlas
    Jake VanderPlas
    Prospects for LSST
    Based on simulated LSST cadence & photometric errors;
    see VanderPlas & Ivezic (2015)
    Fraction Recovered
    6 months
    1 year
    2 years
    5 years
    multiband model
    Oluseyi (2012) approach
    g-band mag
    ~2 mag improvement
    in effective depth of
    LSST!

    View full-size slide

  52. Jake VanderPlas
    Code to reproduce the study & figures
    (including all figures in these slides):
    http://github.com/jakevdp/multiband_LS/
    Python multiband implementation:
    http://github.com/jakevdp/gatspy/
    “If it’s not reproducible, it’s not science.”

    View full-size slide

  53. Jake VanderPlas
    Back to our
    Motivation:
    ? ? ?
    ~100kpc

    View full-size slide

  54. Jake VanderPlas
    Jake VanderPlas
    Other Recent Progress:
    - Long, Chi, & Baraniuk (2015)
    Multiband extension of Lomb-Scargle — uses a
    nonlinear regularization on amplitude & phase
    offset. Better physical motivation, but more
    computationally intensive.
    - Mondrik, Long, & Marshall (2015)
    Multiband extension of Analysis of Variance
    periodogram — also explores dependence of
    multiband detections on survey cadence.

    View full-size slide

  55. Jake VanderPlas
    Jake VanderPlas
    Interesting Pre-LSST Datasets
    - Pan-STARRS
    Natural testing ground for multiband methods,
    though data is very sparse; currently some RR Lyrae
    studies underway (B. Sesar; in prep).
    - SDSS Stripe 82 Reprised
    LSST’s analysis pipeline is capable of going much
    deeper via “forced photometry”. Stripe 82 reanalysis is
    leading to interesting progress in QSO science (Y.
    AlSayyad; in prep) Could we find deeper RR Lyrae in
    re-processed SDSS data?

    View full-size slide

  56. Jake VanderPlas
    Jake VanderPlas
    - Realize that future surveys will likely not be
    optimized for your particular science interests
    - Identify where standard algorithms &
    statistical methods will fail ..
    Astrostatistics: Opening the Black Box
    - Understand the methods you
    want to apply & the
    assumptions behind them.
    - Adapt the methods for use
    with sparse, heterogeneous,
    noisy, large datasets.

    View full-size slide

  57. Jake VanderPlas
    Jake VanderPlas
    Email: [email protected]
    Twitter: @jakevdp
    Github: jakevdp
    Web: http://vanderplas.com/
    Blog: http://jakevdp.github.io/
    Thank You!

    View full-size slide

  58. Jake VanderPlas
    Jake VanderPlas

    View full-size slide

  59. Jake VanderPlas
    Jake VanderPlas
    Astrostatistics: Opening the Black Box
    abstract: The large datasets being generated by current and future
    astronomical surveys give us the ability to answer questions at a
    breadth and depth that was previously unimaginable. Yet datasets
    which strive to be generally useful are rarely ideal for any particular
    science case: measurements are often sparser, noisier, or more
    heterogeneous than one might hope. To adapt tried-and-true
    statistical methods to this new milieu of large-scale, noisy,
    heterogeneous data often requires us to re-examine these methods:
    to pry off the lid of the black box and consider the assumptions they
    are built on, and how these assumptions can be relaxed for use in
    this new context. In this talk I’ll explore a case study of such an
    exercise: our extension of the Lomb-Scargle Periodogram for use
    with the sparse, multi-color photometry expected from LSST. For
    studies involving RR-Lyrae-type variable stars, we expect this
    multiband algorithm to push the effective depth of LSST two
    magnitudes deeper than for previously used methods.

    View full-size slide