Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pangeo at Ocean Science Meeting 2022

Pangeo at Ocean Science Meeting 2022

Slides for the Presentation by Deepak Cherian and Julius Busecke at the Ocean Sciences Meeting 2022

Video Link: https://youtu.be/r82-vVCOuwQ

Julius Busecke

March 23, 2022
Tweet

More Decks by Julius Busecke

Other Decks in Research

Transcript

  1. Cloud-native 
 analysis Machine learning Crunching \Bigg datasets Analysis-Ready 


    Cloud Optimized Data pipelines Teaching earth 
 system data 
 science Cloud-analysis 
 platform Open source 
 scienti fi c 
 software Discourse 
 forum Data catalogs Dask Xarray Zarr
  2. 2010-2016 medium data Laptop/HPC 
 ~ 100 sims = 12

    TB Reinventing many wheels 2017-2019 smol data laptop 
 ~GB 5 moorings! 
 2019- smol & big data MBs & ~TBs 
 laptop/server/HPC/cloud obs / models 

  3. Before: for …: … After: 
 # groupby? (binning) 


    # rolling? (moving window) 
 # … 🤔 2017-2019 smol data laptop 
 ~GB 5 moorings! 

  4. Cloud-native 
 analysis Machine learning Crunching \Bigg datasets Analysis-Ready 


    Cloud Optimized Data pipelines Teaching earth 
 system data 
 science Cloud analysis 
 platform Open source 
 scienti fi c 
 software Discourse 
 forum = peer - to - peer 
 learning Data catalogs Dask Xarray Zarr
  5. Custom Code A common language Project-speci fi c xskillscore …

    … Domain - speci fi c Project-speci fi c A new kind of science discourse.pangeo.io Github Twitter A common language shared Jupyter notebooks Domain - speci fi c
  6. An a lyzing Pet a byte sc a le clim

    a te d a t a in your browser with P a ngeo No Supercomputer, no problem! Custom Analysis applied to each model and member
  7. Di ff erent dimension names in the CMIP data. 


    
 Not quite analysis -ready No! Time to clean data!
  8. There is! + Analysis Ready Data in the cloud Crowd-Sourced

    Data Cleaning 
 (peer-to-peer learning)
  9. There is! + Analysis Ready Data in the cloud Crowd-Sourced

    Data Cleaning 
 (peer-to-peer learning) Less data wrangling, more 💡 =
  10. What’s next? Join the community! discourse.pangeo.io (forum) gallery.pangeo.io (science examples)

    projectpythia.org (open science resources) Github (go where the code lives) Twitter (great for quick questions) Weekly meetings
  11. Dont be afraid to ask and share Nobody is going

    to shame your code. We all started somewhere! 🤗
  12. Start small Start your next project with pangeo tools Stay

    here and and dig into some CMIP6 data if you like
  13. Start small Start your next project with pangeo tools Stay

    here and and dig into some CMIP6 data if you like