a interactively biased retrospective of a (mostly) successful decade Fernando Pérez http://fperez.org, @fperez_org [email protected] PyCon Canada Toronto Nov 11, 2012
computer as a microscope Exploratory: Problem’s definition evolves as we understand it. No ‘requirements’ to build an application against. Mathematica, Maple, Matlab, IDL, etc. All have an interactive environment. Applications Languages FP (UC Berkeley) Python for science 11/11/12 10 / 46
integers Rationals Interval arithmetic Symbolic manipulation FORTRAN Extended precision floating point Text processing Databases Graphical user interfaces Web interfaces Hardware control Multi-language integration Data formats: HDF5, XML, ...
and Lynn Craig, Jet Propulsion Laboratory (NASA/Caltech) From: Name Elided <[email protected]> Date: Oct 2, 2007 7:15 PM Subject: Fwd: matplotlib bug numbers To: John Hunter <[email protected]> One of my lead developers mentioned that they had sent a bug to you about the annotations feature of MatPlotLib. Would you be able to let me know what the timeline is to resolve that bug? The reason is that the feature is needed for the Phoenix project and their arrival at Mars will be in March sometime, but they are doing their testing in the coming few months. This annotation feature is used on reports that present the analysis of the trajectory to the navigation team and it shows up on our schedule. It would really help me to know approximately when it could be resolved. B-plane plots are used to show the trajectory of a spacecraft with respect to the target body (specifically perpendicular to the incoming asymptote of the spacecraft trajectory) and we plot them with the y-axis inverted. The plot is used heavily in flight operations so it is important to our customers. In addition, we have what is called a thundering heard plot where many different trajectory solutions (determined from different measurement sources) are plotted together. The annotations are import there so we can see which plot corresponds to each source of data. I hope it helps to know how your code will be used in spacecraft navigation. Thanks for all your efforts.
a Scientific Idea (schematically) 1 Individual exploratory work 2 Collaborative development 3 Production work (HPC, cloud, parallel) 4 Publication (with reproducible results!) 5 Education 6 Goto 1. The Problem with most tools Barriers and discontinuities in workflow in between all the steps FP (UC Berkeley) Python for science 11/11/12 24 / 46
a Scientific Idea (schematically) 1 Individual exploratory work 2 Collaborative development 3 Production work (HPC, cloud, parallel) 4 Publication (with reproducible results!) 5 Education 6 Goto 1. The Problem with most tools Barriers and discontinuities in workflow in between all the steps FP (UC Berkeley) Python for science 11/11/12 24 / 46
Notebook Format JSON but version control-friendly Easy for machine processing, fixable by hand if need be. Lots of hooks for metadata Not Python-specific (R and Ruby notebooks exist, Julia planned) Produce Markdown, reST, L A TEX, HTML, etc... An open format for sharing, publishing and archiving executable computational work FP (UC Berkeley) Python for science 11/11/12 28 / 46
San Luis Obispo Min Ragan-Kelley - Nuclear Engineering, UC Berkeley Matthias Bussonnier - Physics, Institut Curie, Paris Jonathan March- Enthought Thomas Kluyver - Biology, U. Sheffield Jörgen Stenarson - Elect. Engineering, Sweden. Paul Ivanov - Neuroscience, UC Berkeley. Robert Kern - Enthought Evan Patterson - Physics, Caltech/Enthought Brad Froehle - Mathematics, UC Berkeley Stefan van der Walt - UC Berkeley John Hunter - TradeLink Securities, Chicago. Prabhu Ramachandran - Aerospace Engineering, IIT Bombay. Satra Ghosh- MIT Neuroscience Gaël Varoquaux - Neurospin (Orsay, France) Ville Vainio - CS, Tampere University of Technology, Finland Barry Wark - Neuroscience, U. Washington. Ondrej Certik - Physics, U Nevada Reno Darren Dale - Cornell Justin Riley - MIT Mark Voorhies - UC San Francisco Nicholas Rougier - INRIA Nancy Grand Est Thomas Spura - Fedora project Many more! (~150 commit authors)
Visual Studio integration, Azure (thanks to Shahrokh Mortazavi). DoD/DRC Inc: funding through Sept. 2012 (thanks to Jose Unpingco and Chris Keees). NIH: via NiPy grant NSF: via Sage compmath grant Google: summer of code 2005, 2010. Tech-X Corp., Boulder, CO: Parallel/notebook (previous versions)
me in... But I stayed for the community! Real friendships and incredible people A culture of generous and mutual cross-project collaboration But we have a ton of work to do! The tools we need must be built by scientists. Lots of space for truly innovative thinking, and Python is an expressive tool for the exercise. FP (UC Berkeley) Python for science 11/11/12 45 / 46