Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Platforms for Data Science

Avatar for Deepak Singh Deepak Singh
October 01, 2011
Technology
3
200

Platforms for Data Science

Talk given at the "Computing on the Brink" series

Avatar for Deepak Singh

Deepak Singh

October 01, 2011
Tweet

More Decks by Deepak Singh

See All by Deepak Singh
Changing the Calculus of Containers (Datadog Dash)
Avatar for Deepak Singh mndoci
2
100
Platforms for scientific data analysis
Avatar for Deepak Singh mndoci
3
100
FGED Keynote
Avatar for Deepak Singh mndoci
3
93
Open Mic Science - May 7, 2012
Avatar for Deepak Singh mndoci
4
1.3k
Talk at "Genome Informatics Alliance 2012" meeting
Avatar for Deepak Singh mndoci
1
250
A Platform for Data Science
Avatar for Deepak Singh mndoci
6
14k
Intel Theater Presentation @ SC11
Avatar for Deepak Singh mndoci
6
180
Talk at West Coast Association of Shared Directors meeting
Avatar for Deepak Singh mndoci
3
150
A platform for data science - Systems Bioinformatics Workshop
Avatar for Deepak Singh mndoci
3
110

Other Decks in Technology

See All in Technology
「技術負債にならない・間違えない」 権限管理の設計と実装
Avatar for naro143 naro143
31
9.5k
AI Agentと MCP Serverで実現する iOSアプリの 自動テスト作成の効率化
Avatar for スパイダープラス株式会社 spiderplus_cb
0
290
組織観点から IAM Identity CenterとIAMの設計を考える
Avatar for NRI Netcom nrinetcom
PRO
1
130
LLMアプリケーション開発におけるセキュリティリスクと対策 / LLM Application Security
Avatar for GMO Flatt Security flatt_security
7
1.5k
Go Conference 2025: GoのinterfaceとGenericsの内部構造と進化 / Go type system internals
Avatar for turbofish ryokotmng
3
540
そのグラフに「魂」は宿っているか？ ～生成AI全盛期におけるデータ可視化手法とライブラリ比較～
Avatar for negi111111 negi111111
2
830
WebアプリケーションのUI構築で気を付けてるポイント
Avatar for tomokusaba tomokusaba
0
180
PyCon JP 2025 DAY1 「Hello, satellite data! ～Pythonではじめる衛星データ解析～」
Avatar for ぴっかりん ra0kley
0
730
GopherCon Tour 概略
Avatar for Takuto Nagami logica0419
2
160
Streamlit は社内ツールだけじゃない！PoC の速さで実現する'商用品質'の分析 SaaS アーキテクチャ
Avatar for 𝕂' kdash
2
1k
Sidekiq その前に：Webアプリケーションにおける非同期ジョブ設計原則
Avatar for morihirok morihirok
17
6.2k
kaigi_on_rails_2025_設計.pdf
Avatar for Yasuko Ohba nay3
8
4k

Featured

See All Featured
Designing Experiences People Love
Avatar for Jonathan Moore moore
142
24k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.7k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
140
34k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
34
6.1k
Building Better People: How to give real-time feedback that sticks.
Avatar for Will Jessup wjessup
368
20k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
188
55k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
37
2.9k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
330
21k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
25k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
1.6k
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
180
9.9k
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
45
2.5k

Transcript

  1. There is no magic There is only awesome D e

    e p a k S i n g h Platforms for data science

  2. bioinformatics image: Ethan Hein

  3. 3

  4. collection

  5. curation

  6. analysis

  7. what’s the big deal?

  8. None

  9. Source: http://www.nature.com/news/specials/bigdata/index.html

  10. Image: Yael Fitzpatrick (AAAS)

  11. Image: Yael Fitzpatrick (AAAS)

  12. lots of data

  13. lots of people

  14. lots of places

  15. constant change

  16. we want to make our data more effective

  17. versioning

  18. provenance

  19. filter

  20. aggregate

  21. extend

  22. mashup

  23. human interfaces

  24. None

  25. image: Leo Reynolds

  26. hard problem

  27. really hard problem

  28. so how do get there?

  29. information platforms

  30. Image: Drew Conway

  31. dataspaces Further reading: Jeff Hammerbacher, Information Platforms and the rise

    of the data scientist, Beautiful Data

  32. the unreasonable effectiveness of data Halevy, et al. IEEE Intelligent

    Systems, 24, 8-12 (2009)

  33. accept all data formats

  34. evolve APIs

  35. beyond databases and the data warehouse

  36. data as a programmable resource

  37. data is a royal garden

  38. compute is a fungible commodity

  39. optimizing the most valuable resource

  40. compute, storage, workflows, memory, transmission, algorithms, cost, …

  41. people Credit: Pieter Musterd a CC-BY-NC-ND license

  42. Image: Chris Dagdigian

  43. my bias

  44. cloud services

  45. distributed systems

  46. scale

  47. global

  48. consumption models

  49. on-demand

  50. what is the value of your data?

  51. None
  52. None

  53. Credit: Angel Pizzaro, U. Penn

  54. mapreduce for genomics http://bowtie-bio.sourceforge.net/crossbow/index.shtml http://contrail-bio.sourceforge.net http://bowtie-bio.sourceforge.net/myrna/index.shtml

  55. None

  56. Bioproximity http://aws.amazon.com/solutions/case-studies/bioproximity/

  57. None
  58. None

  59. 30,472 cores

  60. $1279/hr

  61. http://cloudbiolinux.org/

  62. http://usegalaxy.org/cloud

  63. in summary

  64. large scale data requires a rethink

  65. data architecture

  66. compute architecture

  67. distributed, programmable infrastructure

  68. cloud services

  69. remove constraints

  70. can we build data science platforms?

  71. there is no magic there is only awesome

  72. [email protected] Twitter:@mndoci http://slideshare.net/mndoci http://mndoci.com Inspiration and ideas from Matt Wood&

    Larry Lessig Credit” Oberazzi under a CC-BY-NC-SA license