Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Platforms for Data Science

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Deepak Singh Deepak Singh
October 01, 2011
Technology
3
200

Platforms for Data Science

Talk given at the "Computing on the Brink" series

Avatar for Deepak Singh

Deepak Singh

October 01, 2011
Tweet

More Decks by Deepak Singh

See All by Deepak Singh
Changing the Calculus of Containers (Datadog Dash)
Avatar for Deepak Singh mndoci
2
120
Platforms for scientific data analysis
Avatar for Deepak Singh mndoci
3
110
FGED Keynote
Avatar for Deepak Singh mndoci
3
99
Open Mic Science - May 7, 2012
Avatar for Deepak Singh mndoci
4
1.3k
Talk at "Genome Informatics Alliance 2012" meeting
Avatar for Deepak Singh mndoci
1
260
A Platform for Data Science
Avatar for Deepak Singh mndoci
6
14k
Intel Theater Presentation @ SC11
Avatar for Deepak Singh mndoci
6
190
Talk at West Coast Association of Shared Directors meeting
Avatar for Deepak Singh mndoci
3
160
A platform for data science - Systems Bioinformatics Workshop
Avatar for Deepak Singh mndoci
3
120

Other Decks in Technology

See All in Technology
Yahoo!ショッピングのレコメンデーション・システムにおけるML実践の一例 
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
210
[JAWSDAYS2026]Who is responsible for IAM
Avatar for Mizuki TSUJI mizukibbb
0
700
The_Evolution_of_Bits_AI_SRE.pdf
Avatar for 株式会社ヌーラボ nulabinc
PRO
0
220
猫でもわかるKiro CLI(AI 駆動開発への道編)
Avatar for Hiroo Katoh kentapapa
0
220
JAWS Days 2026 楽しく学ぼう！ 認証認可 入門/20260307-jaws-days-novice-lane-auth
Avatar for opelab opelab
11
2.3k
AWS DevOps Agent vs SRE俺 / AWS DevOps Agent vs me, the SRE
Avatar for SMS tech sms_tech
3
840
20260311 技術SWG活動報告（デジタルアイデンティティ人材育成推進WG Ph2 活動報告会）
Avatar for OpenID Foundation Japan oidfj
0
350
GCASアップデート（202601-202603）
Avatar for 高橋広和 techniczna
0
170
Google系サービスで文字起こしから勝手にカレンダーを埋めるエージェントを作った話
Avatar for RisaTube risatube
0
190
JAWS DAYS 2026 楽しく学ぼう！ストレージ 入門
Avatar for Yoshiki Fujiwara yoshiki0705
2
190
Sansanでの認証基盤内製化と移行
Avatar for SansanTech sansantech
PRO
0
490
It’s “Time” to use Temporal
Avatar for Saji sajikix
2
160

Featured

See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
133
19k
Between Models and Reality
Avatar for mayunak mayunak
2
230
The Spectacular Lies of Maps
Avatar for Per Axbom axbom
PRO
1
620
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
140
Applied NLP in the Age of Generative AI
Avatar for Ines Montani inesmontani
PRO
4
2.2k
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8k
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
Avatar for soudai sone soudai
PRO
63
51k
Navigating the moral maze — ethical principles for Al-driven product design
Avatar for Skipper Chong Warson skipperchong
2
290
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.6k
Reflections from 52 weeks, 52 projects
Avatar for Jefferson Lam jeffersonlam
356
21k
Jess Joyce - The Pitfalls of Following Frameworks
Avatar for Tech SEO Connect techseoconnect
PRO
1
110
Amusing Abliteration
Avatar for ianozsvald ianozsvald
0
130

Transcript

  1. There is no magic There is only awesome D e

    e p a k S i n g h Platforms for data science

  2. bioinformatics image: Ethan Hein

  3. 3

  4. collection

  5. curation

  6. analysis

  7. what’s the big deal?

  8. None

  9. Source: http://www.nature.com/news/specials/bigdata/index.html

  10. Image: Yael Fitzpatrick (AAAS)

  11. Image: Yael Fitzpatrick (AAAS)

  12. lots of data

  13. lots of people

  14. lots of places

  15. constant change

  16. we want to make our data more effective

  17. versioning

  18. provenance

  19. filter

  20. aggregate

  21. extend

  22. mashup

  23. human interfaces

  24. None

  25. image: Leo Reynolds

  26. hard problem

  27. really hard problem

  28. so how do get there?

  29. information platforms

  30. Image: Drew Conway

  31. dataspaces Further reading: Jeff Hammerbacher, Information Platforms and the rise

    of the data scientist, Beautiful Data

  32. the unreasonable effectiveness of data Halevy, et al. IEEE Intelligent

    Systems, 24, 8-12 (2009)

  33. accept all data formats

  34. evolve APIs

  35. beyond databases and the data warehouse

  36. data as a programmable resource

  37. data is a royal garden

  38. compute is a fungible commodity

  39. optimizing the most valuable resource

  40. compute, storage, workflows, memory, transmission, algorithms, cost, …

  41. people Credit: Pieter Musterd a CC-BY-NC-ND license

  42. Image: Chris Dagdigian

  43. my bias

  44. cloud services

  45. distributed systems

  46. scale

  47. global

  48. consumption models

  49. on-demand

  50. what is the value of your data?

  51. None
  52. None

  53. Credit: Angel Pizzaro, U. Penn

  54. mapreduce for genomics http://bowtie-bio.sourceforge.net/crossbow/index.shtml http://contrail-bio.sourceforge.net http://bowtie-bio.sourceforge.net/myrna/index.shtml

  55. None

  56. Bioproximity http://aws.amazon.com/solutions/case-studies/bioproximity/

  57. None
  58. None

  59. 30,472 cores

  60. $1279/hr

  61. http://cloudbiolinux.org/

  62. http://usegalaxy.org/cloud

  63. in summary

  64. large scale data requires a rethink

  65. data architecture

  66. compute architecture

  67. distributed, programmable infrastructure

  68. cloud services

  69. remove constraints

  70. can we build data science platforms?

  71. there is no magic there is only awesome

  72. [email protected] Twitter:@mndoci http://slideshare.net/mndoci http://mndoci.com Inspiration and ideas from Matt Wood&

    Larry Lessig Credit” Oberazzi under a CC-BY-NC-SA license