Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intel Theater Presentation @ SC11

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Deepak Singh Deepak Singh
November 19, 2011
Technology
200
6

Intel Theater Presentation @ SC11

Presented at the Intel Theater at SC11

Avatar for Deepak Singh

Deepak Singh

November 19, 2011

More Decks by Deepak Singh

See All by Deepak Singh
Changing the Calculus of Containers (Datadog Dash)
Avatar for Deepak Singh mndoci
2
120
Platforms for scientific data analysis
Avatar for Deepak Singh mndoci
3
110
FGED Keynote
Avatar for Deepak Singh mndoci
3
100
Open Mic Science - May 7, 2012
Avatar for Deepak Singh mndoci
4
1.3k
Talk at "Genome Informatics Alliance 2012" meeting
Avatar for Deepak Singh mndoci
1
270
A Platform for Data Science
Avatar for Deepak Singh mndoci
6
15k
Talk at West Coast Association of Shared Directors meeting
Avatar for Deepak Singh mndoci
3
160
A platform for data science - Systems Bioinformatics Workshop
Avatar for Deepak Singh mndoci
3
120
Platforms for Data Science
Avatar for Deepak Singh mndoci
3
200

Other Decks in Technology

See All in Technology
OCI技術資料 : 証明書サービス概要
Avatar for Oracle Cloud Infrastructure ソリューション・エンジニア ocise
1
7.2k
バックオフィスPJのPjMをコーポレートITが担うとうまくいく3つの理由
Avatar for chama yueda256
1
220
Bref でサービスを運用している話
Avatar for Sugar Sato sgash708
0
220
JAWS DAYS 2026でAIの「もやっと」感が解消された話
Avatar for Makky12 smt7174
1
120
Databricks Appsで実現する社内向けAIアプリ開発の効率化
Avatar for R-Miura r_miura
0
240
Physical AI on AWS リファレンスアーキテクチャ / Physical AI on AWS Reference Architecture
Avatar for shota aws_shota
1
330
20260326_AIDD事例紹介_ULSC.pdf
Avatar for ファインディ株式会社（イベント資料アップ用） findy_eventslides
0
450
トイルを超えたCREは何屋になるのか
Avatar for 弁護士ドットコム bengo4com
0
120
Oracle Cloud Infrastructure：2026年3月度サービス・アップデート
Avatar for oracle4engineer oracle4engineer
PRO
0
340
The essence of decision-making lies in primary data
Avatar for 株式会社カミナシ kaminashi
0
240
Amazon Qはアマコネで頑張っています〜 Amazon Q in Connectについて〜
Avatar for Yuuki Yamashita yama3133
1
170
Databricks Lakehouse Federationで 運用負荷ゼロのデータ連携
Avatar for Kenji Matsuda nek0128
0
110

Featured

See All Featured
Color Theory Basics | Prateek | Gurzu
Avatar for Gurzu gurzu
0
280
Lessons Learnt from Crawling 1000+ Websites
Avatar for Charles Meaden charlesmeaden
PRO
1
1.2k
Exploring the relationship between traditional SERPs and Gen AI search
Avatar for Ray Grieselhuber raygrieselhuber
PRO
2
3.8k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
Avatar for Tammy Everts tammyeverts
2
260
Optimizing for Happiness
Avatar for Tom Preston-Werner mojombo
378
71k
30 Presentation Tips
Avatar for Ian Lurie portentint
PRO
1
270
The Impact of AI in SEO - AI Overviews June 2024 Edition
Avatar for Aleyda Solis aleyda
5
780
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
Avatar for Akash Hashmi akashhashmi
0
310
Primal Persuasion: How to Engage the Brain for Learning That Lasts
Avatar for Mike Taylor tmiket
0
310
KATA
Avatar for Megan Lloyd mclloyd
PRO
35
15k
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
290
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.8k

Transcript

  1. HPC with Amazon EC2 Deepak Singh @mndoci P r i

    n c i p a l P r o d u c t M a n a g e r

  2. Amazon Web Services

  3. 4

  4. 2

  5. 1. Infrastructure

  6. None

  7. ec2-run-instances

  8. None

  9. secure global on demand

  10. programmable

  11. None
  12. None
  13. None

  14. elastic

  15. None

  16. instance types

  17. standard (m1) high memory (m2) high CPU (c1) t1.micro

  18. high performance

  19. “Our 40-instance (m2.2xlarge) cluster can scan, filter, and aggregate 1

    billion rows in 950 milliseconds.” Mike Driscoll - Metamarkets

  20. cluster computing

  21. MPI

  22. bandwidth intensive

  23. Cluster Compute Instance

  24. 2*Intel Xeon 5570 8 cores w/HT 23 GB RAM 1.7

    TB disk HVM cc1.4xlarge

  25. 10 gig E

  26. Placement Group

  27. Placement group full- bisection

  28. linpack

  29. Cores 7040 Rmax 41.82 Rpeak 82.51

  30. 231 November 2010

  31. 451 June 2011

  32. WIEN2K Parallel Performance H size 56,000 (25GB) Runtime (16x8 processors)

    Local (Infiniband) 3h:48 Cloud (10Gbps) 1h:30 ($40) 1200 atom unit cell; SCALAPACK+MPI diagonalization, matrix size 50k-100k Credit: K. Jorissen, F. D. Villa, and J. J. Rehr (U. Washington)

  33. New Cluster Compute Instance

  34. 2*Intel Xeon 16 cores w/HT 60.5 GB RAM 3.4 TB

    disk HVM cc2.8xlarge

  35. linpack

  36. Cores 17024 Rmax 240.09 Rpeak 354.12

  37. 42 November 2011

  38. optimizing costs

  39. on-demand

  40. reserved

  41. spot

  42. None
  43. None
  44. None

  45. 30,472 cores

  46. $1279/hr

  47. 2. Orchestration

  48. None

  49. AWS CloudFormation

  50. bootstrap

  51. Cloud Init

  52. #cloud-config packages: ! - httpd ! runcmd: ! - /etc/init.d

    http start ! - echo "<h1>hello, world"</h1> \ ! ! > /var/www/html/ index.html

  53. #!/bin/sh ec2-run-instances ami-8c1fece5 \ ! -n 1 \ ! -t

    m1.small \ ! -g deesinghdemo-SG \ ! -k deesinghdemo-keypair \ ! --user-data-file \ .\cloudconfig.txt

  54. chef/puppet

  55. familiar tools

  56. LSF

  57. Grid Engine

  58. Bright Cluster Manager

  59. combining worlds

  60. MIT Starcluster

  61. $ starcluster start mycluster $ starcluster listclusters

  62. http://www.bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/

  63. None

  64. Provisions Cluster Shared Storage Monitoring Bootstraps StarCluster Includes 200 GB

    Public Dataset Provisioned Stack = Submit jobs to Grid Engine
  65. None
  66. None
  67. None

  68. Image: Chris Dagdigian