Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Heartbleed at Acquia

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Marc Seeger Marc Seeger
May 20, 2014
Technology
0
15k

Heartbleed at Acquia

A quick presentation on how we handled Heartbleed at Acquia. Held at a DevOps Boston meetup.

Avatar for Marc Seeger

Marc Seeger

May 20, 2014
Tweet

More Decks by Marc Seeger

See All by Marc Seeger
Security in DECT
Avatar for Marc Seeger rb2k
2
180
The DIRAC video codec
Avatar for Marc Seeger rb2k
1
89
Communitygetriebene Android Systemerweiterungen
Avatar for Marc Seeger rb2k
1
57
Alternative infrastructure
Avatar for Marc Seeger rb2k
1
180
NoSQL Lunch and Learn
Avatar for Marc Seeger rb2k
9
8.6k
Lunch and Learn: Cucumber and Capybara
Avatar for Marc Seeger rb2k
7
22k

Other Decks in Technology

See All in Technology
「Blue Team Labs Online」入門 - みんなで挑むログ解析バトル
Avatar for Noriaki Hayashi v_avenger
0
120
Oracle Database@Azure：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
4
1.1k
タスク管理も1on1も、もう「管理」じゃない ― KiroとBedrock AgentCoreで変わった"判断の仕事"
Avatar for Yusuke Shimizu yusukeshimizu
5
1.9k
A Gentle Introduction to Transformers
Avatar for Semantic Machine Intelligence Lab., Keio Univ. keio_smilab
PRO
2
930
楽しく学ぼう！ネットワーク入門
Avatar for Shota Shiratori shotashiratori
0
300
ヘルシーSRE
Avatar for Hiroki Takatsuka tk3fftk
2
240
S3はフラットである –AWS公式SDKにも存在した、 署名付きURLにおけるパストラバーサル脆弱性– / JAWS DAYS 2026
Avatar for GMO Flatt Security flatt_security
0
950
事例に見るスマートファクトリーへの道筋〜工場データをAI Readyにする実践ステップ〜
Avatar for 濱田孝治 hamadakoji
0
210
聲の形にみるアクセシビリティ
Avatar for tomokusaba tomokusaba
0
150
AIエージェント・エコノミーの幕開け 〜 オープンプロトコルが変えるビジネスの未来 〜
Avatar for Shu Kobuchi shukob
0
110
モブプログラミング再入門 ー 基本から見直す、AI時代のチーム開発の選択肢 ー / A Re-introduction of Mob Programming
Avatar for TAKAKING22 takaking22
5
750
研究開発部メンバーの働き⽅ / Sansan R&D Profile
Avatar for Sansan, Inc. sansan33
PRO
4
22k

Featured

See All Featured
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
2
540
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Darren the Foodie - Storyboard
Avatar for Kevinho.art khoart
PRO
3
2.8k
技術選定の審美眼（2025年版） / Understanding the Spiral of Technologies 2025 edition
Avatar for Takuto Wada twada
PRO
118
110k
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
Avatar for Aleyda Solis aleyda
1
1.9k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.8k
sira's awesome portfolio website redesign presentation
Avatar for ElsiraPls elsirapls
0
190
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
82
6.2k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
120
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
Avatar for Aleyda Solis aleyda
1
1.8k
Ethics towards AI in product and experience design
Avatar for Skipper Chong Warson skipperchong
2
220
Lightning talk: Run Django tests with GitHub Actions
Avatar for Sarah Abderemane sabderemane
0
140

Transcript

  1. Marc Seeger (@rb2k)
 Boston Devops Meetup
 May 20th 2014 at

  2. Act 1: Technology

  3. How it all started 7:24 PM

  4. How it all started 7:30 PM

  5. How it all started 7:26 PM

  6. How it all started 7:33 PM

  7. How it all started

  8. Quick risk assessment Lucid: [00:35:27] [email protected]:~# openssl version OpenSSL 0.9.8k

    25 Mar 2009 ! Precise: [00:34:37] [email protected]:~# openssl version OpenSSL 1.0.1 14 Mar 2012

  9. Where’s Waldo OpenSSL 8000 EC2 Machines: - 99.9% of them

    puppetized - Candidates: - Balancers - SVN Servers - Appliances - ELBs - 3rd party AMIs - Unique little snowflakes
 (Jira, Crucible,…)

  10. Let the patching begin

  11. Rollout Australia: ! Con: - Spiders - Snakes ! Pro:

    - Ops is awake

  12. Rollout

  13. Scan www

  14. Waiting on ELBs…

  15. Internal Certificates

  16. Suddenly: “reverse” Heartbleed

  17. Act 2: Communication

  18. Internal • Pre-determined chat rooms • Dial-in conference bridges •

    A communication plan Thanks SSAE-16, PCI and FedRAMP… I guess :)

  19. Statuspage + Twitter * Powered by StatusPage.io *

  20. Documentation https://docs.acquia.com/articles/heartbleed-acquia-cloud

  21. Proactive communication Phone calls by Acquia support, TAMs, …

  22. Since then: Post mortem

  23. Since then: Incident Commander (shamelessly stolen from Heroku) http://en.wikipedia.org/wiki/Incident_command_system

  24. Since then: Dedicated resource to vet security threats

  25. Since then: Clean up intranet docs

  26. Since then: Additional tooling

  27. We’re hiring (shameless self promotion) bit.ly/acquiajobs