Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Heartbleed at Acquia

Avatar for Marc Seeger Marc Seeger
May 20, 2014
Technology
15k
0

Heartbleed at Acquia

A quick presentation on how we handled Heartbleed at Acquia. Held at a DevOps Boston meetup.

Avatar for Marc Seeger

Marc Seeger

May 20, 2014

More Decks by Marc Seeger

See All by Marc Seeger
Security in DECT
Avatar for Marc Seeger rb2k
2
190
The DIRAC video codec
Avatar for Marc Seeger rb2k
1
95
Communitygetriebene Android Systemerweiterungen
Avatar for Marc Seeger rb2k
1
61
Alternative infrastructure
Avatar for Marc Seeger rb2k
1
190
NoSQL Lunch and Learn
Avatar for Marc Seeger rb2k
9
8.6k
Lunch and Learn: Cucumber and Capybara
Avatar for Marc Seeger rb2k
7
22k

Other Decks in Technology

See All in Technology
APIテストとは？
Avatar for 草薙昭彦 nagix
0
160
PHP と TypeScript の型システム比較：AI 時代の「型」は誰のためにあるのか？ #frontend_phpcon_do / frontend_phpcon_do_2026
Avatar for shogogg shogogg
1
190
はじめてのDatadog
Avatar for KairiM kairim0
0
240
Kiro CLI v2.0.0がやってきた！
Avatar for Hiroo Katoh kentapapa
0
240
コードレビューを制するチームがソフトウェアデリバリーのフローを制す / Beyond Code Review: Distributing Its Responsibilities Across the SDLC
Avatar for mtx2s mtx2s
3
500
エンジニアは生成AIと どのように向き合うべきか？ ことばの意味という観点から
Avatar for Hitomi Yanaka verypluming
3
300
オンコールの負荷軽減のためのBits Assistant 活用方法 / How to Use Bits Assistant to Reduce the Workload on On-Call Staff
Avatar for SMS tech sms_tech
1
350
Javaで学ぶSOLID原則
Avatar for Negima negima
1
240
もりもり新機能を一挙紹介！ AgentCoreに入門して、AWS上にAIエージェントを構築しよう
Avatar for みのるん minorun365
PRO
6
350
美味しいスイスチーズを作ろう🧀🐭
Avatar for Taiga Mikami taigamikami
1
190
類似画像検索モデルの開発ノウハウ
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
4
1.1k
サプライチェーンセキュリティの空白地帯 - 信頼できる”依存性”の未来を考える
Avatar for Hiroki Suezawa (@rung) rung
PRO
2
530

Featured

See All Featured
Money Talks: Using Revenue to Get Sh*t Done
Avatar for Nikki Halliwell nikkihalliwell
0
240
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
300
How to train your dragon (web standard)
Avatar for Monica Dinculescu notwaldorf
97
6.6k
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
200
Joys of Absence: A Defence of Solitary Play
Avatar for Sebastian Deterding codingconduct
1
380
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.9k
Abbi's Birthday
Avatar for Violet Bock coloredviolet
2
7.8k
Making Projects Easy
Avatar for Brett Harned brettharned
120
6.7k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
460
Side Projects
Avatar for Sacha Greif sachag
455
43k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
23k
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
5.1k

Transcript

  1. Marc Seeger (@rb2k)
 Boston Devops Meetup
 May 20th 2014 at

  2. Act 1: Technology

  3. How it all started 7:24 PM

  4. How it all started 7:30 PM

  5. How it all started 7:26 PM

  6. How it all started 7:33 PM

  7. How it all started

  8. Quick risk assessment Lucid: [00:35:27] [email protected]:~# openssl version OpenSSL 0.9.8k

    25 Mar 2009 ! Precise: [00:34:37] [email protected]:~# openssl version OpenSSL 1.0.1 14 Mar 2012

  9. Where’s Waldo OpenSSL 8000 EC2 Machines: - 99.9% of them

    puppetized - Candidates: - Balancers - SVN Servers - Appliances - ELBs - 3rd party AMIs - Unique little snowflakes
 (Jira, Crucible,…)

  10. Let the patching begin

  11. Rollout Australia: ! Con: - Spiders - Snakes ! Pro:

    - Ops is awake

  12. Rollout

  13. Scan www

  14. Waiting on ELBs…

  15. Internal Certificates

  16. Suddenly: “reverse” Heartbleed

  17. Act 2: Communication

  18. Internal • Pre-determined chat rooms • Dial-in conference bridges •

    A communication plan Thanks SSAE-16, PCI and FedRAMP… I guess :)

  19. Statuspage + Twitter * Powered by StatusPage.io *

  20. Documentation https://docs.acquia.com/articles/heartbleed-acquia-cloud

  21. Proactive communication Phone calls by Acquia support, TAMs, …

  22. Since then: Post mortem

  23. Since then: Incident Commander (shamelessly stolen from Heroku) http://en.wikipedia.org/wiki/Incident_command_system

  24. Since then: Dedicated resource to vet security threats

  25. Since then: Clean up intranet docs

  26. Since then: Additional tooling

  27. We’re hiring (shameless self promotion) bit.ly/acquiajobs