Share
Embed
ブログサービスのHTTPS化を支えたAWSで作るピタゴラスイッチ / The construction of large scale TLS certificates management system with AWS
talked at builderscon tokyo 2018
aereal
September 08, 2018
Transcript
90 slides
-
͢͜ͱ • ͯͳϒϩάͷৗ࣌HTTPS৴ͷղઆ • എܠͱཁٻ • ࣮ͷհ • ্هࣄྫΛݩʹෳࡶͳόον =
ϐλΰϥεΠονߏஙͷ ΤοηϯεΛߟ͑ͯΈΔ -
ࣗݾհ • id:aereal • GitHub: aereal • Twitter: aereal •
ϒϩά౷߹νʔϜ ΞϓϦέʔγϣϯΤϯδχΞ ςοΫϦʔυ -
Let's Encrypt • ISRG = Internet Security Research Group͕ఏڙ͢Δ ϓϩάϥϚϒϧʹΞΫηεՄೳͳೝূہ
(CA) • ͜Ε·ͰTLSূ໌ॻΛൃߦ͢Δʹ ͦͦ͜͜ͷֹۚͱख͕ؒඞཁ͕ͩͬͨɺͦΕΛม͑ͨCA • LEͷొʹΑΓTLSূ໌ॻͷେྔൃߦ͕ݱ࣮తʹͳͬͨ -
SAN? • = Subject Alternative Names 1ͭͷূ໌ॻʹෳυϝΠϯΛඥ͚Δ֦ு • ͔݁Βݴ͏ͱͯͳϒϩάͷέʔεͰ͍͠ •
LEͰSANΛར༻͢Δ߹ɺACME challengedns-01ͷΈ ར༻Ͱ͖Δ (ݱࡏ) • DNSઃఆ֤ϢʔβʔʹҕͶΒΕΔͷͰࣗಈԽͰ͖ͳ͍ -
ACME? • ACME: Automated Certificate Management Environment • ূ໌ॻൃߦͳͲͷ࡞ۀΛ ࣗಈԽ͢ΔϓϩτίϧΛ·ͱΊ༷ͨ
• ACME challenge: υϝΠϯͷॴ༗ݖݶΛ֬ೝ͢Δํ๏ • Google AnalyticsͷΞϨΈ͍ͨͳͭ • LE͕ࡦఆɾ࠾༻͍ͯ͠Δ -
ACME challenge? • dns-01: υϝΠϯͷTXTϨίʔυʹϫϯλΠϜτʔΫϯΛॻ ͖ࠐΉ • http-01: CAͷϦΫΤετʹର͠ॴఆͷϨεϙϯεΛฦ͢ •
ྫ: /.well-known/TOKEN • (ଞʹ͍Ζ͍Ζ) -
ূ໌ॻൃߦ • Ұ؏ੑɾཏੑ͕ٻΊΒΕΔ • ൃߦʹࣦഊ͠ଓ͚Δͱϒϩά͕ӾཡͰ͖ͳ͘ͳΔ • ແޮͳυϝΠϯΛ์ஔ͍͚ͯ͠ͳ͍ • ཁٻߴ͍͕ෆ࣮֬ੑߴ͍ •
ূ໌ॻΛߋ৽͢ΔࡍɺυϝΠϯʹର͠εέʔϧ͢Δ͜ͱ -
ແޮͳυϝΠϯͷআ • ແޮͳυϝΠϯ = ඞͣACME challengeʹࣦഊ͢Δ • LEʹΞΧϯτ * time
window͝ͱʹࣦഊͷ্ݶ͕͋Δ • ์ஔ͢ΔͱඞͣAPI limitʹ͋ͨͬͯ͠·͏ • ࣦഊͨ͠υϝΠϯඞͣআ -
ূ໌ॻൃߦ: ෆ࣮֬ੑ • υϝΠϯͷ༗ޮੑมΘΓ͏Δ • ՝ۚऴྃ • DNSϨίʔυҟৗ • ֎෦API
= LEͱͷ౷߹ • API Limit • దͳϦτϥΠͱΤϥʔϦΧόϦ͕ඞਢ -
ূ໌ॻൃߦ: εέʔϥϏϦςΟ • ରυϝΠϯͷ૿Ճʹର͠εέʔϧ͢ΔΈʹ͍ͨ͠ • SELECT * FROM custom_domain WHERE
id > ? Έ͍ͨͳΫΤϦආ͚͍ͨ • υϝΠϯ͕૿͑Δͱϖʔδϯά͕ඞཁ • ࣮ߦ్தͰࣦഊͨ͠ΒɺϦτϥΠΩϡʔʹೖΕ͢Α͏ ͳΛڽΒ͞ͳ͍ͱ͍͚ͳ͘ͳΔ -
γεςϜͷཁ݅: ·ͱΊ • ϦΫΤετຖʹূ໌ॻΛऔಘɾ༻ • Ͱ͖Δ͚ͩϨΠςϯγͰ • Τϥʔੑ͕ߴ͍ • ࣦഊͨ͠ΒऔಘରͷυϝΠϯ͔Β֎͢
• ֎෦API௨৴ͷΤϥʔΛదʹॲཧͰ͖Δ • υϝΠϯͷ૿Ճʹεέʔϧ͢Δ -
৴γεςϜ • ngx_mruby: ূ໌ॻಡΈࠐΈ࣌ʹmrubyͷίʔυΛ࣮ߦ • cache gatewayHTTP GET͢Δ͚ͩ • https://github.com/matsumotory/ngx_mruby
• cache gateway (Go): HTTP GET͢Δͱূ໌ॻΛฦ͢ • DynamoDB: ূ໌ॻΛอଘ͢ΔσʔλετΞ -
cache gateway • AWS (DynamoDB) APIݺͼग़͠ΛHTTP APIʹม͑Δ • mrubyʹAWS SDK͕ͳ͍
• ಉډ͢ΔmemcachedʹಡΈॻ͖͠ɺ DynamoDBͷΞΫηεΛͰ͖Δ͚ͩݮΒ͢ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
ূ໌ॻൃߦγεςϜ • cert-updater-state: AWS StepFunctions; ֤LambdaΛىಈ • Τϥʔ༰ʹԠͨ͡ϦΧόϦɾϦτϥΠ (ޙड़) •
cert-updater-function: AWS Lambda; ূ໌ॻΛൃߦɺ DynamoDBॻ͖ࠐΈ • cert-update-notifier: Lambda; ൱Λͯͳϒϩά௨ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
cert-updater-state cert-updater-function cert-update-notifier Let's Encrypt cert-store cert-lifecycle-store Blog HTTP HTTP
࣮ߦ ࣮ߦ UpdateItem UpdateItem ূ໌ॻൃߦ ࣮ߦ ূ໌ॻൃߦ -
AWS SFn: ϦτϥΠ "Retry": [ { "ErrorEquals": ["ErrMaybeRecoverable"], "IntervalSeconds": 1,
"MaxAttempts": 3, "BackoffRate": 2.0 } ], "Catch": [ { "ErrorEquals": ["States.TaskFailed"], "Next": "Notify result to Hatena-Epic" } ], -
AWS SFn: ϦτϥΠ "Retry": [ { "ErrorEquals": ["ErrMaybeRecoverable"], "IntervalSeconds": 1,
"MaxAttempts": 3, "BackoffRate": 2.0 } ], "Catch": [ { "ErrorEquals": ["States.TaskFailed"], "Next": "Notify result to Hatena-Epic" } ], -
AWS SFn: ϦτϥΠ "Retry": [ { "ErrorEquals": ["ErrMaybeRecoverable"], "IntervalSeconds": 1,
"MaxAttempts": 3, "BackoffRate": 2.0 } ], "Catch": [ { "ErrorEquals": ["States.TaskFailed"], "Next": "Notify result to Hatena-Epic" } ], -
AWS SFn: ϦτϥΠ "Retry": [ { "ErrorEquals": ["ErrMaybeRecoverable"], "IntervalSeconds": 1,
"MaxAttempts": 3, "BackoffRate": 2.0 } ], "Catch": [ { "ErrorEquals": ["States.TaskFailed"], "Next": "Notify result to Hatena-Epic" } ], -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
ূ໌ॻൃߦ: ߋ৽ • DynamoDBͷTTL Trigger͕Lambdaܦ༝ͰSFnΛىಈ • cert-reissue-confirmer: ͯͳϒϩάʹυϝΠϯ༗ޮੑΛ ͍߹Θͤͯɺߋ৽͢Δඞཁ͕͋Δ͔Λޙଓʹ͑Δ •
cert-cleanup-function: ແޮͳυϝΠϯΛDynamoDB͔Βফ ͢ -
cert-lifecycle-store (DynamoDB) Domain: ex1.example.com ExpiresAt: 2018-05-23T02:00:00 Domain: ex2.example.com ExpiresAt: 2018-05-23T03:00:00
Domain: ex2.example.com ExpiresAt: 2018-05-23T04:00:00 Domain: ex2.example.com ExpiresAt: 2018-05-23T05:00:00 -
cert-lifecycle-store (DynamoDB) Domain: ex2.example.com ExpiresAt: 2018-05-23T03:00:00 Domain: ex2.example.com ExpiresAt: 2018-05-23T04:00:00
Domain: ex2.example.com ExpiresAt: 2018-05-23T05:00:00 -
Τϥʔॲཧ͕؆ܿʹ • όονॲཧͩͱ: औಘͨ͠ෳͷυϝΠϯΛϧʔϓͰॲཧ • = ॲཧ୯Ґ͕ෳυϝΠϯʹͳΔ • Ұ෦ͷυϝΠϯ͕ࣦഊͨ࣌͠ɺόονॲཧશମͷ εςʔλεͲ͏͢Δ?
ޭ? ࣦഊ? • pub/subͩͱ: Ҿͱͯͬͨ͠υϝΠϯ1ͭΛॲཧ͢Δ • = ॲཧ୯Ґ͕υϝΠϯ1ͭʹͳΔ -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state cert-reissue-confirmer cert-updater-state-caller cert-cleanup-function Blog cert-lifecycle-store cert-update-trigger cert-updater-state cert-store ࣮ߦ
࣮ߦ ࣮ߦ ࣮ߦ ࣮ߦ HTTP TTL Trigger DeleteItem ূ໌ॻൃߦ (ߋ৽) -
cert-reissue-state "Determine next state": { "Comment": "࣍ͷঢ়ଶΛܾఆ͠·͢", "Type": "Choice", "Choices":
[ { "Variable": "$.UpdateRequired", "BooleanEquals": true, "Next": "Call reissue of certificate" }, { "Variable": "$.UpdateRequired", "BooleanEquals": false, "Next": "Clean up of certificate" } ] }, -
cert-reissue-state "Determine next state": { "Comment": "࣍ͷঢ়ଶΛܾఆ͠·͢", "Type": "Choice", "Choices":
[ { "Variable": "$.UpdateRequired", "BooleanEquals": true, "Next": "Call reissue of certificate" }, { "Variable": "$.UpdateRequired", "BooleanEquals": false, "Next": "Clean up of certificate" } ] }, -
cert-reissue-state "Determine next state": { "Comment": "࣍ͷঢ়ଶΛܾఆ͠·͢", "Type": "Choice", "Choices":
[ { "Variable": "$.UpdateRequired", "BooleanEquals": true, "Next": "Call reissue of certificate" }, { "Variable": "$.UpdateRequired", "BooleanEquals": false, "Next": "Clean up of certificate" } ] }, -
ڊେͳόονͷ͠͞ • ࣮ߦεςοϓશ༰ΛѲ͢Δ͜ͱͷ͠͞ • શମͰεςοϓ͕͜Ε͚ͩ͋Δ • Ͳ͜ͷεςοϓͰࣦഊͨ͠ͷ͔ • ॲཧ୯Ґ͕େ͖͘ͳΓ͕ͪ •
ඞવͱ࣮ߦ࣌ؒҾ͖͕ͪ • Ұ෦͚ࣦͩഊͨ࣌͠ɺ࣮ߦͷঢ়ଶޭ? ࣦഊ? -
΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ ͷϐλΰϥεΠον • ϫʔΫϑϩʔΤϯδϯͷಋೖ • ࣮ߦεςοϓશ༰ΛѲ͘͢͠ • ͦΕͱߴʹ౷߹͞Εͨόον࣮ߦڥ͕͋Δͱͳ͓Α͍ • pub/subϞσϧͰରσʔλͷ૿Ճʹର͠εέʔϧͤ͞Δ
• ॲཧ͢Δσʔλ୯ҐΛෳˠ1ͭ • ͍ͭͰʹσʔλετΞঢ়ଶ͕ڽू͞ΕΔ -
// ϫʔΫϑϩʔΤϯδϯͷঢ়ଶ { "domain": "www.example.com", "endpoint": "https://...." } // ͋Δόονͷೖྗ
{ "domain": "www.example.com" } -
// ϫʔΫϑϩʔΤϯδϯͷঢ়ଶ { "domain": "www.example.com", "endpoint": "https://...." } // ͋Δόονͷೖྗ
{ "domain": "www.example.com" } άϩʔόϧঢ়ଶΛҾม͢Δ (όον͔ΒͷมߋෆՄ) -
// ϫʔΫϑϩʔΤϯδϯͷঢ়ଶ { "updateRequired": true, "domain": "www.example.com", "endpoint": "https://...." }
// ͋Δόονͷग़ྗ { "updateRequired": true } -
// ϫʔΫϑϩʔΤϯδϯͷঢ়ଶ { "updateRequired": true, "domain": "www.example.com", "endpoint": "https://...." }
// ͋Δόονͷग़ྗ { "updateRequired": true } όονͷग़ྗΛάϩʔόϧͳঢ়ଶม (વɺग़ྗޙ͔ΒมߋෆՄ) -
΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷ ϐλΰϥεΠον@ͯͳϒϩά • ϫʔΫϑϩʔΤϯδϯ: AWS StepFunctions • ……ͱͦΕΒ͔Β࣮ߦ͞ΕΔAWS Lambda •
pub/sub: DynamoDB TTL Trigger -
࠶: ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ ͷϐλΰϥεΠον • ϫʔΫϑϩʔΤϯδϯͷಋೖ • ࣮ߦεςοϓશ༰ΛѲ͘͢͠ • ͦΕͱߴʹ౷߹͞Εͨόον࣮ߦڥ͕͋Δͱͳ͓Α͍ •
pub/subϞσϧͰରσʔλͷ૿Ճʹର͠εέʔϧͤ͞Δ • ॲཧ͢Δσʔλ୯ҐΛෳˠ1ͭ • ͍ͭͰʹσʔλετΞঢ়ଶ͕ڽू͞ΕΔ -
·ͱΊ • ιϑτΣΞߏஙҰൠͷݪଇ͕͑Δ • άϩʔόϧͳঢ়ଶΛ࣋ͨͳ͍ɾม͑ͳ͍ɾ࣋ͪࠐ·ͤͳ͍ • ॲཧ୯ҐΛͰ͖Δ͚ͩখ͘͞ɺࣦഊΛѲ͘͢͠ • ͜ΕΒΛ࣮ݱ͢ΔͨΊͷҰྫͱͯ͠ •
ϫʔΫϑϩʔΤϯδϯ: AWS StepFunctions • pub/subΛαϙʔτ͢ΔσʔλετΞ: DynamoDB