Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Advanced API Calling in Go
Search
Seiji Takahashi
August 22, 2017
Programming
6
2.9k
Advanced API Calling in Go
Go 1.9 Release Party in Tokyo での発表資料です by @__timakin__
Seiji Takahashi
August 22, 2017
Tweet
Share
More Decks by Seiji Takahashi
See All by Seiji Takahashi
Go Backends for frontends with GraphQL and gRPC
timakin
6
3.7k
Design Pattern for Image and Text Composition in Go
timakin
5
6.4k
Golang API Testing the HARD way
timakin
13
6.3k
Head First Golang Image Package
timakin
2
9.8k
React Native Beyond Prototype
timakin
2
1.6k
Performance Optimization on Google AppEngine
timakin
5
6.1k
testcache.pdf
timakin
1
120
How Go cache
timakin
1
61
How Go cache tests
timakin
1
2.9k
Other Decks in Programming
See All in Programming
What you can do with Ruby on WebAssembly
kateinoigakukun
0
160
Scala におけるコンパイラエラーとの付き合い方
chencmd
2
420
令和トラベルにおけるLLM活用事例:社内ツール開発から得た学びと実践
ippo012
0
150
Amazon BedrockでサーバレスなAIお料理ボットを作成する!!
tosuri13
0
220
LangChainでWebサイトの内容取得やGitHubソースコード取得
shukob
0
160
LangChainの現在とv0.3にむけて
os1ma
4
910
オートマトン学習しろ / Do automata learning
makenowjust
3
130
Patched fetch did not work
quramy
3
180
GenU導入でCDKに初挑戦し、悪戦苦闘した話
hideg
0
160
メモリ最適化を究める!iOSアプリ開発における5つの重要なポイント
yhirakawa333
0
410
GoのIteratorに詳しくなってしまう
inatonix
1
200
Scala アプリケーションのビルドを改善してデプロイ時間を 1/4 にした話 | How I improved the build of my Scala application and reduced deployment time by 4x
nomadblacky
1
180
Featured
See All Featured
It's Worth the Effort
3n
182
27k
The Cost Of JavaScript in 2023
addyosmani
42
5.6k
Statistics for Hackers
jakevdp
794
220k
Designing the Hi-DPI Web
ddemaree
278
34k
Build The Right Thing And Hit Your Dates
maggiecrowley
30
2.3k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
24
610
Documentation Writing (for coders)
carmenintech
65
4.3k
Fontdeck: Realign not Redesign
paulrobertlloyd
80
5.1k
Facilitating Awesome Meetings
lara
49
5.9k
Pencils Down: Stop Designing & Start Developing
hursman
119
11k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.2k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
123
18k
Transcript
Advanced API Calling in Go @__timakin__ / Go 1.9 Release
Party in Tokyo
ࣗݾհ • Github: timakin / Twitter: @__timakin__ • ŧŔŕŪ Go
Ŭ Contribute ŢŦ • Gunosy ৽نࣄۀ։ൃࣨ ॴଐ • Go / Swift • Recently Contributed / Created • giginet/xcprofiler • mercari/gaurun • timakin/gopli • timakin/ssm2env • timakin/md2mid • timakin/gonvert
Copyright© Gunosy Inc. All Rights Reserved 3 Go / Python
ΤϯδχΞืूத ▶https://gunosy.co.jp/recruit/ Gunosyɺ౦ژେֶʹ௨͏3ਓͷֶੜͷ ʮใΛੈքதͷਓʹ࠷దʹಧ͚͍ͨʯͱ͍͏͍͔Β࢝·Γ·ͨ͠ɻ ౦ূϚβʔζ্ɺຊώϧζͷΦϑΟεҠసΛܦͯɺ େ͖͍ͯ͘͠ΔձࣾͰ׆༂͍ͨ͠ϝϯόʔΛืू͍ͯ͠·͢ɻ
ࠓͷςʔϚ
GoͰAPI Client࡞Δͱ͖ͷ ΧελϚΠζྫ
ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ
ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ
ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ
ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ
ࡢࠓͷAPI Client։ൃ • APIΛୟ͍ͯදࣔ͢ΔCLIπʔϧΛ࡞Δ • ։ൃͯ͠ΔAPI͔ΒͲ͔ͬͷAPIΛݺͼ͍ͨ • SDKܦ༝ͰσʔλΛऔ͖͍ͬͯͨ • Proxy࡞Γ͍ͨ
௨ৗͷAPIϦΫΤετ
௨ৗͷϦΫΤετ
ಠࣗAPI Client
ಠࣗͷAPI ClientΛఆٛ͢Δ
ಠࣗͷAPI ClientΛఆٛ͢Δ API ClientͷߏମΛఆٛ͢Δ
ಠࣗͷAPI ClientΛఆٛ͢Δ ίϯετϥΫλΛॻ͘
ಠࣗͷAPI ClientΛఆٛ͢Δ ڞ௨ϝιουΛఆٛ͢Δ
ಠࣗͷAPI ClientΛఆٛ͢Δ Ϧιʔε͝ͱͷAPI ίʔϧΛఆٛ͢Δɻ ͜ͷล௨ৗͷrepositoryͰͬͯΔ͜ͱͱ ͦ͜·ͰมΘΒͳ͍ɻ
goroutine
goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳճAPIίʔϧ ͢Δ߹ͪΖΜgoroutine͍͍ͨ
goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳճAPIίʔϧ ͢Δ߹ͪΖΜgoroutine͍͍ͨ • ྫʣtimakin/ssm2env
goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳճAPIίʔϧ ͢Δ߹ͪΖΜgoroutine͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter
Store)ʹอଘ͞Ε͍ͯΔ ಛఆͷKeyʹͻͮ͘ValueΛऔ͖ͬͯͯɺ ݁ՌΛmapʹೖΕ͍ͨ
goroutineͰͷฒߦcall • Goॻ͍ͯΔΜ͔ͩΒɺෳճAPIίʔϧ ͢Δ߹ͪΖΜgoroutine͍͍ͨ • ྫʣtimakin/ssm2env • AWS (EC2 Parameter
Store)ʹอଘ͞Ε͍ͯΔ ಛఆͷKeyʹͻͮ͘ValueΛऔ͖ͬͯͯɺ ݁ՌΛmapʹೖΕ͍ͨ • Ͱ10ݸ͔ͣͭ͠औಘͰ͖ͳ͍
goroutineͰͷฒߦcall AWS ClientΛ࣋ͬͨߏମΛఆٛ͢Δ
goroutineͰͷฒߦcall AWS SSMʢύϥϝʔλετΞʣ͔Β औͬͯདྷͨValueΛmapʹͯ͠ฦ͢ϝιουΛఆٛɻ
goroutineͰͷฒߦcall x/sync/errgroupͰ ɾContextͷϋϯυϦϯά ɾϦΫΤετࣗମͷΤϥʔॲཧ ɾgoroutineʹΑΔฒߦϦΫΤετ Λཧ͢Δɻ
goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛͪड͚Δchannelʢࠓճ”c”ʣΛ༻ҙ͢Δ
goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛͪड͚Δchannelʢࠓճ”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ ɹɾ͜͏͍͏ׂchannelܦ༝Ͱ݁ՌΛड͚औΔͱ ɹɹͦͷ··rangeʹͤͯศར
goroutineͰͷฒߦcall ɾ֤APIίʔϧͷ݁ՌΛͪड͚Δchannelʢࠓճ”c”ʣΛ༻ҙ͢Δ ɾ10ݸͣͭʹׂͨ͠ϦΫΤετ༻ͷkeyҰཡΛ༻ҙ͢Δ ɹɾ͜͏͍͏ׂchannelܦ༝Ͱ݁ՌΛड͚औΔͱ ɹɹͦͷ··rangeʹͤͯศར ɾeg.Go෦ͰฒߦॲཧΛߦ͏
goroutineͰͷฒߦcall ɾޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖͢
goroutineͰͷฒߦcall ɾޭϨεϙϯεͳΒ݁ՌΛchannelʹҾ͖͢ ɾcontextͷλΠϜΞτ௨Λड͚औͬͨΒͦͷ࣌Ͱऴྃ
goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ
goroutineͰͷฒߦcall ɾeg.Go͔ΒΤϥʔ͕ฦͬͯདྷͨΒclose͢Δ ɾchannelܦ༝Ͱؼͬͯདྷͨ݁ՌΛॱ࣍mapʹ٧ΊΔ
goroutineͰͷฒߦcall x/sync/errgroupͰ ɾContextͷϋϯυϦϯά ɾϦΫΤετࣗମͷΤϥʔॲཧ ɾgoroutineʹΑΔฒߦϦΫΤετ Λཧ͢Δɻ
Configuration
ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ͍ͨͤͨ
ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ
ϦΫΤετ༻ͷৄࡉͳConfig • http.ClientʹΧελϜͳઃఆΛ͍ͨͤͨ • MaxIdleConnɺTimeoutͳͲ • ྫʣmercari/gaurun
ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮Λ࣋ͬͨߏମ)ʹɺ ɹTimeoutConnͷઃఆΛ͢
ϦΫΤετ༻ͷৄࡉͳConfig ɾhttp.Transport(௨৴ॲཧΛ࣮ࡍʹߦ͏࣮Λ࣋ͬͨߏମ)ʹɺ ɹTimeoutConnͷઃఆΛ͢ ɾhttp.ClientͷϑΟʔϧυʹTransportΛͯ͠ΫϥΠΞϯτੜ
Proxy
Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖͍͑ͨ
Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠
Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹΓସ͍͑ͨ
Proxy • API ϦΫΤετͷࡍʹυϝΠϯॻ͖͍͑ͨ • ௨৴ΛΩϟογϡ͓͖͍ͯͨ͠ • SSLɺHTTP2ͳͲΛΑ͠ͳʹΓସ͍͑ͨ • ྫʣkazeburo/chocon
Proxy
Proxy • GoͷHTTPϦΫΤετͷߏཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε
Proxy • GoͷHTTPϦΫΤετͷߏཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport
• http.RoundTripper Λ࣮ͨ͠ߏମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮Λ࣋ͭ
Proxy • GoͷHTTPϦΫΤετͷߏཁૉ • http.RoundTripper • HTTPͷτϥϯβΫγϣϯΛ࣮ߦ͠ɺreq/resΛॲཧ͢Δ ͨΊͷΠϯλʔϑΣʔε • http.Transport
• http.RoundTripper Λ࣮ͨ͠ߏମͰɺ௨৴ʹඞཁͳ ઃఆͱɺRoundTripϝιουͷ࣮Λ࣋ͭ • http.Client • http.Transport Λ෦ʹ࣋ͬͨɺHTTPϓϩτίϧͰͷ௨ ৴Λ͓͜ͳ͏ͨΊͷૉͷΫϥΠΞϯτ࣮ • func (c *Client) Do(…) தͰRoundTripΛݺΜͰ͍Δ
Proxy • kazeburo/choconͰ…
Proxy ϦΫΤετ༰ͷॻ͖͑ɺRoundTripperΛ࣋ͬͨߏମ
Proxy ϦΫΤετ༰ͷॻ͖͑ʢHostͷஔɺBodyͷcopyʣ
Proxy ɾServeHTTPΛ࣮ ɾProxy͕ड͚औͬͨresponseΛɺAPIClient͚ʹcopyͯ͠ ɹฦ٫ͯ͋͛͠Δ
Retry
Retry • ϦΫΤετʹࣦഊͯ͠ɺࢦఆճ·Ͱ ϦτϥΠ͍ͨ͠
Retry • ϦΫΤετʹࣦഊͯ͠ɺࢦఆճ·Ͱ ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go
Retry • ϦΫΤετʹࣦഊͯ͠ɺࢦఆճ·Ͱ ϦτϥΠ͍ͨ͠ • ྫʣaws-sdk-go
AWS-SDK-Go • NinjaΈ͍ͨͳίʔυ͍ͬͺ͍͋Δ
AWS-SDK-Go ಠࣗRetryerΛఆٛ͢Δ
AWS-SDK-Go WithRetryerͰɺΧελϜϦτϥΠϠʔΛઃఆͯ͋͛͠Δɻ ʢcontextύοέʔδͷWithValueతͳʣ
AWS-SDK-Go Requestੜ࣌ʹRetryerΛҾ͖͢
AWS-SDK-Go ϦτϥΠՄೳ͔Ͳ͏͔ΛνΣοΫͯ͠ɺtrueͳΒΧϯτ૿Ճ
AWS-SDK-Go νΣοΫޙɺϦτϥΠՄೳͳΒϦΫΤετ༰Λίϐʔ
ସखஈ • ಠࣗRetryerΛఆٛͤͣʹɺcenkalti/backoffͳͲͷ exponential backoff࣮Λ͏ํ๏͋Γ
ྫʣgo-datadog-api
Paging
Paging • ϖʔδϯά༻ͷAPIΛ࡞ΔͷͰͳ͘ɺ ϖʔδϯάରԠͷAPIʹɺҙͷϖʔδ·Ͱ ϦΫΤετΛ͛ଓ͚͍ͨ • ྫʣEC2ύϥϝʔλʔετΞʹอଘ͞Εͨ ΩʔͷҰཡΛऔಘ͍ͨ͠ɻ͕ɺ 10ݸ͔ͣͭ͠ฦͬͯ͜ͳ͍…
AWS-SDK-Go • ࠶Ninja
Paging request.Paginationʂʂʂʂ ϦΫΤετͷstructʹ߹Θͤͯɺϖʔδϯάॲཧͷ࣮Λ࣋ͭ
Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ͢Δ
Paging ϖʔδϯάՄೳͳΒɺ࣍ͷϦΫΤετΛੜ͢Δ Ҿ͕͗͢ΔͷͰݟ͑ͳ͍͕ɺ ίʔϧόοΫ͕ઃఆͯ͋ͬͯ͠ɺ ݁Ռͱʮ࠷ޙͷϖʔδ͔ʯ ͷϑϥάΛฦ͢
·ͱΊ
·ͱΊ • ੈͷதʹ৭ʑͳAPI Client͕͍Δ ΈΜͳҧͬͯΈΜͳ͍͍ • ඞཁͳΒΧελϜClientɺͳ͍͠ϦΫΤετ ཧ༻ͷstructΛ࣮͢Δ • aws-sdk-go͕͢͞ʹΓ͗͢ͳͷͰɺ
ͪΐ͏Ͳ͍͍ϋϯυϥΛ࣮͠·͠ΐ͏
Thank you!