Perlでも分散トレーシングしたい！AWS::XRay による解析とその実装 / YAPC::Tokyo 2019

YAPC::Tokyo 2019 のトークです

1. PerlͰ΋෼ࢄτϨʔγϯά͍ͨ͠ʂ AWS::XRay ʹΑΔղੳͱͦͷ࣮૷ ɹ 2019.01.26 YAPC::Tokyo @fujiwara
2. @fujiwara ٕज़෦ SRE github.com/fujiwara sfujiwara.hatenablog.com ISUCON8 ຊબग़୊νʔϜ
3. Game & Community
4. Agenda ෼ࢄτϨʔγϯάͱ͸ AWS X-Ray ͷ঺հ X-Ray ʹΑΔ ISUCON8 ຊબ໰୊ͷղੳྫ AWS::XRay ͷ಺෦࣮૷ ίετͷ࿩
5. ෼ࢄτϨʔγϯάͱ͸
6. ෼ࢄτϨʔγϯάͱ͸ GoogleͷDapper࿦จ1͕΋ͱ (2010೥) ෳ਺ͷݴޠͰߏங͞ΕΔେن໛෼ࢄγεςϜͷύϑΥʔϚϯε Λղੳ͢ΔͨΊʹ࡞ΒΕͨ 1 https://ai.google/research/pubs/pub36356
7. ෼ࢄτϨʔγϯάͷඞཁੑ ෼ࢄγεςϜ(ϚΠΫϩαʔϏε)Ͱ͸ͻ ͱͭͷॲཧ͕ෳ਺ͷίϯϙʔωϯτʹ· ͕ͨΔ ύϑΥʔϚϯεͷ໰୊͕෼͔ΓͮΒ͍ ো֐Λݟ͚ͭΔͷ΋೉͍͠ ਤ͸ Dapper࿦จ 1 ΑΓ 1 https://ai.google/research/pubs/pub36356
8. ෼ࢄτϨʔγϯάͰՄࢹԽ ෳ਺ͷίϯϙʔωϯτʹ·͕ͨͬͯॲཧ ͞Εͨ݁ՌΛՄࢹԽͯ͠೺ѲΛ༰қʹ ֤ίϯϙʔωϯτ͕ݺͼग़͞Εͨॱংɺ λΠϛϯάɺཁ͕ͨ࣌ؒ͠ݟ͑Δ
9. ֓೦ Trace͸ෳ਺ͷSpanΛؚΉ༗޲ඇ८ճάϥϑ5 5 https://github.com/opentracing/specification/blob/master/specification.md
10. ֓೦ Span ͸ҎԼͷཁૉΛ΋ͭ ໊લ ։࢝࣌ࠁ ऴྃ࣌ࠁ Context (ID, Trace-ID, Parent) Tag {Key: Value},... Log {Key: Value},...
11. ෼ࢄτϨʔγϯάͷ࣮૷ͱ࢓༷ Zipkin(OpenZipkin)2: Twitter → OSS 2012 Jeager3: Uber → CNCF OSS 2016 OpenTracing4: ෼ࢄτϨʔγϯάͷ࢓༷ CNCF 2016 AWS X-Ray: AWS͕ఏڙ ϚωʔδυαʔϏε 2016 StackDriver Trace: Google͕ఏڙ ZipkinΫϥΠΞϯτޓ׵ 2017 4 https://opentracing.io/ 3 https://www.jaegertracing.io/ 2 https://zipkin.io/
12. Ͳ͏΍ͬͯτϨʔε͢Δ͔ ΞϓϦέʔγϣϯͰ ΞϓϦέʔγϣϯ(SDK)ͰτϨʔεσʔ λΛ࡞੒ͯ͠సૹ༻ΤʔδΣϯτʹૹ৴ ͔ͦ͜Βαʔό(ίϨΫλ)ʹૹΒΕΔ ! ΞϓϦέʔγϣϯ಺෦ͷڍಈ΋τ ϨʔεͰ͖Δ " ݴޠʹґଘ(SDK)
13. Ͳ͏΍ͬͯτϨʔε͢Δ͔ αʔϏεϝογϡͰ αʔϏεϝογϡͷσʔλϓϨʔϯ(ྫ Envoy) Ͱߦ͏ ! ݴޠʹґଘ͠ͳ͍ " ΞϓϦέʔγϣϯ಺෦ͷڍಈ͸τ ϨʔεͰ͖ͳ͍
14. AWS X-Ray
15. AWS X-Ray AWS ʹΑΔ෼ࢄτϨʔγϯάͷϚωʔδυαʔϏε
16. AWS X-Ray ͷߏ੒ཁૉ6 X-Ray API AWS͕ఏڙ͍ͯ͠ΔAPI X-Ray daemon τϨʔεసૹ༻σʔϞϯ X-Ray SDK X-Ray daemonʹτϨʔεΛૹ৴͢Δͨ ΊͷϥΠϒϥϦ X-Ray console τϨʔεΛՄࢹԽ͢ΔViewer 6 https://docs.aws.amazon.com/ja_jp/xray/latest/devguide/aws-xray.html
17. ϚωʔδυαʔϏεͳͷͰτϨʔεΛૹΔ͚ͩͰ͙͢࢖͑Δ σʔλͷอଘ΍αʔόͷՄ༻ੑΛؾʹ͢Δඞཁ͕શ͘ͳ͍
18. ͲͷΑ͏ʹτϨʔεΛૹ৴͢Δ͔ X-Ray daemon7 ͕ UDP port 2000 Λ listen ΞϓϦέʔγϣϯ(SDK)͔ΒτϨʔεσʔλΛUDPͰ౤͛Δ {"format": "json", "version": 1} { "trace_id": "1-5759e988-bd862e3fe1be46a994272793", "id": "defdfd9912dc5a56", "start_time": 1461096053.37518, "end_time": 1461096053.4042, "name": "MyApp" } 7 https://github.com/aws/aws-xray-daemon
19. ͲͷΑ͏ʹτϨʔεΛૹ৴͢Δ͔ X-Ray daemon ͕όοϑΝϦϯάͯ͠όονͰ X-Ray API ʹૹ৴ X-Ray API ʹಧ͍ͨ͋ͱ͸׬શʹϚωʔδυ σʔλอ࣋ 30೔(ݹ͍ͷ͸ফ͑Δ) τϨʔε਺ͱදࣔ࣌ͷεΩϟϯͨ͠ྔʹΑΔ׬શैྔ՝ۚ
20. X-Ray ͷΞʔΩςΫνϟͷ͍͍ͱ͜Ζ τϨʔεσʔλ͸େྔʹൃੜ͢Δ 1ϦΫΤετॲཧதʹଟ͍ͱ਺ेҎ্ ຖճΞϓϦέʔγϣϯ͔Βಉظ௨৴(TCP) → ϨΠςϯγ૿Ճʹ௚݁͢Δ ಛʹো֐࣌ X-Ray daemon ʹ͸UDPͰ౤͛ͬͺͳ͠ → ΞϓϦέʔγϣϯ΁ͷӨڹ͕ܰඍ X-Ray daemon ͕མ͍ͪͯͯ΋ΞϓϦʹӨڹ͠ͳ͍
21. X-Ray ༻ޠͰ͸ εύϯ → ηάϝϯτ ࠷ॳʹઆ໌ͨ͠Ұൠͷ༻ޠ(εύϯ)ͱ͸ҧ͏͚Ͳಉ͡΋ͷ Ҏ߱͸ X-Ray ͷ࿩ͳͷͰηάϝϯτͱݺͼ·͢
22. AWS X-Ray Λ Perl Ͱ࢖͓͏
23. X-Ray SDK τϨʔεΛ X-Ray daemon ʹૹΔϥΠϒϥϦ AWS͕ఏڙ͍ͯ͠Δͷ͸ Java, Go, Node, Python, Ruby, .NET ͋Εʁࠓ೔͸ YAPC (Yet Another Perl Conference)…
24. ! Perl SDK ͸ެࣜʹ͸ͳ͍ CPAN ʹ΋ݟ౰ͨΒͳ͍ X-Ray API Λୟ͘ϥΠϒϥϦ͸͋Δ (Paws::XRay)
25. ͍͍ͩͨશ෦ Perl ࠷ۙͷ৽͍͠ϓϩδΣΫτ͸ Go ͚ͩͲ௕೥ͷப͸ Perl
26. AWS::XRay Λॻ͖·ͨ͠ ཁ͢Δʹ JSON Λ UDP Ͱ౤͛Ε͹͍͍͚ͩ ͳͷͰ؆୯ 300ߦ͙Β͍͔͠ͳ͍ capture $name, sub {} ͰηάϝϯτΛ࡞ͬͯૹ৴ use AWS::XRay qw/ capture /; capture "myApp", sub { capture "foo", sub { # do something ... capture "bar", sub { # ... }; }; };
27. Plack::Middleware::XRay ૊ΈࠐΉͱτϨʔεΛ։࢝ͯ͘͠ΕΔ Plack Middleware 100ߦ͙Β͍ .psgi Ͱ enable ͢Δ͚ͩ (Φϓγϣϯ͸ޙड़) use Plack::Builder; builder { enable "XRay", name => "myApp"; $app; };
28. AWS::XRay ։ൃܦҢ Perl Ͱ΋ X-Ray ࢖͍͍ͨ… → JSONΛUDPͰ౤͛Δ͚ͩͩ͠؆୯ͩΑͶʁ → Ͱ΋τϨʔε࢓ࠐΉͷΊΜͲ͍͘͞ͳ… → ! Devel::KYTProf ͕औͬͨ݁ՌΛૹΕ͹ʂ
29. Devel::KYTProf::Logger::XRay Devel::KYTProf ͱ͍͏ϓϩϑΝΠϥͷϩΨʔͱͯ͠ಈ࡞ͯ͠ τϨʔεσʔλΛૹ৴ Devel::KYTProf ͸ use ͢Δ͚ͩͰDBIͳͲͷݺͼग़͠Λܭଌ ϩάΛग़ྗͯ͘͠ΕΔศརϓϩϑΝΠϥ use Devel::KYTProf; # your code 315.837 ms [DBI::st] select * from table where name = ? (1 rows) | main:23 1464.204 ms [LWP::UserAgent] GET http://www.hatena.ne.jp/ | main:25
30. Devel::KYTProf::Logger::XRay use Devel::KYTProf::Logger::XRay; Devel::KYTProf->logger("Devel::KYTProf::Logger::XRay"); ͜ΕͰDevel::KYTProfͰܭଌͰ͖Δ΋ͷ͸શͯτϨʔεର৅ʹ DBI, LWP::UserAgent, Furl::HTTP, Cache::Memcached::Fast, MogileFS::Client ࣗ෼Ͱର৅ϞδϡʔϧΛ௥Ճ͢Δ͜ͱ΋Մೳ (Redis::Fastͱ͔)
31. AWS::XRay ։ൃܦҢ Perl Ͱ΋ X-Ray ࢖͍͍ͨ… → JSONΛUDPͰ౤͛Δ͚ͩͩ͠؆୯ͩΑͶʁ → Ͱ΋τϨʔε࢓ࠐΉͷΊΜͲ͍͘͞ͳ… → ! Devel::KYTProf ͕औͬͨ݁ՌΛૹΕ͹ʂ → Logger ࠩ͠ସ͑Ͱ͖Δ KYTProf++ ਖ਼௚ KYTProf ͕ͳ͔ͬͨΒ࡞ͬͯͳ͔͔ͬͨ΋͠Εͳ͍
32. ISUCON 8 ຊબ໰୊ͷղੳ
33. ISUCON is ͳʹ ͍͍ײ͡ʹεϐʔυΞοϓίϯςετͷུ ఏڙ͞Εͨαʔό্Ͱಈ͘WebΞϓϦέʔγϣϯΛߴ଎Խ !!!!!!!!!! ༏উ৆ۚ100ສԁ 2011೥͔Βຖ೥։࠵͞Ε͍ͯΔ (ओ࠵ LINEࣾ) 2018೥ͷ ISUCON 8 Λ DeNA ͞ΜͱΧϠοΫͰग़୊ ࣍ͷ࿮͸ karupanerura ͞ΜʹΑΔ ʮISUCON8༧બ໰୊࡞੒ͷཪଆʯͷτʔΫͰ͢
34. ISUCON 8 ຊબ໰୊ Ծ૝௨՟औҾαʔϏε ISUCOIN ۜߦAPI, ϩάه࿥API͸ӡӦ͕༻ҙͨ͠ ϒϥοΫϘοΫεͰఏڙ͞Εɺhttps Ͱ ϦΫΤετΛૹΔඞཁ͕͋Δ ֎෦APIݺͼग़͠Λ͍͔ʹ্ख͘ॲཧ͢ Δ͔͕ϙΠϯτ
35. ISUCON 8 ຊબ໰୊
36. ISUCON 8 ຊબ໰୊Λ X-Ray Ͱղੳͯ͠ΈΔ
37. Perl ࣮૷ʹ AWS::XRay Λ૊ΈࠐΉ ఏڙ͞Εͨݴޠ࣮૷͸ Go, Perl, PHP, Python, Ruby YAPC ͳͷͰ౰વ Perl ࣮૷Ͱ https://github.com/isucon/isucon8-final
38. X-Ray daemon Λ༻ҙ ॳظ࣮૷͸ docker-compose ͳͷͰద౰ʹ Dockerfile Λ༻ҙ8 FROM amazonlinux RUN yum install -y unzip RUN curl -o daemon.zip https://s3.dualstack.us-east-2.amazonaws.com/ aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-linux-3.x.zip RUN unzip daemon.zip && cp xray /usr/bin/xray ENTRYPOINT ["/usr/bin/xray", "-b", "0.0.0.0:2000", "--local-mode"] EXPOSE 2000/udp 8 https://docs.aws.amazon.com/ja_jp/xray/latest/devguide/xray-daemon-local.html
39. docker-compose.yml services: isucoin: environment: # ུ AWS_XRAY_DAEMON_ADDRESS: 'xray:2000' links: - mysql - xray xray: build: xray environment: AWS_REGION: "ap-northeast-1" AWS_ACCESS_KEY_ID: "000000000000" AWS_SECRET_ACCESS_KEY: "****"
40. AWS::XRay ૊ΈࠐΈ app.psgi ʹ5ߦ௥Ճ + cpanfile # app.psgi use Devel::KYTProf::Logger::XRay; use AWS::XRay; Devel::KYTProf->logger("Devel::KYTProf::Logger::XRay"); builder { enable 'XRay', name => 'isucoin'; # ... # cpanfile requires 'AWS::XRay'; requires 'Plack::Middleware::XRay'; requires 'Devel::KYTProf::Logger::XRay';
41. ϕϯνΛ࣮ߦ͢Δͱ…τϨʔε͕ʂ
49. ϘτϧωοΫൃݟ SELECT * FROM trade ORDER BY id DESC 23ສߦΛಡΈऔͬͯ1ߦ࢖ࣺ͍ͬͯͯͯΔ LIMIT 1 ͢Ε͹ OK ! ʮ͜Ε͸΂ͭʹ෼ࢄτϨʔγϯά͍Βͳ͍ΑͶ?ʯ ! ʮslowlog ݟΕ͹෼͔Δ΍Ζʯ
50. ֎෦APIݺͼग़͠Λղੳ
54. ֎෦ͷHTTP APIݺͼग़͠ͷதͰ஗͍΋ͷΛಛఆ (ग़୊ҙਤͰ) ֎෦API͝ͱʹϨΠςϯγʹࠩΛ෇͚͍ͯͨ ! ʮ͜Ε΋ผʹʰ෼ࢄʱͰ͸ͳ͍ΑͶʯ ! ʮDevel::KYTProf ͰϩάΛग़ͤ͹෼͔Δ΍Ζʯ
55. ֎෦API࣮૷ʹ X-Ray Λ૊ΈࠐΜͰΈΔ (ڝٕͰ͸࣮ࡍ͸ϒϥοΫϘοΫε͚ͩͲ) xray-sdk-go Λ૊ΈࠐΈ // sql.Open -> xray.SQL db, err := xray.SQL("mysql", dsn) // http.Handler Λ xray.Handler Ͱ wrap return xray.Handler( xray.NewFixedSegmentNamer("Isubank"), myHandler, )
56. Perl ଆͷ HTTPϦΫΤετૹ৴΋ capture ݺͼग़͠ݩΛ఻͑ΔͨΊʹ X-Amzn-Trace-ID ϔομΛ෇༩ capture "externalHTTP", sub { my $segment = shift; $res = $self->client->post( $self->endpoint . $p, [ "Content-Type" => "application/json", "Authorization" => "Bearer " . $self->app_id, "X-Amzn-Trace-ID" => $segment->trace_header, ], $body, ); };
57. Isucoin(ΞϓϦ)→Isubank(API)
60. ֎෦APIݺͼग़͠ͷதͰߦΘΕͨSQLΫΤϦ΋ಉ͡ը໘Ͱ
61. ʰ෼ࢄʱτϨʔγϯάʂ
62. ϓϩηεΛ·͕ͨͬͨ௥੻ͷ࢓૊Έ Isucoin(Perl) ϦΫΤετ։࢝࣌ʹ X-Amzn-Trace-ID Λൃߦ 1-5c4902d7-e8fe620cab66e01e57df8a12 1-[timestamp]-[unique_id] sprintf( "1-%x-%s", CORE::time(), unpack("H*", Crypt::URandom::urandom(12)), );
63. ֎෦ϦΫΤετݺͼग़͠ʹϔομΛ෇༩ use AWS::XRay qw/capture/; capture "externalHTTP", sub { # Ͳ͜Ͱ΋ݺ΂Δ(਌͕͍ͳ͚Ε͹RootʹͳΔ) my $segment = shift; $furl->POST( "https://example.com", [ "X-Amzn-Trace-ID" => $segment->trace_header ], $body, ); }; POST / HTTP/1.1 Host: example.com X-Amzn-Trace-ID: Root=1-5c4902d7-e8fe620cab66e01e57df8a12;Parent=851cf9ec07b528c2 ...
64. ϔομΛड৴ͨ͠API(Isubank)͕Ҿ͖ܧ͙ Root=1-5c4902d7-e8fe620cab66e01e57df8a12;Parent=851cf9ec07b528c2 # ࣮ࡍ͸Go͚ͩͲઆ໌ͷͨΊʹPerl AWS::XRay::Segment->new( id => new_id(), trace_id => "1-5c4902d7-e8fe620cab66e01e57df8a12", parent_id => "851cf9ec07b528c2", ); ͦͷτϨʔεΛ X-Ray daemon ΁ૹ৴ → X-Ray consoleͰτϨʔε͕࿈݁ͯ͠දࣔ͞ΕΔ
65. AWS::XRay ͷ಺෦࣮૷
66. τϨʔε͍ͨ͠ͱ͜ΖͰʮݱࡏͷʯRoot΍Parent͕ඞཁ package App; sub main { capture "App::main", sub { # Root my $segment = shift; Model::call($param); # ֎෦ݺͼग़͠Λ൐͏ॲཧ }; }; package Model; sub call() { my $param = shift; # ࣮ࡍʹ֎ΛݺͿͷ͸͕ͩ͜͜ݺͼग़͠ݩͷsegmentΛ஌Βͳ͍ͱRoot͕Θ͔Βͳ͍ $furl->post($url, [ "X-Amzn-Trace-ID" => "????", ], $param); } call($param)Λcall($segment,$param)ʹॻ͖׵͑ͨ͘ͳ͍
67. AWS::XRay Ͱ͸Ҿ਺Λ౉͞ͳ͍͍ͯ͘Α͏ʹͳͬͯΔ package App; sub main { capture "App::main", sub { # Root my $segment = shift; Model::call($param); # ֎෦ݺͼग़͠Λ൐͏ॲཧ }; }; package Model; sub call() { my $param = shift; # ਌ͷ͜ͱ͸஌Βͳ͍͚Ͳͱʹ͔͘ capture ͢Ε͹Α͍ capture "Model::call", sub { my $segment = shift; # ਌͔Βݺ͹Ε͍ͯΕ͹ࣗಈతʹࢠʹͳΔ $furl->post($url, [ "X-Amzn-Trace-ID" => $segment->trace_header, ], $param); }; }
68. Ͳ͏΍࣮ͬͯ૷͞Ε͍ͯΔ͔ Perl ͷຐ๏ͷͻͱͭ local perldoc -f local ͋ͳ͕ͨຊ౰ʹ๬ΜͰ͍Δͷ͸ my ͷํͰ͠ΐ͏; local ͸΄ͱΜ Ͳͷਓʑ͕ʮϩʔΧϧʯͱߟ͑Δ΋ͷͱ ҧ͏͔ΒͰ͢ɻ "local" ͸ϦετΞοϓ͞Εͨม਺Λɺғ͍ͬͯΔϒϩοΫɺ ϑΝΠϧɺeval ͷதͰɺϩʔΧϧͳ΋ͷʹ͠·͢ɻ !!!!!!!
69. local → μΠφϛοΫείʔϓ (ݺͼग़͠ݩͰείʔϓ͕ܾ·Δ) my → ϨΩγΧϧείʔϓ (ιʔείʔυͷࣈ໘Ͱείʔϓ͕ܾ·Δ) package main; our $Foo = "A"; # ύοέʔδม਺ sub foo { print $Foo; } foo(); #= A { local $Foo = "B"; # ͜ͷϒϩοΫ͔Βݺ͹Ε͍ͯΔؒ͸B foo(); #= B } foo(); #= A ϒϩοΫΛൈ͚Δͱࣗಈతʹ໭Δ
70. Կճωετͯ͠ݺͼग़ͯ͠΋ॱ൪ʹ্ॻ͖ͯ͠ɺ΋ͲΕ͹෮ݩ foo(); #= A { local $Foo = "B"; foo(); #= B { local $Foo = "C"; foo(); #= C { local $Foo = "D"; foo() #= D } foo(); #= C } foo(); #= B } foo(); #= A
71. AWS::XRay Ͱ͸۩ମతʹ͸͜Μͳײ͡ ݱࡏͷ Trace-ID, Segment-ID Λ local Ͱอ࣋ package AWS::XRay; our $TRACE_ID; # Root our $SEGMENT_ID; # ݱࡏͷSegmentID sub capture { my ($name, $code) = @_; # $TRACE_ID͕ະఆٛͳΒRootͳͷͰ৽نIDൃߦ local ʹ local $TRACE_ID = $TRACE_ID // new_trace_id(); # ৽͍͠ηάϝϯτΛ࡞Δ # $SEGMENT_ID ͕͋Ε͹ parent_id => $SEGMENT_ID ʹͳ͍ͬͯΔ my $segment = AWS::XRay::Segment->new({ name => $name, type => $SEGMENT_ID ? "subsegment" : undef, parent_id => $SEGMENT_ID, });
72. local $SEGMENT_ID = $segment->{id}; # ৽͍͠ηάϝϯτIDΛ্ॻ͖ͯ͠localʹଋറ # ίʔυΛ࣮ߦ # ͜ͷதͰ͸ $SEGMENT_ID ͕ઃఆ͞Ε͍ͯΔͷͰ # ͞Βʹ capture ͨ͠Β $segment ͷࢠڙʹͳΔ $ret = $code->($segment); $segment->close(); # ηάϝϯτσʔλૹ৴ return $ret; # $segment ͸ফ͑Δ(my͔ͩΒ) # $SEGMENT_ID ΋ݺͼग़࣌͠ͷ஋ʹ໭Δ # RootͳΒ $TRACE_ID ΋ফ͑Δ(ݩ͕ະఆ͔ٛͩΒ) }
73. Perl ʹ͓͚Δ local ͦͷϒϩοΫ(౳)͔Βݺͼग़͞Ε͍ͯΔ͚ؒͩม਺ͷ஋Λมߋ ྫ֎Ͱ΋ԿͰ΋ൈ͚ͨΒݩʹ໭Δ Կஈ֊Ͱ΋ωετͰ͖Δ ݴޠͰαϙʔτ͞Ε͍ͯΔͷͰʮ࣮֬ʹʯ໭Δ Perl ࠷ߴศརʂʂ
74. ༨ஊ Go Ͱ X-Ray SDK Λ࢖͏৔߹ ϦΫΤετͰൃੜͨ͠ context.Context ΛͲ͜·Ͱ΋ Ҿ਺ʹ౉͍͔ͯ͠ͳ͍ͱτϨʔεͰ͖ͳ͍ ్தʹ ctx ΛҾ͖ܧ͛ͳ͍ϥΠϒϥϦ(ORMͱ͔)͕ڬ·Δͱ (Isubank ΋࣮ࡍ͸40ՕॴҎ্ ctx Λ౉͢Α͏ʹॻ͖׵͑ͨ)
75. ! ίετͷ࿩
76. ! ίετͷ࿩ ʮX-Ray ͸Αͦ͞͏͚ͩͲɺ͓ߴ͍ΜͰ͠ΐ͏…?ʯ ௚઀ίετ = X-Ray ར༻ྉۚ ؒ઀ίετ = ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ (݁Ռతʹ࢖༻Ϧιʔε͕૿͑ΔɺϢʔβମݧ͕ଛͳΘΕΔͳͲ)
77. ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ τϨʔεΛ૊ΈࠐΉ͜ͱʹΑͬͯͲΕ͙Β͍஗͘ͳΔ͔ • 1ॲཧͰ100ݸͷ capture (ίʔυ͸ۭ sub{}) Λੜ੒ • αϯϓϦϯά͠ͳ͍ͰશͯͷτϨʔεΛૹ৴ Λ 500/sec ఔ౓࣮ߦͰ͖Δ (macbook pro 13 2017) ͭ·Γ1ॲཧ͋ͨΓ2msఔ౓ͷΦʔόʔϔου τϨʔεΛૹ৴͠ͳ͍(captureΛωετ͢Δ͚ͩ)ͳΒ 10,000/sec, 0.1msఔ౓ͷΦʔόʔϔου
78. ΞϓϦέʔγϣϯͷύϑΥʔϚϯεྼԽ ͍͍ͨͯͷWebΞϓϦέʔγϣϯʹରͯ͠͸໰୊ͳ͍଎౓ ࣮ࡍʹ͸αϯϓϦϯάΛߦ͏ͷͰߋʹӨڹ͸ܰඍ αϯϓϦϯά1%Ͱຊ൪౤ೖͯ͠ύϑΥʔϚϯε΁ͷӨڹͳ͠ ࣮ࡍͷܭଌίʔυ͸ xt/01_overhead.t10 ࢀর 10 https://github.com/fujiwara/AWS-XRay/blob/master/xt/01_overhead.t
79. X-Ray ར༻ྉۚ ຖ݄ͷແྉ࿮ τϨʔεͷه࿥͸ 10 ສճ·Ͱແྉ τϨʔεͷऔಘͱεΩϟϯ͸߹Θͤͯ 100 ສճ·Ͱແྉ ௥ՃͰ τϨʔεͷه࿥ 100 ສ݅͋ͨΓ 5 USD τϨʔεͷऔಘͱεΩϟϯΛ߹Θͤͯ 100 ສ݅͋ͨΓ 0.50 USD
80. ࣮ࡍͲΕ͙Βֻ͍͔Δͷʁ ฏۉ 100 req/sec ͷ৔߹ 100 * 86,400 * 30 / 1,000,000 * 5 USD = 1,296 USD ฏۉ 1,000 req/sec ͳΒ 12,960 USD / month શͯͷϦΫΤετʹ͍ͭͯه࿥͢Δͷ͸ίετ͕ߴ͍
81. αϯϓϦϯά͢Δ ద౰ʹαϯϓϦϯά͢Ε͹܏޲͸௫ΊΔ ͋·Γൃੜ͠ͳ͍Τϥʔɺslowdown ࣌ͷௐࠪʹ͸࢖͍ͮΒ͍
82. αϯϓϦϯάͷ޻෉ ྫɿ1ඵ͝ͱʹ࠷ॳͷϦΫΤετͱ ௥ՃϦΫΤετͷ5%Λه࿥ (ެࣜSDKͷσϑΥϧτઃఆ) 100 req/sec ͷ৔߹ 1 + 100 * 5% = 6 req/sec ෼͕ه࿥͞ΕΔ ϦΫΤετ͕গͳͯ͘΋ 1 req/sec ͸࣮֬ʹه࿥ ެࣜ SDK ͸ X-Ray console ͔ΒαϯϓϦϯάϧʔϧΛઃఆՄ (AWS::XRay Ͱ͸ະରԠ…)
83. X-Ray console ͰͷαϯϓϦϯάઃఆ
84. Plack::Middleware::XRay ͰͷαϯϓϦϯάઃఆ ݻఆϨʔτ enable "XRay" name => "myApp", sampling_rate => 0.05; # 5%
85. αϯϓϦϯάϧʔϧΛίʔυͰهड़ PSGI $env Λड͚औͬͯਅِΛฦ͢ίʔυΛࢦఆͰ͖Δ enable "XRay" name => "myApp", sampler => sub { my $env = shift; state %paths; if ( $paths{$env->{PATH_INFO}++ == 0 ) { # ಛఆͷPATHΛॳΊͯड৴ͨ͠৔߹͸ඞͣه࿥ return 1; } rand() < 0.05; # ͦΕҎ֎͸ 5% };
86. Τϥʔ΍஗͍ϦΫΤετ͸ඞͣه࿥͍ͨ͠ 5% αϯϓϦϯάˠ ΤϥʔΛه࿥Ͱ͖Δ֬཰΋5% શϦΫΤετͰϝϞϦ্ʹτϨʔεΛอଘ͓ͯ͘͠ ϨεϙϯεΛฦ࣌͢఺ͰτϨʔεΛૹ৴͢Δ͔Ͳ͏͔ΛܾΊΔ enable "XRay" name => "myApp", response_filter => sub { my ($env, $res, $elapsed) = @_; # 5xx OR 1ඵҎ্ֻ͔ͬͨ৔߹ʹૹ৴ return $res->[0] >= 500 || $elapsed >= 1; };
87. ·ͱΊ
88. ·ͱΊ ෼ࢄτϨʔγϯά͸ύϑΥʔϚϯεɺো֐ղੳʹେม༗༻ ର ISUCON ༻ܾઓฌثͱͯ͠΋༗༻ AWS X-Ray ͸ϚωʔδυͰಋೖ͕༰қ αϯϓϦϯάઃఆʹ͸ཁ஫ҙ ! Perl SDK ͸ͳ͔ͬͨͷͰ AWS::XRay Λ࡞ͬͨ पลϞδϡʔϧؚΊͯ΋਺ඦߦͰ͍͍͔Μ͡ʹ