Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Guzzle Promiseを使った
非同期処理によるAPIコールの高速化

Guzzle Promiseを使った
非同期処理によるAPIコールの高速化

JavaScript で非同期処理を実現する Promise という機構はご存知でしょうか?

今回は「Promise の考え方を PHP で実装した Guzzle Promise」を使って、大量の API コールを高速化したときの経験についてお話してみたいと思います。

Talked:
- http://phpcon.fukuoka.jp/

Sample codes:
- https://github.com/suzuki/guzzle-promise-sample

Norio Suzuki

May 21, 2016
Tweet

More Decks by Norio Suzuki

Other Decks in Technology

Transcript

  1. Think of the system • ࣮૷ݴޠ • PHPͰ͸ແཧʁ • Go΍Scala౳͕ద੾ͩͱͯ͠΋࣮૷ܦݧ͸ͳ͍

    • ίʔϧઌͷAPI • ॳΊͯར༻͢ΔAPI • ΞαΠϯ࣌఺Ͱ͸ಈ࡞͕೺ѲͰ͖͍ͯͳ͍ → ॳΊͯͷݴޠͰॳΊͯͷAPIίʔϧ͢Δඞཁ͕ʂʁ
  2. Other magic methods $response = $client->get('http://localhost/'); $response = $client->delete('http://localhost/'); $response

    = $client->head('http://localhost/'); $response = $client->options('http://localhost/'); $response = $client->patch('http://localhost/'); $response = $client->post('http://localhost/'); $response = $client->put('http://localhost/');
  3. Using Request object use GuzzleHttp\Client; use GuzzleHttp\Psr7\Request; $client = new

    Client(); $request = new Request( 'GET', 'http://localhost:8008' ); $response = $client->send($request);
  4. server.js var http = require('http'); var sleep = require('sleep-async')(); var

    server = http.createServer(); var sleepMiliSecond = 1000; server.on('request', function(req, res) { console.log('Received.'); sleep.sleep(sleepMiliSecond, function() { res.writeHead(200, {'Content-Type': 'text/json'}); res.write('{"status": "ok"}'); res.end(); }); }); server.listen(8008, 'localhost');
  5. sample01.php use GuzzleHttp\Client; $client = new Client(); $response = $client->request(

    'GET', 'http://localhost:8008/' ); echo $response->getBody() . PHP_EOL;
  6. sample02.php for ($i = 0; $i < 10; ++$i) {

    $response = $client->request( 'GET', 'http://localhost:8008' ); echo $response->getBody() . PHP_EOL; }
  7. ղܾํ๏͸ʁ • σʔλΛ෼ׂ͢Δʁ • ෼ׂ਺ͷ਺͚ͩεΫϦϓτΛىಈʁ • ෳ਺ϓϩηεͰ࣮ߦ͢Δʁ • forkͯ͠ؤுΔʁ •

    ඇಉظϦΫΤετ͕͋ͬͨɺ͜ΕͰͳΜͱ͔ͳΔʂʁ → Guzzle ͷ Async ϦΫΤετΛ࢖ͬͯΈΔ͜ͱʹ
  8. sample03.php for ($i = 0; $i < 10; ++$i) {

    $response = $client->requestAsync( 'GET', 'http://localhost:8008/' ); echo $response->getBody() . PHP_EOL; }
  9. sample04.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); // echo $response->getBody() . PHP_EOL; }
  10. Summary of sample04.php • PHP Fatal error͸ग़ͳ͘ͳͬͨ • ࣮ߦʹ͸੒ޭ͍ͯ͠Δʁ •

    αʔόଆʹʮReceived.ʯͷද͕ࣔग़͍ͯͳ͍ • ϦΫΤετ͕ಧ͍͍ͯͳ͍ʁʁ • Promise ͬͯͳʹʁ ͋ͷ Promise ͷ͜ͱʁʁ
  11. Promise overview var promise = new Promise(function(resolve, reject) { //

    Async routines if (isSuccess) { resolve(value); } else { reject(error); } });
  12. Method Chains sample var promise = new Promise(...); promise .then(function(value)

    { if (isSuccess) { resolve(value); } else { reject(error); }; }) .then(function(value) { if (isSuccess) { resolve(value); } else { reject(error); }; }) .catch(function(error) { // handle error }) ; 1SPNJTF 1SPNJTF 1SPNJTF 1SPNJTF then then catch
  13. Guzzle Promise overview use GuzzleHttp\Promise\Promise; $promise = new Promise(function() use

    (&$promise) { // Async routine if ($isSuccess) { $promise->resolve($value); } else { $promise->reject($error); } });
  14. Async Request using Promise use GuzzleHttp\Client; use Psr\Http\Message\ResponseInterface; use GuzzleHttp\Exception\RequestException;

    $client = new Client(); $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ->otherwise(function(RequestException $e) { echo $e->getMessage() . PHP_EOL; }) ; $promise->wait();
  15. sample05.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promise->wait(); }
  16. sample05.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promise->wait(); } Not Async
  17. sample06.php use GuzzleHttp\Promise; $promises = []; for ($i = 0;

    $i < 10; ++$i) { $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }); $promises[] = $promise; } Promise\all($promises)->wait();
  18. sample07.php $promises = []; for ($i = 0; $i <

    10000; ++$i) { $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promises[] = $promise; } Promise\all($promises)->wait();
  19. Summary of sample07.php PHP Fatal error: Allowed memory size of

    134217728 bytes exhausted (tried to allocate 72 bytes)
  20. Why exhausted memory limit ? • $promises ഑ྻʹ10000ݸͷ Promise Φϒ

    δΣΫτ͕ೖͬͨ··ղ์͞Ε͍ͯͳ͍ • ͦΕͧΕͷ Promise ΦϒδΣΫτ͸ঢ়ଶΛอ ͍࣋ͯ͠ΔͷͰͦΕͳΓʹॏ͍σʔλྔ → Կ͔ղܾํ๏͸ʁ
  21. Generator $requests = function() { for ($i = 0; $i

    < 10; ++$i) { yield new Request( 'GET', 'http://localhost:8008/' ); } };
  22. How to use Generator sample08.php use GuzzleHttp\Client; use GuzzleHttp\Psr7\Request; $client

    = new Client(); $requests = function() { for ($i = 0; $i < 10; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; foreach ($requests() as $request) { $response = $client->send($request); echo $response->getBody() . PHP_EOL; }
  23. Pool overview use GuzzleHttp\Pool; $pool = new Pool($client, $requests(), [

    'concurrency' => 5, 'fulfilled' => function ($response, $index) { // when success }, 'rejected' => function ($reason, $index) { // when failed }, ]); Generator
  24. Pool config LFZ UZQF QVSQPTF DPODVSSFODZ JOUFHFS NBYDPODVSSFODZQBSBNFUFS GVMpMMFE DBMMBCMFGVODUJPO

    $BMMXIFO1SPNJTFJTSFTPMWFE SFKFDUFE DBMMBCMFGVODUJPO $BMMXIFO1SPNJTFJTSFKFDUFE
  25. sample09.php $client = new Client(); $requests = function() { for

    ($i = 0; $i < 10; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; $pool = new Pool($client, $requests(), [ 'concurrency' => 5, 'fulfilled' => function(ResponseInterface $response, $index) { echo $response->getBody() . PHP_EOL; }, ]); $promise = $pool->promise(); $promise->wait();
  26. Concurrency = 10 sample10.php $pool = new Pool($client, $requests(), [

    'concurrency' => 10, 'fulfilled' => function(ResponseInterface $response, $index) { echo $response->getBody() . PHP_EOL; }, ]);
  27. sample11.php $client = new Client(); $requests = function() { for

    ($i = 0; $i < 10000; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; $pool = new Pool($client, $requests(), [ 'concurrency' => 1000, 'fulfilled' => function(ResponseInterface $response, $index) { echo sprintf("%5d: %s\n", $index, $response->getBody()); }, ]); $promise = $pool->promise(); $promise->wait();
  28. ࣮Ҋ݅Ͱͷ Pool ར༻ • concurrency = 100 Λ࠾༻ • APIαʔό΁ͷಉ࣌ΞΫηε਺͸ڐ༰ൣғ

    • ϝϞϦͷ࢖༻ྔ΋ڐ༰ൣғ • ໿50ສϦΫΤετΛ30෼͔͔Βͣʹ׬ྃՄೳ → 1࣌ؒ100ສϦΫΤετΛΫϦΞ
  29. PHP side links • Guzzle Documentation • http://docs.guzzlephp.org/en/latest/ • Guzzle

    Promises • https://github.com/guzzle/promises • PHP Documentation: Generator • http://php.net/manual/en/language.generators.overview.php • AWS SDK for PHP • https://docs.aws.amazon.com/aws-sdk-php/v3/guide/
  30. JavaScript side links • JavaScript Promiseͷຊ • http://azu.github.io/promises-book/ •Promise/A+ •

    https://promisesaplus.com/ •ECMAScript 2015 (ES6) • http://www.ecma-international.org/ecma-262/6.0/ •MDN Generator • https://developer.mozilla.org/en-US/docs/Web/JavaScript/ Reference/Global_Objects/Generator
  31. References • Asynchronous API Interaction with Guzzle • https://speakerdeck.com/jeremeamia/asynchronous-api- interaction-with-guzzle

    • JavaScript Promises - Thinking Sync in an Async World • https://speakerdeck.com/kerrick/javascript-promises- thinking-sync-in-an-async-world • guzzleͰඇಉظϦΫΤετΛฒྻॲཧͰ͍ͬͺ͍౤͛Δ • http://hacknote.jp/archives/16095/