Consuming APIs: reporting from the trenches (PHPBenelux Conference 2019)

Integrating multiple API endpoints into a single application can be challenging. In this talk I will go over a lot of problems and how the can be solved. Going from easy authentication to locally caching calls via middlewares and using webhooks (or callbacks) for notification, this talk covers it all (or at least tries to).

Transcript

1. Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group jachim.coudenys@combell.group @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj
2. APIs? Web API Request - Response Content-Type: * SOAP
3. file_get_contents() fopen()
4. <?php $response = json_decode( file_get_contents( 'https://api.meetup.com/php-wvl' ) ); dump($response);
5. {#3 +"id": 12568822 +"name": "PHP-WVL" +"status": "active" +"link": "https://www.meetup.com/php-wvl/" +"urlname": "php-wvl" +"description": "<p>A PHP Usergroup for all PHP devs based in West-Vlaanderen.</p>" +"created": 1390997367000 +"city": "Torhout" +"untranslated_city": "Torhout" +"country": "BE" +"localized_country_name": "Belgium" +"localized_location": "Torhout, Belgium" +"state": "" +"join_mode": "open" +"visibility": "public" +"lat": 51.07 +"lon": 3.1 +"members": 427 +"organizer": {#2 +"id": 84915262
6. <?php $response = json_decode( file_get_contents( 'https://api.meetup.com/mqskjdfqsdjjfaoijqmodiffqsjdf' ) ); dump($response);
7. failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found PHP Warning: file_get_contents(https://api.meetup.com/mqskjdfqsdjjfaoijqmodiffqsjdf): failed to open stream in 01/2_file_get_contents_exceptions.php on line 9 PHP Stack trace: PHP 1. {main}() 01/2_file_get_contents_exceptions.php:0 PHP 2. file_get_contents() 01/2_file_get_contents_exceptions.php:9 PHP Fatal error: Uncaught TypeError: json_decode() expects parameter 1 to be string, boolean given in 01/ Stack trace: #0 01/2_file_get_contents_exceptions.php(9): json_decode(false) #1 {main} thrown in 01/2_file_get_contents_exceptions.php on line 9
8. allow_url_fopen This option enables the URL-aware fopen wrappers that enable accessing URL object like files. Default wrappers are provided for the access of remote files using the ftp or http protocol, some extensions like zlib may register additional wrappers.
9. We need something more robust
10. Jachim Coudenys
13. Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs Consuming APIs reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches reporting from the trenches
14. Combell Internal Project Evolution in project Started small, escalated quickly Average of 120 API calls per request
15. guzzlehttp/guzzle
16. Guzzle is a PHP HTTP client that makes it easy to send HTTP requests and trivial to integrate with web services.
17. Simple interface for building query strings, POST requests, streaming large uploads, streaming large downloads, using HTTP cookies, uploading JSON data, etc... Synchronous and asynchronous requests. Uses PSR-7 interfaces for requests, responses, and streams. This allows you to utilize other PSR- 7 compatible libraries with Guzzle. No hard dependency on cURL, PHP streams, sockets, or non-blocking event loops. Middleware system allows you to augment and compose client behavior.
18. Guzzle basics Define request headers Inspect response headers Stream body Exception handling
19. php 02/*.php
20. Guzzle abstracted Connection class Abstract away specific calls Add multiple API endpoints
21. php 03/*.php
22. logic
23. Authentication Every API has its own way of auth Most of them use Authorization header getToken() in subclasses
24. php 04/*.php
25. Traceability We have a 'generic' request method Let's add logging And keep track of performance
26. php 05/*.php
27. Middleware
28. https://stackphp.com/img/onion.png
29. HTTP message interfaces Request messages Response messages Headers PSR-7
30. Server use Tari\ServerMiddlewareInterface; use Tari\ServerFrameInterface; use Psr\Http\Message\ServerRequestInterface; use Psr\Http\Message\ResponseInterface; class Foo implements ServerMiddlewareInterface { public function handle( ServerRequestInterface $request, ServerFrameInterface $frame ): ResponseInterface { if ($this->isBadRequest($request)) { return $frame->factory() ->createResponse("Bad Request", 400); } return $frame->next($request);
31. Server A/B Testing Debugging Caching CORS CSRF Protection HTTP Basic Auth OAuth 2.0 OpenID Rate Limiting Referrals IP Restriction
32. Client function (callable $handler) { return function (RequestInterface $request, array $options) use ($handler) { return $handler($request, $options); }; }
33. StackPHP
34. HTTP Server Request Handlers PSR-15
35. Guzzle Handlers and Middleware
36. php 06/*.php
37. GuzzleHttp\Middleware
38. https://www.google.be/search? q=guzzle%20middleware
39. Local caching layer and rfc7234 Cache Docs
40. https://www.sitepoint.com/getting-started-with-varnish/
41. Local caching layer ModerateCacheStrategy * kevinrob/guzzle-cache-middleware PrivateCacheStrategy PublicCacheStrategy GreedyCacheStrategy
42. \GuzzleHttp\Middleware ::mapResponse Add mapResponse middleware (so caching works as expected) 200 -> 404 403 500 -> 404 etc...
43. Automatically retries HTTP requests when a server responds with a 429 or 503 status Sets a retry delay based on the Retry-After HTTP header, if it is sent, or automatically backs off exponentially if no Retry-After header is sent Optionally retries requests that time out (based on the connect_timeout or timeout options) Set an optional callback when a retry occurs (useful for logging/reporting) Specify a maximum number of retry attempts before giving up (default: 10) caseyamcl/ guzzle_retry_middleware
44. \GuzzleHttp\Middleware ::mapResponse Add mapResponse middleware (so retry works as expected) 500 -> 429 500 -> 503 etc...
45. Webhooks / Callbacks Don't poll Register callback url (https://app.local/callback/34dl2F4v) https://swagger.io/docs/specification/callbacks/
46. @coudenysj jachim.coudenys@combell.group Thank you https://joind.in/talk/f71d9