Map, Filter, and Reduce in PHP

Even though Map Reduce was created for parallelization, which PHP does not natively support, the syntax is supported. In this talk I will outline how to use map, filter, and reduce with PHP collections and the benefits you can gain over traditional for loops.

In this talk we will explore different methods of processing and manipulating collections of data giving you new tools in your developer’s toolbox

Transcript

1. @DerekB_WI Map, Filter, and Reduce in PHP Presented by Derek Binkley
2. @DerekB_WI About Me • Father of 3, Husband • Software Consultant with Spark Labs • PHP, Java, Groovy, JavaScript … • He/Him/His
4. @DerekB_WI
5. @DerekB_WI
6. @DerekB_WI Google in 2004 • MapReduce: Simplified Data Processing on Large Clusters, Jeffrey Dean and Sanjay Ghemawat
7. –Oscar Stiffelman - early Google engineer “Only when something is simple can you make it really efficient”
9. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457, 111233, 6);
10. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457, 111233, 6); List<Integer> filteredNumbers = (List<Integer>) numbers.stream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList());
11. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457, 111233, 6); Integer sum = numbers.stream() .reduce(0, (subtotal, element) -> subtotal + element);
12. @DerekB_WI Java Data Streams List<Integer> numbers = Arrays.asList(5, 22, 457, 111233, 6); List<Integer> filteredNumbers = (List<Integer>) numbers.parallelStream() .filter(num -> (int)num > 10) .sorted() .map(num -> (int)num * -1) .collect(Collectors.toList()); Integer sum = numbers.parallelStream() .reduce(0, (subtotal, element) -> subtotal + element);
14. @DerekB_WI JavaScript Promise function loadImage() { let imageUrl = 'https://live.staticflickr.com/ 65535/47813616341_6063c7b420_b_d.jpg'; fetch(imageUrl) .then((data) => { // Save image and display ... }) .catch((error) => { // Alert user of error ... }); }
16. @DerekB_WI Benefits • No Parallel Processing, So what else? • Functional, no side-effects • Easy to read • Single Responsibility • Reduce boilerplate
18. @DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = []; foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }
19. @DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {
 return $place['state'] == 'AK';
 }
 );
20. @DerekB_WI Filter $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 function($place) {
 return $place['state'] == 'AK';
 }
 );
21. @DerekB_WI Filter - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_filter($placesArray,
 fn($place) => $place['state'] == 'AK'
 );
23. @DerekB_WI Traditional Foreach $placesArray = $reader->getFileContents();
 
 $filteredLocations = []; foreach ($placesArray as $place) {
 if ($place['state'] == 'AK') {
 $filteredLocations[] = new Place($place);
 }
 }
24. @DerekB_WI Map $placesArray = $reader->getFileContents(); $modifiedList = array_map( function($place) {
 return PlaceMapper::parse($place);
 }, 
 $placesArray);
25. @DerekB_WI Map - 7.4 $placesArray = $reader->getFileContents(); $filteredAlaskaList = array_map(
 fn($place) => PlaceMapper::parse($place), 
 $placesArray);
27. @DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance" => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";
28. @DerekB_WI Reduce $myTrips = [
 new Trip("name" => "school", "distance" => 1.5),
 new Trip("name" => "the store", "distance" => 2.5)]; $totalDriven = array_reduce($myTrips,
 function ($total, $trip) {
 return $total + $trip->distance;
 },
 0 // initial
 ); echo "I biked {$totalDriven} kilometers today.";
29. @DerekB_WI Reduce - 7.4 $totalDriven = array_reduce($myTrips,
 fn($total, $trip) => $total + $trip->distance,
 0);
30. @DerekB_WI Traditional Loop for ($i = 0; $i < count($placesArray); $i++) {
 $stateName = $placesArray[$i]['state']; if (!array_key_exists($stateName, $stateCounters)) 
 {
 $stateCounters[$stateName] = 0;
 } $stateCounters[$stateName]++; }
31. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
32. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
33. @DerekB_WI Reduce $stateCounters = array_reduce($placesArray,
 function($totals, $location) {
 $totals[$location['state']] =
 isset($totals[$location['state']])
 ? $totals[$location['state']] + 1 : 1;
 return $totals;
 });
34. @DerekB_WI Use an outside variable $searchWord = 'Bridge'; $placesWithBridges = array_filter($placesArray,
 function($location) use($searchWord) {
 return strpos($location['name'], $searchWord);
 });
35. @DerekB_WI Patterns ApiPlacesController extends AbstractController { … }
36. @DerekB_WI Patterns /**
 * route - /api/place
 */ public function allPlaces() {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, $places); return convertToJson($placesDtos); }
37. @DerekB_WI Patterns /**
 * route - /api/user/$userId/place
 */ public function placesByUser($userId) {
 $places = $this->repository->retrieveAll(); 
 $placesDtos = array_map(
 function($place) {
 return PlacesTransformer::toDTO($place);
 }, array_filter($places, function($place) {
 return LegacyMiddleware::shouldInclude($place);
 }
 )); return convertToJson($placesDtos);
 }
39. @DerekB_WI Immutable $searchWord = 'Bridge'; $placesWithBridges = array_filter($placesArray,
 function($location) use($searchWord) { $searchWord = ‘something else’;
 return strpos($location['name'], $searchWord);
 });
40. @DerekB_WI Mutable array_walk(
 $placesArray,
 function(&$location, $key) {
 $location['distance'] = calcDistance($location);
 }); array_multisort(
 array_column($placesArray, 'distance'),
 SORT_ASC,
 $placesArray);
41. @DerekB_WI Generators function location_file_generator($filename)
 {
 $file = fopen($filename, 'r'); while(($fileData = fgetcsv($file, 1000, "\t")) !== FALSE) { yield $fileData;
 }
 }
42. @DerekB_WI Generators 
 foreach(location_file_generator($filename) as $line) 
 {
 ...
 }
43. @DerekB_WI Generators • Pro: Reduced Memory Usage • Pro: Easy syntax • Con: Cannot use array functions
44. @DerekB_WI Generator Map function map(Traversable $data, Callable $func) { foreach($data as $item) {
 yield call_user_func($func, $item);
 }
 }
45. @DerekB_WI Generator Map foreach(map(location_file_generator($filename), 
 function($item) {
 return $item[1];
 }) as $line) { … }
46. @DerekB_WI Generator Filter function filter(Traversable $data, Callable $func) { foreach($data as $item) {
 if (call_user_func($func)) {
 yield $item;
 }
 }
 }
47. @DerekB_WI Generator Filter foreach(filter(location_file_generator($filename), 
 function($item) {
 return $item[1] == 1;
 }) as $line) { … }
48. @DerekB_WI Data Structures • Standard PHP Library - SPL • Common Structures - widely used in other languages • Many have filter and reduce functions • apt install php-ds • pecl install ds
49. @DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI', PlaceFactory::make('Wisconsin')); $map->filter(function($key, $value) {
 return $value->population > 5000000;
 });
50. @DerekB_WI Hash Map $map = new DS\Map();
 $map->put('AK', PlaceFactory::make('Alaska'));
 $map->put('WI', PlaceFactory::make('Wisconsin')); $map->reduce(function($carry, $key, $value) {
 return $carry + $value->population;
 }, 0);
52. @DerekB_WI Iterators class MyIter implements Iterator {
 protected $data;
 protected $index;
 protected $valid = true; … }
53. @DerekB_WI Iterators public function __construct($inArray) {
 $this->data = $inArray;
 $this->index;
 }
54. @DerekB_WI Iterators public function current() {
 return $this->data[$this->index];
 }
55. @DerekB_WI Iterators public function next() {
 $this->index++; if ($this->index >= count($this->data)) {
 $this->valid = false;
 return null;
 } return $this->current(); }
56. @DerekB_WI Iterators public function key() {
 return $this->index;
 } public function valid() {
 return $this->valid;
 } public function rewind() {
 $this->index = 0;
 }
57. @DerekB_WI Invoke Iterator foreach(new MyIter($someArray) as $it) {
 …
 }
58. @DerekB_WI Invoke Iterator + Map foreach(map(new MyIter($someArray), function($item) {
 …
 }) as $it) {
 …
 }
59. @DerekB_WI Laravel Collections
60. @DerekB_WI Laravel Collections
61. @DerekB_WI CakePHP MapReduce
62. @DerekB_WI CakePHP MapReduce
63. @DerekB_WI Doctrine Collections
64. @DerekB_WI Doctrine Collections
65. @DerekB_WI Async/Process • Symfony process component • Promise library • Job Queues
66. @DerekB_WI Promise use GuzzleHttp\Promise\Promise; $promise = new Promise(function() { … }); $result = $promise->then(
 function($value) {
 // handle result
 }
 );
67. @DerekB_WI Resources • https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge • https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf • https://medium.com/@oscarstiffelman/a-brief-history-of-mapreduce-97aec97df8ff • https://www.sitepoint.com/parallel-programming-pthreads-php-fundamentals/ • https://www.dragonbe.com/2015/07/speeding-up-database-calls-with-pdo-and.html • https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/Collection.php