Validating JSON with JSON Schema

My talk from Berlin PHP Usergroup, December 2013, about the benefits of using JSON schema for validating complex JSON data

Transcript

1. Validating your JSON
2. About me → Anne-Julia Scheuermann → Creating Web-Applications since 2004 → Developing information literacy solutions for
3. Short Story API DB JSON
4. Short Story API DB JSON
5. Custom validation
6. Custom validation protected function isValidUnformatted(array $data) { if (empty($data['data'])) { return false; // no data sent } $data = $data['data']; $source = $data['source']; $style = empty($data['style']) ? null : $data['style']; $structure = $this->getStructure($source, $style); $allowedTopLevelFields = $this->getOtherValidFields(); $ignoredDataFields = $this->getIgnoredFields(); // source if (!empty($structure['source_data'])) { if (empty($data[$source])) { $this->errors[] = 'needed source data missing'; return false; } $notAllowed = array_diff_key($data[$source], array_flip($structure['source_data'])); // +300 lines of code more
7. Many lines of complex code
8. PHPMD warnings /!\ /** * @param array $data * * @Todo Fix suppressed warnings * @SuppressWarnings( * PHPMD.CyclomaticComplexity, * PHPMD.NPathComplexity * ) * @return bool */ protected function isValidUnformatted(array $data) { //...
9. Validation only on server side API DB JSON
10. Other technologies have solved this
11. DTD for SGML <!DOCTYPE HTML PUBLIC ”-//W3C//DTD HTML 4.01//EN” ”http://www.w3.org/TR/html4/strict.dtd”> <!--=================== Generic Attributes ===============================--> <!ENTITY % coreattrs ”id ID #IMPLIED -- document-wide unique id -- class CDATA #IMPLIED -- space-separated list of classes -- style %StyleSheet; #IMPLIED -- associated style info -- title %Text; #IMPLIED -- advisory title --” >
12. XSD for XML (XHTML) <?xml version=”1.0”?> <xs:schema xmlns:xs=”http://www.w3.org/2001/XMLSchema”> <xs:element name=”note”> <xs:complexType> <xs:sequence> <xs:element name=”to” type=”xs:string”/> <xs:element name=”from” type=”xs:string”/> <xs:element name=”body” type=”xs:string”/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema>
13. Short Story API DB JSON
14. JSON Schema → http://json-schema.org/
15. Hello World! 42 ”42” {} true
16. JSON Schema specification Validation Core
17. JSON Schema specification Validation Core
18. type {“type”: “string”} {“type”: “integer”} {“type”: “array”} {“type”: “object”} {“type”: “boolean”} {“type”: “null”} {“type”: “number”}
19. string
20. string ”Hello World!” ”42” {”type”: ”string”} 42
21. ”php” string length { ”type”: ”string”, ”minLength”: 2, ”maxLength”: 3 } ”i <? php” ”p”
22. string pattern { ”type”: ”string”, ”pattern”: ”^and.+$” } ”iphone” ”android”
23. numeric
24. integer -1 42 0.5 “42” { ”type”: ”integer” }
25. number -1 42 0.5 2.99792458e8 { ”type”: ”number” }
26. number range 10 11 40 { ”type”: ”number”, “minimum”: 10, “maximum”: 40, “exclusiveMinimum”: true, “exclusiveMaximum”: false }
27. array
28. array [1, 2, 3, 4, 5] [1, “2”, {“3”:”4 and 5”}, [42]] {”type”: ”array”} {“this”: “is an object”}
29. array items [] [1, 2, 3, 4, 5] [1, 2, 3, 4, “5”] { ”type”: ”array”, “items”: {“type”: “number”} }
30. array length [] [1] [1, 2] { ”type”: ”array”, “minItems”: 1, “maxItems”: 2 } [1, 2, 3]
31. object
32. object { ”key”: ”value”, ”anotherKey”: ”anotherValue” } {”type”: ”object”} ”42” [“hello”, “world”]
33. {”author”: ”A. A. Milne”, ”year”: 1882} properties { ”type”: ”object”, ”properties”: { ”author”: {”type”: ”string”}, ”year”: {”type”: ”number”} } } {”year”: ”1882”} {”author”: ”A. A. Milne”} {}
34. {”author”: ”A. A Milne”, ”title”: 1882} required properties { ”type”: ”object”, ”properties”: { ”author”: {”type”: ”string”}, ”title”: {”type”: ”string”}}, ”required”: [”title”] } {} {”author”: ”A. A Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926} {”author”: “A. A Milne”}
35. additional properties { ”type”: ”object”, ”properties”: { ”author”: {”type”: ”string”}, ”title”: {”type”: ”string”} }, ”additionalProperties”: false } {”title”: []} {} {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926} {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”}
36. {”a”: ”aa”, “b”: “bb”} number of properties { ”type”: ”object”, ”minProperties”: 1, ”maxProperties”: 2 } {} {”a”: ”aa”} {”a”: ”aa”, “b”: “bb”, “c”: “cc”}
37. dependencies
38. {”name”: ”Alan”, “birthday”: “1882-01-18”} property dependencies { ”type”: ”object”, ”properties”: {“name”: {“type”: “string”}, “birthday”: {“type”: “string”}}, ”dependencies”: { “birthday”: [“name”]} } {”name”: ”Alan”} {“birthday”: “1882-01-18”}
39. {”name”: ”Alan”, “birthday”: “1882-01-18”} schema dependencies { ”type”: ”object”, ”properties”: { “birthday”: {“type”: “string”}}, ”dependencies”: { “birthday”: { “properties”: {“name”: {“type”: “string”}}}} } {”name”: ”Alan”} {“birthday”: “1882-01-18”}
40. {”PHPUnit”: ”testing”, “PHPCS”: “styling”} pattern properties { ”type”: ”object”, ”patternProperties”: { “^PHP”: {“type”: “string”}}, “additionalProperties”: {“type”: “integer”} } {”BeHat”: ”accepting”} {“PHPLOC”: 9001} {“LOC”: 9001}
41. boolean
42. boolean true false {”type”: ”boolean”} “true” 0
43. null
44. null null false {”type”: ”null”} “” 0
45. More features
46. “one” “eleven” 1 -11 Combining with oneOf, anyOf, allOf { ”anyOf”: [ {“type”: “string”, “maxLength”: 5}, {“type”: “number”, “minimum”: 0} ] }
47. not “42” {“key”: 42} 42 { ”not”: {“type”: “number”} }
48. available formats: email, date-time, hostname, ipv4, ipv6, uri ”ab@mail.de” ”ab@mail.” built-in formats { ”format”: ”email” }
49. JSON Schema specification Validation Core
50. title, description & default { ”title”: ”Expressive title”, ”description”: “Take time to explain”, ”default”: “Validators ignore this keyword” }
51. $schema { “$schema”: “http://json-schema.org/draft-04/schema#”, ”title”: ”Expressive title”, ”description”: “Take time to explain”, ”default”: “Validators ignore this keyword” }
52. definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “definitions”: { "address": { "type": "object", "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } } }
53. $ref & definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “definitions”: { "address": { "type": "object", "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } } "type": "object", "properties": { "billing_address": { "$ref": "#/definitions/address" }, "shipping_address": { "$ref": "#/definitions/address" } } }
54. $ref & definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “description”: “This schema defines an address”, "type": "object", "properties": { "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } { “$schema”: “http://json-schema.org/draft-04/schema#”, “description”: “This schema describes an order”, "type": "object", "properties": { "billing_address": { "$ref": "address.json#" }, "shipping_address": { "$ref": "address.json#" } } } address.json order.json
55. Breathe!
56. Validation Libraries
57. Validation Libraries geraintluff/jsv4-php 11 commits, 1 contributor hasbridge/php-json-schema 28 commits, 2 contributors, not yet feature complete justinrainbow/json-schema 202 commits, 21 contributors
58. Integration of schema validation
59. Preparing validation with json-schema // Get the schema as object $retriever = new JsonSchema\Uri\UriRetriever; $schema = $retriever->retrieve('file://' . realpath('schema.json')); // Resolve references $refResolver = new JsonSchema\RefResolver($retriever); $refResolver->resolve($schema, 'file://' . __DIR__);
60. Validation with json-schema /** * Validate json against schema * * @param string $data * @param string $schema * * @return bool */ public function isValid($data, $schema) { $data = json_decode($data); $this->validator->check($data, $schema); $this->errors = $this->validator->getErrors(); return $this->validator->isValid(); }
61. The benefits of using JSON Schema
62. Less code /** * Validate json against schema * * @param string $data * @param string $schema * * @return bool */ public function isValid($data, $schema) { $data = json_decode($data); $this->validator->check($data, $schema); $this->errors = $this->validator->getErrors(); return $this->validator->isValid(); }
63. No custom validation
64. Complexity Complexity of validation and document structure is independent
65. Validation on client and server side API DB JSON
66. Implementation in many languages PHP, JavaScript, Java, Python, Ruby, Perl, .NET, C, Haskell, Erlang, Go
67. Better user experience Clear, human- and machine-readable documentation of our data structure
68. Thank You! • http://json-schema.org/ • http://tools.ietf.org/html/draft-zyp-json-schema-04 • https://en.wikipedia.org/wiki/JSON • http://spacetelescope.github.io/understanding-json-schema/ • https://packagist.org/packages/justinrainbow/json-schema • http://getcomposer.org
69. Questions?