hyperjson - the journey towards faster, safer JSON parsing in Python using Rust

During a recent Python Hackathon in Düsseldorf, Matthias somehow managed to sneak in some Rust code while writing a fast, safe Python module for encoding and decoding JSON. It passes (most of) the Python test suite for the json module and was unreasonably pleasant to write. Listen carefully as he tells the story of a little side-project that got out of hand and how Rust can help speed up even the most boring, highly-optimized tasks like parsing file formats in the future.

Transcript

2. hydrojson
3. EXPECTATION... REALITY...
5. "Hey, I wrote a JSON encoder!"
7. https://www.cartoonstock.com/cartoonview.asp?catref=rdin355
9. http://seriot.ch/parsing_json.php
10. https://bugs.python.org/issue21529
13. Parsing JSON - Hieronymus Bosch, 1501
20. ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 3 0 0 1087568 Python 15 404 148 1331 Rust 2 63 65 468 Markdown 1 46 0 107 Bourne Shell 4 17 7 54 make 1 9 0 32 YAML 1 6 2 23 TOML 1 5 0 20 ------------------------------------------------------------------------------- SUM: 28 550 222 1089603 -------------------------------------------------------------------------------
21. >>> import lenrs >>> len("hello") 5 >>> len(["this", "is", "a", "test"]) 4 >>> len(1) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'int' has no len()
22. #![feature(proc_macro)] #![feature(proc_macro_path_invoc)] extern crate pyo3; use pyo3::prelude::*; #[py::modinit(_lenrs)] fn init(py: Python, m: &PyModule) -> PyResult<()> { #[pyfn(m, "len")] fn len(py: Python, obj: PyObject) -> PyResult<PyObject> { if let Ok(s) = obj.extract::<String>(py) { return Ok(s.len().to_object(py)); } if let Ok(v) = obj.extract::<Vec<String>>(py) { return Ok(v.len().to_object(py)); } Err(PyErr::new::<pyo3::exc::TypeError, _>("Not supported")) } Ok(()) } Requires nightly Rust Module name in Python Function name in Python
23. Write your own Python Extension in Rust!
 
 https://www.youtube.com/watch?v=D9r__qxtRMQ