Upgrade to Pro — share decks privately, control downloads, hide ads and more …

hyperjson - the journey towards faster, safer J...

hyperjson - the journey towards faster, safer JSON parsing in Python using Rust

During a recent Python Hackathon in Düsseldorf, Matthias somehow managed to sneak in some Rust code while writing a fast, safe Python module for encoding and decoding JSON. It passes (most of) the Python test suite for the json module and was unreasonably pleasant to write. Listen carefully as he tells the story of a little side-project that got out of hand and how Rust can help speed up even the most boring, highly-optimized tasks like parsing file formats in the future.

Matthias Endler

August 01, 2018
Tweet

More Decks by Matthias Endler

Other Decks in Technology

Transcript

  1. ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 3 0

    0 1087568 Python 15 404 148 1331 Rust 2 63 65 468 Markdown 1 46 0 107 Bourne Shell 4 17 7 54 make 1 9 0 32 YAML 1 6 2 23 TOML 1 5 0 20 ------------------------------------------------------------------------------- SUM: 28 550 222 1089603 -------------------------------------------------------------------------------
  2. >>> import lenrs >>> len("hello") 5 >>> len(["this", "is", "a",

    "test"]) 4 >>> len(1) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'int' has no len()
  3. #![feature(proc_macro)] #![feature(proc_macro_path_invoc)] extern crate pyo3; use pyo3::prelude::*; #[py::modinit(_lenrs)] fn init(py:

    Python, m: &PyModule) -> PyResult<()> { #[pyfn(m, "len")] fn len(py: Python, obj: PyObject) -> PyResult<PyObject> { if let Ok(s) = obj.extract::<String>(py) { return Ok(s.len().to_object(py)); } if let Ok(v) = obj.extract::<Vec<String>>(py) { return Ok(v.len().to_object(py)); } Err(PyErr::new::<pyo3::exc::TypeError, _>("Not supported")) } Ok(()) } Requires nightly Rust Module name in Python Function name in Python