@ymyzk • Site Reliability Engineer @ Indeed • Personal projects • wsgi_lineprof: line-by-line profiler • mypy Playground: mypy on web browsers • tox-gh-actions: smooth integration of tox with GitHub Actions • See ymyzk.com for more details
What is WSGI middleware? • Wrapper for WSGI app to add features • Why WSGI middleware? • Keep track of HTTP requests • Support multiple web application frameworks • Easily enable/disable a middleware
a function) • WSGI middleware can be implemented as a higher-order function WSGIApplication = Callable[ [WSGIEnvironment, StartResponse], Iterable[bytes]] WSGIMiddleware = Callable[ [WSGIApplication], WSGIApplication]
extension to track line-by-line execution • What is tracing? 1. Register a callback function to CPython 2. CPython calls the callback function on specific events like executing a line, calling a function, raising exception…
of APIs for tracing • trace module • setprofile/settrace functions in sys module • PyEval_SetTrace/PyEval_SetProfile functions in Python/C API import sys sys.settrace(lambda frame, event, arg: ...) See extensions/extensions.pyx
Convert to parse tree 3. Convert to AST 4. Convert to CFG 5. Convert to byte code • Byte code retains corresponding line number 6. Evaluate byte code • Call tracing function when evaluating byte code See Python Developers’s Guide
depending on platforms for measuring time • Why measuring time is not simple? • Needs to use monotonic timer • time.time() / datetime.now() are not • Reduce overhead of getting time • Use of time.monotonic/perf_counter() incurs additional overhead
accessing source code efficiently • threading/queue: for writing results efficiently • wheel: for packaging with compiled extensions • Airspeed Velocity: for measuring overhead Other Technologies
technologies, though making an efficient profiler requires additional efforts Future plan of wsgi_lineprof: • Show result via HTTP endpoint • Support ASGI (asyncio) / multithread • Reduce overhead wsgi_lineprof is available on GitHub