PyCon Ireland 2013: Gccpy - Python Front-end to GCC

(via http://python.ie/pycon/2013/talks/gccpy_python_frontend_to_gcc/)
Speaker: Philip Herron (https://twitter.com/redzor)

Gccpy is an ongoing project to allow users to compile Python Code as an AOT (ahead of time) compiled language. I am the author and was mentored by Ian Lance Taylor and was inspired by Dubliner Paul Biggar's PHD on PHC php compiler.

ve been steadily gaining more and more contributions from GCC enthusiasts overt the last year. And i haven't even made a release yet and its been in development since early 2010.

http://gcc.gnu.org/wiki/PythonFrontEnd

Transcript

1. Gccpy – Python Front-end to GCC Philip Herron - @redzor redbrain@gcc.gnu.org - http://redbrain.co.uk
2. Introduction • Fully ahead of time compiler for Pure Python code (no Rpython) • Currently targeting Python 2.4 syntax (simple exception handling) • Implemented on-top of GCC for a feature rich Middle and backend. • Not using LLVM as it wasn't as mature back in 2009 when I was developing the idea • I didn't like LLVM's code base at the time.
3. What is a Front End?
4. Why develop this • GOAL: Is Jit compilation better than Ahead of time with current technologies and languages? – No one has really tested this fully but ends up in flame wars. • GCCPY vs PyPy Basically. • To learn and gain a deep understanding in Compilers and code- generation - Python argued to be slow • Inspired by PHC a php compiler Paul Biggar's PHD optimizing dynamic languages • Surely making python code native is faster right...? – This isn't necessarily true
5. History • Always wanted a deep full project to get into and got accepted to Gsoc 2010 • PHC - http://www.phpcompiler.org/ Php compiler: Paul Biggar – PHD thesis: http://paulbiggar.com/research/#phd-dissertation • Wanted to learn how compilers and languages really worked • Python is awesome, Paul also wanted to do this exact project from his google talk on his phd its on youtube search his name. • In the end PHC Failed for many reasons I believe mostly because it was a home-brew compiler! – If you are to reuse GCC with all its lovely middle/back end optimizers we could have something special – Also why I can't stand compiler's which output C because its LAZY
6. Hello World $ gccpy -g -O2 -fpy-gen-main helloworld.py -o helloworld $ ./helloworld
7. Compiling Python – DOT IL • Developed DOT_IL (Python with braces ;) ) – It is a dynamic typed language with some optimizations – constant folding at a highlevel – Sits on-top of GCC GENERIC which is very C like no control structure almost 3-address-code – Lowered to Gimple and into GCC middle-end optimizers and finally thr backend with target specific optimizations.
8. The whole Pipeline together!
9. Dynamic Typing • What something points to! • Basically looks similar to Cpython gpy_object_t, PYObject are very similar structures in principle. • The runtime is more optimized to Ahead of time languages more runtime behaviour is compiled in rather than dynamically figured out at runtime. • Objects are allocated at runtime into gpy_object_t
10. Memory Addressing
11. Issues • Empty class declarations such as: class myClass: pass • Won't be supported by gccpy
12. Latency and Speed • Programmers have forgot how powerful C is! • Programmers have forgot how powerful old hardware is. • Add more resources argument is broken.
13. Demo 1 Hello World!..........
14. Your gonna make this face next
15. Structure of the Gccpy Front-End
16. Gccpy VS other's • Nuitka is pretty much a clone of ShedSkin.... – They generate optimized, embedded python/C++ depending on libpython.so. • Cython is awesome for mixing pure python + C – Dialect of python supports C types – Can be used to 'compile' python but its not really • PyPy Jit'd python boasting lots of runtime optimization. • Numba and parakeet extremely DSL specific
17. Compile Time vs Runtime • EVERYTHING is already Jit'ed before running your program – Reduces the size of the software stack no intermediate Virtual Machine. • Runtime optimizations on the language very hard to do. – Jit optimization mostly revolves around target cpu funky'ness • GCC is already aware of this -mtune – Languages and Architectures don't lend themselves to current state of dynamic languages. –
18. Good and the Bad • Developing this is a huge task! – Takes long time everything from scratch • Generating sensible code – Jit's in principle are better idea for the future of dynamic languages but I don't agree with this in our current landscape – PyPy has a budget! – Ahead of time binaries are always going to be around – Embedded systems will always benefit from this type of project. – Native applications without need to change language
19. Gccpy Optimizations • Global Dictionary Lookup offset's all calculated at compile time – No runtime lookups! • Jump tables calculated at compile time with labels and goto's in the compiled code! – No runtime jump tables • No middle-ish crap to manage like parsing • No need to pseudo compile .pyc or manage the jit.
20. Project Status • First milestone (lucy) accomplished – All basic core principles in place – List/dict/integers/string all types need their api filled out more community has been helping with that – Stdlib needs done (huge task) going to implement enough to bootstrap the rest of Cpython stdlib. • __builtin__ sys etc...
21. Major Projects • Module compilation with __init__.py (should modules be .so or .a or both?) • Threading and multi-core no GIL (only should affect the gvs (gccpy global virtual stack)) • Need a garbage Collector – bohem-gc used in gcj (gnu-java and objc/objc++) is meant to be slow but will do in the mean time I think)
22. Thanks! Twitter @redzor redbrain@gcc.gnu.org http://redbrain.co.uk https://github.com/redbrain/gccpy http://gcc.gnu.org/wiki/PythonFrontEnd Questions?!