Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cython - Making Python as Fast as C

Cython - Making Python as Fast as C

Is your Python program too slow, or planning to integrate some C libraries?

Cython is a transcompiler to compile your Python source into C source. Then compile the C source into binary. It implies your program is able to take the benefits of the optimization from compiler. Cython is also a programming language which defined extra features from Python. The extra features make it is easier to access C, even C++ library from Python.

This talk will introduce you to Cython and cover how to set up, the extra features in Cython, examples, and many tips.

Mosky Liu

April 24, 2014
Tweet

More Decks by Mosky Liu

Other Decks in Programming

Transcript

  1. Cython ➤ Cython is a source-to-source compiler (a.k.a. transcompiler). ➤

    Cython is a superset of Python. ➤ Provides optional static type declarations. ➤ Makes writing C extensions for Python easier. ➤ Makes Python faster by pre-compiling and static type. ➤ Sometimes faster by orders of magnitude. ➤ The pandas uses Cython! 2
  2. Mosky ➤ Python Charmer at Pinkoi. ➤ Has spoken at:

    PyCons in 
 TW, MY, KR, JP , SG, HK,
 COSCUPs, and TEDx, etc. ➤ Countless hours 
 on teaching Python. ➤ Own the Python packages like ZIPCodeTW. ➤ http://mosky.tw/ 4
  3. Outline 1. Setup 2. Foundation 3. Practicing 4. Tips 5.

    Uncovered Topics 6. Other Solutions 5
  4. Install C Compiler ➤ Mac: ➤ $ xcode-select --install ➤

    Ubuntu / Debian: ➤ $ sudo apt-get install build-essential ➤ Other: ➤ http://docs.cython.org/src/quickstart/install.html 7
  5. Install Cython ➤ Mac / Ubuntu / Debian: ➤ $

    pip3 install cython ➤ Other: ➤ http://docs.cython.org/src/quickstart/install.html 8
  6. The setup.py from distutils.core import setup from Cython.Build import cythonize

    setup( name='cython-lab', ext_modules=cythonize( '*.pyx', language_level='3' ) ) 9
  7. cdef struct Grail: int age float volume cdef union Food:

    char *spam float *eggs cdef enum CheeseType: cheddar, edam, camembert cdef enum CheeseState: hard = 1 soft = 2 runny = 3 14
  8. cdef struct Point: int x int y # either `struct`

    or `ctypedef` is not need cdef Point p 16
  9. cdef object add(object a, object b): return a+b cdef int

    add(int a, int b): return a+b cpdef int add(int a, int b): return a+b 19
  10. .pxd Exposes cdef Func # mylib.pxd cdef say_hello(char* name=?) #

    another.pyx from mylib cimport say_hello 20
  11. Using C Lib from libc.math cimport sin # or cdef

    extern from "math.h": double sin(double x) 21
  12. Function Visibility 22 Same 
 File Other .pyx .py Funcs

    in .h/.c Visible
 directly Visible
 via cdef extern Invisible cdef Visible
 via .pxd & cimport cpdef Visible
 via import def
  13. Binding During 23 Same 
 File Other .pyx .py Funcs

    in .h/.c compile-time compile-time x cdef cpdef run-time def
  14. Type Conversions 24 C From Py To Py short, int,

    long int int float, double int/float float char* str/bytes bytes array iterable list struct dict
  15. The Overflow ➤ Static types may also overflow in Cython

    silently. ➤ Try to make an overflow! ➤ In C: ➤ http://j.mp/test_overflow_in_c_c ➤ In Cython: ➤ http://j.mp/overflow_in_pyx_pyx 27
  16. Using C Functions ➤ Try to use the functions in

    C. ➤ Playing with fork, the system call, may be fun. ➤ In C: ➤ http://j.mp/test_fork_c ➤ In Cython: ➤ http://j.mp/fork_in_pyx_pyx 28
  17. The Functions ➤ Write three functions defined in 
 def,

    cdef, and cpdef. ➤ Try to call them in: ➤ The same file, ➤ Another .pyx file, ➤ And a .py file. ➤ Refer to the table, “Function Visibility”. ➤ The Cython code: ➤ http://j.mp/ lib_in_pyx_pyx ➤ http://j.mp/ use_lib_in_pyx_pyx ➤ http://j.mp/ test_lib_in_pyx_py 29
  18. cython -a ➤ cython -a NAME.pyx ➤ cython -a -3

    NAME.pyx ➤ open NAME.html 31
  19. Uncovered Topics ➤ Differences between C and Cython expressions ➤

    http://docs.cython.org/src/userguide/ language_basics.html#differences-between-c-and-cython- expressions ➤ Propagating Exceptions in cdef ➤ http://docs.cython.org/src/userguide/ language_basics.html#error-return-values 35
  20. ➤ Extension Type — cdef class ➤ http://docs.cython.org/src/userguide/ extension_types.html ➤

    Generic programming using Cython's Template ➤ http://docs.cython.org/src/userguide/fusedtypes.html ➤ Conditional Compilation ➤ http://docs.cython.org/src/userguide/ language_basics.html#conditional-compilation 36
  21. ➤ Profiling ➤ http://docs.cython.org/src/tutorial/profiling_tutorial.html ➤ Parallelism (No GIL + OpenMP)

    ➤ http://docs.cython.org/src/userguide/parallelism.html ➤ Using C++ in Cython ➤ http://docs.cython.org/src/userguide/ wrapping_CPlusPlus.html 37
  22. Other Solutions ➤ Numba: translates Python to optimized machine code.

    ➤ @jit ➤ CFFI: calls C code in Python dynamically. ➤ Cython versus CFFI – fuzzy notepad ➤ pybind11: makes C++ code available in Python. ➤ Cython, pybind11, cffi – which tool should you choose? – Setfans Welt ➤ Dask: provides advanced parallelism for analytics. ➤ Comparison to Spark – Dask ➤ panda and Enhancing performance – pandas. 40
  23. Cool Down ➤ Did you use NumPy and SciPy efficiently?

    ➤ Profile your program. ➤ Consider the portability — you are writing C programs! ➤ Algorithm still does matter. ➤ Pick the most suitable solution. 41
  24. Recap ➤ static types by cdef and in function definition.

    ➤ cdef function & cpdef function. ➤ .pxd exposes cdef functions. ➤ cdef extern for C functions. 42