Interesting Things With Python

This was originally written for my talk in SyPy September 2013. It was originally written as HTML slides. I've converted it to PDF for SpeakerDeck as I have had the time now

1. Interesting Things With Python SyPy  September  2013  
2. Zip tricks •  [x amount of] things that I think is interesting (YMMV) •  Hacks in Python •  Based on my working experience •  Do NOT put these code into production
3. THE BASICS
4. The Basics These are things I have found helpful: 1.  In place value swapping 2.  Zip tricks 3.  Splat 4.  Enumerate 5.  SimpleHTTPServer
5. In-place value swapping >>> a = 1! >>> b = 2! >>> a, b = b, a! >>> print(a, b)! 2, 1
6. While we're on that topic... >>> a, b, c, d = 1, 2, 3, 4! >>> a! 1! >>> b! 2
7. Zip tricks Zip is really more useful than expected. Take this 2 x 4 matrix! 1 2 3 4! 5 6 7 8! Represented in Python: m = [[1,2,3,4], [5,6,7,8]]! Make it a 4 x 2 matrix
8. Zip tricks >>> zip(*m)! [(1, 5), (2, 6), (3, 7), (4, 8)]! It essentially is this: 1 5! 2 6! 3 7! 4 8
9. Splat Operator •  That's the name in Perl. •  There isn't a name in Python. •  Hard to find information about for beginners. •  I thought it was pointers when I started. •  It unpacks arguments (equivalent to Go's ...something and something...)
10. Splat Operator def fn(arg1=None, arg2=None, arg3=None, arg4=None):! print(arg1, arg2, arg3, arg4)! ...! >>> l = [1,2,3,4]! >>> fn(l)! ([1, 2, 3, 4], None, None, None)! >>> fn(*l)! (1, 2, 3, 4)
11. Splat Operator Now you can take any amount of arguments! ! def fn(*args):! print args! ! >>> l = [1,2,3,4]! >>> fn(l)! ([1, 2, 3, 4])! >>> fn(*l)! (1, 2, 3, 4)
12. Enumerate enumerate() is a generator that returns the index and the value of a iterable. Instead of: for i in range(0, len(l)):! v = l[i]! # do something with the index and value! Do this: for i, v in enumerate(l):! # do something with index and value
13. Enumerate One more trick with enumerate: Start at any number! for i, v in enumerate(l, start=2):! # do something with index and value! Do this, and you will eliminate many off-by- one errors.
14. SimpleHTTPServer python -m SimpleHTTPServer 8080! That's it! You have a simple HTTP Server running.
15. Bonus: Pydoc I love Godoc. Then I discovered Godoc was inspired by Pydoc pydoc -p 6060! Now go to http://localhost:6060 for documentation!
16. The Fun Stuff
17. Bit Twiddling What does this code do? def fn(x): ! return (3435973837*x)>>35! And this? def fn2(y): ! return ((1717986919*y)>>34)-(y>>31)
18. Bit Twiddling Answer: They both divide x by 10 Can you figure out what the magic numbers 3435973837 and 1717986919 are?
19. Little Known/Used Libraries
20. Arrays There is a difference between lists and arrays. Python has a built-in array library. import array
21. When are arrays useful? •  When you have a list, and you know the list will only ever be integers, characters or bytes. •  When your list does not change much. •  When you don't need to do math (if you want to do math, use Numpy's Array).
22. Guido's Example Compare these two functions: import array! def f7(l):! return array.array('B', list).tostring()! ! def f1(l):! string = ""! for item in l:! string = string + chr(item)! return string! The one with arrays is 15x faster than the other function.
23. Caveat Don't be seduced, however. Here's the performance of sorting an array vs sorting a list of 1 million integers (naive sorts) ! timeit.timeit("sorted(l)", setup="from __main__ import l", number=100)! 54.20818901062012! ! ! timeit.timeit("sorted(a)", setup="from __main__ import a", number=100)! 93.52262997627258
24. Heap The data structure, not the memory term. Very useful as a priority queue. Here's Guido's code to sort 1 million integers with an array and a heap queue: ! ! import sys, array, tempfile, heapq! assert array.array('i').itemsize == 4! ! def intsfromfile(f):! while True:! a = array.array('i')! a.fromstring(f.read(4000))! if not a:! break! for x in a:! yield x! ! iters = []! while True:! a = array.array('i')! a.fromstring(sys.stdin.buffer.read(40000))! if not a:! break! f = tempfile.TemporaryFile()! array.array('i', sorted(a)).tofile(f)! f.seek(0)! iters.append(intsfromfile(f))! ! a = array.array('i')! for x in heapq.merge(*iters):! a.append(x)! if len(a) >= 1000:! a.tofile(sys.stdout.buffer)! del a[:]! if a:! a.tofile(sys.stdout.buffer)
25. Sorting Sorting is a matter of context. timeit.timeit("sorted(l)", setup="from __main__ import l", number=100)! 54.20818901062012! ! timeit.timeit("sorted(a)", setup="from __main__ import a", number=100)! 93.52262997627258! ! timeit.timeit("customSortBA(l)", setup="from __main__ import l, customSort", number=100)! 40.209283113479614! ! timeit.timeit("customSortPP(a)", setup="from __main__ import a, customSort", number=100)! 41.524425983428955! HOW?
26. Sorting Python uses something called TimSort as it's default sort. ! import random! import array! ! l = [x for x in range(0, 1000000)] # generate 1 million integers! ! # shuffle a few times! random.shuffle(l)! random.shuffle(l)! random.shuffle(l)! ! # create an array! a = array.array('i', l)
27. Sorting By  now  it  should  be  something  obvious  about  the   integers  to  be  sorted:   –  There  are  no  repeats   –  There  is  a  range  (specifically  from  0-­‐1000000)   –  Sound  familiar?   Sounds  like  database  indices?   A  real  life  example   [insert  example  about  url  classificaHon]  
28. Sorting The Sorting Algorithm from bitarray import bitarray! ! def customSortBA(iterable):! ba = bitarray('0' * len(iterable))! for i in iterable:! ba[i] = True! return [j for j,k in enumerate(ba) if k]! Want to do it in pure Python? def customSortPP(iterable):! a = [None for i in range(0, len(iterable))]! for i in iterable:! a[i] = True! return [j for j,k in enumerate(ba) if k]!
29. Sorting •  Lots  of  opHmizaHons  can  be  done.   •  Limited  in  applicaHon  though.   •  O(3N).  (Technically  O(2M  +  N),  but  in  this  case  M  =  N)   •  But  Timsort  has  a  best  case  of  O(N)  too!   •  Not  a  general  purpose  sort  
30. Fun (and stupid) things you can do with Python
31. Try this! def fn1():! print("Hello World")! ! def fn2():! print("EXTERMINATE! EXTERMINATE!")! ! fn1.__code__ = fn2.__code__! ! exec(fn1.__code__)  
32. Q is fun! import q! ! @q! def fn():! q("In fn()")! print("Hello, I'm the Doctor. Run.")! ! fn()! Then chewxy@chewxy-TARDIS-40:~$ tail -f /tmp/q! 19.5s <module>: <function fn>! 93.1s fn: 'In fn()'  
33. When in doubt, inspect! import inspect! import pprint! ! import random! ! pprint.pprint(inspect.getmembers(random))  
34. THANK YOU Follow me on Twitter: @chewxy