A short presentation I held at the Zurich Python User Group in December 2014. It introduces Celery and the different task workflows (canvas) that can be implemented with it.
types into a format that can be stored in the queue. The serialized data can be compressed and signed cryptographically. Supported serializers: • Pickle • JSON • YAML • MessagePack
without Celery: >>> add(19, 23) 42 Or you can call it asynchronously through Celery: >>> add.delay(19, 23) <AsyncResult: 257d9f84-cd4b-4248-b7ba-93ed24ce6804> Celery: Calling a Task
method: >>> add.apply_async(args=(19, 23)) <AsyncResult: df5abf5e-737a-4a36-acd2-c0ffe8bad305> You can also delay the execution of a task (and many other things): >>> add.apply_async((19, 23), countdown=10) <AsyncResult: def99720-ea55-4280-957b-23bbcbc7646c> Celery: Calling a Task
add.delay(19, 23).get() 42 Or you can poll the result asynchronously: >>> res = add.delay(19, 23) >>> res.status, res.ready(), res.result 'SUCCESS', True, 42 Celery: Retrieving a Result
to call a function. You can manually create a signature for a task: >>> from celery import signature >>> signature('tasks.add', args=(2, 2), countdown=10) tasks.add(2, 2)
are essentially tasks that execute tasks. >>> add.subtask((2, 2), countdown=10) tasks.add(2, 2) There is also a shortcut using star arguments. >>> add.s(2, 2, countdown=10) tasks.add(2, 2)
of 10 lists containing 10 numbers each. We want to flatten this to a single list containing all 100 numbers. >>> sig2 = flatten.s() Finally we want to sum all numbers in that list: >>> sig3 = sum_list.s()
a signature, a list of instructions on how to calculate the result. Nothing has been evaluated yet. Now let’s actually run those signatures and get the final result: >>> sig.delay().get() 9900
dynamic tasks – modifying a task in a chord or group at runtime. • replace(sig): Replace the current task with a new task inheriting the same task id. • add_to_chord(sig): Add a signature to the chord the current task is a member of.