GTasklets: Python generator based pseudo-threads for PyGTK


Table of Contents

Project moved
Motivation
Solution
Related Work
Bibliography

Project moved

Warning

The gtasklets module has now been merged into kiwi. Check the new documentation here.

Motivation

Threads are evil.

Oh, need I say more?... Well, lots of nasty things happen with threads, such as deadlocks and race conditions. OK? Are you happy now?

The typical PyGTK solution to avoid threads is to use callbacks. It is a fine solution, the only problem being that sometimes you end up with too many callbacks, one after the other, and that hurts the readability of a program, just as a bunch of goto's in C make what we usually call a sphagetti program. Not to mention the difficulty for some newbies to think in terms of callbacks; one has to do some mental gymnastics to split a long operation in a series of steps, put each step in a different function, and figure out which data needs to be passed from one function to the next one. Not to mention handling errors and disconnecting callbacks...

Since Python 2.2, a new language feature has been available called generator[pep255]. Generators are objects that are defined as functions, and when called produce iterators that return values defined by the body of the function, specifically yield statements.

The neat thing about generators are not the iterators themselves but the fact that a function's state is completely frozen and restored between one call to the iterator's next() and the following one. This is just the right thing we need to turn our functions into pseudo-threads, without actually incurring the wrath of the gods of software.

Solution

The module gtasklet has support for what is called tasklets, which is like a thread, except that:

  • Only one tasklet can run execute at any given time;

  • Tasklets should give away control from time to time, specifying which events must happen for it to regain control;

With this model, the programmer is given full control of execution. Tasklets decide at which points preemption can occur, and why. With threads, preemption can occur any point/time, causing major headaches.

Giving control back to the "system" is done inside the tasklet function using the yield keyword, with the wait conditions as arguments. A wait condition is an object that indicates an event a tasklet is waiting for in order to receive back control. Any number of wait conditions can be specified as an argument to yield. After yielding, the tasklet should call get_event() to find out which event actually took place that caused it to regain control. Unlike Twisted, there's no distinction between normal and error conditions; it's up to the programmer to make that distinction. For example, it could be that an object signal is a normal event, but a timeout an error one.

Here's an example of a Tasklet, based on the demo program.

def counter(task, dialog):
    timeout = gtasklet.WaitForTimeout(1000)
    msgwait = gtasklet.WaitForMessages(accept='quit')
    for i in xrange(10, 0, -1):
        dialog.format_secondary_markup("Time left: <b>%i</b> seconds" % i)
        yield timeout, msgwait
        ev = self.get_event()
        if isinstance(ev, gtasklet.Message) and ev.name == 'quit':
            return
        elif ev is timeout:
            pass
        else:
            raise AssertionError

In the above code, the tasklet counts down from 10 to 1 and changes a message dialog's text. Between each step, it yields control for one second. After each yield, it checks for the received event. The WaitForMessages instance passed to yield declares that we are also interested in receiving messages with name 'quit'. It is also possible to use WaitForMessages to indicate messages that we want to defer (queue) or discard (drop). The message 'quit' is used in this tasklet with the meaning that it should stop. To launch the above tasklet, one would do (including the main loop):

if __name__ == '__main__':
    dialog = gtk.MessageDialog(...)
    gtasklet.Tasklet(counter, dialog)
    gtk.main()

Related Work

Twisted Deferreds [twisted-deferreds] are like pygtk callbacks, except that it always has two parallel callbacks, one for the normal flow of control, the other one for reporting errors. So nothing really new (in my opinion) relative to pygtk, just a big dependency for pygtk programs.

Arjan Molenaar submitted [molenaar] a very useful recipe to the Python Cookbook, which uses generators as idle functions. Unfortunately, it is limited to idle functions, and doesn't other event sources such as timeouts I/O events.

The PEAK framework [peak] does exactly what is right. The only problem is that this is a huge dependency and is over-engineered (interfaces and protocols, anyone?). Anyway, I only found out about it after writing gtasklets :P

Finally, Stackless Python [stackless] already has support for what they call greenlets, which is very similar to the tasklets described here.

Bibliography

[pep255] Simple Generators. PEP255

[stackless] Stackless Python.