Python DevCenter
oreilly.comSafari Books Online.Conferences.

advertisement


What's New in Python 2.5
Pages: 1, 2, 3, 4

Generators that Return Values

Generators entered the language with Python version 2.3. Version 2.5 extends the capabilities of generators.



Generators are functions that can be exited and re-entered, and are useful for iterators. For example, here's a class that includes a generator called iterate:

class MyStuff:
   def __init__(self):
       self.a = [2,4,6,8,10,12]
   def iterate(self):
       i = 0
       while i < len(self.a):
           yield self.a[i]
           i += 1

This code uses of the generator, demonstrating the iterator feature:

obj = MyStuff()
iterate = obj.iterate()
print iterate.next()
print iterate.next()
print iterate.next()

The first line creates an instance of the class MyStuff. The second line retrieves an iterator from the instance. Notice that the iterator itself isn't the value obtained from the generator; instead, you call next to get the value. If you're in the interactive shell, you can see what happens if you try to take the iterator's value:

>>> print iterate
<generator object at 0x009EB698>
>>>

This is different from iterators in C++. Though you can dereference an iterator in C++ to obtain the value, in Python the generator is a built-in generator object.

The idea behind the generator is that you call the function that includes a yield statement. The yield statement causes execution of the function to suspend and return. You can then go back into the function, picking up where it left off, by calling next(). Thus, when you called:

iterate.next()

... the execution stepped into the function, and the first two lines ran (the assignment statement and the first line of the while loop). The next line is the yield statement, which caused execution to return to the caller, temporarily returning the value self.a[i].

Then, when you called iterate.next() again, execution resumed after the yield statement with the line that increments i. The while loop continued again until the next yield statement.

Note that creating the generator with iterate = obj.iterate(), doesn't actually call into the function. The Python compiler saw the yield statement inside the function and identified it as an iterator, causing the call to the function to create a generator object instead. You can verify this if you add some print statements to the iterate function:

def iterate(self):
   print "Creating iterator"
   i = 0
   ...rest of code...

You won't see the output "Creating iterator" when you call iterate = obj.iterate(), but you will see it when you make the first next call, print iterate.next().

That's a quick introduction to generators, and is not new to Python 2.5. However, people have identified shortcomings to the generator feature. In languages that provide for generators, an important feature is the ability to pass a value back into the generator. This allows for supporting a programming feature called coroutines.

In order to make the generators more powerful, the designers of Python have added the ability to pass data back into the generator. If, from the perspective of the generator, you think of the yield as calling something, then the concept is easy: You just save the results of yield:

x = yield y

From the perspective of the caller of the generator, this statement returns control back to the caller, just as before. From the perspective of the generator, when the execution comes back into the generator, a value will come with it--in this case, the generator saves it into the variable x.

Where does the value come from? The caller calls a function called send() to pass a value back into the generator. The function send() behaves just like the function next(), except that it passes a value.

Here's some modified code that demonstrates how to use send(). In this case, I use send() to let the user of the generator reposition the index in the array. However, I'm making sure that the value sent in is within the bounds of the array. Otherwise I set it to the closest boundary.

class MyStuff:
   def __init__(self):
       self.a = [2,4,6,8,10,12]

   def iterate(self):
       i = 0
       while i < len(self.a):
           val = yield self.a[i]
           if val == None:
               i += 1
           else:
               if val < 0:
                   val = 0
               if val >= len(self.a):
                   val = len(self.a) -1
               i = val

Notice how I first test what value the generator received and stored in val. If val is None, that means the generator received no value (the calling code used next(), or perhaps send(None)). However, if a value was passed, I limit it to the bounds of the array, and then save it to the index.

Incidentally, I first tried using the new conditional expression feature to set the value of i, instead of using two if statements. You can condense the lines following the else line in the preceding code to:

i = val if (val >= 0 and val < max) else (0 if val < 0 else max)

Personally, I think all of that on a single line is a bit unreadable and suffers from the same obfuscation that plagues so much old C code. Thus I opted for the longer version. (Although I must say this condensed line is more readable than the equivalent C code.)

When the generator code runs out, it throws an exception. If you want, you can loop through your next() calls, and wrap them in a try/catch block. Alternately, you can set up your yield call to return a value that indicates the end of the iteration. Here's a modified form of the constructor in the previous code:

   def __init__(self):
       self.a = [2,4,6,8,10,12, None]

I simply added a None to the end of the list. Then I can watch for that to test for the end of the list without having to catch an exception:

obj = MyStuff()
iterate = obj.iterate()
a = iterate.next()
while a != None:
   print a
   a = iterate.next()
iterate.close()

The final line calls the close() function, which is also new to generators with Python 2.5. This frees up the resources for the generator. If you call iterate.next() again after calling iterate.close(), you'll get a StopIteration exception.

If you're familiar with recent versions of Python, you might wonder where the built-in Python iter() function fits in. The answer is that if you name your generator function __iter__(), then your generator will automatically work with the iter() function. This feature also existed in Python 2.4, but with Python 2.5, you can now use the send() function in conjunction with iter(). If I change the name of the iterator() function in the preceding MyStuff function to __iter__, and keep everything the same, then the iter function works:

>>> obj = gen1.MyStuff2()
>>> i = iter(obj)
>>> i.next()
2
>>> i.send(3)
8
>>> i.next()
10

Pages: 1, 2, 3, 4

Next Pagearrow





Sponsored by: