ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


What's New in Python 2.5

by Jeff Cogswell
10/26/2006

It's hard to believe Python is more than 15 years old already. While that may seem old for a programming language, in the case of Python it means the language is mature. In spite of its age, the newest versions of Python are powerful, providing everything you would expect from a modern programming language.

This article provides a rundown of the new and important features of Python 2.5. I assume that you're familiar with Python and aren't looking for an introductory tutorial, although in some cases I do introduce some of the material, such as generators.

Changes and Enhancements to the Language Itself

Python 2.5 includes many useful improvements to the language. None of the changes are huge; nor do they require changes to your existing 2.4 code.

Conditional Expressions

Other languages, particularly C and C++, have for years allowed conditional expressions, which are expressions that include an if statement. The idea is simple: You have a variable, and you want to set the variable to one value if some condition is true; otherwise, you want to set the variable to another variable.

Certainly, you could always do this in Python using a block if statement like so:

if x > 15:
   a = 1
else:
   a = 2

Many programmers prefer shortcuts, however. The C language introduced the idea of doing the same assignment on a single line:

a = x > 15 ? 1 : 2;

In the C code, if the condition x > 15 evaluates to true, the expression evaluates to 1. Otherwise it evaluates to 2.

Python 2.5 now has a similar feature. The Python form is rearranged a bit:

a = 1 if x > 15 else 2

Although conditional expressions exist in other languages such as the statically typed C and C++, Python's type system is different. While exploring this new feature, I was curious if I could modify types in the middle of the construct. Here goes:

a = "abc" if x > 15 else 10

This ran just fine. Of course, it's probably not the safest code in terms of introducing bugs, but you can use different types if you have a compelling reason to do so.

Slight Changes to Exception Handling for the Long Term

Python 2.5 includes changes to the Exception class hierarchy. In general, the changes are mostly transitional changes in preparation for Python 3.0, with certain pre-2.5 features having been deprecated. That means you can still use the previous features, but the language will discourage you from doing so, as they will no longer work once Python 3.0 eventually arrives.

One newly-deprecated feature is the use of raising strings as exceptions. Previously (I used Python 2.4.3), you could write code such as:

>>> raise "Invalid argument"
Traceback (most recent call last):
 File "<stdin>", line 1, in ?
Invalid argument
>>>

With Python 2.5, you can still raise strings as exceptions, but you also get a deprecation warning:

>>> raise "Invalid argument"
__main__:1: DeprecationWarning: raising a string exception
is deprecated
Traceback (most recent call last):
 File "<stdin>", line 1, in
<module>
Invalid argument
>>>

However, you only get the warning the first time per Python session. Subsequent raises don't result in the warning. Python 2.5 encourages you to stop writing code like this, because when Python 3.0 arrives, your code will not compile.

In addition to the deprecation of raising strings as exceptions, Python 2.5 also rearranged the exception class hierarchy. The new BaseException class is the base class for Exception. You should now derive all your custom exception classes from Exception, or from a class derived from Exception.

Two other exception classes, KeyboardInterrupt and SystemExit, also extend BaseException, but not Exception. Thus, you can trap your own exceptions, and then trap all other exceptions derived from Exception. This will trap every exception except KeyboardInterrupt and SystemExit. Why would you not want to trap those two exceptions? That way your program will end when the user presses Ctrl+C without you having to manually catch and respond to Ctrl+C, or without you having to manually skip catching Ctrl+C to let the system handle it. The same is true with the SystemExit exception, which should force the program to stop.

This particular new feature, then, is less of a feature and more of a requirement: Recode your exception classes to extend Exception. Then you'll be fine when Python 3.0 arrives.

Finally, there is now another short warning about future exceptions in Python. Presently, exception classes have both a message and an args member. The general idea was that message would be a string, and args would be a list of any additional arguments to the exception. However, this is redundant, because the message member doesn't have to be a string. Thus, Python 3.0 will have no more args member, so you should stop using it.

Better Exception Handling

In addition to the changes to the hierarchy, the try mechanism has improved with Python 2.5. Previously, you could not mix except and finally blocks without using two layers of exceptions. Now you can use them both in a single block, which makes life easier. The order matters, however. You must have your try block first, then optionally your exception handlers, then optionally an else block that runs if no exceptions occurred, and then optionally your finally block, which runs in all cases. Here's an example:

try:
   test()
except ZeroDivisionError:
   print "The expression resulted in a division-by-zero error"
except Exception:
   print "Oops"
else:
   print "All is good"
finally:
   print "And everything is finished"

In previous versions, you had to use two try blocks, with one inside the other, and the outer having the finally block. Now you can use a single block for all of it.

Generators that Return Values

Generators entered the language with Python version 2.3. Version 2.5 extends the capabilities of generators.

Generators are functions that can be exited and re-entered, and are useful for iterators. For example, here's a class that includes a generator called iterate:

class MyStuff:
   def __init__(self):
       self.a = [2,4,6,8,10,12]
   def iterate(self):
       i = 0
       while i < len(self.a):
           yield self.a[i]
           i += 1

This code uses of the generator, demonstrating the iterator feature:

obj = MyStuff()
iterate = obj.iterate()
print iterate.next()
print iterate.next()
print iterate.next()

The first line creates an instance of the class MyStuff. The second line retrieves an iterator from the instance. Notice that the iterator itself isn't the value obtained from the generator; instead, you call next to get the value. If you're in the interactive shell, you can see what happens if you try to take the iterator's value:

>>> print iterate
<generator object at 0x009EB698>
>>>

This is different from iterators in C++. Though you can dereference an iterator in C++ to obtain the value, in Python the generator is a built-in generator object.

The idea behind the generator is that you call the function that includes a yield statement. The yield statement causes execution of the function to suspend and return. You can then go back into the function, picking up where it left off, by calling next(). Thus, when you called:

iterate.next()

... the execution stepped into the function, and the first two lines ran (the assignment statement and the first line of the while loop). The next line is the yield statement, which caused execution to return to the caller, temporarily returning the value self.a[i].

Then, when you called iterate.next() again, execution resumed after the yield statement with the line that increments i. The while loop continued again until the next yield statement.

Note that creating the generator with iterate = obj.iterate(), doesn't actually call into the function. The Python compiler saw the yield statement inside the function and identified it as an iterator, causing the call to the function to create a generator object instead. You can verify this if you add some print statements to the iterate function:

def iterate(self):
   print "Creating iterator"
   i = 0
   ...rest of code...

You won't see the output "Creating iterator" when you call iterate = obj.iterate(), but you will see it when you make the first next call, print iterate.next().

That's a quick introduction to generators, and is not new to Python 2.5. However, people have identified shortcomings to the generator feature. In languages that provide for generators, an important feature is the ability to pass a value back into the generator. This allows for supporting a programming feature called coroutines.

In order to make the generators more powerful, the designers of Python have added the ability to pass data back into the generator. If, from the perspective of the generator, you think of the yield as calling something, then the concept is easy: You just save the results of yield:

x = yield y

From the perspective of the caller of the generator, this statement returns control back to the caller, just as before. From the perspective of the generator, when the execution comes back into the generator, a value will come with it--in this case, the generator saves it into the variable x.

Where does the value come from? The caller calls a function called send() to pass a value back into the generator. The function send() behaves just like the function next(), except that it passes a value.

Here's some modified code that demonstrates how to use send(). In this case, I use send() to let the user of the generator reposition the index in the array. However, I'm making sure that the value sent in is within the bounds of the array. Otherwise I set it to the closest boundary.

class MyStuff:
   def __init__(self):
       self.a = [2,4,6,8,10,12]

   def iterate(self):
       i = 0
       while i < len(self.a):
           val = yield self.a[i]
           if val == None:
               i += 1
           else:
               if val < 0:
                   val = 0
               if val >= len(self.a):
                   val = len(self.a) -1
               i = val

Notice how I first test what value the generator received and stored in val. If val is None, that means the generator received no value (the calling code used next(), or perhaps send(None)). However, if a value was passed, I limit it to the bounds of the array, and then save it to the index.

Incidentally, I first tried using the new conditional expression feature to set the value of i, instead of using two if statements. You can condense the lines following the else line in the preceding code to:

i = val if (val >= 0 and val < max) else (0 if val < 0 else max)

Personally, I think all of that on a single line is a bit unreadable and suffers from the same obfuscation that plagues so much old C code. Thus I opted for the longer version. (Although I must say this condensed line is more readable than the equivalent C code.)

When the generator code runs out, it throws an exception. If you want, you can loop through your next() calls, and wrap them in a try/catch block. Alternately, you can set up your yield call to return a value that indicates the end of the iteration. Here's a modified form of the constructor in the previous code:

   def __init__(self):
       self.a = [2,4,6,8,10,12, None]

I simply added a None to the end of the list. Then I can watch for that to test for the end of the list without having to catch an exception:

obj = MyStuff()
iterate = obj.iterate()
a = iterate.next()
while a != None:
   print a
   a = iterate.next()
iterate.close()

The final line calls the close() function, which is also new to generators with Python 2.5. This frees up the resources for the generator. If you call iterate.next() again after calling iterate.close(), you'll get a StopIteration exception.

If you're familiar with recent versions of Python, you might wonder where the built-in Python iter() function fits in. The answer is that if you name your generator function __iter__(), then your generator will automatically work with the iter() function. This feature also existed in Python 2.4, but with Python 2.5, you can now use the send() function in conjunction with iter(). If I change the name of the iterator() function in the preceding MyStuff function to __iter__, and keep everything the same, then the iter function works:

>>> obj = gen1.MyStuff2()
>>> i = iter(obj)
>>> i.next()
2
>>> i.send(3)
8
>>> i.next()
10

Context Management

Python now has a new statement called with. Many languages have a with statement, but they are usually different from the with statement in Python. In Python, the purpose of with is to let you set up a block of code that runs under a certain context. When that context finishes after your code runs, it runs some cleanup code.

What is a context? A common example is that of database transactions. With database transactions, you start a transaction, and then make all your changes to the database. Then you can decide whether to undo (or "roll back") all the changes or to commit them (that is, make them permanent), thus finishing the transaction.

While you're making the changes to the database, you're operating under the context of the particular database transaction. The context performs some code when the transaction starts (by creating the transaction and allocating the necessary resources) and then some code at the end (either by committing the changes, or by rolling back the changes). The context is the transaction itself, not the rolling back or committing. Contexts don't necessarily involve rolling back and committing changes.

Another example is that of a file being open. You can open a file, write to the file, and close the file. While writing to the file, you can think of your code as operating under the context of the file. The context, then, would open the file and, when you finish working with it, can handle the task of closing the file for you.

To create a context, create a class that includes the __enter__() and __exit__() member functions. In the __enter__() function, write the code that initializes the context (such as opening a file, or starting a database transaction), and in the __exit__() function either handle any exceptions or just perform the cleanup code. In the case of an exception, the __exit__() function receives three parameters describing the exception. (Note: For further details on creating your own contexts, see Writing Context Managers in PEP 843.

At first, it could seem like contexts are a bad idea. It seems like they hide away certain vital code (such as closing a file), leaving you unsure of whether the file actually closes. However, that's the case with any library that you use; abstraction hides away certain behavior, but you know that the library is doing the necessary work. The library lightens your work. Think of a context as simplifying your code, taking care of extra details that you otherwise would have to do yourself, as well as ensuring such behavior, much like garbage collection.

Several built-in Python modules and classes now have context management built in. To use them, use the new with keyword. Before you can use with, however, enable the with feature:

from __future__ import with_statement

(You won't have to do this starting with version 2.6.)

These built-in Python objects supporting contexts now have the necessary __enter__() and __exit__() functions to provide the context management. This will make life much easier in terms of cleanup. Look at the code:

with open('/test.txt', 'r') as f:
   for line in f:
       process(line)

By using the with statement, I don't need to worry about closing the file myself. The code does it for me. Indeed, if after this code runs I try to write the value of f, I'll see that the file is closed:

>>> print f
<closed file '/test.txt', mode 'r' at 0x009E6890>

For me personally, using a context such as this will require a slight shift in my thinking as I write such code, but in the end, it will help us all write code that has fewer bugs, because we won't have to worry about forgetting to close the files or performing other required cleanup code.

A Few Minor Language Changes

Here are a few more changes to the language.

Python 2.5 includes a few more minor changes that have short descriptions. Rather than reiterate them here, Other Language Changes discusses:

New or Improved Libraries

The core modules have seen many changes. The full details are available in New, Improved, and Removed Modules. Here are some highlights.

Calling Arbitrary Shared Libraries and DLLs

Python 2.5 now has a nice package called ctypes. (This package has been around for awhile, but now has a new version and is included as part of Python.) ctypes lets you load external shared libraries and then call functions that are inside the libraries. Here's an example on Windows where I'm calling a function inside the system library USER32:

>>> import ctypes
>>> libc = ctypes.CDLL('c:/windows/system32/user32.dll')
>>> print libc.GetDesktopWindow()

Although I've refrained from personal opinion throughout this article, I have to say, this module is a godsend for those of us who have written code that calls into third-party libraries.

Working with XML through ElementTree

In the old days, parsing XML files was a nightmare. Generating XML files was a bit easier, although you had to do a lot of the work yourself. As the use of XML has grown and evolved, people have found better programming paradigms for managing XML (without XML itself changing). The ElementTree module greatly simplifies XML processing:

>>> from xml.etree import ElementTree as ET
>>> tree = ET.parse('test.xml')
>>> r = tree.getroot()
>>> r
<Element doc at aeeb70>
>>> def trav(node, indent=0):
...     for c in node.getchildren():
...         print ' '*indent, c.tag, ':', c.text
...         trav(c, indent+1)
...
>>>
>>> trav(r)
header1 : abc
text1 : This is some
    b : text
text1 : This is some more text
>>>

The classes in ElementTree let you read in your XML and traverse the XML hierarchy. This sample code I wrote uses recursions to traverse down through the children. For much of my own work with XML, I find that parsing the XML hierarchically like this rather than linearly is much more useful.

SQLite Support for Simple SQL Data Storage

The sqlite3 module is an interface to the SQLite library. This library provides SQL access to a single file for data storage, and I find it to be extremely useful for storing small amounts of local data without the need for large external database systems. Here's a sample:

import sqlite3
conn = sqlite3.connect('data.dat')
c = conn.cursor()
c.execute('''
   create table members(
       id int,
       name varchar,
       login timestamp
   )
''')
c.execute('''
   insert into members
   values (1,'jeff','2006-10-01')
''')
c.execute('''
   insert into members
   values (2,'angie','2006-10-02')
''')
c.execute('''
   insert into members
   values (3,'dylan','2006-10-03')
''')

res = c.execute('select * from members')
for row in res:
   print row

This code connects to a database (which is simply a file, data.dat), then creates a new table in the database using standard SQL CREATE TABLE syntax, then inserts data, again using standard SQL syntax. Then the code finally selects data from the table, again with SQL, and iterates through the selected data.

This module is especially useful for data that isn't big enough to justify a large database system, but is relational in nature and requires more than simple data types.

Summary

Python 2.5 includes many welcome changes. For me personally, I will get great use out of the new ElementTree classes for XML processing and the SQLite classes for storing data.

The changes to the language itself, particularly generators and contexts, will help me write more robust code, taking away the emphasis on how to do something, and focusing more on what to do. For my own work, I know that moving the emphasis in that direction always helps reduce bugs.

As you explore the changes to version 2.5, remember that many of the changes are precursors of what will come when the big Python 3.0 eventually appears. If you start adopting the changes now, you'll be better prepared for Python 3.0, requiring less porting and changing of code. I encourage you to explore these changes and use them, and, most importantly, have fun!

Jeff Cogswell lives near Cincinnati and has been working as a software engineer for over 15 years.


Return to the Python DevCenter.

Copyright © 2009 O'Reilly Media, Inc.