It's hard to believe Python is more than 15 years old already. While that may seem old for a programming language, in the case of Python it means the language is mature. In spite of its age, the newest versions of Python are powerful, providing everything you would expect from a modern programming language.
This article provides a rundown of the new and important features of Python 2.5. I assume that you're familiar with Python and aren't looking for an introductory tutorial, although in some cases I do introduce some of the material, such as generators.
Python 2.5 includes many useful improvements to the language. None of the changes are huge; nor do they require changes to your existing 2.4 code.
Other languages, particularly C and C++, have for years allowed conditional expressions, which are expressions that include an
if statement. The idea is simple: You have a variable, and you want to set the variable to one value if some condition is true; otherwise, you want to set the variable to another variable.
Certainly, you could always do this in Python using a block if statement like so:
if x > 15: a = 1 else: a = 2
Many programmers prefer shortcuts, however. The C language introduced the idea of doing the same assignment on a single line:
a = x > 15 ? 1 : 2;
In the C code, if the condition
x > 15 evaluates to true, the expression evaluates to 1. Otherwise it evaluates to 2.
Python 2.5 now has a similar feature. The Python form is rearranged a bit:
a = 1 if x > 15 else 2
Although conditional expressions exist in other languages such as the statically typed C and C++, Python's type system is different. While exploring this new feature, I was curious if I could modify types in the middle of the construct. Here goes:
a = "abc" if x > 15 else 10
This ran just fine. Of course, it's probably not the safest code in terms of introducing bugs, but you can use different types if you have a compelling reason to do so.
Python 2.5 includes changes to the
Exception class hierarchy. In general, the changes are mostly transitional changes in preparation for Python 3.0, with certain pre-2.5 features having been deprecated. That means you can still use the previous features, but the language will discourage you from doing so, as they will no longer work once Python 3.0 eventually arrives.
One newly-deprecated feature is the use of raising strings as exceptions. Previously (I used Python 2.4.3), you could write code such as:
>>> raise "Invalid argument" Traceback (most recent call last): File "<stdin>", line 1, in ? Invalid argument >>>
With Python 2.5, you can still raise strings as exceptions, but you also get a deprecation warning:
>>> raise "Invalid argument" __main__:1: DeprecationWarning: raising a string exception is deprecated Traceback (most recent call last): File "<stdin>", line 1, in <module> Invalid argument >>>
However, you only get the warning the first time per Python session. Subsequent raises don't result in the warning. Python 2.5 encourages you to stop writing code like this, because when Python 3.0 arrives, your code will not compile.
In addition to the deprecation of raising strings as exceptions, Python 2.5 also rearranged the exception class hierarchy. The new
BaseException class is the base class for
Exception. You should now derive all your custom exception classes from
Exception, or from a class derived from
Two other exception classes,
SystemExit, also extend
BaseException, but not
Exception. Thus, you can trap your own exceptions, and then trap all other exceptions derived from
Exception. This will trap every exception except
SystemExit. Why would you not want to trap those two exceptions? That way your program will end when the user presses Ctrl+C without you having to manually catch and respond to Ctrl+C, or without you having to manually skip catching Ctrl+C to let the system handle it. The same is true with the
SystemExit exception, which should force the program to stop.
This particular new feature, then, is less of a feature and more of a requirement: Recode your exception classes to extend
Exception. Then you'll be fine when Python 3.0 arrives.
Finally, there is now another short warning about future exceptions in Python. Presently, exception classes have both a message and an args member. The general idea was that message would be a string, and args would be a list of any additional arguments to the exception. However, this is redundant, because the message member doesn't have to be a string. Thus, Python 3.0 will have no more args member, so you should stop using it.
In addition to the changes to the hierarchy, the try mechanism has improved with Python 2.5. Previously, you could not mix
finally blocks without using two layers of exceptions. Now you can use them both in a single block, which makes life easier. The order matters, however. You must have your
try block first, then optionally your exception handlers, then optionally an
else block that runs if no exceptions occurred, and then optionally your
finally block, which runs in all cases. Here's an example:
try: test() except ZeroDivisionError: print "The expression resulted in a division-by-zero error" except Exception: print "Oops" else: print "All is good" finally: print "And everything is finished"
In previous versions, you had to use two
try blocks, with one inside the other, and the outer having the
finally block. Now you can use a single block for all of it.
Generators entered the language with Python version 2.3. Version 2.5 extends the capabilities of generators.
Generators are functions that can be exited and re-entered, and are useful for iterators. For example, here's a class that includes a generator called iterate:
class MyStuff: def __init__(self): self.a = [2,4,6,8,10,12] def iterate(self): i = 0 while i < len(self.a): yield self.a[i] i += 1
This code uses of the generator, demonstrating the iterator feature:
obj = MyStuff() iterate = obj.iterate() print iterate.next() print iterate.next() print iterate.next()
The first line creates an instance of the class
MyStuff. The second line retrieves an iterator from the instance. Notice that the iterator itself isn't the value obtained from the generator; instead, you call
next to get the value. If you're in the interactive shell, you can see what happens if you try to take the iterator's value:
>>> print iterate <generator object at 0x009EB698> >>>
This is different from iterators in C++. Though you can dereference an iterator in C++ to obtain the value, in Python the generator is a built-in generator object.
The idea behind the generator is that you call the function that includes a
yield statement. The
yield statement causes execution of the function to suspend and return. You can then go back into the function, picking up where it left off, by calling
next(). Thus, when you called:
... the execution stepped into the function, and the first two lines ran (the assignment statement and the first line of the while loop). The next line is the yield statement, which caused execution to return to the caller, temporarily returning the value
Then, when you called
iterate.next() again, execution resumed after the
yield statement with the line that increments i. The while loop continued again until the next
Note that creating the generator with
iterate = obj.iterate(), doesn't actually call into the function. The Python compiler saw the
yield statement inside the function and identified it as an iterator, causing the call to the function to create a generator object instead. You can verify this if you add some
def iterate(self): print "Creating iterator" i = 0 ...rest of code...
You won't see the output "Creating iterator" when you call
iterate = obj.iterate(), but you will see it when you make the first
That's a quick introduction to generators, and is not new to Python 2.5. However, people have identified shortcomings to the generator feature. In languages that provide for generators, an important feature is the ability to pass a value back into the generator. This allows for supporting a programming feature called coroutines.
In order to make the generators more powerful, the designers of Python have added the ability to pass data back into the generator. If, from the perspective of the generator, you think of the
yield as calling something, then the concept is easy: You just save the results of
x = yield y
From the perspective of the caller of the generator, this statement returns control back to the caller, just as before. From the perspective of the generator, when the execution comes back into the generator, a value will come with it--in this case, the generator saves it into the variable x.
Where does the value come from? The caller calls a function called
send() to pass a value back into the generator. The function
send() behaves just like the function
next(), except that it passes a value.
Here's some modified code that demonstrates how to use
send(). In this case, I use
send() to let the user of the generator reposition the index in the array. However, I'm making sure that the value sent in is within the bounds of the array. Otherwise I set it to the closest boundary.
class MyStuff: def __init__(self): self.a = [2,4,6,8,10,12] def iterate(self): i = 0 while i < len(self.a): val = yield self.a[i] if val == None: i += 1 else: if val < 0: val = 0 if val >= len(self.a): val = len(self.a) -1 i = val
Notice how I first test what value the generator received and stored in
None, that means the generator received no value (the calling code used
next(), or perhaps
send(None)). However, if a value was passed, I limit it to the bounds of the array, and then save it to the index.
Incidentally, I first tried using the new conditional expression feature to set the value of
i, instead of using two if statements. You can condense the lines following the
else line in the preceding code to:
i = val if (val >= 0 and val < max) else (0 if val < 0 else max)
Personally, I think all of that on a single line is a bit unreadable and suffers from the same obfuscation that plagues so much old C code. Thus I opted for the longer version. (Although I must say this condensed line is more readable than the equivalent C code.)
When the generator code runs out, it throws an exception. If you want, you can loop through your
next() calls, and wrap them in a
catch block. Alternately, you can set up your
yield call to return a value that indicates the end of the iteration. Here's a modified form of the constructor in the previous code:
def __init__(self): self.a = [2,4,6,8,10,12, None]
I simply added a
None to the end of the list. Then I can watch for that to test for the end of the list without having to catch an exception:
obj = MyStuff() iterate = obj.iterate() a = iterate.next() while a != None: print a a = iterate.next() iterate.close()
The final line calls the
close() function, which is also new to generators with Python 2.5. This frees up the resources for the generator. If you call
iterate.next() again after calling
iterate.close(), you'll get a
If you're familiar with recent versions of Python, you might wonder where the built-in Python
iter() function fits in. The answer is that if you name your generator function
__iter__(), then your generator will automatically work with the
iter() function. This feature also existed in Python 2.4, but with Python 2.5, you can now use the
send() function in conjunction with
iter(). If I change the name of the
iterator() function in the preceding MyStuff function to
__iter__, and keep everything the same, then the iter function works:
>>> obj = gen1.MyStuff2() >>> i = iter(obj) >>> i.next() 2 >>> i.send(3) 8 >>> i.next() 10
Python now has a new statement called
with. Many languages have a
with statement, but they are usually different from the
with statement in Python. In Python, the purpose of
with is to let you set up a block of code that runs under a certain context. When that context finishes after your code runs, it runs some cleanup code.
What is a context? A common example is that of database transactions. With database transactions, you start a transaction, and then make all your changes to the database. Then you can decide whether to undo (or "roll back") all the changes or to commit them (that is, make them permanent), thus finishing the transaction.
While you're making the changes to the database, you're operating under the context of the particular database transaction. The context performs some code when the transaction starts (by creating the transaction and allocating the necessary resources) and then some code at the end (either by committing the changes, or by rolling back the changes). The context is the transaction itself, not the rolling back or committing. Contexts don't necessarily involve rolling back and committing changes.
Another example is that of a file being open. You can open a file, write to the file, and close the file. While writing to the file, you can think of your code as operating under the context of the file. The context, then, would open the file and, when you finish working with it, can handle the task of closing the file for you.
To create a context, create a class that includes the
__exit__() member functions. In the
__enter__() function, write the code that initializes the context (such as opening a file, or starting a database transaction), and in the
__exit__() function either handle any exceptions or just perform the cleanup code. In the case of an exception, the
__exit__() function receives three parameters describing the exception. (Note: For further details on creating your own contexts, see Writing Context Managers in PEP 843.
At first, it could seem like contexts are a bad idea. It seems like they hide away certain vital code (such as closing a file), leaving you unsure of whether the file actually closes. However, that's the case with any library that you use; abstraction hides away certain behavior, but you know that the library is doing the necessary work. The library lightens your work. Think of a context as simplifying your code, taking care of extra details that you otherwise would have to do yourself, as well as ensuring such behavior, much like garbage collection.
Several built-in Python modules and classes now have context management built in. To use them, use the new
with keyword. Before you can use with, however, enable the
from __future__ import with_statement
(You won't have to do this starting with version 2.6.)
These built-in Python objects supporting contexts now have the necessary
__exit__() functions to provide the context management. This will make life much easier in terms of cleanup. Look at the code:
with open('/test.txt', 'r') as f: for line in f: process(line)
By using the
with statement, I don't need to worry about closing the file myself. The code does it for me. Indeed, if after this code runs I try to write the value of
f, I'll see that the file is closed:
>>> print f <closed file '/test.txt', mode 'r' at 0x009E6890>
For me personally, using a context such as this will require a slight shift in my thinking as I write such code, but in the end, it will help us all write code that has fewer bugs, because we won't have to worry about forgetting to close the files or performing other required cleanup code.
Here are a few more changes to the language.
dict, you can define a
__missing__(self, key)function to handle retrievals when the key doesn't exist in the dictionary.
rpartitionmethods of the string class. These functions let you split a string on a particular substring, retrieving a tuple containing the substring to the left of the separator, the separator itself, and the substring to the right of the separator.
endswith. The string methods
endswithmethods already exist for testing whether a string starts or ends with a given substring. Previously, you could only test for the existence of a single substring. Now, with version 2.5, you can pass a tuple to test for the existence of any of a given set of substrings.
Key functions with
max. You can now pass the name of a function to the
max function to evaluate the ordinality of each function. For example, you might determine a string's ordinality with:
def myord(astr): try: num = int(astr) return num except: return len(astr)
Then, given a list of strings, you can evaluate the maximum with the function:
print max(['abc','100','a','xyz','1'], key=myord)
More binary expression functions. Python 2.5 now includes two built-in functions that are useful for binary expressions. The function
any() returns true if any of its items are true. The function
all() returns true only if all items are true. Here's some sample code that shows
any() in action:
>>> a = 10 >>> b = 20 >>> any([a>5, b>5]) True >>> any([a>50, b>50]) False >>> any([a>50, b>5]) True
Python 2.5 includes a few more minor changes that have short descriptions. Rather than reiterate them here, Other Language Changes discusses:
quit()function in the interpreter
The core modules have seen many changes. The full details are available in New, Improved, and Removed Modules. Here are some highlights.
Python 2.5 now has a nice package called ctypes. (This package has been around for awhile, but now has a new version and is included as part of Python.) ctypes lets you load external shared libraries and then call functions that are inside the libraries. Here's an example on Windows where I'm calling a function inside the system library USER32:
>>> import ctypes >>> libc = ctypes.CDLL('c:/windows/system32/user32.dll') >>> print libc.GetDesktopWindow()
Although I've refrained from personal opinion throughout this article, I have to say, this module is a godsend for those of us who have written code that calls into third-party libraries.
In the old days, parsing XML files was a nightmare. Generating XML files was a bit easier, although you had to do a lot of the work yourself. As the use of XML has grown and evolved, people have found better programming paradigms for managing XML (without XML itself changing). The
ElementTree module greatly simplifies XML processing:
>>> from xml.etree import ElementTree as ET >>> tree = ET.parse('test.xml') >>> r = tree.getroot() >>> r <Element doc at aeeb70> >>> def trav(node, indent=0): ... for c in node.getchildren(): ... print ' '*indent, c.tag, ':', c.text ... trav(c, indent+1) ... >>> >>> trav(r) header1 : abc text1 : This is some b : text text1 : This is some more text >>>
The classes in
ElementTree let you read in your XML and traverse the XML hierarchy. This sample code I wrote uses recursions to traverse down through the children. For much of my own work with XML, I find that parsing the XML hierarchically like this rather than linearly is much more useful.
sqlite3 module is an interface to the SQLite library. This library provides SQL access to a single file for data storage, and I find it to be extremely useful for storing small amounts of local data without the need for large external database systems. Here's a sample:
import sqlite3 conn = sqlite3.connect('data.dat') c = conn.cursor() c.execute(''' create table members( id int, name varchar, login timestamp ) ''') c.execute(''' insert into members values (1,'jeff','2006-10-01') ''') c.execute(''' insert into members values (2,'angie','2006-10-02') ''') c.execute(''' insert into members values (3,'dylan','2006-10-03') ''') res = c.execute('select * from members') for row in res: print row
This code connects to a database (which is simply a file, data.dat), then creates a new table in the database using standard SQL
CREATE TABLE syntax, then inserts data, again using standard SQL syntax. Then the code finally selects data from the table, again with SQL, and iterates through the selected data.
This module is especially useful for data that isn't big enough to justify a large database system, but is relational in nature and requires more than simple data types.
Python 2.5 includes many welcome changes. For me personally, I will get great use out of the new
ElementTree classes for XML processing and the
SQLite classes for storing data.
The changes to the language itself, particularly generators and contexts, will help me write more robust code, taking away the emphasis on how to do something, and focusing more on what to do. For my own work, I know that moving the emphasis in that direction always helps reduce bugs.
As you explore the changes to version 2.5, remember that many of the changes are precursors of what will come when the big Python 3.0 eventually appears. If you start adopting the changes now, you'll be better prepared for Python 3.0, requiring less porting and changing of code. I encourage you to explore these changes and use them, and, most importantly, have fun!
Jeff Cogswell lives near Cincinnati and has been working as a software engineer for over 15 years.
Return to the Python DevCenter.
Copyright © 2009 O'Reilly Media, Inc.