Making the Python Cookbook
by Bruce Stewart
O'Reilly has just released the Python Cookbook, a collection of problems, solutions, and practical examples written by a large cast of Python programmers. The book includes over two hundred recipes, with contributions from Python luminaries Guido van Rossum, Tim Peters, Paul Prescod, and Mark Hammond. Putting together such a collective work presented some unique challenges, and we recently spoke to the book's two editors, Alex Martelli and David Ascher, about the process.
Stewart: What inspired you to write the Python Cookbook?
Ascher: As I describe in the preface, the history of the book is quite convoluted. My late friend Frank Willison, who had edited "Learning Python", had been talking to me about my "next Python book" for some time. We'd batted around the idea of a JPython book (the name Jython wasn't around yet), and other topics, and then I got hired by ActiveState, which resulted in a significant decline in the number of hours I could spend writing Python books (luckily, the number of hours writing Python /code/ increased dramatically!). Months later, Frank and ActiveState's CEO Dick Hardt figured out that it was in both companies' interest to cooperate on a cookbook--ActiveState would run the online submission process, and O'Reilly would print it. Neither of them had a clue how much work would be involved, I'm sure--although maybe Frank did, and simply decided to avoid warning me!
Martelli: I knew a cookbook was just what Python needed--when David Ascher and Laura Lewin invited me to join the existing project I jumped at the opportunity.
Actually, I wanted to do a cookbook even before I knew O'Reilly was already planning one with ActiveState--I had sent a proposal for one, although ActiveState's "community-written" idea had rightly been preferred, and by then I had changed my plans to write the Python Nutshell book instead.
Stewart: How does Python lend itself to the cookbook approach?
Martelli: Learning the language is like learning to drive; the cookbook is like a series of maps accompanied by tips from expert drivers who often go there and can tell you which intersection to avoid as overcrowded, what tempting shortcuts to take, and which ones to avoid and why. Makes your driving, oops programming, far more enjoyable and productive than having to find it out all by yourself by unaided trial and error!
Stewart: How many recipes are there in the Python Cookbook, and how many different people contributed to the final product?
Martelli: Over 200 recipes, from over 100 contributors.
Stewart: Was it difficult to work with so many contributors? What were some of the challenges of putting together this kind of book?
Martelli: It was a real challenge to both respect the individuality and diversity of each contributor's style and ideas, and the uniformity of presentation and style expected from an O'Reilly cookbook. We did our best to meet these two apparently conflicting goals, keeping in mind that there's a meta-goal they both stem from: give the reader of the book the maximum value, in terms of both clarity and ease of reading (best served by uniformity) and also exposure to a wide range of individuals' approach to Python (best served by diversity). So, each time an editing decision had to be made, the meta-goal of "maximum value to the reader" was always foremost in our mind.
Our execution of these principles may not have been always impeccable--we were, after all, breaking new ground with this "multiple-authorship, open source cookbook," and, as the saying goes, "You can always tell the pioneers by the arrows in their backs." But I think the principles are correct and valuable, and worth keeping in mind for every editor and author: "you're doing all of this for readers--so the right choice in each and every case is the one that maximizes benefits to readers".
Ascher: Alex's answer is a good one. I'll just add the following:
One of the challenges that Alex was (happily, I'm sure) not exposed to was the difficulty of designing a Web site that was a "book entry form". In most "edited volumes," the editors get to coordinate with all of their preselected authors, assigning a chapter and topic to each, setting general editorial guidelines, and ensuring uniformity of coverage and equal representation to the important voices in the field is relatively easy.
Here, authors were self-selected, and wrote about the topics that they cared about. While we managed to do more of the traditional author-management when it came to finding introduction authors, the pool of recipes was very uneven. Some sections were extremely rich (for example, the "Algorithms" and "Object-Oriented" chapters), while other sections had to be dispersed throughout other chapters. One such example of this is Jython because for whatever reason, few recipes specific to Jython were submitted. Having Alex around to help fill in the gaps was crucial to the book's successful completion.
Stewart: Why is your book especially important now?
Martelli: Python use is growing fast, and a foremost need is to expose new users to examples of how-tos and good practices, just like the recipes in the cookbook.
The theme of "the right moment is right now" has been growing in my eyes. The Python Business Forum has just been formed, and one of its main plans is to identify, exhaustively test, and take over maintenance, with Guido's blessing, of one chosen Python version, to be known as "Python in a Tie"--so as to provide business users (the Forum is a consortium of firms for whom Python is mission-critical) with a solid version of Python that will change only rarely, while leaving the Python core group of developers free to move forward at their chosen, faster pace.
The current, brightest prospect for "Python in a Tie" is exactly the Python version that the Python Cookbook ended up covering so well--Python 2.2. I predict a surge in business use of Python as the Forum executes and advertises their plans--and the new business users will all be users of Python 2.2, at which the Cookbook (and the coming Nutshell) are exactly targeted. Solid, field-proven examples coming from the vast and varied Python community, with discussion of pro's and con's, applicability and alternatives, advice and tips-- that's what the Cookbook is all about, and what every newcomer to Python and existing user of Python who's looking for a leg up in making even more effective use of this wonderful language needs.
Stewart: What makes your book unique compared with other Python books?
Martelli: It's a vastly collective work: the recipes come from hundreds of authors who freely donated it for the good of Python, so the book inherently deals with what current Pythonistas consider useful, interesting, and important. Organization and introduction of each chapter was then added by an expert in each field. We edited quite a bit, selected recipes, organized them, added discussion and a few recipes for completeness, all to make it into a consistent whole, but it remains a collective work, unique today in its being a book written by a whole software community.
Stewart: Who is your intended audience?
Martelli: People who know at least a little Python but haven't yet had the good fortune of working side by side with real experts of the language and its many fields of application--the book gives them the same kind of know-how such experience would give.
I would advise the book even to somebody who's not directly interested in coding in Python, but just to grasp what Python is all about. For example, the Cookbook can help make a strategic decision about what language will be used in a given project--even without following the details, the many recipes, and complete examples of both modules and programs, and the tone of the discussion.
Stewart: Do you have any interesting anecdotes about writing the book?
Martelli: My father jocularly threatened to bill me for the coffee he was making me, as I was consuming so much more--writing this book meant more long nights of work than I had ever had to do in programming!.
I paid him back by doing him homage--the last and most important thanks in my acknowledgments in the Preface ... not just for the coffee, but for the lifelong example of dedication and hard work, never without enthusiasm. He turned 80 after these words were written, and he's still working, both as a university teacher and in professional practice, in his chosen field (medicine), though fortunately, not full-time any more, so he now has the time to make me all of that excellent "espresso".
Stewart: Any anecdotes about the language?
Martelli: Shortly after I was introduced to Python, my girlfriend became worried--many little signs in my attitude hinted to her that I had fallen in love, and of course, she thought she had a rival since I had not mentioned the programming language I had been newly studying. She had seen me enthusiastic about technical stuff many times before, but never anywhere close to that extent!
We still kid that Python is "her rival" to this day, most particularly as my love for it is proving as solid and deep-rooted as it is for her. These days I'm even "commuting" (almost a week per month, the rest of the time telecommuting via the Net), from my home in Bologna to the Gothenburg, Sweden, offices of AB Strakt, a Python-centered firm for which I now work....
Stewart: What was ActiveState's role in the Python Cookbook project?
Ascher: Three major parts: First, ActiveState ran the Web site, which means that our Web team was responsible for designing and implementing the online cookbook (which is still available, and used for multiple languages--check it out. Then ActiveState and O'Reilly both got the word out about the Cookbook, which was essential for getting high-quality submissions. Finally, ActiveState pays my salary.
Stewart: David, give us an update on ActiveState. What are you folks up to these days?
Ascher: A whole bunch of neat projects are in the works right now--summertime is probably our most intense coding season, as we were rushing to get things in shape to show them off at last month's Open Source Convention. PerlMx is our Perl-based Spam, virus, and general mail-filtering software, which is making lots of mail users happy again; our Visual Studio.NET plug-ins are being updated with cool new features; however, my personal favorite is the project I'm working on, which is our next version of our cross-platform, multilanguage integrated development environment (Komodo for short)--I can't talk about the new features yet but I'm quite pleased with what we've done. Also, most of the interesting parts of Komodo are written in Python, which is still as much fun to write in as it was the first day.
Stewart: What do you think is on the horizon for Python?
Martelli: I think Python's current rapid evolution is due to slow down pretty soon as the language comes closer and closer to its designer's ideals (it's already quite close). Growth in the user base is going to continue, so more and more prospective users will need to "learn the ropes".
Stewart: You're both involved in the Python Software Foundation. Tell us a little about that organization, and how people can get involved.
Martelli: I'll let David take that one, but I think readers will be interested in hearing that some of what would normally be author royalties for the Cookbook will go to the Python Software Foundation, a nonprofit endeavor whose goal is promoting Python and supporting its use.
Ascher: I'm a director of the PSF, which means that I am part of the board--the governing body of the PSF. We are building an organization that can serve the interests of our membership--people like Alex who have significantly contributed to Python, and who want to ensure that Python thrives in the long term. To that end, we are setting up a legal and financial infrastructure that will make it easy for the PSF to responsibly own the intellectual property in Python, as well as foster Python's growth. Individual members are invited to join the PSF thanks to their contributions--however, corporations who share in the PSF's goals are encouraged to apply for sponsor membership. Information about the PSF is available at the Python Software Foundation Home Page. In addition, I wrote an article for the Perl Review about the PSF (a little joke, but the article is serious!)--the relevant issue is available online.
Stewart: David, Learning Python is one the most popular Python books on the market. Is an update in the works?
Ascher: Now that the Cookbook is out of the way, it's my next book project. It's an exciting update, since so much has changed, what with Python 2.0, 2.1, 2.2, and the 2.3 release currently being planned. Luckily, almost all of the content in the first edition still works and is still useful. But it's definitely time for an update.
Stewart: Alex, you're working on Python in a Nutshell for O'Reilly. How's that coming?
Martelli: When I started working on the Cookbook it was almost suspended from October 2001 to March , and then when I changed my job in order to be able to work on Python full-time. But I put the delay to good effect, by revamping the whole Nutshell so it now covers all the new developments in Python 2.2, the current and most significant version of Python--just as the Cookbook now does. The Nutshell is now due to come out in October 2002, and we're on schedule to meet that goal.
Bruce Stewart is a freelance technology writer and editor.
Return to the Python DevCenter.
Copyright © 2009 O'Reilly Media, Inc.