An Interview with the Author of Practical mod_perl by chromatic 07/03/2003

Editor's Note: Stas Bekman is a long-time contributor to mod_perl. In addition to writing the mod_perl guide, he's also coauthor of the recently released Practical mod_perl. Stas recently agreed to a brief email interview about his work, mod_perl 2, and what it's like to be sponsored to work on free software full-time.

O'Reilly Network: You've been involved in mod_perl as long as I can remember (five years). How did you get your start?

Stas Bekman: Back in 1997 I was playing with WDBI (Web DataBase Interface, used to be wdbi.net, now only in the archive) and using it as a core part of the web product I was developing for one of my clients. Jeff Rowe, the lead WDBI developer, has mentioned that WDBI also works under mod_perl. So I decided to check it out. That's the prelude to the story.

Phase I: Discovering the mod_perl magic

After playing a bit with mod_perl I've learned that mod_perl was the thing I was dreaming about all my previous "CGI life." :) All the complaints about CGIs written in Perl being slow, mainly because on every request the Perl interpreter was reloaded and all of the code recompiled, were irrelevant now. mod_perl has solved all the problems, and even more — I could now write Perl code that could intervene at any stage of the Apache request processing and do virtually anything I want, something that you normally needed to write in C. This led to faster and easier coding, and more third-party code written, since more people are familiar with Perl and it's easier to code in Perl than in C.

Phase II: Getting involved in the mod_perl mailing list activity

I've joined the mod_perl list and spent months as a silent lurker, trying to learn as much as possible from the list's discussions. After awhile, I had enough courage to try and answer some simple questions. With time, I've gained the confidence and started to answer all the questions I could answer. Sometimes I'd even answered questions I didn't have answers for. I've learned that in many cases, people just need support and direction, so many times I've just suggested things to look at and people were solving the problems by themselves. All they need is support and knowledge that someone out there listens to their problems.

Related Reading Practical mod_perl By Stas Bekman, Eric Cholet Table of Contents Index Sample Chapter Read Online--Safari Search this book on Safari:   Only This Book All of Safari Code Fragments only

Phase III: Coping with broken-record discussion threads; the birth of the guide.

Back then, people tended to ask the same questions again and again, which resulted in lots of negative waves on the list, as the inhabitants of the list were bored to answer the same questions again and again, and many people were popping in to ask some question without reading the mailing list archives. There were a few documents written that answered on quite a few common questions, but it wasn't enough. That's when I've decided to start summarizing the answers and put them into a mod_perl guide. It used to be called mini-guide, even when it was about 650 pages long. perl.apache.org/'s documentation volume has long surpassed 1000 pages since then. There are several new mod_perl 2.0 documents and we started to host general web-technologies tutorials useful to mod_perl users. Started as a personal project, it became a community project with more than 200 contributors helping to improve the documentation. I must single out Ged Haywood and Mark Summerfield, who have reviewed the whole mod_perl 1.0 guide. I'd like to use the opportunity to thank all those helpful folks.

The main advantage of having the guide was and is the ability to post a direct link to the section documenting the answer to a question. So even if the questioning person didn't bother to look for an answer, or simply couldn't find it, we can point to the place where the answer is. This has cut a lot of the wasteful traffic and the mod_perl list is a much happier place to be in since then.

The online documentation being still a bit of a mess, I have decided to rewrite it as a book, building on the existing information, but restructuring it for easier reading; providing more examples, diagrams, etc. Linda Mui, the ORA editor, liked the idea, and that's how I started working on the Practical mod_perl book. It was a very hard project for me, since English isn't my mother tongue and I needed to have a real job and I also wanted to hack my own things. Therefore, somewhere in the middle of the road, I asked Eric Cholet to join my efforts. He kindly accepted the offer, but not before I've convinced him that it's going to be a very easy project to work on. Yeah, right ...

Oh, and the book has now been finally published.

One nice side effect of working on the documentation is that I was invited to teach at conferences, user group meetings, and companies. Trying to produce good presentation materials led to better online documentation.

Phase IV: Working on the core mod_perl

I didn't quite want to remain a documentation guy forever, since after all, no programmer loves writing documentation. I was writing mod_perl code and third-party modules all that time, but didn't have much chance to hack on the core. The main obstacles were unfamiliarity with Apache 1.3 guts, rusty C, and lack of XS and Perl guts knowledge. That was about the time when Doug had released the first prototype of mod_perl 2.0. I've decided that rather trying to dive into mod_perl 1.0, where almost everything has been already done, I can start fresh with mod_perl 2.0.

Getting started with the mod_perl 2.0 core was a very frustrating time for me. At the very beginning, I'd try to add a very simple feature and I'd spend a week trying to solve a segfault my several lines code was causing. The main problem is that mod_perl is not a stand-alone thing — it's based on Apache and Perl. Therefore, in order to code mod_perl 2.0 guts, one has to have a pretty intimate knowledge of Apache 2.0 and Perl 5.6.x-5.9.x guts, staying up to date with all of the new developments on the cutting edge of Apache, APR, and Perl projects, making sure that newly introduced features of these projects don't break mod_perl. Not only did I have to follow Perl core development, but also modules like MakeMaker and ithreads, upon which mod_perl heavily relies. Many times I had to debug Perl and Apache, because the bugs were there and not in mod_perl. These bugs had to be solved, and many times I had to produce the fix myself and then bug those with commit access to accept my patches. I have spent hundreds of hours stepping through the code with debugger and now ddd (the gdb GUI front end) is my best friend, and I'm no longer terrified of copying with segfaults.

ORN: Ticketmaster hired you recently (last year?) to work on mod_perl full time. How did that come about? What have you been building? I see your patches on p5p occasionally. Have they given you any specific direction for new features, or is it all up to you?

SB: About the time I decided to work on the mod_perl 2.0 core, I was quite busy writing the book, maintaining the documentation, hacking third-party modules, and working a full-time job. I've realized that what was missing is life, when the screen is off. So on one hand, I was trying to squeeze into 25 hours day yet another thing to do with the screen on, the 2.0 core development, at the same time trying to get a life. I've realized that the easiest solution is to drop my day job, turning and expanding all my writing and development after-hours activities to be my day job, and using the freed time to have a life. There was a little problem remaining — how do I get my ends together at the end of each month? No fear, I said to myself. Let's ask if somebody is willing to sponsor my work on mod_perl core. And here on 27 Apr 2001 I'm posting this bold request to the mod_perl list. The post has generated a lot of interesting ideas, but no concrete proposals from companies. And then all of a sudden it happened — a month later Craig McLane, from Ticketmaster, contacted me, asking whether I'm still interested in being sponsored. I don't have to tell you that I didn't refuse the offer. ;)

Not only did Ticketmaster extend this great offer to me, they gave me the freedom to hack on whatever I want. Therefore, all of the patches that you see me posting to p5p, httpd-dev, and apr-dev lists are to solve the problems discovered while developing and using mod_perl.

Starting from Sep 2001 I've spent the first year working on the same old thing — documentation, while wrapping my head around mod_perl, Perl and Apache guts, and contributing a bit to the core. The main achievement of that year was a complete remake of perl.apache.org/ in addition to improving the documentation and helping on with the mailing list. As with the documentation, credits are due to quite a few contributors, especially to Allan Juul, Bill Moseley, Thomas Klausner, and Per Einar Ellefsen.

After I've returned from my first year's burn-out-recovery-get-away around Everest, I have finally started to spend the majority of the time on writing the mod_perl 2.0 code, initially helping the project architect Doug MacEachern, who wrote most of its code. And I'm sort of leading it, while Doug is busy with his day job. Luckily Geoffrey Young, Philippe M. Chiasson, and Randy Kobes are helping a lot with it.

ORN: What's your favorite new feature of Apache 2? mod_perl 2?

SB: I hate both. ;) Now I have to write all the documentation from scratch, as both come loaded with many new features that didn't exist in the first generation.

Seriously, though, it's hard for me to pick the best new feature, as currently I'm developing mod_perl without really using it in the real world. Once I return to yet another crazy startup and put mod_perl 2.0 to use, I'll be better able to tell. I think that the threading support is going to be the main decision factor for people to upgrade to mod_perl 2.0, as we expect a big performance boost and memory savings with it.

I/O filters take mod_perl's flexibility to the next level. Protocol modules is another new venue to explore; imagine mod_perl handling HTTP, IMAP/POP3, SMTP, SVN, and many other protocols in one setup, reusing the same authentication, compression, and admin tools.

ORN: I've seen performance numbers where you say that mod_perl can serve a request in as little as 25msec — that's 40 requests per second. What kind of application is this?

SB: A well-tuned mod_perl CPU-bound application can give similar results. Once all the code is compiled, the performance is not far from C-compiled code. Also as you can read in the Practical mod_perl book, Chapter 13, it's quite easy to rewrite the bottleneck parts of the program in C, while having the rest of the program in Perl.

ORN: One feature I really liked about the mod_perl guide is that it mixed configuration and administration and coding. What's the breakdown in Practical mod_perl?

SB: Both the mod_perl guide and Practical mod_perl reflect the reality of mod_perl daily usage. One has to be proficient in administration and coding in order to get the best our of mod_perl, because these areas are so tightly related. For example sometimes a simple configuration tuning (Chapter 11), server setup (Chapter 12), or compilation change (Chapter 15) may give a better performance than a much longer time spent to optimize the code. You have to learn to look at the big picture and not just at the code.

Of course, coding is still the most important area, but luckily there are two great books, Writing Apache Modules with Perl and C (by Doug MacEachern and Lincoln Stein) and The mod_perl Developer's Cookbook (by Geoffrey Young, Paul Lindner and Randy Kobes), which concentrate on coding. Believe me, you really need these two books and Practical mod_perl to have a complete mod_perl reference.

Our book is called Practical mod_perl not only because it's a cool name, but mainly because it comprises six years of practical mod_perl community and our experiences.

ORN: Aside from Apache 2 necessitating mod_perl 2, what's the biggest change you've seen in mod_perl since you've been working with it?

The biggest change was in the mod_perl community. I think the mod_perl list is one of the most helpful and friendly lists in the open source domain. Considering the current economy downturn, it keeps its spirit high and most questions are answered (unless they are off-topic, which we try to kill as soon as possible). I believe that as the economy gets better, the community will become even stronger, and I hope that mod_perl 2.0 is going to boost people's productivity and free their hands to help others to make our tribe bigger and happier.

ORN: What's your favorite new module or feature?

SB: My favorite new module is not so much a mod_perl module, but an idiom coming from Perl and introduced by Doug MacEachern. It goes like this:

use strict;
use warnings FATAL => 'all';

It keeps me on my toes, when I develop my code and I get almost no bug reports since I've started using it.

chromatic promotes free and open source software for O'Reilly's Open Technology Exchange.

Return to ONLamp.com.