oreilly.comSafari Books Online.Conferences.


The Best of ONLamp 2003

by chromatic

Before you receive your official journalist card, you have to promise to write at least one retrospective article and one predictive article every December.

OK, that's not true. It is true that I rarely get the chance to talk about the articles we've published and what I like about them. This is my one chance this year for extremely shameless site promotion. Be prepared: a quick query shows more than 200 articles this year. No wonder they talk about a "long winter's nap!"

Popular Articles

Without further ado, here are the 25 most popular articles we published in the past year, in approximate order of popularity. I'm ranking them based on our internal statistics of page views, not any inherent goodness, controversy, or number of people who agreed with the views in the articles. You'll see why I say that in a moment.

My What I Hate About Your Programming Language set many people talking; talkbacks still trickle in seven months later. Possibly the two most important points of the article are that complexity in a language has to come out somewhere -- in the syntax, in the libraries, or in the way you solve complex problems -- and that the philosophy of the language designer, vocalized or not, guides all design and implementation decisions.

KIVILCIM Hindistan's Video Playback and Encoding with MPlayer and MEncode was a sort of mini-cookbook for video playback and manipulation. Though the developers have strong (and possibly strange) opinions, multimedia on Linux is the better for their work.

OSCON 2003 expanded upon Perl's popular "State of the Onion" address by inviting leaders from five other important open source projects to speak. Steve Holden's Guido van Rossum Speaks follows up on the Python creator's talk, especially his new job.

Rob Flickenger, ex-O'Reilly Network system administrator and now Hacks editor and author, contributed Creating Your Own CA. Well, actually, it was a promo piece out of Linux Server Hacks -- but, true to form, it's readable and highly useful.

O'Reilly Open Source Convention.

Glenn Graham's Synchronizing Networks with NTP explained how the Network Time Protocol worked and how it can keep all hosts on a network in sync. It may be a little thing, but knowing that my machines are all accurate to within microseconds is quite nice.

Æleen Frisch, author of Essential System Administration, revealed that CFEngine was her pick for Top Five Open Source Packages for System Administrators (#1). One of the benefits of Unix and Open Source is indeed choice, but unification provides a lot of convenience. If you read David N. Blank-Edelman's Perl for System Administration, you'll pick up a lot of other ideas for automating and simplifying your life.

Rob Flickenger's Cheap IP Takeover is yet another excerpt from Linux Server Hacks. You might be surprised to know how many of these cheap but effective and clever hacks we actually use here -- and I mean that in a positive way.

Glenn Graham's Using NFS for Networked Backups has all of the characteristics of a good hack: it uses pieces already in place on your network, it combines them in an interesting way, and, even if you wouldn't do it that exact way, it makes you think "Hey, why didn't I think of that?" You might already have a good backup solution in place, but if not, here's one way to build your own simply.

Clancy Malcolm's Ten Security Checks for PHP, Part 1 is a sort of security checklist, not in the sense that "if you do these ten things, you'll be secure," but "check these ten things as they're often insecure." As usual, the answer to many security questions starts by asking yourself, "How could this possibly go wrong?"

Adam Trachtenberg's PHP 4.3 and Mac OS X brought together two popular technologies. Though Mac OS X is chock full of Unixy-goodness, it does behave a little differently in places. If you remember when Jaguar was the screen-licking champion, this article took away some of the pain.

Dustin Puryear's Building an Address Book with OpenLDAP grew out of a question he asked himself -- "It's easy to find instructions on installing and configuring LDAP, but where are the guides to using it in applications?" It's always nice to be the first to publish on a subject, especially when it's useful and well-written.

Howard Wen's interview with Bunnie Huang, The Hacker Behind "Hacking the XBox", garnered no little attention. Games are cool and hacking is cool, so when you put them together -- and skirt an unpopular law in the process -- you have something doubly cool. We hope to work with Bunnie more in the future.

I've long believed that a little more discipline in software development will go a long way to improving the programs we use. It was nice to install the Neon library and see that the developers were inspired to write a test suite in part due to my OSCON 2001 talks. Five Lessons You Should Learn from Extreme Programming, an article written to promote my Extreme Programming Pocket Guide, explained five techniques you can start to practice right now that will improve your software and your development experience.

Miod Vallat claims not to be a compiler guy. When he saw OpenBSD walking away from a platform he liked due to lack of resources, he took matters into his own hands. Diving into Gcc: OpenBSD and m88k explains how motivation, a debugger, and a lot of thought helped him find and fix bugs in gcc. This is almost a detective story; it's very cool.

My Myths Open Source Developers Tell Ourselves grew out of conversations with fellow programmers. One useful problem-solving technique is to ignore the obvious answer that first comes to mind, looking for different approaches. I tried to take "conventional wisdom" and figure out when, if ever, it failed. You can find several projects that practice one or more of these behaviors. Some are successful anyway.

"What's New in Blank" articles are nearly always successful, and Alex Martelli's What's New in Python 2.3? was no exception. There's something satisfying and fun about applying new solutions to old problems, even if it's only in your head. "Oh yeah, I could have used THAT back then...."

Howard Wen's Falcon's Eye: The Making-Over of Nethack proves that, though you don't need fancy graphics and sounds to make a great game, they can add quite a bit to an already-great game. Sadly, Falcon's Eye has seen little progress since this article. Hopefully someone will take up the reins.

Sometimes, an article hits the sweet spot for beginners. John Coggeshall is good at that. His Working with Forms in PHP, Part 1 remains popular because it's patient, thorough, and easy to read. When it's time to work with user-submitted data, start here.

OK, so the idea of a year-ending wrap up is nothing new. Dru Lavigne did it last year in her FreeBSD Odds and Ends installment of FreeBSD Basics. Scrambling for column ideas every couple of weeks (though I've never had the impression that Dru has to scramble for column ideas), you're bound to come across several interesting corners of your system that may not be worth a full column. As seen recently in the Hacks book series, it's interesting and fun to share interesting and fun things, even if they don't immediately have a practical benefit.

George Belotsky impressed a lot of people with the depth and scope of his research and knowledge in C++ Memory Management: From Fear to Triumph (Part 1) -- as well as in parts two and three. I've not seen a more comprehensive guide to the subject.

It sometimes seems like capabilities are the next big thing in security. That's part of the appeal of systrace: you can specify which files a program can open, which system calls it can use, and the arguments they can take. Michael Lucas' Systrace Policies went into detail about how to accomplish this. Systrace is a very cool utility.

My Five Lessons Open Source Developers Should Learn from Extreme Programming went into more detail on the kinds of problems open source developers face. I truly believe we could produce more reliable, useful software more quickly with less effort if we coded and designed with a little more discipline. Alas, not everyone agrees -- yet.

OpenBSD's pf is an amazing success story. It's a utility that should never have needed to be written, but its adoption rate, power, and flexibility are hard to deny. Jacek Artymiak's NAT with pf, one of a four-part series in his Securing Small Networks with OpenBSD column, explained how to use the packet filter with Network Address Translation. His configuration model is particularly useful.

Secure Programming Techniques, the first excerpt from the third edition of Practical Unix and Internet Security, gave several tips on avoiding security-related bugs. Again, the simple phrase "Know about what you're doing and how it really works" sums it up. That doesn't preclude more specific suggestions, though.

Tony Stubblebine's Five Habits for Successful Regular Expressions, written to promote his Regular Expression Pocket Guide, started by explaining that, though powerful, regexps are compact, subtle, and hard to maintain. That's no reason to avoid them. That's good reason to take advantage of implementation features that increase maintainability and clarity. Finally!

Editor's Choice Articles

Some of my favorite articles didn't make the top 25. That's OK; I'm satisfied with the authors and articles of the past year. Still, it's worth choosing a few articles that deserve more recognition. This is not to slight the dozens of other articles that made neither list; I'm very pleased with what we covered this year.

KIVILCIM Hindistan proposed Client-Side Mail Filtering with SaveMyModem just as my parents had asked for help getting rid of spam and viruses in their email. Since they use their ISP's mail service, client-side filters are the way to go. Most of the spam-filtering tutorials I've seen focus on server-side features. This was our attempt to buck the trend. (It's working well for my father.)

Andrew Odewahn's RouteWord: An Interesting Diversion came to us almost as the idea came to Andrew -- out of the blue, with no warning. Another editor sent me the draft article, saying, "There's something here. Are you interested?" This article appeals to me because it's fiendishly clever; Andrew combined two geeky ideas -- graph theory and word puzzles -- and polished the idea until it shone.

Howard Feldman's Distributed Computing Sanity Checking feels almost like a project post-mortem. Don't worry; the Distributed Folding Project is still alive and well. Howard's overview of the things that can go wrong in a distributed computing project is exactly what I wanted to read from an expert offering advice. (Pity about the URL, though.)

I make a habit of asking "Why?" and sometimes it doesn't work out. Sometimes it does, though. Steven Hauser's Quick and Dirty RDBMS Tuning said, "Before you trudge through your schema and add another index, look for the real culprit!" Another good title might have been "Database Tuning Without the Database".

Michael Lucas' Expanding Small NetBSD Systems was nominally about installing software on a palmtop computer running NetBSD, but the principles apply to any other small system. Since several of my friends and colleagues have small BSD systems serving as firewalls and routers, these ideas have a broader audience.

I was very happy to talk Bob Pendleton into writing a series on animation in SDL, and not just because I've been dabbling in it lately. Though his examples are in C, his explanations (and his code) are valuable to other languages, specifically Python, Perl, and Ruby. Animation in SDL: OpenGL is the most recent example; Bob went above and beyond the call of duty, explaining OpenGL jargon in words even a word-slinging editor could understand. (Fortunately, I also program.)

We promote Open Source (and, unrepentantly, Free Software) here, but in a pragmatic fashion. I usually prefer to leave the advocacy arguments to others. Certainly, freedom is important, as are price, quality, and utility. When we do jump into the realm of advocacy, we like to do it well and slightly differently. That's why I was pleased to read Bernard Golden's Open Source: The Whole Product. Bernard's analysis explains why certain companies jump at the chance to adopt open source and why others are reluctant. Better yet, he explains a way of thinking that can only help our goal of spreading good software.

The "web services" buzz phrase unfortunately conjures up images of acronym soup. Life seems so much simpler when you replace it with "making automated queries of web sites." Adam Trachtenberg's PHP Web Services Without SOAP pointed out that most queries use the REST interface. Oops, there's an acronym again. Don't worry; it's easier than you think.

As a programmer and Unix fan, I like the idea of tying together small utilities and libraries to make bigger and better programs -- or, to make my life easier by hiding small tasks and simplifying bigger ones. Garrett Rooney's Using the Subversion Client API, Part 1 (and Part 2) demonstrated how to embed the Subversion client library in your own programs. Good version control on the cheap? I want that!

The day after SQL Slammer went public (and earned its name), we'd somehow roped Iljitsch van Beijnum into studying the situation and writing Network Impact of the MS SQL Worm. (I credit editor Tara McGoldrick; even if it wasn't her, she's sweet-talked enough book authors into writing articles that she deserves the credit.) I liked this article because it's an insight into the mind of an expert. Iljitsch noticed something odd and set to work investigating it. That's always fun -- and illuminating.

Speaking of investigation, when DARPA abruptly pulled funding for OpenBSD, speculations flew. Ian F. Darwin was quick to summarize the theories in Soldiers Renege on Hackers (itself a play on the title of Cameron Laird's and George Peter Staplin's Hackers Meet Soldiers). We don't normally present opinion pieces, but the suddenness and the quick speculations made for an interesting story. It's probably due for a follow-up investigation.

Robert Bernier's Command-Line Email combined two interesting ideas. First, it gives a history of email, for readers who aren't familiar with how we got here today. Second, it demonstrates command line tools that are still useful. There's not a lot in here new to graybeards, but cheaply available and hard-working free Unixes have provided lots of people with tools they don't necessary know are there. I like digging back in computer history occasionally to see what's still useful.

Jacek Artymiak is better known as the Securing Small Networks with OpenBSD columnist. He's also managed to teach multiple BSD classes in Poland, impressing BSD-loving teachers in North America. I was slightly surprised to see him submit Building Unix Tools with Ruby, but I eventually ran it anyway. Ruby's a nice language, combining some of the best parts of Perl and Smalltalk, so it's nice to give it more press. For people who weren't Ruby aficionados, I liked this article as it gave a good overview of how Unix applications should behave. It's always nice to syncretize two separate ideas into one gestalt.

Though our audience attracts quite a few dedicated beginners (and, I suspect, many self-taught people), I like to run advanced articles every now and then. David Mertz has been writing a series on Advanced Object-Oriented Programming. His Multimethods article is a good example. Though any good CS program should expose students to "exotic" languages -- that is, anything besides C, C++, and Java -- not everyone remembers dabbling in Lisp or ML and, again, not everyone has (or needs) a CS application. I like how David explains the concepts and then demonstrates them in Python, a clear and popular language. It's also nice that his examples are clear enough to port easily to other languages.

Email had a lot of coverage this year. The shining gem in our article set may have been Joe Stump's three-part series on Building an Advanced Mail Server. He started with installation and configuration and, along the way, covered virtual domains, web mail, client communication, and server-side filtering. What else is there to say?

Looking into 2004

That's a wide range of subjects and authors, and that's only one out of five articles we ran!

The year 2004 will likely look similar to 2003. We do have a new domain name for the Linux DevCenter: As well, our producers and graphics people have been tweaking the look of our sites for clarity and beauty. Miky Vacik's header images are amazingly cool.

Subjectwise, we'll cover open source desktop applications more seriously. I'm looking for articles on applications such as Gnumeric and We're not giving up the developer and system administrator focus; we're just adding another focus. How much more is there to say about installing software? It's time to start using it.

I'd also like to continue discussing good software development practices. One of the advantages of open source is, as always, "Not all the smart people work at your company." There's a tremendous pool of volunteer talent available. How can individual projects harness that talent and focus it in proper directions? Sure, you can accomplish great things by throwing piles and piles of cash and time at a problem, but think what you could do if you aimed those resources carefully?

I'm also trying to talk a couple of our frequent authors into starting projects that will run through the year, writing weblogs on their progress and articles on design decisions and clever code techniques. That should be interesting.

2003's been quite a year, with consolidations, legal threats, and, above all, new releases and new projects. We'll keep covering the world of open source usage, administration, and development.

chromatic manages Onyx Neon Press, an independent publisher.

Return to

Sponsored by: