Building a Mailing List
Pages: 1, 2
Users can sign up for the list through any page on WiFiNetNews.com using a small form in which they need to provide as little information as their email addresses. The form is preset to digest mode, but users can click the radio button to switch to individual messages. They can also choose to enter their first and/or last names. The current software stores this information but does not use it.
A user can also subscribe by sending email to either email@example.com (digest) or firstname.lastname@example.org (individual posts). That information isn't as widely disseminated, but it does appear on a page on the site that explains in more detail how to subscribe.
Both the web page form and the wifi-sub addresses trigger the same script, which confirms that the address isn't already in the database, then inserts a new record with a number of defaults, including setting a flag to indicate that the subscription is pending.
The script now sends out the second opt-in request via email using the Perl
If that request bounces, a suspend flag is set for that email. The system
retains the record to forestall the potential of someone making a second
attempt. (Someone has, of course, tried to subscribe
email@example.com.) A message will stay pending forever if no
one ever responds to it.
The body of the message contains a message that alerts the recipient either to respond to the email with any or no contents or to follow a web link. Either action triggers a confirmation script that updates the pending flag to confirmed. The next time an appropriate action comes along — a new post for individual messages or a new digest for daily subscribers — that subscriber receives the message.
Every message sent to the list has a lengthy post-amble where
CODEHERE is the unique code assigned to a user on
-- You are subscribed as firstname.lastname@example.org You can unsubscribe at any time by sending email to <mailto:wifiunsub+CODEHERE@wifinetnews.com> or visiting <http://wifinetnews.com/cgi-bin/listunsub?confirm=CODEHERE> To change from individual messages to the daily digest, just send email to <mailto:email@example.com> You can reach Wi-Fi Networking News at <firstname.lastname@example.org> Our postal address: 115 N. Any St., Suite 1000, Seattle WA 98999
It's long, but it's also prevented the Angry Subscriber responses described early in this article. All unsubscribers have managed to figure out how to use this multi-tiered approach. Both the unsubscribe methods use the same script: the script sets the unsubscribe flag and stores a date stamp for that action. The address remains in the system as a historical record and for statistics tracking.
I've included the postal address (obscured here) in each message to demonstrate a good faith effort to comply with the U.S. CAN-SPAM legislation that requires a postal address in advertising messages in certain circumstances. Even though I'm sending out editorial that meets the highest tests in the law, I still include it just to keep readers happy. I'm not worried about someone stopping by or sending nasty-grams via snail mail.
If you examine the stream of replies that seemingly identical mail servers send back when they bounce mail, you laugh and cry. Because the Internet is an anarcho-syndicalist union in which we each agree to have our own rules in our little domains as long as when we meet, we don't interfere, mailers produce a host of bizarre, funny, and misleading responses. There's some standardization, but the exceptions are quite broad.
If you look at the code in
wifibounce, you'll see my best analysis and
attempts to deal with bounces. This seems to cover now about 99.5 percent of
all bounces to the list. I have to examine the bounce raw logs once or twice a
month for exceptions.
Not processing bounces can be one of the biggest failures of a mailing list system. Among other problems, repeated mailings to non-existent users can cause some administrators or systems to mark you as a spammer and blacklist you. It also keeps a higher bandwidth and system load running when you mail out messages — and it's not neat. For larger lists with tens of thousands of subscribers, you could generate 100,000 bounce messages a day without proper trimming.
My assumption is that it's better to suspend an account aggressively rather than to send out unwanted mail indefinitely. For certain kinds of bounces that include complete failures (user unknown or blacklisted, for example), the script suspends a user's account immediately. There's a separate suspend flag so that we can occasionally restore suspended accounts to test whether they're working again.
Accounts that generate transient failures aren't suspended until the system processes five in a row within a day. Unfortunately, my list typically sends out several messages a day, so a transient problem can result in an unnecessary suspension. That's a longer-term issue to solve.
Occasionally, I reset all of the suspend flags from no to yes and let the system re-calibrate against which accounts were transiently dead and which were permanently. One improvement in my system would handle suspension more gracefully, using a cron job to restore only soft bounced accounts every few days.
Since setting up this system in January, I've saved several thousand dollars and, with very few additional inputs, I've developed something I'm secure enough about that I'm ready to share it under a Creative Commons license. I don't have any interest in making a buck off this system, but would hope that Movable Type users will develop the MT hooks better than I can while perfecting some of the more obvious bounce situations. I'm not an efficient coder, and I know that a motivated Perl hacker could trim the scripts down enormously. It could also broaden to work with Postfix, Qmail, and other MTAs, or to use PHP on the backend instead of Perl.
The list statistics speak for themselves. With several thousand daily unique visitors to the web site, the list currently has nearly 1,400 subscribers, a good ratio of visitors to list members. There are just under 1,100 subscribe to the digest and just under 300 to individual messages.
Another cron job produces statistics every morning for me, and I've tracked how well the list has performed on a variety of measures. Fewer than one percent of subscribers unsubscribe each week, while the list grows by three to five percent each week with a few growth spurts when the news site has additional traffic or publicity. Usually, less than one percent of the list is suspended in an average week due to hard bounces or for exceeding the soft bounce count.
As noted earlier, I've had to remove a single person manually due to a bug, since fixed, and she wasn't angry. I never have to add subscribers: they all figure it out for themselves. I've received no complaints from ISPs, spam services, or other engines of protection. I may be restricting the list's growth a tad, but with no complaints it's hard for me to have cavils, either.
Some of you will surely point out that I could have turned to Mailman, Majordomo, or other commercial or free mailing packages and tuned them to my needs. But given the success of converting a little bit of starter code in under 10 hours using LAMP and sendmail into an almost perfectly operating customized list, it's hard to feel too bad about writing my own code and now sharing it.
Example Code for this article is available for download.
Glenn Fleishman is a freelance technology journalist contributing regularly to The New York Times, The Seattle Times, Macworld magazine, and InfoWorld. He maintains a wireless weblog at wifinetnews.com.
Return to ONLamp.com.