Untwisting Python Network Programming

Networking is an essential task in software applications nowadays. Many programming languages have support for network programming to various extents. While the core libraries of most languages allow low-level socket programming, other libraries and third-party extensions often facilitate higher-level Internet protocols. For example, Java has a standard API to access sockets and send emails (via the URL class), but other common Internet protocols are available through external libraries such as JavaMail, JTelnet, and JTA. Perl has a native Unix-style interface to sockets and convenient core modules such as IO::Socket, Net::POP3, Net::SMTP, and Net::FTP. To access Telnet programmatically in Perl, the CPAN module Net::Telnet is a good option.

Python is an exception--it has very good built-in support for both socket and various Internet protocols, including POP3, SMTP, FTP, Telnet, and Gopher. The Python core distribution contains many networking modules, such as socket, poplib, ftplib, smtplib, telnetlib, and gopherlib. Being components in a high-level programming language, these modules encapsulate the complexity of the underlying protocols and are very convenient to use. Twisted is also another powerful networking framework, which, unlike the core networking modules, adopts an asynchronous approach in the networking programs.

This article introduces basic client-side networking using both core Python modules and the Twisted framework. For its example, I will show how to send, receive, and delete emails, and conduct Telnet sessions. I have written two functionally equivalent examples, one using the core modules (mail-core.py) and another using Twisted (mail-twisted.py), with both start, stop, and interact with a server to process emails. These programs work with any standard-compliant SMTP and POP3 servers in sending and retrieving of emails. The starting and stopping of server are specific to the Apache James mail server, which I choose as a local testing server due to its ease of installation and its shutdown procedure in a Telnet session.

Sending Mails with smtplib

The core module smtplib provides a SMTP class that encapsulates the interactions to a SMTP server to send emails. Essentially, you create an SMTP instance with the address of the server specified, invoke sendmail to send the mail(s), and finally close the SMTP connection by the quit method:

def sendMail(host, addr, to, subject, content):
    import smtplib
    from email.MIMEText import MIMEText

    print "Sending mail from", addr, "to", to, "...",
    server = smtplib.SMTP(host)
    msg    = MIMEText(content)

    msg["Subject"] = subject
    msg["From"]    = addr
    msg["To"]      = to

    server.sendmail(addr, [to], msg.as_string())
    server.quit()
    print "done."

The sendmail method takes parameters of the sender's address, receivers's addresses in a list, and the content of the message. Because the message content should be in the MIME format, use the convenient class MIMEText (from email.MIMEText) to create a text message. Create a MIMEText with the message body, specify the subject, from, and to addresses with the dictionary-like syntax, and take it as a string when passing to sendmail.

This testing server accepts and relays emails from anyone, and this is the default configuration of the Apache James server. Although it is fine for a local testing server, most, if not all, SMTP servers in the Internet mandate certain security measures to fight spam. To use a SMTP object with a server that requires authentication, invoke the login(username, password) method before sending any message. This method keeps silent on success and raises an exception when the authentication fails.

Retrieving Emails with poplib

Retrieving emails is inherently more complex than sending: it involves identification of the user, getting the number of messages, and retrieving or deleting the messages. The POP3 class in the poplib module provides methods to do this:

def display(host, user, passwd, deletion = 0):
    import poplib, email
    pop3 = poplib.POP3(host)
    pop3.user(user)
    pop3.pass_(passwd)

    num = len(pop3.list()[1])
    print user, "has", num, "messages"

    format = "%-3s %-15s %s"

    if num > 0:
        if deletion:
            print "Deleting", num, "messages",
            for i in range(1, num+1):
                pop3.dele(i)
                print ".",
            print " done."
        else:
            print format % ("Num", "From", "Subject")
            for i in range(1, num+1):
                str = string.join(pop3.top(i, 1)[1], "\n")
                msg = email.message_from_string(str)
                print format % (i, msg["From"], msg["Subject"])
    pop3.quit()

Like the SMTP class, you can create a POP3 instance by specifying the mail server. Terminate the POP3 session with the quit method. Log in to the POP3 account with the methods user and pass_ with parameters of the user name and password respectively.

To get the number of messages in the server, you may use either the stat method or the list method. The stat method returns a tuple of two values: the number of messages and the total mailbox size. The list method, used in the example, returns the message list in the following form:

(response, ['mesg_num octets', ...], octets)

Here the second value is a list of strings, each stating the number and size of a message in the mailbox. The number of messages is the size of this list of strings.

Retrieve a message wholly with the method retr(message_index) or partially with top(message_index, number_of_lines). In both methods, the message index is 1-based and returns a message in the following form:

(response, ['line', ...], octets)

Typically, the second value in the results is the most interesting: it is a list of the lines of the message. The example program needs only to retrieve the header information, so it gets string containing all header plus one line of body text with string.join(pop3.top(i, 1)[1], "\n"), then uses email.message_from_string to parse the string to build a MIME message object. From the MIME message, you can fetch the email subject, sender address, and receiver address(es) via the standard dictionary syntax.

To delete a message in the mailbox, use the method dele(message_index). It sets the deletion flag of the specified message and then the server actually does the deletion when you close the POP3 session. If you have a program calling dele on a message but the message persists, check that the program actually invokes quit.