Retrieving Mails with Twisted
Programming a POP3 client to retrieve mails in the Twisted framework is more
complex and takes more code. The logic to retrieve mails is all in a class that
Due to the event-driven nature of Twisted, it's easier to define a method for
each step, register the success, and failure callbacks to the method to enter
the next step and handle any error, respectively.
from twisted.internet.protocol import ClientCreator from twisted.mail.pop3client import POP3Client class MyPOP3Client(POP3Client): def serverGreeting(self, msg): POP3Client.serverGreeting(self, msg) self.login(self.myuser, self.mypass).addCallbacks( self.do_stat, errorHandler) def do_stat(self, result): self.stat().addCallbacks(self.do_retrieve, errorHandler)
In the class
MyPOP3Client, the first step to get mails is the
serverGreeting method, which Twisted will invoke when the client
starts. This method invokes the superclass's
then logs in to the POP3 server with a user name and password. The
login method returns a
Deferred object, invoking the
addCallbacks method to register the
(called upon successful login), and the
(called on login error).
do_stat method invokes
stat method to perform a POP3 STAT command, and registers the next
do_retrieve. Because the call to method
is asynchronous, it cannot return its results to the caller with return values.
Instead, it passes the results as arguments to the success callback registered
stat method. The second parameter to the
do_retrieve method is a list, of which the first element is the
number of messages in the POP3 account.
def do_retrieve(self, stats): self.format = "%-3s %-15s %s" self.num_messages = stats self.cur_message = 0 print self.myuser, "has", self.num_messages, "messages" if self.num_messages > 0: if deletion: print "Deleting", self.num_messages, "messages", self.delete(0).addCallbacks(self.do_delete_msg, errorHandler) else: print self.format % ("Num", "From", "Subject") self.retrieve(0).addCallbacks(self.do_retrieve_msg, errorHandler) else: reactor.stop() def do_retrieve_msg(self, lines): msg = email.message_from_string("\r\n".join(lines)) print self.format % (self.cur_message, msg["From"], msg["Subject"]) self.cur_message += 1 if (self.cur_message < self.num_messages): self.retrieve(self.cur_message).addCallbacks( self.do_retrieve_msg, errorHandler) else: reactor.stop()
If there is no message in the mailbox, the code calls
reactor.stop to tell Twisted to shutdown. Otherwise, it invokes
retrieve(0) to get the first message. Its success callback,
do_retrieve_msg, handles the message by displaying its summary,
and then retrieves the next message. Because the method
do_retrieve_msg gets invoked for all subsequent messages, the code
uses an instance variable,
cur_message, to keep track of the
current message number and to determine when it has handled all messages. When
it has processed everything, it stops the Twisted main loop.
Because the logic of mail retrieval is similar to deletion, both features
are in the same class,
MyPOP3Client. The instance variable
deletion denotes the current mode of working. You can see its
__init__, along with the user name and password.
More interesting is the setting of
allowInsecureLogin to true,
which allows login to a server without authentication challenge non-encrypted
def __init__(self): self.myuser = user self.mypass = passwd self.deletion = deletion self.allowInsecureLogin = True def do_delete_msg(self, str): print ".", self.cur_message += 1 if (self.cur_message < self.num_messages): self.delete(self.cur_message).addCallbacks( self.do_delete_msg, errorHandler) else: print " done." q = self.quit() q.addCallbacks(lambda _: reactor.stop(), errorHandler)
To delete a mail, call the
delete method of class
POP3Client. Similar to the core
poplib module, this
method just marks the mail for deletion, and the actual deletion occurs when
you send the POP3 command
QUIT to the server, as with the
quit method. Finally, Twisted's execution thread stops when the
quitting action completes.
pop3 = ClientCreator(reactor, MyPOP3Client) d = pop3.connectTCP(host, 110) reactor.run()
With the implementation of the desired mail handling in class
MyPOP3Client, you can launch the client. With its descriptive
name, the class
twisted.internet.protocol) provides a convenient way to start a
communication client. This code passes the
reactor and the
MyPOP3Client class to create a
begins the mail retrieval by calling
connectTCP with the specified
server and port number. Twisted's execution loop then kicks off by
do_retrieve_msg repeatedly for all messages is
conceptually tedious and lengthy, when compared to the
DeferredList mechanism which keeps track of several actions and
gets notified when all actions complete, as in the case of sending mails.
However, collecting the
Deferreds of multiple calls to
POP3Client in a
simply does not work in Twisted (Versions 2.2.0 and 2.4.0). The success
callback never gets invoked (see mail-twisted.py).
Doing Telnet with Twisted
Twisted can power a Telnet client in a way similar to, but simpler than, the
POP3 client. the Telnet conversation logic goes in a subclass of
def stop(host): from twisted.internet.protocol import ClientCreator from twisted.conch.telnet import Telnet class MyTelnet(Telnet): def dataReceived(self, data): if "Login id:" in data: self._write("root\n") elif "Password:" in data: self._write("root\n") elif "Welcome" in data: d = self._write("shutdown\n") def connectionLost(self, reason): reactor.stop() print "done." mytelnet = ClientCreator(reactor, MyTelnet) d = mytelnet.connectTCP(host, 4555) reactor.run()
MyTelnet overrides two methods of class
Telnet. The first method,
dataReceived, is called
when data arrives at the client. It checks the data received and calls the
_write method to send the user name, password, or the shutdown
command accordingly to the server. The second method is
connectionLost, which Twisted calls when the server closes the
telnet session. In that case, the program simply terminates the Twisted
execution loop. The Telnet client starts by using the
class, connected to port 4555 of the James mail server.
When to Be Twisted?
The two functionally equivalent programs, one using Python core modules and the other using the Twisted framework, significantly differ from each other in terms of programming style and the amount of code. Then when should you use either of the two options?
For basic programs such as the command-line client of this example, the Python core networking modules are more desirable due to the simplicity and performance advantages. However, most real-world networking programs are very complex, and Twisted's asynchronous programming model is more effective. For example, BitTorrent, the popular peer-to-peer file sharing client that performs massive parallel downloading of data chunks from different sources, uses Twisted. Twisted also works well in programs with graphical user interface (GUI), because its asynchronous nature fits more seamlessly with the event-driven programming models of modern GUI frameworks. In fact, Twisted has integration with popular GUI frameworks including PyGTK, Qt, Tkinter, WxPython, and Win32.
The other area where Twisted shines is in server programming. A typical network server uses multithreading so that it can handle multiple clients concurrently. The asynchronous mechanism of Twisted alleviates the creation and handling of threads by server programs. In addition, Twisted provides several protocols on which to build new networking services, enabling rapid development of complex servers. One such project is Quotient, which adopts Twisted to build a multiprotocol messaging server that supports a variety of protocols and services including SMTP, POP3, IMAP, webmail, and SIP.
Kendrew Lau is a consultant in Hong Kong, with focus on Java, Linux, and other OSS technologies.
Return to the Python DevCenter.