Which Method to Use?
After looking at the various techniques for network I/O, it is time to formulate some guidelines about which method best fits a particular situation.
For simple clients, plain synchronous I/O is probably the best choice. For example, if you need to extract information from a single web page, the added complexity of threads or asynchronous I/O is not likely to provide many advantages. At most, you might use one additional thread so that the GUI remains responsive even if the network I/O takes a long time.
Synchronous I/O has simplicity on its side. It is the least subtle, most straightforward method. This significantly reduces the chances of programmer error — a powerful advantage that should never be overlooked.
Of course, a server that needs to handle multiple concurrent requests or a complex client (such as a web spider) that queries many separate servers to generate a result, need something more than plain synchronous I/O.
One excellent source of ideas on network I/O is the ADAPTIVE Communication Environment (ACE) project. ACE is a highly mature Open Source system used in a variety of demanding applications (including medical imaging, military avionics, and industrial process control).
In many striking ways, ACE did for C++ what Twisted has, more recently, done for Python. For example, the Reactor design pattern — an important part of Twisted — appeared earlier in ACE. While Twisted is strongly focused on asynchronous I/O, it, like ACE, also provides support for threads (including the thread pool model).
ACE is a giant, comprehensive, and portable framework with many significant capabilities, including real-time performance. In the process of creating this system, the ACE team has published many papers detailing their insights into network I/O. These papers — regardless of whether you use ACE — are very helpful in planning your project.
The ACE paper comparing the performance of various network I/O models (PDF) is of particular interest. This is quite an accessible work, which presents several highly informative results. The following list provides a brief summary:
- Thread-per-request is generally not an efficient model.
- Thread pools are best for medium to small requests (particularly in the range of a few hundred bytes to several kilobytes).
- Asynchronous I/O is superior for large requests (especially ones requiring multiple megabytes to be transferred).
The paper contains many graphs of experimental data, which makes it easy to compare the characteristics of different I/O techniques. In addition, the paper presents several interesting variations (such as thread pools that use asynchronous I/O) on the standard I/O methods.
When using Python, however, the Twisted team has raised some concerns about the efficiency of threads. This has to do with the internals of the Python interpreter, particularly the Global Interpreter Lock (GIL). Python itself is not fully thread safe; the GIL is therefore used to protect critical regions. As mentioned earlier, locking has an adverse effect on concurrency. It is therefore reasonable that the Twisted team chose asynchronous I/O as the basis for their framework.
Guidelines for Choosing the I/O Model
Based on the observations presented thus far, here are some useful guidelines for choosing a network I/O model:
- Use synchronous I/O for testing and basic clients; in general, whenever the simplest solution is sufficient.
- When concurrent network I/O operations are required, use either thread pools or asynchronous I/O, depending on the amount of data to be transferred.
- Keep in mind the Twisted team's warnings about threads and Python; if you are building a performance-critical system for which thread pools are particularly applicable, it may be worthwhile to use C or C++.
If you decide to use C or C++ in order to create fast thread-based implementations, keep in mind that you may not have to give up Python altogether. There are many ways of combining Python with C and C++. See Further Reading for details.
Sometimes, an approach combining multiple languages is advantageous because it becomes possible to emphasize the strengths of each in a hybrid system. In particular, Python/C++ hybrids are well suited for creating elegant, very high-performance systems that are also easy to modify and extend.
Comprehensive Frameworks or Lightweight Libraries?
There is no question that comprehensive frameworks, such as ACE and Twisted, can make it significantly easier to create networked applications. When should you go with such a framework, and when are simpler tools (such as asyncore) more appropriate?
While the ultimate choice depends on the analysis of your specific needs, here are some suggestions:
For small utilities, lightweight libraries such as asyncore are likely the best choice. There is less for the user to set up and install; you also eliminate a complex dependency. In the case of asyncore, it is part of the standard Python library. Thus, users need only Python to run your asyncore-based application (barring any other dependencies you might introduce). These are important considerations for the general ease with which your utility might be adopted, as well as the effort required of system administrators to keep your software running on a large number of machines.
For complex applications, the parameters shift markedly in favor of comprehensive frameworks such as Twisted. Complex programs already require that users or administrators spend considerable time installing, configuring, and learning the software. The additional dependency on something like Twisted is minor by comparison. On the other hand, rewriting and maintaining the functionality that a framework already provides is probably a wasteful duplication of effort. It is a significant advantage if the framework authors fix bugs, patch security problems, and add new features for you. The overall result is also highly likely to be more secure, because a popular framework is constantly being reviewed by many developers and users.
In very rare cases, your system's purpose will actually be to break new ground in fundamental network I/O techniques. In those situations, working at the lowest levels would probably be required. With the rise of open source software, however, there is now an exciting alternative. Instead of doing everything yourself, you can choose another project (ACE, Twisted, asyncore, the Linux or BSD kernels, etc.) to modify.
Don't Forget About UDP
No discussion of Internet-based network I/O would be complete without mentioning the UDP protocol. This often overlooked facility can provide enormous performance advantages (10 times or more over TCP virtual circuits in the author's experience).
If you are designing your own network protocol, do not automatically assume that you need TCP. There are situations for which the tradeoffs made by TCP are not the right ones. This is why UDP exists.
For example, a system that samples data at a high rate could benefit greatly from a UDP-based approach. In this case, there is no need to resend lost messages; they would arrive too late anyway. Instead, the system can resynchronize itself during a subsequent successful sample. A simple alarm mechanism whenever too many samples are missing may actually be all that is required for error handling.
This appendix lists various sources where you can find more information on the topics covered in the article. If you are interested in Python, how to install it, how to install Twisted and similar material, the previous article provides an overview and many links. It also contains more introductory discussion on network I/O.
The Wikipedia page on Computer
Multitasking provides a good overview of this topic. Another good source of
information is the documentation for Python's standard thread modules: the
and the higher-level
The book Programming Python,
2nd Edition (March 2001) also offers a very helpful discussion of threads
in section 3.3. The book is available through Safari Online (there is a 14-day free
trial if you never had an account) as well as in print form. If you just want
to read the section on threads, then Safari Online is probably your best
Queues are a very useful construct, especially for implementing networked applications. Queues are often quite easy to work with, but are still a very active research area with many unsolved problems. The study of how queues behave is formally known as queueing theory. MIT OpenCourseWare offers a lot of freely accessible information if you are interested in the theoretical foundations of the subject. You may also download SimPy, an easy-to-use simulation package that includes models of queues as examples.
asyncore provides a good overview of asynchronous I/O.
Twisted also includes a useful asynchronous
I/O document. You may also be interested in one of the ACE papers
Proactor design pattern. The Proactor is somewhat
similar to the Twisted
Deferred. You should likewise take a look
overview if you plan to use Twisted.
The ACE project is a major source of experimentally verified data on network I/O. There are also several other projects related to ACE, including TAO, a real-time capable CORBA ORB. The ACE homepage provides links to TAO and other ACE-related work.
There are many ACE-related books, manuals, and papers that you may find helpful regardless of which particular library you choose for your project. Of course, if you are programming a networked application in C++, you should seriously consider using ACE itself.
Several projects deal with creating systems using Python and C/C++. The frontrunner for Python/C++ hybrids appears to be Boost.Python. There is also PyCXX (although its author is now urging users to consider Boost.Python instead), Pyrex (which supports C only, not C++), and SCXX (a lightweight approach originally inspired by PyCXX).
Python itself includes support for writing C/C++ extensions, as well as embedding the Python interpreter in a C/C++ program (see Extending and Embedding the Python Interpreter). The SWIG project is also well-known; it combines C/C++ with a variety of other languages (including Python, Perl, and Tcl).
Finally, see section 5 of The Design Philosophy of the DARPA Internet Protocols for brief but insightful examples in which UDP is preferable to TCP. This paper was also mentioned in the first article, because it is overall very worthwhile reading for anyone dealing with networked systems.
George Belotsky is a software architect who has done extensive work on high-performance internet servers, as well as hard real-time and embedded systems.
Return to the Python DevCenter.