ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


XML-RPC in Python

by Dave Warner
11/22/2000

The Internet, with its simple protocol and ubiquity, has opened up huge opportunities for programs to communicate between computers, a task that always seemed complex and daunting in the past. Now there is a new dilemma. Which framework should you use for automating communication?

The two most popular frameworks, Enterprise JavaBeans and Microsoft's COM, are highly expressive and robust, but also complex. They carry an enormous amount of processing overhead. Other requirements, such as cross-platform and cross-language capability, only add to the dilemma. The Simple Object Access Protocol (SOAP) has recently come to the forefront, promising to solve many of these problems. In active development, SOAP has several partial implementations available but is still a moving target. Some implementations cannot yet talk to each other.

So what's a developer to do?

What many developers have done is to use what is in place and what works now. And what works is XML-RPC. It's easy, quick, and consistent. XML-RPC fits a "glue" role that its big brother, SOAP, cannot. If you are looking for a large-scale framework for communicating between diverse servers and enterprise functionality like automated discovery of services, then XML-RPC will not fit. XML-RPC is, however, a quick, easy-to-use, reliable means of communicating between distributed locations, without worrying about what language the "other guy" has used. XML-RPC is a hybrid solution. It's not strictly object-oriented. The "RPC" portion of the acronym stands for Remote Procedure Call. RPC is very simple -- a user sends a request over the network to a specific location and receives a reponse. If this sounds suspiciously familar, it is -- the Web works this way. The death-knell for RPC was heard in the 1990s. Its developers were unable to standardize on the structure of data. Sun did try to implement a standard in XDR. It never quite caught on. With the rise of object-oriented programming, RPC fell into disfavor. By adding XML to the RPC portion, a standard, popular structure for data has been added to the process, breathing new life into RPC. So why use it?

Related:

Java and XML

Java and XML
By Brett McLaughlin

The major, incontestable reason for using XML-RPC is that it works, and works well. With a very minimum of effort, a developer can install and use Python xmlrpclib, communicating with servers built in Python, PHP, Java, Perl, C++, and even COM. An entire chapter devoted to XML-RPC in the O'Reilly's best-selling Java and XML brought great attention to XML-RPC. In the Python world, the largest use of XML-RPC is via Zope. Zope has been able to send and respond to XML-RPC since mid-1999.

Enter PythonWare's xmlrpclib

Related Articles:

Web Client Programming in Python

Meerkat: The XML-RPC Interface

Unlike other languages in its generation, Python has always had the ability to create persistent objects. It uses increasingly complex mechanisms to do so. These mechanisms range from the standard library package, "marshall," which converts simple objects for storage in flat files, to the "Zope Object Database," which rivals commercial Object-Oriented Databases (OODB) in its robustness. To "marshall" an object is to convert it into a different format. Within the context of xmlrpclib, marshalling a Python object means to convert it into an XML-RPC call. Unmarshalling is the reverse of that step.

Written by Fredrik Lundh of PythonWare, xmlrpclib hides all of the complexity in marshalling and unmarshalling objects. It goes beyond this to also hide the details of connecting to a server, sending a request, and receiving a response.

You can download the latest version of xmlrpclib.py (and get more information about the library) at the XML-RPC for Python page.

XML-RPC and Meerkat

Editor's note: Meerkat predated the popularity of syndication, feed services, and feed readers. Now that other groups are providing this service, we have removed Meerkat in favor of their better solutions. We maintain these articles for the sense of historical interest.

Rael Dornfest, a driving force in the adoption of XML in the PHP arena, has quite naturally constructed an XML-RPC interface to Meerkat, an open wire service that collects news information from many sources using RSS. Let's use this readily available server for our examples. First we'll import the pretty print (pprint) module from the Python Standard Library. Some of the output from Meerkat is quite lengthy.


>>> from pprint import pprint

After importing xmlrpclib, connecting to the server using xmlrpclib consists of one line:


>>> meerkatsvr = xmlrpclib.Server("http://www.oreillynet.com/meerkat/xml-rpc/server.php")

Here we create an instance of an XML-RPC server that understands how to connect, send, and receive XML-RPC. Through recursive "sleight-of-hand," this server can even understand nested XML-RPC requests.

Next, let's request a list of the methods that are available from the server:

>>> pprint(meerkatsvr.system.listMethods())
['meerkat.getChannels',
'meerkat.getCategories', 
'meerkat.getChannelsByCategory',
'meerkat.getItems',
'system.listMethods',
'system.methodHelp',
'system.methodSignature']

You have just used XML-RPC to connect to a server across the Internet and retrieve a result set. Besides the simplicity of the approach, notice that the form of the result set returned by xmlrpclib is a native Python list.

Getting a list of channels from Meerkat

Meerkat groups RSS channels into topics, called categories. For example, channels related to XML are in category 23. If you want to get a list of channels for a specific topic, you can query Meerkat this way:

>>> pprint(meerkatsvr.meerkat.getChannelsByCategory(23))
[{'title': '<XML>fr', 'id': 2991},
{'title': '4xt', 'id': 2559},
{'title': 'Eclectic', 'id': 555},
{'title': 'eXploringXML', 'id': 4471},
{'title': 'eXploringXML Channel', 'id': 1105},
{'title': 'finetuning com', 'id': 107},
{'title': 'finetuning.com', 'id': 4628},
{'title': 'Free XML tools', 'id': 906},
{'title': 'JabberCentral Recent Jabber News', 'id': 4655},
{'title': 'Moreover XML and metadata news', 'id': 2243},
{'title': 'moreover... XML and metadata news', 'id': 683},
{'title': 'My Userland', 'id': 1945},
{'title': "O'Reilly Network XML FAQs", 'id': 2365},
{'title': 'oreillynet.xml', 'id': 989},
{'title': 'oreillynet.xmldev', 'id': 990},
{'title': 'SOAP Webservices Resource Center', 'id': 2022},
{'title': 'XML About com', 'id': 2435},
{'title': 'XML News from PerlXML.com', 'id': 718},
{'title': 'XML XSL Portal', 'id': 4460},
{'title': 'XML.com', 'id': 47},
{'title': 'XML.com Resource Guide', 'id': 4637},
{'title': 'xmlhack', 'id': 724},
{'title': 'xmlTree Newsletter', 'id': 413}]

Deeper into XML-RPC and Meerkat

In my web client programming article, I demonstrated how we could easily download a listing of all Meerkat articles related to Linux, using Python's powerful URLLIB and a very simple script. Let's revisit that task using xmlrpclib. Employing an even simpler script, we can eliminate some of the ambiguities and awkwardness of the former approach and make a more robust application.

Fire up a new interactive session in Python, import xmlrpclib, and create a server instance for interacting with the Meerkat server:

>>> import xmlrpclib
>>> meerkatURI = http://www.oreillynet.com/meerkat/xml-rpc/server.php
>>> meerkatsvr = xmlrpclib.Server(meerkatURI)

Before we begin coding our updated solution, let's explore some features Rael Dornfest has built into the Meerkat XML-RPC server to document its capabilities. We've seen system.listMethods(). It returns an up-to-date list of the methods available on the server. You can obtain the method signature of any of these methods by calling methodSignature()


>>> meerkatsvr.system.methodSignature(methodname)

This returns a two-element array containing the return value and parameter(s) of the method named. Note that you should not include enclosing parentheses after the method name.

Dornfest also provides a means of obtaining help about any method on the server


>>> meerkatsvr.system.methodHelp(meerkat.getItems)

To see these and all the other methods available to XML-RPC clients of Meerkat, go to the Meerkat XML-RPC Interface Test page. Selecting introspection from the drop-down list will give you all methods available, with their documentation. The URI, as its name suggests, also tests the Meerkat XML-RPC Server, returning both the method call and the server response. Nice.

Obtaining a list of Linux articles from Meerkat

As you may recall, the objective of the program in my web client programming article was to obtain links to Linux articles from Meerkat over the past hour. After building the framework to access the server through URLLIB, we constructed the URI:


http://www.oreillynet.com/meerkat/?p=5&t=1HOUR&_fl=minimal&_de=0&_ca=0&_ch=0&_da=0.

Imagine having to maintain the above line in an application! As a refresher, the URI tells Meerkat to give us all stories in the Linux category that have been posted in the last hour, using the minimal flavor. We want to further restrict the information coming back by eliminating the description, category, channel, and date fields. Let's replicate this functionality using xmlrpclib.

Declare a parameter list by constructing a Python dictionary (the same as a structure in XML-RPC):

>>> params = {
... 'category' : 7,
... 'time_period' : '1HOUR',
... 'descriptions' : 0,
... 'categories' : 0,
... 'channels' : 0,
... 'dates' : 0
    }

The first two parameters define the scope of the information we desire. The last four act as switches, turning off the various components returned. Next, we'll call a method on the server to obtain the desired information


>>> results = meerkatsvr.meerkat.getItems(params)

This returns a list of dictionaries containing both the title and an HTML link for all Linux articles that have appeared on Meerkat over the last hour. If your request returns an empty list, increase the time period (e.g. 2HOUR or 1DAY).

Now, let's loop through this list and construct full-fledged HTML links to the articles:

>>> for d in results:
...     print <A HREF=%s>%s</A> % (d[link],d[title])

What we have gained

This approach has given us several benefits. We have enhanced readability. We have less code. Presentation has been separated from data, allowing you to choose alternatives easily. Best of all, this approach has a distinctly Pythonic feel to it. Thanks to the functionality built into XML-RPC and xmlrpclib, the response from Meerkat is now Python data, with all the attendant flexibility: Pickle it, store it in a database, show it in a wxPython or Tkinter widget. Or even send it to another client via XML-RPC!

In the next article, I will dig deeper into the internals of xmlrpclib to show how Python objects are converted into XML-RPC requests and how return values are unmarshalled. We will also build an XML-RPC server that includes a testing framework.

Dave Warner is a senior programmer and DBA at Federal Data Corporation. He specializes in accessing relational databases with languages that begin with the letter P: Python, Perl, and PowerBuilder.


Discuss this article in the O'Reilly Network Python Forum.

Return to the Python DevCenter.

 

Copyright © 2009 O'Reilly Media, Inc.