Envisioning RSS as a Web 2.0 platform

Email.Email weblog link
Blog this.Blog this
Mark Sigal

Mark Sigal
Aug. 09, 2005 12:31 PM

Atom feed for this author. RSS 1.0 feed for this author. RSS 2.0 feed for this author.

RSS began its life as a really simple way for content providers to syndicate their content and for content consumers to subscribe to their favorite providers. When the blogosphere emerged, RSS really took off. Now, just as its “simple” technology cousin, HTML, provided the underpinnings of the Web 1.0 technology platform, RSS is emerging as a platform for delivering the broadband and mobile ready applications of a Web 2.0 enabled world.

From this vantage point, RSS evolves beyond simple publish and subscribe to become more akin to web services. The concept of a feed is extended to support both a diverse range of data and content types, and feeds can contain rich “payloads.” Furthermore, feeds gain the ability to expose well-formed methods providing the intelligent “glue logic” for building loosely coupled applications. Backed by two application examples, this blog presents a thesis of the key moving parts integral to the RSS platform and how they come together. (Note: this is a continuation of an earlier O’Reilly blog that I wrote and postings on my digital media blog, The Network Garden.)

Beginning With the End in Mind
When talking about innovative technologies, it is sometimes tempting to get sucked into the “WHAT” (is it) and the “HOW” (do you use it) without satisfying the “WHY” (it is a big deal) side of the equation. Therefore, let’s begin by imagining a world where RSS is mature and ubiquitous to see if the sample applications that can be built around this platform are truly compelling.

Sample Application One: The Community Calendar
In this example, I use The Community Calendar to plan my Friday evening in terms of things to do in San Francisco. To do this, I define the activities of interest to me, flag the people in my social network whose recommendations I trust, and the people that I want to connect with while I am out on the town.

To figure out dinner plans for Friday night, I configure and subscribe to an RSS feed from OpenTable.com of Italian restaurants in SOMA (South of Market) that have 8PM reservations available for a party of two. These results are then automatically filtered according to restaurants most consistently tagged as “good restaurant” by food lovers that I trust. As reservations become available (or unavailable), they are updated in my calendar in the form of a list of verified restaurant reservation options. At any time, I can book a reservation with a click. Similarly, the calendar can be extended to include bars and nightclubs that people in my social circle are planning to be at within a given time range of the evening, including whether they will definitely or tentatively be there. I can even set a parameter so that as people in my social circle change their destination plans, I am automatically notified by SMS on my mobile phone.

There are interesting opportunities for advertisers to tap into this model. For example, once a restaurant, club or bar has been added to my calendar as a prospective destination, the retailer can selectively entice me to go to their venue with offers of free drinks or specials. Similarly, local content providers can automatically infill this information with reviews, which is of high value both to me as a consumer and to them as highly relevant real estate that they can sell advertising on.

Sample Application Two: Video Clip Channel
In the case of the Video Clip Channel (VCC), I go to a web site where I define the categories of video content that are interesting to me. To do this, I build a structured query of the specific content that I am interested in (e.g., Los Angeles Lakers basketball highlights), the content sources that I want to cull from (e.g., ESPN, LA Times, the video “clip miners” community site) and my content filtering parameters (e.g., aggregate storage limits, automatic removal of duplicate clips or content sources that I consistently fail to view content from, and auto-inclusion of all Lakers clips in my friend Leon’s video content store). VCC then dynamically builds a play list for me, and associates the content with my video player (or devices) of choice before downloading the applicable content locally or readying a streaming version of same. In turn, VCC exposes the play list for syndication to consumers that are interested in my Lakers clip feed channel.

This brings up an interesting point. As video iPods take hold, the presence of a service like VCC enables legions of micro broadcasters to propagate. Building upon The Long Tail model of market demand and demand fulfillment economics, as espoused by Chris Anderson, editor-in-chief of Wired Magazine, micro-broadcasters will fall into three different buckets: content producers (they actually create new content), content aggregators (they stock the virtual shelves with content from a multitude of content sources) and content filters (they categorize the surplus of content available and separate the wheat from the chaff).

Moving from “WHY” to “WHAT”
When considered as part of the same application platform, the above two examples suggest a few things. One is the premise that RSS is best thought of as an intelligent message exchange that can support not only the syndication and subscription of feeds (like headlines and blogs), but the actual delivery of rich content payloads as well.

Two is that payloads can consist of a broad range of data and content types, including blogs, photos, podcasts, videos, flash applications, news content, calendar items, TV programs, local movie listings, reviews, stuff for sale, special offers, product listings, and dynamic lists, like wish lists.

Three is that content items will evolve to expose “handles” such that the available actions associated with a given piece of content can become quite robust. For example, in The Community Calendar application referenced earlier, a calendar item can recognize the event “available restaurant reservation,” pipe the reservation data in from a remote provider based on well-formed parameters, filter that data according to social networking constructs, aggregate the data with third party content like a restaurant review and keep other members of the reservation planning process apprised as plans solidify and change.

Four is that to support these capabilities, RSS will become more like publish and subscribe middleware, gaining the ability to perform message queuing and routing functions, as well as handling of data transformations with and between consumer applications and disparate devices types.

Finally, in terms of deciding what gets pushed into or filtered out of feed channels and how such content items are categorized and ranked, it seems clear that there is the need for more systematic ways to define and negotiate sharing of information based on user profile data.

The ideal here is to disseminate between both user defined contexts, such as metadata “tags” (e.g., “good restaurant” and “bad restaurant” in The Community Calendar example), ratings and notes, and autonomous, algorithmically-defined ones, such as newest, most viewed and most downloaded. Among the major portals, Yahoo is taking some innovative, albeit wobbly, steps with My Web 2.0, and del.icio.us has emerged as a surprising powerful social bookmarks manager in the startup realm.

A key part of this challenge, however, is working through a model for defining how content feeds are categorized both on the syndication side and on the subscriber side of the information pipe. This is non-trivial because approaches that can systematically watch “what I do” and intelligently optimize feedback accordingly (such as what Amazon does with the feature “People who bought this book, also bought this one.”) have the benefit of being incredibly easy to use. By contrast, ones that can be customized based on direct user feedback are very powerful but more complex to use and more subject to be “gamed” by spammers.

Netting it out, for developers embracing RSS as a platform, there is tremendous value to add in terms of flagging, aggregating and filtering the stuff that I care about and the people or sources that I respect. This is one part information management, one part markup, one part ranking and one part algorithmic search. Similarly, there is a lot of value to be realized in terms of the handling logic for autonomously organizing, archiving and deleting the stuff I don't get around to clicking on, as well as dealing with duplicates.

And from “WHAT” to “HOW”
So where do we go from here? In thinking about how to approach RSS as a development platform one has to consider Microsoft’s recent announcement that they “get” RSS and are embracing it in a big way within Longhorn/Windows Vista. Specifically, my knee jerk is that Microsoft is going to focus on getting the “message router” portion of feed management right and ensuring that hand-offs between a central feed repository and the consumers of those feeds (people and applications) occurs in a highest common denominator fashion.

They will also ensure that Visual Studio makes it easy to create custom applications around such a model. Microsoft understands that the best way to ensure broad adoption of Longhorn is to hook the developers first, as they drive the ecosystem that will compel consumers and enterprise to toggle over.

In parallel, one would expect that the open source community tackles many of the same issues. Towards that end, I am sure that multiple existing projects are logical candidates, depending on whether you view this as a middleware, content management, or application server problem.

Similarly, I would expect that fierce competition for developer mindshare between Microsoft, Google, Yahoo, Amazon and eBay will continue to push these folks to open up more and more of their APIs. For example, Google, Yahoo and Microsoft all have both deep investments in algorithmic search technology, and in case of Google and Yahoo today (and Microsoft soon), contextual advertising networks. They also all seem to get the premise that by exposing APIs to these functions, new and interesting applications can be born that extend their reach as a platform. My bet is that before too long, the filtration, personalization and ad serving functions get reduced to an API that a developer can plug into their application.

Finally, one area that will be interesting to see how it plays out is how much Web 2.0 looks like Web 1.0 in terms of continuing to be web browser based (especially as Ajax driven Web application models evolve) versus being powered by a next generation uber client application or specialized standalone applications.

As a blogger and an entrepreneur, as well as someone perennially on the prowl for better ways to manage online information in this age of digital media, I can say without hesitation that I believe that the emergence of RSS as a platform is one of the remaining gates to Web 2.0 taking flight. Or, as Yogi Berra might have said, the future of the Web is destined to be just like the past. Only different.

Mark Sigal is a seven-time entrepreneur whose focus these days is on digital media services; namely the re-envisioning of traditional video, audio, and advertising channels in a mobile broadband-enabled world.