oreilly.comSafari Books Online.Conferences.
Articles Radar Books  

Free Radical: Ian Clarke has Big Plans for the Internet
Pages: 1, 2, 3

Q: Peer-to-peer is the hot topic of the year. What do you make of that?

A: I think it's interesting. It is slightly strange since nobody really seems to know what peer-to-peer is, yet everyone seems to be raving about it. Everybody seems to be trying to nail down peer-to-peer, or at least some people are, yet other people are quite keen to broaden the definition as much as possible, so it will encompass what they're doing. I personally am not convinced of the value of the term "peer-to-peer" anymore, given that it seems to apply to everything from distributor processing, which has been around for a long time, to data distribution, which is perhaps a little bit more recent.

Q: Isn't there a parallel between sharing PC cycles and sharing PC storage?

A: There is a parallel but there are also a couple of important differences. First, in my view, one of the most important features of the Freenet network is that it's completely decentralized. Generally distributed processing relies on some form of centralization in order to orchestrate what's going on, so if your view of what is interesting by peer-to-peer is "Oh, well, we get to use all of these kind of client PCs," then distributor processing is quite a close analogy. But in fact what's interesting to me about these technologies is the fact that they're completely decentralized, and you're no longer beholden to some form of centralized server that allows the system to function.

Q: I wonder if you would just start at the beginning and describe your early thinking that turned into Freenet and how you envisioned this network at first.

A: There were two sides to my early thinking. There was a kind of philosophical interest, and there was also a technical interest. Philosophically I was very interested in the whole idea of freedom of information, and I was somewhat concerned by what I saw as increasing moves to impose censorship on the Internet. While in 1998, when I first started to think about this, this hadn't really begun in earnest, my fears have really been justified in the past two or three years in terms of a number of Western governments making increased efforts to both monitor and censor the Internet in ways that simply wouldn't be tolerated if applied to more conventional means of communication, such as the postal service or the telephone networks.

Q: What specific incidents were of great concern to you?

A: The first Western country to really impose what I viewed as somewhat Draconian censorship on the Internet was Australia, which came up with these laws whereby it had a list of Web sites that were censored and any Internet service provider in Australia that did not restrict access to that list of Web sites could be subjected to huge fines. The way that that list was generated was - in terms of the accountability of the people who were coming up with this list of what should and shouldn't be censored on the Internet - extremely dubious.

Subsequently, the United Kingdom had a Regulation of Internet Powers bill, which has now become law, that allows the security services to monitor all Internet traffic, and that was extremely worrying.

Q: You mentioned censorship in Western nations and I wonder if you have been thinking also about censorship in more totalitarian countries.

A: Well, certainly. In countries like China and Saudi Arabia, the Internet is very, very heavily censored. Certainly Freenet could still be used there to communicate securely and to share information securely. But whereas in Western countries it's very unlikely that encryption, for example, would be banned, that is possible in countries like China. Now in terms of their ability to enforce that ban - it will be extremely costly to do that, they could just ban Freenet full stop.

Q: What were your technical interests in creating a fully decentralized network?

A: If you look at the way the Internet works, it's very centralized in terms of its architecture. It's very inefficient. If you look at the Web, if 1,000 people in the U.K. request the same document from the United States, the same information travels across the Atlantic 1,000 times. That and other examples of centralization struck me as being highly inefficient in terms of network-bandwidth usage, which was, and still is, a somewhat valuable commodity.

So one of the things that Freenet does is it actually moves information around and dynamically replicates information to reduce the load on the network bandwidth. So in that specific example, if 1,000 people in the U.K. request the same document from the U.S. and they were using Freenet, it would only need to travel over the Atlantic once, and thereafter it would be stored locally and distributed within the U.K. - or within Europe, depending on where the demand was.

Q: And the other problem with the way the Web is organized is that the more popular a piece of information is the less available it becomes.

A: That's certainly true, and that is the other real deficiency that Freenet addresses - in that popular information does become more widely distributed, which means that you're not going to get what some people call "the slashdot effect" whereby an extremely popular piece of information becomes unavailable. The availability of information on Freenet increases in proportion to its popularity.

Q: Can you just explain a bit technically how that works? How the system is architected?

A: You could look at it like an ant colony where instead of food you have pieces of information, and instead of ants you have requests, which travel around this network. Freenet, when you request a piece of information on Freenet, you ask your local Freenet node for that information. If it has the information itself, it will obviously return it to you. If not, it will forward that request on to another node that is more likely to have that information - and nodes in the network actually learn with time how to better route information through the network - so they additionally move information closer to where the demand for that information is, so that when you request a piece of information, immediately after you requested it a copy of that information will reside on your computer and the computers close to you for a short amount of time. If you or other people close to you then request that information, they will receive that information immediately. So this is really the way that it dynamically moves information closer to demand.

Q: And how do you accomplish the spawning of copies of the information based on interest in it or requests for it?

A: Well, the actual act of requesting the information results in the information being propagated further, because the request will pass through a number of computers in order to reach the computer that actually has the information, and when that is passed back, a copy of it is stored on each of the computers which participated in the request, so the more requests there are for a given piece of information, the more widely distributed it becomes.

Q: And those intermediary nodes store this information for a substantial amount of time or just temporarily?

A: They store the information in least-recently-used cache, which means that the more frequently a piece of information is requested the longer it is stored by the node. If a piece of information - if demand for a piece of information drops off, then that information will effectively be removed from the node.

Q: So is your system of nodes similar to the e-mail system of servers where messages bounce from one server to the next until they get closest to where the recipient lives?

A: Yes, that's a reasonably close analogy. Of course e-mail relies on the domain-name system in order to route messages, and the domain-name system is kind of fundamentally centralized in its architecture. But there are similarities.

Pages: 1, 2, 3

Next Pagearrow

P2P Weblogs

Richard Koman Richard Koman's Weblog
Supreme Court Decides Unanimously Against Grokster
Updating as we go. Supremes have ruled 9-0 in favor of the studios in MGM v Grokster. But does the decision have wider import? Is it a death knell for tech? It's starting to look like the answer is no. (Jun 27, 2005)

> More from O'Reilly Developer Weblogs

More Weblogs
FolderShare remote computer search: better privacy than Google Desktop? [Sid Steward]

Data Condoms: Solutions for Private, Remote Search Indexes [Sid Steward]

Behold! Google the darknet/p2p search engine! [Sid Steward]

Open Source & The Fallacy Of Composition [Spencer Critchley]