oreilly.comSafari Books Online.Conferences.


Inside Prelude, an Open Source IDS

by KIVILCIM Hindistan

Are Bad Guys In?

Today organizations, companies, countries, and ordinary individuals have reflections or even a point of presence in another medium, the Internet. In some cases this point of presence is more important than many real world assessments.

The Internet is built on TCP/IP, not the most secure of protocols. It is never easy to know who is trying to endanger your online presence or when. There's no single do-it-all piece of software. If you don't know what you're doing, a whole suite of firewalls, honeypots, and Intrusion Detection Systems (IDS) may not be enough.

The quest for security is hot, and it's recently focused on IDSes. An IDS is a virtual watchdog which watches the network and specific hosts for suspicious behavior. Then, it barks or bites. There are plenty of IDSes to choose from, both commercial and open source. Snort, a network intrusion detection system (NIDS) released under the GPL, is one popular example. Snort has reached maturity in its 2.0 release.

The open source community has another runner in the race. Prelude is a very promising and featureful IDS. It differs from other IDSes, being a hybrid system. Prelude operates both as a host and network IDS.

Prelude has five main parts:

  • Sensors are detection entities deployed at strategic points on the Network. The sensors are able to report detailed security alerts to a Prelude Manager.
  • Managers are centralized data processors for Sensor data. Managers can forward alerts to other managers or to Counter Measure Agents. In a distributed environment several managers can act as relays to a CentralManager.
  • Counter Measure Agents receive data from a Manager concerning a particular anomaly and perform an action designed to stop or otherwise thwart the anomaly.
  • Frontend is the centralized viewing point for anomaly and attack data. Data is archived, normalized, and processed into a format which allows easy administration and understanding of the current security status.
  • The Prelude Library was created to ease the effort of sensor developers by providing common features and a standard API. It is used by all the Prelude programs. libprelude makes it easy to port existing applications so that they can report to the whole Prelude system. (Snort, honeyd, nessus, nagios, systrace, libsafe, and hogwash were modified to include Prelude support.) Patches are available on the Prelude site.

Related Reading

Incident Response
By Kenneth R. van Wyk, Richard Forno

This modular infrastructure allows Prelude to be manipulated and deployed to answer different needs.

Let's ask what really matters to the project founder and lead developer, Yoann Vandoorselaere.

O'Reilly Network: Hello Yoann, before we hear about Prelude, I'd like to learn what is hot about IDSes. Security people just seem unable to live without them anymore.

Yoann Vandoorselaere: The popularity of IDS software can be considered to be a trend of the moment, with all the positive and negative side effect this can bring. The bad point about this is that we see more and more companies deploying an IDS without curing their security problem.

Having a good IDS is worth nothing if you don't have a good security policy on your network. You might detect some of the attack, but there is no way you'll detect them all. Furthermore, when an attack is detected, you want to be sure it's not a false positive, and you may then have to react.

Automated reaction can be used in certain cases, but it must be applied with lot of caution because of the implied risk of self Denial Of Services. This mean you can't react against every attacker or every kind of attack. You might defeat some of the attacks, but you won't defeat them all. And even if you forget about all these problems, the time needed to take counter measures against a given attacker, which is present in every IDS product, might be enough for the attacker to gain permanent access to the system by using different techniques.

Detecting and Reacting is not enough and will never be enough. You also need a rock solid security policy. On the other side, when a system administrator gets the security approach right, then an IDS plays an important role in the security infrastructure.

As for what's hot in the IDS field, it seems to me the latest marketing buzzword is "IPS" (Intrusion Prevention System). IPSes are a new kind of insect that will save the world from black hats wishing to destroy it. IPS is a technical term that doesn't invent anything new: IPS is the combination of IDS real time detection capability with full control over what will or will not be forwarded to the destination of the data. IPSs prevent an attack from actually happening because they will detect the attack before it reaches the target.

Most IDS can easily be modified so that they become IPS. For example, you could take whatever NIDS you'd like to, make sure it runs on an OS where the application is able to decide what can be forwarded from an interface to another and what will not, modify it so that it uses the provided OS hook to do so, and, at that point, decide or not to drop the packet if you detect an attack.

IPSs just make the IDS active, instead of being passive like they are right now. However, you get the same set of problems that I mentioned above. IPSes just remove the race between the time you detect the attack and the time you'll react against it.

ORN: What are the main differences of Prelude over, say, Snort or Symantec's Net Recon? Can you explain a bit about the whole architecture?

YV: I don't know about Symantec's Net Recon, but I can give you a fairly descriptive explanation about the difference between Prelude and most other IDSes available. Prelude is an hybrid IDS. This mean Prelude provides you with the ability to centralize events gathered across your network by different sensors.

A sensor is software whose main task is to react to various stimuli in order to detect different kind of events. There are several types of sensors, all detecting different things and analyzing different pieces of information.

All the events (for example, an alert) generated by these distributed sensors are sent to a component called the prelude-manager which is responsible for centralizing them in a single place. From there, you can correlate events generated by these different sources and deduce what has to be deduced.

Moreover, we often see people referring to Prelude as being a "meta" IDS because of its inherent capability at providing hooks to port existing security applications so that they can report events to the Prelude IDS system through the Prelude Library.

For example, the latest Libsafe releases include Prelude support. And we distribute patches for Snort, Honeyd, Systrace, Nagios, Nessus, and Hogwash so that they can be made Prelude-aware.

You might want to have a look at a description of Prelude's architecture.

ORN: In Prelude, you have come across some very nice ideas. Did you ever think it would be better to contribute to Snort, rather than undergoing such a big task as writing a complete new IDS with a new hybrid model?

YV: I originally started Prelude in 1998 because of the lack of IDSes in the free software world. I was not aware of any other IDS at this time. I was a system administrator at the time, and we were in need of such tools. I also did it to learn. I think that there is no better way than experience for learning. I like rewriting something in an empirical fashion until I think it's perfect (well there's nothing perfect, so let me say "good enough").

Moreover, Snort is a Network Intrusion Detection system, so there is really no point in comparing both product. Snort focuses on just one of the areas of intrusion detection: the network part. Also Snort can be modified so that it become a Prelude sensor. A patch is available on the Prelude website.

Two others components we could mention are the Manager and the Prelude library, which are the main pieces behind the whole suite. You can see these as the bridge between sensors and users.

ORN: We know that you consider an IDS a must. Can you state the importance of other complementary components, such as a firewall, honeypot, or a security savvy admin?

YV: The most information you can pull to the IDS the better. All these systems have their pros and cons, but all of these provide you part of the infrastructure needed to secure and monitor your network.

The more you can glue security related application together, the more you'll be able to get accurate report of the systems activity. Thus the reason for Prelude, whose job is to provide the needed hook to glue them together.

ORN: What is the biggest mistake people planning the IT security infrastructure make? What do they lack? Why are there so many successful attacks?

YV: I'm not sure. Maybe system administrators don't consider security to be their job. So some of them may not monitor security mailing list accurately and might delay upgrade in a security insane manner.

ORN: Let's talk about the license of Prelude. It is licensed under the GPL. Did you ever consider any other option?

YV: The whole Prelude suite is licensed under the General Public License. It happened that some company proposed to pay me to work on a fully proprietary version of Prelude. This is an idea I am against, even through we are currently lacking sponsors. I tend to think open source is something that helps people understand how things work and helps them to develop things in an efficient fashion. I think it tends to forge better software engineers. I learned a lot through open source.

Today we are facing a problem due to the lack of sponsors. The development effort is slower than it was previously, when I was sponsored by MandrakeSoft through December 2002. Since then I'm seeing more and more companies who are making money out of Prelude or who depend on it for their activity but are not willing to give money to support the project.

It also appear that some of them would be willing to use our API for their proprietary softwares to benefit from the Prelude architecture. This is not possible currently as the APIs in question are GPL. I'm not planning to move them to LGPL anytime soon, as it would allow more companies to benefit from our swirl without giving anything back to the project. That's why I am currently investigating the possibility of dual licensing these API. These companies would then have to pay for a license in order to make proprietary softwares interoperate with Prelude.

ORN: How about the team? You are the lead developer, but there are fifteen or more developers. How do you coordinate? What makes this team go?

YV: These days, due to my lack of time, resulting in the need of sponsors, I mostly manage the team and the development effort. I'd like to spend more time coding for Prelude, but I guess it won't be possible until I find a sponsor.

We mostly use IRC, mail, as well as a Bug Tracking System in order to communicate. Coordination is not always easy because some people tend to have a big ego, and I have a big ego also. People might not easily accept it when you say "no". I guess managing a team always requires a lot of time, and I think I'll never apply for that vacancy when looking for a job. As for what make this team go, I guess the team is just a set of programmers interested in security and with full confidence in the way Prelude does things.

ORN: The Prelude project is looking for sponsorship. What kind of a sponsorship are you interested in? Funding, employment, or any other model?

YV: I'm currently trying to setup a legal structure (association, company) around the Prelude project which should bring an official structure on top of the project and, I hope, would encourage company in funding the project.

However, there is really no straightforward way to gather funds for the project legally other than being employed by a company. At least, there are many difficulties here in France. The law doesn't provide any facility for people developing free software. For example, it would be very complicated if I just created a Paypal account for people to send me, or other developers, money for helping out the project.

In France, you can't gather money without declaring it to the government, meaning you'd have to issue a bill for each transaction. People would have to pay for something well defined, not only to help the project. Apart from that, you also have to deal with a lot of paper in order to work as an independent person. Donations exist, but they are taxed at a very high rate for the person receiving the money, meaning they are not worth the effort.

I think it's a shame that free software is not recognized by the government. I hope one day the law will provide facilities for free software developers to get funds for the projects they are working on, through I really doubt it, seeing how things are currently evolving in Europe (see patents and DMCA-like debates).

So at the moment I'm hoping a company interested in Prelude will show up and salary me, which would be the better short term solution. This shouldn't forbid companies that cannot afford to pay someone a salary from funding the project: the better solution in this case would be to contact me directly so that I can provide up-to-date information about the status of the legal structure creation. Once it's done, I can offer instruction about the way to send funds to the Prelude project.

ORN: Thank you, Yoann. I hope we will be hearing more about Prelude and you will have better conditions to develop Prelude.


KIVILCIM Hindistan works as a full time computer security consultant with a CISSP, using Linux and Free Software as weapons of choice.

Return to

Sponsored by: