O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


Network Tool Development with hping3

by Federico Biancuzzi
10/07/2004

In 1998, Salvatore Sanfilippo, also known as antirez, presented a new network scanner based on the IP ID field. While working as a security researcher, he started learning C to develop a tool called hping. That tool, then called hping2, is so good that it's still No. 6 in the Top 75 Security Tools of the Nmap Hackers mailing list.

However, Salvatore was not satisfied with the code, and he started a new project. The result is so good that it could influence the way network security specialists work. Until today, we have used tools such nmap or hping2 for specific tasks--automated scans with nmap, manual probes with hping2, and so on. People who needed particular features had only two options: writing a patch for an existing tool or developing a brand-new tool, probably based on libpcap, libnet, libdnet, or lib.

Maybe hping3 will revolutionize this approach. It's everything from a simple tool to a complete and scriptable framework for network analysis. The secret is the inclusion of a Tcl interpreter that interacts with the C core. Anyone, even a newbie programmer, can develop one of the famous nmap scan types with a simple Tcl script. The revolution has passed from writing a famous tool to developing a means of creating infinite features.

Federico Biancuzzi recently interviewed Salvatore by email about the development of hping3.

Could you present yourself?

Sure. Hello, I'm Salvatore Sanfilippo, an Italian open source developer in my spare time, mainly interested in security and programming languages. I'm 27 and work as a freelance consultant developing applications for my customers. Most applications are web-related, or low-level things like microcontrollers and network daemons.

How did your interest in network security start?

Related Reading

Network Security Hacks
100 Industrial-Strength Tips & Tools
By Andrew Lockhart

It all started when I installed Linux for the first time and joined IRC; this was around 1997 IIRC, I started with lame things like IRC wars :) Fortunately, after some time I got more interested in the technical side of the game, and started to study protocols and security. At some point I moved to Milan to work for a security company, where I found a lot of interesting people to talk with and to experiment with some new ideas. I didn't last more than four or five months in Milan because I'm from the south of Italy (Sicily) and found Milan not very good in terms of quality of life, but I learned many interesting things during this period.

If I'm not wrong, hping was your first open source project and one of your first programming projects. What have your learned developing the first version of hping?

When I wrote the first version of hping I had only five months of experience with the C language, so there was a lot to learn ... the first thing I discovered was the world of Unix system calls, and how a simple program like the first hping.c was only able to work on Linux and not on other Unix systems we had in the lab at Milan.

hping was also an experience regarding user interfaces. The first hping did less than many other programs already available, like the powerful ipsend, but the user interface was different, because hping is like ping in that everyone knows and can use it: you can see the replies from the target host, and to modify outgoing packets you just have to add or change command-line switches.

In my successive programming I have always tried to pay attention to user interfaces. I actually think that reinventing the wheel is not a bad idea if there is an effort to make programs simpler to use and more intuitive.

Why have you chosen the GNU General Public License to distribute your code?

Because I learned a lot using and looking at GPL code, and at the same time I don't like the idea that third parties can redistribute modified binary-only versions of hping.

What types of differences are there in the development process and code organization between hping2 and hping3?

The main difference is probably that hping2 was conceived as a tool, while hping3 adds to this tool a development system, something like a scriptable TCP/IP implementation in userspace. Another difference is that hping2 evolved incrementally, while in hping3 there was somewhat a goal from the beginning. At the moment, hping3 includes the support for hping2 command-line options; however, I'd like to remove that code from the C core and provide the same compatibility layer as an .htcl script.

About the code organization, hping3 is mainly a collection of libraries that may live apart from hping itself; hping2 instead is in the form of a single program that's very hard to reuse for something different.

From the user point of view, hping3 should be both simpler and more powerful, assuming that there will be two different classes of users. Programmers will be able to exploit the full power of a real programming language and a flexible packet construction/analysis sytem. On the other hand, it should be much easier for nondevelopers [to] run hping3 scripts developed by others than to use hping2. For example, one could develop a hping3 script to audit a firewall without doing all the common stuff by hand.

Why have you chosen Tcl?

Because I like "programmable programming languages" like Tcl and Lisp a lot. The programmer is free to reinvent the language, write new control structures, and so on. Tcl is very powerful, but for some reason few programmers fully understand it, so it's often regarded as a toy language. I use Tcl for everything not involving low-level things or speed. For the rest, I tend to use C.

What particular limit or feature does the language itself bring?

One of the best features that Tcl gives you is that you can specialize it so much to have a domain-specific language designed to play with packets. The limitation of all this flexibility is speed. For example, in theory you can use hping3 to write a firewalling engine, but in reality you can't go beyond a "prototype" because a firewall needs to be fast.

How does the TCL scripting engine communicate with the C core?

Related Reading

Network Security Assessment
Know Your Network
By Chris McNab

Tcl is designed to communicate with C, so it's pretty simple. The entire hping engine is exported to Tcl as a single Tcl command, so hping3 scripts are just Tcl scripts with a unique new command able to do a lot of new things related to packets and TCP/IP that are not possible with plain Tcl.

How does the C core work?

The C core is the sum of the hping2 code, used to provide command-line compatibility in this initial stage, and the new hping3 code. I'll focus on the latter. The core is comprised of different layers; the first layer is called ARS, which provides low-level packet construction and analysis. It works describing packets as different layers. For example, a packet can have two layers: IP and TCP. These layers are then "compiled"--the compilation will take care of computing checksums and dependencies between layers, so that, for example, the IP layer will get the total length field set according to the following layers, and so on. This way users of the library don't need to do too much and can just create a packet adding layers, and finally compiling the packet. The packet can be created as you like, but still the checksums, option paddings, and lengths will be OK. As I stated, ARS is also able to do the reverse--given a binary packet, it can be split into layers, modified, and then recompiled and possibly re-sent.

The higher level API is APD (ARS packet description). Basically this layer uses strings to represent ARS layers, so complex packets can be generated starting from human readable descriptions of layers like ip{saddr=1.2.3.4,daddr=5.6.7.8}+icmp{type=3,code=3} and so on, or binary packets can be translated into the string representation.

Given that APD packets are just strings, they are very natural to bind to Tcl, which is a language mainly based on strings.

Does hping3 take advantage of multi-CPU hardware?

No, for now at least, but there aren't plans to add support for threads because Tcl is mainly event driven in design. (Actually, Tcl is able to support threads pretty well; I just like more the event-driven way.)

I saw a lot of names in the credits page; could you tell us who did what?

Most of the contributions are limited to bug fixes or the addition of minor features, so I don't remember exactly who did what, with the exception of Nicolas Jombart, who provided significant help and code for the BSD port.

I tested hping3-alpha2 on FreeBSD and OpenBSD, and I found out that some scripts and docs are completely Linux oriented. Can we say that there's space in the development team for some BSD experts to improve the portability, especially on *BSD systems?

The goal of hping3 is to fully support at least Linux, *BSD, and Mac OS X. Currently most of the work is focused on Linux because that's the main development platform I use, but as happened for hping2, the BSD port will become more stable after some time. Actually hping3 is already working on BSD and Mac OS X, but there was far less testing on those platforms--what I mean is that most of the BSD port is already in the source code.

Do you plan to port hping3 to commercial systems like Solaris, Mac OS X, and Windows?

OS X for sure; it already works as well as the BSD port itself. I received some reports that hping3 is almost working on Solaris, which will eventually be semi-supported, but we will never claim to aim for Solaris support. Windows is a much more interesting target because of the big user base, and because there is already a working hping2 port (wiki.hping.org is where to go if you want to try it in your Win32 box), and most of hping3's OS-dependent code is either on the Tcl side, which is perfectly supported on Win32, or in functions for packet transmission and reception that can be easily extracted from hping2.

Are you looking for any developer? Which area would you like to add manpower to?

Now that hping3 is a programming environment, I'm looking for developers to provide interesting scripts written utilizing hping3 itself. Another interesting area is IPv6. hping3 is based on the ARS library, which is designed to be modular, so adding IPv6 shouldn't be hard.

Do you plan to create a repository for .htcl scripts created by the community like Nessus does?

Yes, it's one of my main goals once hping3 is widely used enough.

I've read some short docs on wiki.hping.org and some in the hping tarball. Are you looking for people to write more extensive documentation?

Yes! The goal of the Wiki is just that, to build a collaborative site in order to document hping directly online.

When do you think that hping3 will be stable enough to be released? I'm a little suspect that the release process could require a mythological time like Doom 3 did.

[Laughing] Actually it has already taken a lot of time, but now it's in the classic 98 percent done stage. The last 2 percent is very hard to do, but I think that the next time I don't have a big workload it will take another little jump. I hope that in 6 to 12 months at most I'll find the time to push hping to 3.0 stable.

Have you ever thought that hping3 could become an easier, alternative way to build low-level tools that today are written with C and libpcap, libnet, or libdnet?

This is the main goal of hping3 itself. Developers and security researchers are wasting a lot of time with low-level tools; they will go faster using advanced tools with a higher level of abstraction. If security researchers have better tools, they will be able to do more interesting research. Often programmers and researchers come up with a new idea, but if the efforts needed just to test whether the idea works in practice are too big, there is a good chance that the idea will remain untested for a long time.

In 1998 you posted on Bugtraq a short explanation of a new type of network scan called idle scan. It's amazing that it still works. Today we have networks and hosts that use multiple technologies to protect themselves, and then old and vulnerable hosts that permit spoofed scans. How can we make the Internet a better place if the gap among hosts' security keeps increasing? Trusting nothing? Paranoia by default?

In the case of the "idle scan," the problem is in the TCP/IP protocol: fortunately there is a fix you can apply in software (that is, unguessable IP IDs). Still, correct implementations of TCP/IP have this bug. So my guess is that Internet security should be more a collective and technological goal, rather than something focused on a few hosts on the Internet.

Part of the "secure host" idea is related to business, in my opinion. If you want to sell a security product, you have to create the (wrong) impression that your security fully depends on the amount of money you invest in security products. Unfortunately, the real world works in a different way--to be secure you need secure code for your applications, but these applications are in most cases developed by others, and code security in turn depends on software culture, libraries, programming languages, and other things related to the "community." Second, you need good and secure networking protocols, again something about the Internet community that you can't buy for yourself.

Last year nmap included the support for idle scan. What do you think about its implementation?

That's very good and smart, as nmap itself generally is. idle scan is simple to describe, but very hard to implement in a real-world program; it's not by accident that nmap is the first usable implementation of idle scan. I'm very happy that Fyodor [the author of nmap] did this great work, and I think he reached one of his goals: to make idle scan available enough to the masses in order to force TCP/IP implementations to fix this problem.

Have you ever noted that some of the most known open source networking tools (ettercap, ntop, hping, WinDump, and WinPcap) are developed by Italians? Why do you think we are so interested in networking and security?

It's hard to tell. I guess that in part it's because security is a stimulating thing. It's possible that another reason is that research is not well funded in Italy, yet there is still a lot of desire to do something new and innovative--and one of the rare fields where you can do a lot of interesting new things with little money is computer security.

I live in Italy like you, and I know that in our country it's nearly impossible to find any economic support for this type of project. How could the community help you? Are you looking for Internet services, hardware, or maybe a job?

I agree that in Italy it's very hard to get economic support :) But I think I found a way in order to have economic support (very minimal, but still enough to do a bit less work as a freelancer and invest more time in free software): advertising. I think that banners may provide enough economic support for many little projects, at least. hping gets quite a lot of unique visitors every day, so the best way to help hping is to put links to the hping web site where appropriate, if you own a web site.

Users will get software for free from your pages; I think that a little banner is acceptable. On the other hand, if open source software is funded by a company, it's possible the developer will not be free to implement what he likes, but what the company wants. Advertising doesn't have this problem; you get the money and can use the saved time to implement the features you want.

I also applied the advertising strategy to Visitors, a GPL weblog analyzer you can find at www.hping.org/visitors, and it is working pretty well even for this less known software.

Federico Biancuzzi is a freelance interviewer. His interviews appeared on publications such as ONLamp.com, LinuxDevCenter.com, SecurityFocus.com, NewsForge.com, Linux.com, TheRegister.co.uk, ArsTechnica.com, the Polish print magazine BSD Magazine, and the Italian print magazine Linux&C.


Return to the Security DevCenter

Copyright © 2009 O'Reilly Media, Inc.