oreilly.comSafari Books Online.Conferences.


An Interview with OpenBSD's Marc Espie
Pages: 1, 2

FB: Could you explain how W^X works?

ME: In simple words, if you can write stuff into memory, and then tell the processor, "Go execute that," then you have the basis for a security hole. You just need a buffer overflow or something similar to be able to write arbitrary code in memory, and go execute it.

W^X is, basically, being very selective about what areas get written to, and what areas are marked as executable. Make sure those areas are, as far as possible, distinct.

Traditional Unix already says that program text should not be writable. W^X involves changes in various areas of the system, most notably the kernel, and the dynamic loader, so that other areas are NOT left writable and executable at the same time.

For instance:

  • The stack is normally not executable.
  • Locations in a program's code that refer to external libraries, relocations, indirect function calls, are only writable while is doing its job, then they're marked executable and not writable.

In order to implement this, it has been necessary to have program sections that differentiate between "normal program text" and "relocation area" in a very fine way. This has driven the switch of i386 a.out to ELF, because ELF allows for this kind of distinction, and a.out does not.

On many modern processors, one can directly mark memory as writable or executable.

Some other less-gifted processors (with a history of design bugs that plague them to this day), like Intel's family, don't have this distinction. That's to be expected: this distinction is only useful for security, and security is not a selling argument for Intel. The OpenBSD kernel then uses a hack that allows memory over a given boundary to be effectively marked as non-executable. Thus allowing the implementation of W^X. Neat trick, that.

FB: Someone of the PaX team asked for credits on your W^X system. What happened?

ME: Independent invention. Nobody in OpenBSD was even aware of PaX until *after* W^X happened. There are even substantial differences that show this.

FB: Systrace is included since the 3.2 release, but it seems not used anywhere. Only since 3.4 is there an option to protect ports building. Why?

ME: There are a few reasons for that. First, Systrace was developed by Niels Provos, who is no longer developing code for OpenBSD due to disagreements with our project leader, Theo de Raadt. Pity.

Second, Systrace has taken a bit of time to mature. There were a few issues with it that needed to be solved. Then, it's hard to have genuine example uses of Systrace in a "generic" system.

OpenBSD more or less provides Systrace as part of a toolbox that you can use to secure the system. Also, Systrace can be used to change the security model of Unix in a rather cataclysmic way. And that we won't do. We still run normal Unix, and changing the semantics of Unix privileges in such a fundamental way may have wide-ranging and unpredictable consequences.

Finally, ports-building under Systrace has been a long endeavor that has taken several people dozens of days to make work (most notably Nikolay Sturm). It's not surprising it is only now robust enough to actually be usable.

One small detail you forgot to mention is that a *lot* of system users and groups have appeared in OpenBSD over the last few releases... If you look very carefully, there are very few programs that are still setuid root in the system. This might look like a quiet change, but this is more important, security-wise, than Systrace.

FB: FreeBSD 5.x and NetBSD-current have introduced new code to handle threads. What is the plan for OpenBSD?

ME: 3.4 has switched from libc_r to libpthread. Most of the effort is focused on userland currently. Thread support in 3.4 is MUCH better than it was in 3.3, a lot of programs that need threads (MySQL, KDE, etc.) work MUCH better than they used to be. This is mostly due to Marco S. Hyman's work.

FB: What is a priority for future releases and what's going on in 3.4-current branch?

ME: The priority for future releases is to get stuff more stable, more usable, and more secure ... as usual.

Beyond that, gee.... My personal priority is to finish replacing the old pkg_* tools with a new set of improved pkg_* tools. The switch has already occurred in OpenBSD post-3.4, and so far, people are happy with the new tools. The new tools have a simpler design, are easier to debug, are more powerful, and will permit lot of things that were simply not possible to implement under a reasonable timeframe with the old tools.

Already, the new tools do more checks for consistency, have better behavior with respect to dependencies, and can extract stuff in place with the need for a temporary area.

Hopefully, there will be on-the-fly package signature checking, smart policies for adding packages through ftp or scp, and system updates before 3.5 is released. Some of these issues are hard to tackle, so don't hold your breath. It may take some time to be ready for public usage.

Speaking for myself, this is part of a larger plan to improve our ports and packages. I've been rewriting our infrastructure from the ground up, transforming into something manageable, efficient, sensible, and useful. Adding features such as fake building (ensure packages and ports install do the same thing), flavors and multi-packages (uniform encoding of variations in building), and, lately, full documentation for (there's a manpage) and fast computation of dependencies (new in 3.4!!!) are just the tip of the iceberg.

The next natural step was to replace the pkg_* tools, as the old ones were hindering progress. This plan is advancing nicely.

FB: Are they developed with Perl?

ME: About the new pkg_* tools, yes, they are in Perl. And they're in the tree now, just grab a copy of current, and you'll be able to see them.

FB: Why does OpenBSD keep a branch tag also for ports? Don't you think that most people would prefer FreeBSD-like "always" updated ports?

ME: Two reasons:

  • We don't have the manpower to check that everything works.
  • It's much simpler to be able to move forward when you don't have to keep track of old tools.

The result is that, for each release, a very large subset of our ports collection builds correctly on a large array of machines. And we keep the infrastructure simple.

It's as simple as that -- we focus on quality and robustness. And we don't have the resources to keep it working over long periods.

Keeping important patches alive for past releases already takes enough resources as it is. We don't have enough competent people that would simply build the ports tree over several releases, analyze errors, and ensure that the ports tree builds correctly ... realize that there are over 2,000 packages built from the ports tree. At any one point, there are normally less than 10 packages broken on i386. A little bit more for PowerPC and sparc64, but not that many. And still a bit more on other architectures.

There is a core team (in an informal sense, there is no core in OpenBSD, just people that you notice because they do most of the work) responsible for handling most packages and keeping them working. You can see their work in commits, and that's at most two handfuls of people at any given point. Which amounts to each person being roughly responsible for tracking problems in 200 ports or more. Yes, we have a much larger number of maintainers. But most of these people don't have the resources nor the will to make sure their ports work and keep working on a wide set of architectures. If we could keep the number down to 10 for all architectures, then we'd be very happy already.

Several well-wishers have come forward in the past, and offered to do that.

Nothing but words. And we work with hard facts.

In comparison to FreeBSD, how many FreeBSD ports work at any given time? How many are broken? Looking at the FreeBSD mailing lists, I see a lot of lists of broken ports they are going to remove if no one steps forward and adopts them. And how many architectures does FreeBSD truly support?

I don't want to disparage their efforts, but I think our relative situations are very comparable. They chose to have an eternal ports collection and they pay the price in complexity and stability and manpower.

We have made a different compromise, and I much prefer our results: lots of reliable packages all the time.

FB: Is there any plan to provide binary updates, maybe using the ports system?

ME: We're thinking about it. It's not going to happen yet.

FB: Binary compatibility is strategic for a niche OS. It seems that writing compatibility would be a good way to discover what features you don't yet support. Of course, OpenBSD is a mature OS, but have you found any improvements that needed to be made?

ME: Binary compatibility is necessary to run a select few programs that software houses have not seen fit to port to OpenBSD. For instance, I run maple for Linux, adom for Linux, acroread (though gv works fine), and when I'm sick, Linux Java machines. I used to run Netscape, but we have a working Mozilla AND a working Konqueror now. I used to run RealPlayer, but MPlayer now does a fine job of it.

As far as I'm concerned, improvements that need to be made are more along the lines of actual standard compliance. You know, the stuff that is called POSIX, or ISO C, or Single Unix...

FB: With the many different machines supported by OpenBSD, how was the binary distribution available on the FTP server created? Did you have all machines available and do all the builds yourself?

ME: The binary distributions were made by following release(8) and ports(7) (see BULK PACKAGE BUILDING) and loads of careful testing. A few people did the builds for this release, mostly using their own machines, often donated to them by people friendly to the project, or bought using project's money. I personally didn't have time this round.

Obviously, there are some huge variations in build-time between a modern i386, which can rebuild src in under one hour, and an elderly Sun3 that needs a week to achieve the same result.

FB: It seems that there will be two new supported platforms: Pegasos and cats. Isn't this a resources dispersion? Have you enough manpower to handle this and still improve the most-used platforms (i386 and macppc and sparc)?

ME: OpenBSD is still improving, so I guess having two more platforms doesn't detract.

FB: What is the plan for SMP support?

ME: Some people are working on it. As usual with OpenBSD, we don't make announcements until things are ready. On the performance front, you can see interesting advances in Asymmetric Multiprocessing, more specifically the growing support for crypto-accelerators.

FB: Any plan to develop something like user-level Linux or virtualization extensions for the FreeBSD kernel? Or maybe add support for xen? (xen is a fast and free PC virtualizer, like VMware.)

ME: Not to my knowledge. But OpenBSD is a lot of volunteers' work, so I can't know what everybody is working on.

FB: A lot of attention is given to fix licenses and reduce the amount of GPLd software in the base system. Is there any chance for big-and-bad software like sendmail to be replaced? And gcc?

ME: Yes. This has already happened, in fact. We have replaced some fairly major pieces of non-BSD software with BSD stuff. To wit:

  • diff
  • patch
  • pf
  • gzip
  • dc
  • tar (we use pax)
  • m4 (I added enough functionality to our m4 so that it can run autoconf).

We might get traditional nroff at some point, and some other stuff, who knows?

FB: Another license war has started and it seems worse than before. Does OpenBSD really want to fork XFree starting from the last 4.4.0-RC2?

ME: Yes.

FB: Why do you think a lot of open source projects are modifying their license, limiting users freedom?

ME: Because they're not doing their technical job in the first place, and playing with the license gets them more attention, in the short term.

XFree changed their license because they thought they were not getting enough recognition. And they don't get enough recognition because they don't communicate enough. Also, because they have internal trouble.

I don't know all that much about the Apache license. What I know is that their code is not that great, from a security standpoint, and that we haven't updated to the new license.

The main issue is not exactly the *freedom* of the license, but the *proliferation* of different licenses. And the fact that most of them have not been written by lawyers. The XFree people tell us they haven't really changed their license, that what they do is not even new. Well, the wording is new. It isn't a tested license, and it's not 100% obvious what it says, exactly. A bit similar to the Apache license.

You've got two different worlds, here: the corporate world, where law is very important; and a portion of the free software world, who drafts licenses based on goodwill. Well, law is NOT goodwill. We don't know how the new XFree license will stand in court. The old one is better tested. And besides, there is not even a consensus in the XFree license.

Another issue is that the groups who write this software are not even closed. We do resent the fact that some member of the group (David Hawes) changed the license, almost unilaterally. There are hundreds of people, all over the place, who contribute to XFree routinely. Heck, I contribute to XFree: I test stuff, I find bugs, I give back changes to the imake infrastructure in there. None of us were consulted, not really.

If you allow the first change, who knows where the NEXT change will go?

Federico Biancuzzi is a freelance interviewer. His interviews appeared on publications such as,,,,,,, the Polish print magazine BSD Magazine, and the Italian print magazine Linux&C.

Return to the BSD DevCenter.

Sponsored by: