ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


NetBSD 2.0 Rendezvous

by Federico Biancuzzi
02/24/2005

In December 2004, the NetBSD Project released the feature-rich NetBSD 2.0. Even after such a masterpiece, developers kept working on improvements, new features, and new ports following the new development roadmap. Federico Biancuzzi recently interviewed them to find out what they are working on and how they plan to promote their project in the near future.

NetBSD's goal is to port the OS to as many platforms as it can. Which missing platforms would you like to support?

Christos Zoulas: We are currently working on IA64 and we should have something to show soon. As far as other platforms go, it is quite random. We need to have access to hardware or to a simulator, and find a developer that has time to do the port. I'd personally like to see a port on the Zaurus (should be easy because of our existing arm support), the IBM360 (which is a lot harder without help from IBM), and some of the big SPARC SMP systems that we don't support (there has been some work on that).

Is anyone already working on new technologies like 10Gigabit Networking, PCI-Express, and ExpressCard?

Hubert Feyrer: NetBSD already has support for these devices: the dge(4) driver supports cards based on the Intel i82597EX chipset, like the Intel PRO/10GbE LR. The i82597EX chipset supports IPv4/TCP/UDP checksumming in hardware, as well as TCP Segmentation Offloading (TSO). The driver does currently only support the hardware checksumming features, which can be enabled with the ifconfig(8) command. The driver also makes use of the link flags link0 and link1 to set the PCIX burst size; see the dge(4) manpage for more information.

Support for PCI-Express is already available in NetBSD's machine independent PCI bus interface, which is used for all hardware platforms NetBSD runs on, including, but not limited to, Intel- and AMD-based PCs, Opterons, Sun UItraSPARCs, and Apple PowerPC machines.

An example of NetBSD's performant support of this hardware can be seen in the Internet2 Land Speed Record achieved by SUNET. Using two off-the-shelf PCs with the above-mentioned cards made it possible to transfer data in a production network between Lulea, Sweden, and San Jose, U.S. at an average rate of 4.3GBit/s over the duration of an hour, with peak throughput at about 8GBit/s for a single stream. See SUNET Internet2 Land Speed Record: 124.935Pbmps (single stream) and SUNET Internet2 Land Speed Record: 122.367Pbmps (multiple stream) for more information, including details on the network infrastructure and endpoints and their configuration.

FreeBSD 5.3 and DragonFlyBSD 1.0 provided a multi-threaded network stack, and now their developers are working on other sub-systems too. Do you plan to follow the same path?

Christos Zoulas: We are definitely going to multi-thread the network stack very soon. This is necessary in order to achieve wire-speed performance on gigabit+ networks. I don't think that this will make it for 3.0, because it is scheduled to be branched in February. This release will incorporate significant fixes to our NewReno code, SACK, and TCP Westwood+ system.

FreeBSD 5 introduced the new scheduler called ULE, and Linux 2.6 included a new O(1) scheduler. What about NetBSD?

Luke Mewburn: No specific plans at this time. We will observe FreeBSD's experiences with ULE and consider importing it if it is beneficial in the long run.

Hubert Feyrer: As far as I understand, there is a certain desire to have pluggable scheduling modules in NetBSD to accommodate different schedulers than the classic round-robin scheduler NetBSD uses right now. If and when this happens to materialize is not yet known due to limited capacities available in volunteer projects like NetBSD.

FreeBSD developed a framework to support binary drivers for MS Windows. Is this something interesting for NetBSD, too?

Hubert Feyrer: It is interesting to NetBSD, of course, as it enables using proprietary, closed-source/binary-only drivers at least on one platform, where no driver at all would be available otherwise. Of course, we much prefer proper drivers based on NetBSD's driver framework, which requires vendors to release documentation along with the hardware they sell. That way, people would benefit not only on one hardware platform, but any of their own choice.

As it currently stands, it's nice to have Atheros, Nvidia and NTFS drivers available that way on PCs, but leaving out all the fine Mac and UltraSPARC hardware seems suboptimal from NetBSD's point of view.

Luke Mewburn: Possibly. We'll observe FreeBSD's success with the concept.

OpenBSD started an effort to include in the main code base drivers for E1/T1 or T3/E3 to be a valid and cheap alternative to Cisco products. Is this be something interesting for NetBSD, which is known to be easily portable to embedded devices?

Luke Mewburn: Sure. There is a lot of "cross pollination" of drivers and subsystems between the BSD projects, which is good.

I've read some epic threads about the inclusion of PF in the main NetBSD code base. Now that Itojun has imported it, how will PF and IPF interact?

Luke Mewburn: There are some issues remaining with the PF integration, especially in terms of the PF + ALTQ interactions.

A longer-term goal is to separate out a packet classification so that the code can be shared between systems such as IPF, PF, and ALTQ without unnecessary replication.

James Chacon: PF is being included as another packet filtering choice for users to have available to them. In the end it's expected users can pick/chose whichever software best meets their needs here, so being able to offer both is a big win.

Christos Zoulas: PF and IPF are competing products, each one having its strengths are shortcomings. NetBSD will support both, and may the best of breed win.

Is anyone working on virtualization extensions to have multiple independent systems running at the same time?

Hubert Feyrer: Yes, in two ways: on a kernel level, there is NetBSD/xen, which acts as a virtual machine that can run several concurrent operating systems, with NetBSD being one of them (besides Linux and Windows). NetBSD was especially prepared to run as "Domain 0"; i.e., as the first machine on top of the Xem VM.

On a userland level, work is underway to do something like Usermode Linux on steroids: not only are system calls, etc. trapped, but also CPU instructions, which allows running non-native binaries. An example implementation of NetBSD/arm32 binaries running pkgsrc bulk builds on a NetBSD/i386 system was demonstrated at the pkgsrcCon in May 2004 in Vienna, and it was very impressive to see the system compile packages in near-realtime.

Do you have any plans to include or develop a kernel framework for clustering like DragonFlyBSD?

Christos Zoulas: We would like to eventually, but this is a longer-term goal for us. Improving the filesystems and realtime support are higher priority for us, and easier to achieve.

Do you plan to use systrace to alter the common Unix permissions model without introducing MAC extensions like FreeBSD 5 did?

Niels Provos: Well, that would be one application that you could use it for. I think about it as a tool that should be easy to understand and deploy. In many cases, MAC systems are very complex and very difficult to understand. systrace is not perfect, but provides some easy-to-deploy security enhancements. I think that it is going to be important to make it work well for the average user; e.g., many users download untrusted software and run it. systrace could prevent any malicious code that might be hidden in there causing damage to the system.

Do you have any plan to replace the old plain-text logs format with something like Bruce Schneier's Cryptographic Support for Secure Logs? Tampering with logs is pretty easy, and this has gone unaddressed for years.

Christos Zoulas: We have not considered this yet. While it sounds like a good idea, it has the drawback that the log files are not simple text anymore. I personally think that it is better to centralize log collecting in a secure machine, because cryptographically protecting the logs only helps you to verify that the logs have been tampered with. It does not help you to recover the information that has been potentially lost. We do have append-only files right now, and this seems to serve us well so far.

Do you plan to use Verified Exec to distribute binary updates?

Brett Lymn: I really cannot speak for the project. It would be nice to have as an option, but verified exec is not really something that I would recommend for a general-purpose workstation because it can get really annoying very fast--it is more aimed at providing an extra level of protection for machines like firewalls, web servers, and the like. Distributing the fingerprint file is a tricky thing to do; issues like making sure it has not been modified in transit, who is allowed to generate the file, and where the file is generated are all difficult questions that need to be solved before an official fingerprint file can be produced. At the moment, I have no firm plans to push this now but I may ask the project about doing this later.

Christos Zoulas: No, we are going to be using the standard hash signature support for that (MD5, SHA1 etc). Note though that we have /etc/mtree/set.* (per-set mtree specfiles) that contain a SHA1 hash suitable for more easily generating veriexec hash lists and/or validating a system.

What is the status of package views support?

Christos Zoulas: Some packages have been converted to use views, but not all. Package views are targeted for the pkgsrc-2005Q2 release, which will take place in June 2005, but may be brought forward if possible. There has also been a lot of pkgsrc work in porting to other platforms--the list of currently supported platforms looks like:

There's also been a lot of work in cutting the number of broken packages via regular bulk builds on a number of platforms, by a strict monitoring of any new features in the infrastructure, and by rapid fixing of anything that is broken.

Alistair Crooks: The infrastructure for package views is in place, and a number of packages (maybe 30-40 percent) have been converted to be able to take advantage of package views. We are targeting Q1/Q2 2005 for having all packages converted to be able to take advantage of package views, although we'll retain the existing "overwrite" functionality (it is desirable in a number of situations, such as on embedded machines).

Subversion 1.1 has been released. Do you have any plan or desire to adopt it instead of CVS?

Hubert Feyrer: Not yet. There is no experience with a repository as big as the NetBSD one, and the import routines from CVS to Subversion need a lot more work before all the information kept--and used!--in the NetBSD CVS repository can be transferred into Subversion easily.

Luke Mewburn: Not at this time. We need to further analyze the long-term benefits and disadvantages in migrating our version control system from CVS to another technology. It may be that Subversion doesn't provide us with enough benefits over CVS to be worth the hassle of converting. There are other open source version control systems available as well that advertise as being "CVS replacements."

NetBSD is the only BSD project that accepted the new XFree 4.4 license and imported the code as it is. Why?

Matthias Scheler: The new XFree86 license is a normal BSD-style license with and advertisement clause. Because most of our own source code uses a similar license, we didn't see any reason not to accept the XFree86 one.

We imported the code because it offered the technically best solution available at that point of time. It fixed many bugs that existed in the previous XFree86 4.x release and offered support for new hardware, which various NetBSD users were waiting for.

Related Reading

The Complete FreeBSD
Documentation from the Source
By Greg Lehey

Luke Mewburn: The NetBSD Foundation license is still based on the 4-clause BSD license. A significant chunk of our codebase is also under 4-clause BSD licenses from other copyright holders. It would be hypocritical of us to not import XFree 4.4 solely on their current license given that it's so close to existing licenses that we use. That's not to say we won't consider changing on technical grounds in the future, just like any other component of our software release.

Why have you registered the "NetBSD" trademark?

Luke Mewburn: To protect the name "NetBSD" from use or abuse by external parties who do not share the goals of the NetBSD Foundation and our members (developers with commit privileges). We are not unique in the open source world for feeling the need to do this; the Linux Mark Institute (LMI) was established a few years ago specifically for providing a means of sublicensing the Linux(R) trademark.

Does this means that since the application was completed, we should always use the trademark symbol after the name?

Luke Mewburn: As "NetBSD" is a registered trademark of the NetBSD Foundation, the "(R)" symbol is more appropriate than "(TM)." The NetBSD Foundation is in the process of composing a trademark usage policy, similar to LMI as mentioned above.

A new logo, the "NetBSD" trademark, and a new major release number: why? Is this a new marketing strategy or a way to make the project look more professional?

Luke Mewburn: Both.

Does the NetBSD Foundation plan to certify the operating system for Common Criteria or other international standards?

Christos Zoulas: Certifications usually cost a lot of money that can be better spent in other places. Of course, we are not going to complain if a corporation who wants NetBSD certified pays for it. Finally, we have not had a certification related question in years, which means that NetBSD users do not care enough about them.

Why did you choose to change the NetBSD version numbering scheme?

Christos Zoulas: There was a lot of confusion between release versions, which were of the form of X.Y.Z (e.g., 1.6.2), and NetBSD current versions, which were of the form X.Y[A-Z]* (e.g., 1.6D). To solve that, we decided to make current X.99.V (e.g., 2.99.15), which is going to be branched to become [X+1].Y.Z (e.g., 3.0.0). This way, all non-current versions of release X will have a version number less than the one of NetBSD-current for that release.

NetBSD 2.0 was released on December 2004. What is the next stable step, 2.0.1, 2.1.0, or both?

Christos Zoulas: 2.0.1 will precede 2.1.0, because there are some pullups from head that are required for 2.1.0. We are scheduling 2.0.1 for the end of February because we need the new build machines online to build the release, and the build machines will be coming online this week. We are scheduling 2.1.0 for the end of March, because of the time it will take to integrate the pullups mentioned above and the ones currently in the queue (more than 100).

Do you plan to modify the release cycle too, maybe to something like one release every six months, as OpenBSD does?

Christos Zoulas: Yes, we are planning to have more frequent releases. We have made a lot of mistakes in the past that have delayed the release schedules, but hopefully we have learned from them. We were very aggressive about committing large sub-system upgrades just before a release, and that added a lot of bugs to the code that needed to be fixed for the release. This not only delays the releases, but also adds extra work, since each bug needs to be tested on head and pulled up via patches by the release engineering group.

Having said that, I don't think that we want to have too-frequent releases, because we don't want to force frequent upgrades on our production users. We believe that production users are better served by a stable platform that gets enhanced with security fixes, bug fixes, and limited scope functionality improvements. This way, a production user can upgrade from 2.0 to 2.0.1 to get security fixes, or to 2.1.0 to get, in addition to security fixes, other bug fixes and functionality improvements. Both paths, though, are well-tested and should have no negative impact.

I think that most software vendors don't have a fixed release schedule these days, and their releases are mostly driven by market needs. We are going to try to have at least one release per year from now on. This is because we don't want the step between releases to become too large, and because it makes release engineering simpler. For example, our current target is to release 3.0 by mid-year.

The Netfilter web site features some pages about all the companies that were not fulfilling the obligations imposed by the GPL and were sued, or that signed a "Declaration to Cease and Desist" from further distributing their product (Access Points) without adhering to the license terms. My own opinion is that most of those companies and a lot of smaller ones around the planet keep choosing Linux and other GPLed software because it's "trendy," not because they have found it better than BSD. Do you think that NetBSD should improve its marketing especially for this type of embedded projects, like AccessPoints?

Luke Mewburn: NetBSD is more than appropriate for use by companies in embedded applications; our license goal permits closed source enhancements, and our portability goal provides a portable operating system that is one of the easiest systems to cross-compile from other operating systems that third-party developers feel more comfortable developing in.

James Chacon: NetBSD follows the BSD copyright and not the GPL (though there are pieces of the toolchain and some userland tools that are GPLed). But as far as the main system itself, it's completely free for any given company to use for whatever purposes they choose without restrictions (within the boundaries of the license depending on 2 vs. 3 clause). As a result, I'd expect companies to continue to take a serious look at all the BSDs when deciding on an OS to use in an embedded product.

Sometimes, I read complaints about the big number of committers in some BSD projects. Could you tell us the current total number for each section (pkgsrc, src, xsrc, docs)? How many of them are really active?

Luke Mewburn: There are approximately 250 people with commit privileges to NetBSD. We don't have a further breakdown for each section, although it would be possible to grab the mailing list archives of the source-changes mailing list and perform your own analysis on it.

James Chacon: While there are a large number of people with commit access to the source tree, it's also the case (at least for NetBSD) that these tend to break into various areas of interest/ability and historically, the number hasn't been an issue.

The largest number of committers we have is probably in the pkgsrc area and even there, problems don't often arise, as changes are coordinated for any large scale modification/additions. With quarterly freezes and releases, it's been a very stable source tree.

As far as the rest of the tree goes, while a specific commit may cause discussion/changes, I've almost never seen any problems arise regardless of the numbers of people with access. In most cases, that ends up as a benefit, with more eyes able to look over a given change and provide feedback.

Hubert Feyrer: After some recent looking at source-changes posts, it can be said that an average of about 90 developers commit to src+xsrc on a regular basis, while there are about 60 people working on pkgsrc. These numbers are per month, and not each developer may commit changes every month (think of people working on modules or package updates and the committing them when ready), so the actual number of active developers is most likely higher--IIRC, the amount of "active" developers per year is something like 250.

What events will feature NetBSD developers in the future?

Luke Mewburn: I expect that NetBSD will have developers and users presenting tutorials and papers at various UNIX and BSD [events] internationally. That has been the situation in the past, and continues to be the case.

Hubert Feyrer: Please check out our Events page:

We try to do booths to display NetBSD, answer questions, and offer limited amounts of merchandising material.

Federico Biancuzzi is a freelance interviewer. His interviews appeared on publications such as ONLamp.com, LinuxDevCenter.com, SecurityFocus.com, NewsForge.com, Linux.com, TheRegister.co.uk, ArsTechnica.com, the Polish print magazine BSD Magazine, and the Italian print magazine Linux&C.


Return to the BSD DevCenter.

Copyright © 2009 O'Reilly Media, Inc.