BSD DevCenter
oreilly.comSafari Books Online.Conferences.


OpenBSD 3.8: Hackers of the Lost RAID
Pages: 1, 2, 3, 4, 5, 6

A new tool called stat has been included. It displays file status obtained from stat(2) or lstat(2). What can you do with it?

Otto Moerbeek: stat(1) is a tool to display the raw information on a file: its modification date, link count, owner, and more such things. Of course, there's this other well-known command, ls(1), that offers basically the same information. But ls(1) is mostly intended for human consumption, while stat(1) is nicer for scripting: it is capable of producing all available information in one call using output format, which is directly digestible by shell scripts. And if you, like me, are working on userland tools, you sometimes need the raw information to check if a command like tar(1) is doing the right thing.

stat(1) comes from the NetBSD project. We did some rewriting to make it use safe string operations like strlcpy and snprintf. We do not want unsafe string operations in the base source tree.

What is the purpose of xidle?

Federico G. Schwindt: Back in 2002, xautolock was removed from the base X system due to license issues. The author didn't want to change it, so we've been forced to take that direction. Shortly after, were added back to the ports tree but I missed its functionality from the base set. At the same time, I wanted to do some X programming, so xidle(1) was born.

Basically, it locks your X session when you're away. It also allows you to lock it by moving the mouse to a predefined corner. In reality it can run any program, but xlock(1) is the default, hence the term locking. And of course, it has a BSD license.

It seems you fixed several bugs in pax. This is an ancient tool, but you still find something to fix inside it. Were these pathname races and potential buffer handling problems a heritage of Berkeley's codebase?

Otto Moerbeek: Partly. Some of the fixes were done to fix bugs that were not in the original but introduced later. The pathname race fixes were to get remove the unsafe open(), write(), close(), chown(), chmod() constructs and replace them by the safer open(), write(), fchown(), fchmod(), close() sequence. The first is a variation on the more well-known traditional symlink race, which happens between the test for existence and actual creation of a file.

Looking at the release page, I read, "libc(3) source code has been converted to ANSI C." What type of nonstandard code was cleaned? And by the way, what is the origin of OpenBSD's libc?

Otto Moerbeek: There are three C standards: the original from Kernighan and Ritchie from 1978 (K&R), the ANSI C standard from 1989, and C99, the most recent. A large part of the OpenBSD source tree were originally written in K&R C, but a lot of files have been converted to ANSI C in the recent years. The C library (libc) was done for this release.

An important feature ANSI C adds are prototypes. Prototypes are vital to catch "misunderstandings" between the caller of a function and the implementation. While this problem was partly solved by introducing prototypes in the header files, the actual implementation of libc was mostly written without them. The C library mostly originates from the BSD4.4 Lite distribution, which was developed before ANSI C compilers were generally available.

Converting code to ANSI C is not only important to catch errors in argument handling. Having our code in a uniform format and style also helps in auditing; every program uses libc, and making it as bug-free as possible is very important. Auditing should not be hindered by having it in a nonstandard form, which might hurt the developer's eyes ;-)

What is the status of wide-character and locale support?

Marc Espie: Half the work is done. We stopped quite a bit before 3.8, in order to allow for reasonably comprehensive testing.

Right now:

  • The wide-char infrastructure is in. There are conversion functions from normal characters. C++ knows about it. The ports tree recognizes and uses these functions.
  • There is ctype-style locale support for 8-bit locale: lowercase/uppercase conversions are fully localized, and character classification works.
  • Multibyte character locales are currently disabled. This includes Japanese, Chinese, UTF8 ... all the multibyte to wide-char conversion simply converts each character to a big integer. The remaining code is currently being tested.
  • Like other BSD, we do not have full C99 locale support yet. Functions like wfprintf do not exist. We do not have collate functions or monetary support yet. The aim is first to have full wide-char and ctype support, then to add the rest.

I should add that implementing locale support looks a lot like a Chinese puzzle: you've got all these pieces that have to fit, and you have to find a correct building order, figuring out what you can have without destabilizing the whole. In the case at hand, this involved making sure that GNU-configure would not think we have full locale support yet, and also making some VERY intrusive changes to the libc (much to Theo's worries). But ... it works.

The utility gzsig lets you create and verify cryptographic signatures built into gzip file headers. I'm wondering if you plan to use it in the future to provide binary patches, or sign packages and release sets.

Marc Espie: No, I don't. Frankly, the concept of gzsig is something I stumbled upon a few years ago, and there was the beginning of a gzsig implementation in the old package tools. But it's gone from the new tools. gzsig has one major limitation: you have to download the whole archive in order to verify the signature. You can't do anything with a partial package, can't start scripts. So, the solution pkg_add is heading towards is signing the packing list, and making sure there's enough info in it so that you can check things one component at a time. Basically, you don't need a full sig for everything. If the packing list is signed, and it contains crypto hashes for all files, then it will work. And you have just-in-time signing, which is a very important feature.

Parts of it are working, parts are not finished, and I'm waiting to see what crypto hash we're going to use, since md5 and sha1 are basically out now.

It's just one feature among a lot of things happening in pkg_add. In 3.8, you'll notice the tools have become MORE sturdy. We've weeded out a lot of fringe cases, and the tools behave correctly in a lot more cases. And there's a new update option. In 3.8, it only helps you figure out what packages to replace. In -current, it's fully enabled, and it works.

We've got more checks. The packages have gone through extensive automatic consistency checks. And there's more planned. The quality is going up. Crypto signatures is just one component. The framework is designed to incorporate it when it's ready. But it won't be gzsig.

You worked on signal delivery, and improved signal handlers' reliability. What type of problems did you fix?

Otto Moerbeek: These fixes were typical teamwork. We had a known problem on sparc64 that sometimes cron(8) would not run after booting the machine. This has annoyed me for a long while, so at some point I decided to hunt the problem down. I changed rc to ktrace cron, and began a rebooting session to catch the error. After I had a ktrace of a failing cron startup, I saw that the problem occurs if a signal was delivered to a child process before the child returned from fork(). That enabled me to write a regress test that demonstrated the bug (called earlysig) without requiring a reboot, which put Mark Kettenis on track to actually analyze and fix the problem.

The macppc and i386 problems had to do with signal handlers using floating point. Matthieu Herrb had been debugging a problem with the X server on i386 without much progress, and at one point the hypothesis was that if floating-point computations were done inside a signal handler, the main program's floating-point registers could get trashed. Again I was able to write a regress test (fpsig) to show the problem, and further testing showed that it also occurred on macppc. Michael Shalayeff and Dale Rahn made the actual fixes for these bugs.

Pages: 1, 2, 3, 4, 5, 6

Next Pagearrow

Sponsored by: