BSD DevCenter
oreilly.comSafari Books Online.Conferences.


chrooted ntpd in NetBSD
Pages: 1, 2

A word on the ioctl(2) system call and its use in clockctl: before talking about the changes to ntpd, it is worth explaining what the ioctl(2) system call does. On Unix systems, all objects are seen as files. This includes device files, terminals, and so on. Of course there are some object-specific operations that cannot be done through a file interface (read, write, lseek, etc.). These operations include, for instance, getting terminal characteristics when the file is the standard output, or ejecting the disk when the file is a a removable disk's device. Nearly everything that cannot be done through the standard file-related system calls is done using ioctl(2) calls.

Here is ioctl(2)'s prototype:

int ioctl(int d, unsigned long request, void *argp);

d is the file descriptor on which we operate, request is a value indicating which command we want to perform, and argp is an optional argument pointer. The structure of the argument itself depends on the request. Here is an example of ioctl use in a userland program, for getting the terminal width in columns:

/* col.c -- print the terminal width */
#include <stdio.h>
#include <err.h>
#include <sys/ioctl.h>

main(void) {
	struct winsize ws;

	if (ioctl(1, TIOCGWINSZ, (void *)&ws) == 0) 
		printf("terminal width = %d\n", ws.ws_col);
		err(1, "ioctl failed");
	return 0;

In ioctl() first's argument, we have 1, standard output, which is attached to the controlling terminal. TIOCGWINSZ is a macro defined in <sys/ttycom.h> for getting window information on terminals. The third argument here is a pointer to a struct winsize where ioctl(TIOCGWINSZ) will write its data.

Of course, TIOCGWINSZ will only work if the standard output is attached to a terminal. It's possible to check this:

$ cc -o col col.c
$ ./col
terminal width = 80
$ ./col > toto
col: ioctl failed: Inappropriate ioctl for device

For the clockctl device, we use four ioctl commands, one for each of our system calls. All are defined in <sys/clockctl.h>: CLOCKCTL_SETTIMEOFDAY, CLOCKCTL_ADJTIME, CLOCKCTL_CLOCK_SETTIME, and CLOCKCTL_NTP_ADJTIME. Each command uses a pointer to a structure holding the system call arguments for its arguments.

To keep things simple, we use exactly the same structures as kernel when passing arguments to system calls. They are defined in <sys/syscallargs.h>:

struct sys_settimeofday_args {
	syscallarg(const struct timeval *) tv;
	syscallarg(const struct timezone *) tzp;

struct sys_adjtime_args {
	syscallarg(const struct timeval *) delta;
	syscallarg(struct timeval *) olddelta;

struct sys_clock_settime_args {
	syscallarg(clockid_t) clock_id; 
	syscallarg(const struct timespec *) tp;

syscallarg() is a macro that deals with machine-dependent alignment and endianness issues. It enables us to deal with machine-independent, system call argument structure declarations, whereas in fact these are really machine-dependent.

There is a special case for ntp_adjtime, which needs to set the value returned to userland. Since ioctl(2) already uses it to indicate error conditions, it is not possible for an ioctl command to set ioctl's return value. We work around this by including the return value in the ioctl argument (this is from <sys/clockctl.h>:

struct clockctl_ntp_adjtime_args {
	struct sys_ntp_adjtime_args uas;
	register_t retval; 

The sys_ntp_adjtime_args struct is defined in <sys/syscallargs.h>. The kernel uses it to store ntp_adjtime(2) arguments:

struct sys_ntp_gettime_args {
	syscallarg(struct ntptimeval *) ntvp;

Now that we have a precise idea of how the alternate entry points to the time-related kernel functions are made, let us move to kernel changes.

Kernel Changes: clockctl Device Driver Implementation

The clockctl device driver is a plain pseudodevice driver. There is some psuedodevice documentation explaining how to introduce such a driver into the NetBSD kernel. Since the kernel registration process is well described in the document, I will not cover it here. Let us focus on the driver structure itself. It can be found within the NetBSD sources in src/sys/dev/clockctl.c.

Each driver provides a set of functions, known as methods. The kernel calls the driver methods to execute operations such as open, read, write, ioctl, and so on. In NetBSD, the method names must be the name of actual operation prefixed with the driver name. For clockctl, we have clockctlopen(), clockctlread(), and so on.

When the user does an open(2) system call on the clockctl device file, the kernel will use the device major number to identify that the operation must be serviced by the clockctl driver. For character devices such as clockctl, this is done by reading the cdevsw array, which is defined in a machine-dependent file. (Unfortunately, driver major numbers are not unified on different NetBSD ports.) For the i386 port, the array is defined in src/sys/arch/i386/conf/majors.i386.

Once the kernel knows which driver is to service the open request, it just calls the driver's open method. For read, write, ioctl, poll, and other operations, the process is the same. The actual code path is a bit complicated, because there are two abstraction layers before reaching the driver methods: the first makes any object appear to be a file to userland (this is done with struct file, defined in <sys/file.h>), and the second, known as the Virtual File System or VFS, enables the transparent use of different filesystem types (this is done using struct vnode, as defined in <sys/vnode.h>). An in-depth explanation of what happens exactly is out of the scope of this article, but it might pop up in an upcoming part of my series on IRIX binary compatibility on NetBSD.

For clockctl, most methods are meaningless; only ioctl actually contains more than just return 0;. The ioctl method understands four commands, which we described in the section about libc.

The job of the driver is really simple. A code snippet might say more than an explanation:

	struct sys_settimeofday_args *args = (struct sys_settimeofday_args *)data;

	error = settimeofday1(SCARG(args, tv), SCARG(args, tzp), p);
	if (error)
		return (error);

SCARG() is another macro which deals with machine-dependent differences in the way system call arguments are structured. The clockctl driver just calls the function the settimeofday(2) system call normally would have called. The only difference is that clockctl does not check if the user is root, since the permissions are enforced at the filesystem level. To request this ioctl(2) command, you must have opened /dev/clockctl for writing.

Userland Modification: ntpd

As we said, we wanted to make as few modifications to ntpd as possible. This goal was achieved, since we only added command-line options to specify the UID/GID to run the process as, and the directory to chroot to after initialization, using the following two new flags: ntpd [-u user[:group]] [-i /path/to/jail].

There is very little to tell about these changes. There are probably some OSes with ACLified system calls where it was already possible for a non-root user to set the time. Therefore the changes are not really NetBSD-specific. This is why they have been sent to the NTP team in order to be included in the next NTP release. Propagating this change to the NTP team also ensures that the -i and -u flags will not be used for something else in the future. Having to face a conflict between new ntpd flags from the NTP distribution and the NetBSD locally-patched ntpd would be quite uncomfortable.


This work enables ntpd to be chrooted. The method we chose to do this is not perfect. One could argue (and in fact some have) that it is bad to introduce magic into libc. The best solution to chroot ntpd would indeed be to introduce capabilities, which are kind of ACLified system calls. Some Linux distributions now ship a ntpd daemon running under a non-root UID, and they do this using capabilities. This is a much better approach.

However, capabilities alone are a huge project. TrustedBSD is a subproject of FreeBSD that is aimed toward the implementation of filesystem access lists, capabilities, and other security features. The project was started years ago and is not finished yet. On NetBSD, nobody is working on capabilities, and in fact, people are waiting for TrustedBSD to settle before importing some code. It could be a very long time before NetBSD would have capabilities available. In the meantime, clockctl appears to be a good solution for chrooting any time-related daemon.

The advantages of clockctl are simplicity and the fact that we do not modify any existing APIs. The ntpd modifications are only about chrooting, not about the way time is controlled. The day we want to replace clockctl by capabilities, there is nothing to change in ntpd; it will work immediately.

Finally, it is worth mentioning orthogonal efforts to improve general daemon security. systrace was introduced by Matthieu Herrb and Niels Provos from the OpenBSD project, and was integrated into NetBSD by Christos Zoulas. It is now maintained by Niels Provos, who joined the NetBSD team in the meantime. systrace enables the system administrator to write a list of allowed system calls for a given daemon. The kernel will ensure the daemon does not do any other operations. That way, if the daemon gets compromised, it will not be able to execute things like system("/bin/sh"), even if it runs as root.

On another orthogonal direction, Jason Thorpe has made changes to NetBSD-current in order to remove the need for an executable stack. On processors that support it (which is not the case for the old 80x86), the stack can therefore be set non-executable, thus making impossible the whole class of exploits that use stack buffer overflows. The non-executable stack is not a new idea; it can be found in various OSes, but it is extermely effective at reducing security holes, at least on machines with a processor modern enough to be able to set memory as non-executable.


Thanks to John Klos, Simon Burge, and Christos Zoulas for reviewing this article.

Emmanuel Dreyfus is a system and network administrator in Paris, France, and is currently a developer for NetBSD.

Return to the BSD DevCenter.

Sponsored by: