BSD DevCenter
oreilly.comSafari Books Online.Conferences.


IRIX Binary Compatibility, Part 3

by Emmanuel Dreyfus

IRIX Oddities: system calls that you will not see anywhere else!

Now that we are able to launch dynamic binaries, the goal is to get them linking. The dynamic linker has to do a lot of system calls before actually launching the program. Most of them are plain SVR4, and hence are taken from sys/compat/svr4. Here, we will deal with IRIX-specific system calls.

syssgi() Overview

One of the very first things on which we fail when running IRIX 6.5 binaries is the syssgi(2) system call. In fact, syssgi(2) is more like a meta-system call. Its first argument is an int named request. Depending on request's value, syssgi(2) will run literally dozens of different commands. The remaining arguments to syssgi(2) are interpreted according to the request argument.

syssgi(2) commands include some quite standard functionality that is implemented in plain system calls on NetBSD, such as getgroups(2), getsid(2), or getpid(2). There are also some SGI-specific things, such as commands to get hardware inventory, system configuration, or NVRAM values.

In This Series

IRIX Binary Compatibility, Part 6
With IRIX threads emulated, it's time to emulate share groups, a building block of parallel processing. Emmanuel Dreyfus digs deep into his bag of reverse engineering tricks to demonstrate how headers, documentation, a debugger, and a lot of luck are helping NetBSD build a binary compatibility layer for IRIX.

IRIX Binary Compatibility, Part 5
How do you emulate a thread model on an operating system that doesn't support native threads (in user space, anyway)? Emmanuel Dreyfus returns with the fifth article of his series on reverse engineering and kernel programming. This time, he explains thread models and demonstrates how NetBSD emulates IRIX threads.

IRIX Binary Compatibility, Part 4
Emmanuel Dreyfus tackles the chore of emulating IRIX signal handling on NetBSD.

IRIX Binary Compatibility, Part 2
Emmanual Dreyfus shows us how he implemented the things necessary to start an IRIX binary. These things include the program's arguments, environment, and for dynamic binaries, the ELF auxiliary table, which is used by the dynamic linker to learn how to link the program.

IRIX Binary Compatibility, Part 1
This article details the IRIX binary compatibility implementation for the NetBSD operating system. It covers creating a new emulation subsystem inside the NetBSD kernel as well as some reverse engineering to understand and reproduce how IRIX internals work.

The big question is why SGI decided to fold so much functionality into a single system call. There must be a good reason for doing so, but it is not easy to guess. The only thing that is obvious when you are doing some reverse engineering on syssgi(2) is that for many requests, you do not know what arguments are used when calling syssgi(2). The only information available is the name of the request, and it makes things much more difficult.

syssgi(2) emulation in NetBSD is done in sys/compat/irix/irix_syssgi.c:irix_sys_syssgi(). It is just a giant switch on the request value, which will branch to various kernel functions implementing the request. All standard features, such as getpid(2), are quite easy to implement. Others are more tricky.

The first difficulty with syssgi(2) is the ELFMAP request. The dynamic linker invokes syssgi(2) with this request, and all we can find in the syssgi(2) man page is that this is an interface to implement a system library function, and that this interface is subject to change. Not very helpful.

Fortunately, Linux already tried to get there, and the person that worked on it managed to discover that ELFMAP takes a file descriptor, an ELF program header array, and the array length, and then maps the ELF sections described in the array in the calling process' user space. Information on this can be found inside Linux kernel sources, in linux/arch/mips/kernel/sysirix.c:irix_syssgi() and linux/arch/mips/kernel/irixelf.c:irix_mapelf().

In fact, syssgi(ELFMAP) is a kernel implementation of a part of the dynamic linker. Native binaries on a NetBSD system map each code section doing a mmap(2). Here again, one could wonder what the reasons are for pushing that code from userland to the kernel. One reason could be to improve performance by saving system calls: an IRIX binary can map a library with only one system call.

Reverse Engineering syssgi(ELFMAP)

Another way of guessing what the syssgi(ELFMAP) function does is to use the par(1) command in IRIX. This command is similar to ktrace(1) on NetBSD: it reports the system call activity of a user program. Fortunately, syssgi(ELFMAP) gets disassembled into a more system-call-looking presentation:

31mS          : open("/lib/rld", O_RDONLY, 04) = 3
31mS          : read(3, <7f 45 4c 46 01 02 01 00 00 00>..., 512) = 512
32mS          : elfmap(3, 0x7fff2d98, 2) = 0xfb60000

Here is the ktrace(1)/kdump(1) output on NetBSD for this:

1343 ftp      CALL  open(0xfb3509c,0,0x4)
1343 ftp      NAMI  "/emul/irix/lib/rld"
1343 ftp      NAMI  "/emul/irix"
1343 ftp      NAMI  "/emul/irix/lib/rld"
1343 ftp      RET   open 3
1343 ftp      CALL  read(0x3,0x7fffe76c,0x200)
1343 ftp      RET   read 512/0x200
1343 ftp      CALL  syssgi(0x44,0x3,0x7fffe7e0,0x2,0,0xfb3509c)

NB : 0x44 is the request code for ELFMAP. This is defined in IRIX's <sys/syssgi.h>.

Here we get the confirmation that:

  • We should never really trust par(1) about what is going on, because it does masquerade on some system calls.
  • syssgi(ELFMAP) really expects three arguments.
  • The first one is very likely to be the file descriptor just acquired from open(2).
  • It is much more difficult to guess what the second and third arguments are.
  • We need to discover the returned value.

Pages: 1, 2, 3

Next Pagearrow

Sponsored by: