BSD DevCenter
oreilly.comSafari Books Online.Conferences.


IRIX Binary Compatibility, Part 3
Pages: 1, 2, 3

irix_syssgi_mapelf() Implementation

On NetBSD, syssgi(ELFMAP) is implemented through the irix_syssgi_mapelf() function. Let us now talk about what this function does.

We already have some support in the kernel for mapping ELF program sections: the kernel needs to load the ELF program sections of the executable and the interpreter.

The code to do this is split into two parts: the first part is the elf32_load_psection() function, from sys/kern/exec_elf32.c. This function takes a program header and builds a set of virtual memory (VM) commands that will load the code section. The VM commands are described by the struct exec_vmcmd, which is defined in sys/sys/exec.h. One struct, exec_vmcmd, contains a pointer to a function and holds its arguments. The functions that can be used are in sys/kern/exec_subr.c:

vmcmd_map_pagedvn(): maps a file area into user space.
vmcmd_map_readvn(): reads a file area into user space.
vmcmd_map_zero(): zeroes a user space area.

The function elf32_load_psection() builds VM commands that use vmcmd_map_pagedvn() when it has to load a section that fits within memory page range. For pages that are not completely filled, the data is copied instead of being mapped, and this is done using vmcmd_map_readvn(). vmcmd_map_zero() is then used to zero the end of the page.

The set of VM commands is returned by elf32_load_psection() in a struct exec_vmcmd_set (defined in /sys/sys/exec.h). Once we have the struct exec_vmcmd_set filled, we can use the second part of the ELF section load, which consists of running the VM commands found in the set.

Although this works well for loading just an executable and its interpreter, calling elf32_load_psection() and running the VM commands does not work very well for the syssgi(ELFMAP) implementation. The reason is that when the kernel loads an executable and its interpreter, it doesn't have to deal with the possibility that the virtual address range where a section was to be loaded is already mapped to another object. This is because the process address space is completely unused at that time.

When mapping several shared libraries, the likelihood that the load address of an object is already allocated to another object is is very high. In fact, it does happen for any X11-related o32 IRIX binary:

The load addresses of the program sections of libX11 overlap with the load addresses of libXaw:

$ objdump -p /usr/lib/
    LOAD off    0x00000000 vaddr 0x0f5b0000 paddr 0x0f5b0000 align 
         filesz 0x000f1000 memsz 0x000f1000 flags r-x
    LOAD off    0x000f4000 vaddr 0x0f6a4000 paddr 0x0f6a4000 align 
         filesz 0x0000a000 memsz 0x0000a000 flags rw-

$ objdump -p /usr/lib/
    LOAD off    0x00000000 vaddr 0x0f5a0000 paddr 0x0f5a0000 align 
         filesz 0x00041000 memsz 0x00041000 flags r-x
    LOAD off    0x00044000 vaddr 0x0f5f4000 paddr 0x0f5f4000 align 
         filesz 0x00005000 memsz 0x00005000 flags rw-

The first section of loads at 0x0f5a0000 and is 0x00041000 bytes long. Therefore, the section is loaded from 0x0f5a0000 to 0x0f5e1000, whereas the first section of wants to be loaded at 0x0f5b0000, which falls into that range.

Using par(1) on IRIX, it is possible to check what IRIX does to work around this: the value returned by syssgi(ELFMAP) is not the default load address of the first LOAD section, but another place. By building test programs linked with libX11 and libXaw, it is possible to check that libX11 is indeed loaded at the address returned by syssgi(ELFMAP). The library has been relocated in memory.

elf32_load_psection() contains no code to check if the address range requested by the program header is available. We then have to check this in our irix_syssgi_mapelf() function. This is done using the uvm_findspace(9) function. uvm_findspace(9) can be used in several ways. Given an area's virtual address, an area's length, and the UVM_FLAG_FIXED flag, it will tell if the area can be allocated at the given virtual address or not. Without the UVM_FLAG_FIXED flag, uvm_findspace(9) will find a virtual address where the area can be allocated.

uvm_findspace(9) is first used to check that there is enough free space to load each program section. If there is a problem with any of them, then we will have to relocate all the sections from this shared object.

Relocation is a difficult job. syssgi(ELFMAP) only returns the virtual address of the first section. If the sections are relocated, the only way for the calling program to find them is by using offsets from the first section. If the first section is moved by 0x4000 bytes, all of the other sections should be moved by 0x4000 bytes.

We want to keep the code in irix_syssgi_mapelf() simple, so that it has some chance to work correctly. We do this by making a few assumptions:

  • The section described by the program header array will never overlap.
  • The load address of a section in the program header array is always higher than the section described by the previous entry.

Since syssgi(ELFMAP) is used to map shared libraries, the first assumption is likely to be okay: no shared library will come with overlapping code sections. The second assumption seems okay, but one could build a bad binary with program headers reverse-ordered. At least this nasty kind of object does not seems to exist in a real IRIX system.

Once we have made the two assumptions, we compute the section union area. This is an area enclosing all of the code sections described in the program header array. Then we use uvm_findspace(9) without the UVM_FLAG_FIXED flag to find a place for this area. Once we have the address of a free place, we just have to add the offset to this new location to the load addresses of all entries in the program header array. The elf32_load_psection can do its job; the load addresses are not already used.

Here is a quick summary of irix_syssgi_mapelf() behavior:

  • Copy the program header array into kernel memory.
  • For each section, check that:
    • The section is loadable.
    • The section's load address is bigger than the previous section.
    • There is some free space to load the section at the default address.
      • If not, we will need a relocation.
  • If we need a relocation:
    • Compute the section union size.
    • Find some free place for the section union.
    • For each section:
      • Apply the relocation offset to the section entry in the table.
  • For each section:
    • Run elf32_load_psection().
    • For each VM command in the returned set:
      • Run the VM command.

One reading the Linux implementation of syssgi(ELFMAP) might wonder why the NetBSD version is that much more complicated. This is because the Linux version does not handle relocations, nor does it properly handle the loading of sections that are not aligned on a page boundary.

Other IRIX Oddities

There are a few other IRIX-specific system calls that are used in nearly every IRIX binary: sysmp() and prctl(). Both are meta-system calls like syssgi().

sysmp() is supposed to gather various multiprocessor-related functionality. The most commonly-used request is PGSIZE, which returns the memory page size. There are also requests to get well-known kernel structure offsets in /dev/kmem (KERNADDR), or the number of available processors (NPROCS). All of the requests are defined in IRIX's <sys/sysmp.h>.

prctl() implements functions related to multi-threading. The most-used request is LASTSHEXIT, which tells the kernel that the caller is the last thread of the process. Every IRIX process calls this before terminating. The emulation of this command is simple, for now: we just do nothing. All of the commands of prctl() are defined in IRIX's <sys/prctl.h>.

Finally, sginap() is another widely used, IRIX-specific system call. It is an equivalent of sleep(2) that returns the number of ticks elapsed. This was easy to emulate by checking the system clock before and after a sleep(9), and then returning the difference.

Emmanuel Dreyfus is a system and network administrator in Paris, France, and is currently a developer for NetBSD.

Return to the BSD DevCenter.

Sponsored by: