Meanwhile, in the offices of the planners for the Defense Advanced Research Projects Agency (DARPA), discussions were being held that would have a major influence on the work at Berkeley. One of DARPA's early successes had been to set up a nationwide computer network to link together all their major research centers. At that time, they were finding that many of the computers at these centers were reaching the end of their useful lifetime and had to be replaced. The heaviest cost of replacement was the porting of the research software to the new machines. In addition, many sites were unable to share their software because of the diversity of hardware and operating systems.
Choosing a single hardware vendor was impractical because of the widely varying computing needs of the research groups and the undesirability of depending on a single manufacturer. Thus, the planners at DARPA decided that the best solution was to unify at the operating systems level. After much discussion, Unix was chosen as a standard because of its proven portability.
In the fall of 1979, Bob Fabry responded to DARPA's interest in moving towards Unix by writing a proposal suggesting that Berkeley develop an enhanced version of 3BSD for the use of the DARPA community. Fabry took a copy of his proposal to a meeting of DARPA image processing and VLSI contractors, plus representatives from Bolt, Beranek, and Newman, the developers of the ARPAnet. There was some reservation whether Berkeley could produce a working system; however, the release of 3BSD in December 1979 assuaged most of the doubts.
With the increasingly good reputation of the 3BSD release to validate his claims, Bob Fabry was able to get an 18-month contract with DARPA beginning in April 1980. This contract was to add features needed by the DARPA contractors. Under the auspices of this contract, Bob Fabry sets up an organization which was christened the Computer Systems Research Group, or CSRG for short. He immediately hired Laura Tong to handle the project administration. Fabry turned his attention to finding a project leader to manage the software development. Fabry assumed that since Joy had just passed his Ph.D. qualifying examination, he would rather concentrate on completing his degree than take the software development position. But Joy had other plans. One night in early March he phoned Fabry at home to express interest in taking charge of the further development of Unix. Though surprised by the offer, Fabry took little time to agree.
The project started promptly. Tong set up a distribution system that could handle a higher volume of orders than Joy's previous distributions. Fabry managed to coordinate with Bob Guffy at AT&T and lawyers at the University of California to formally release Unix under terms agreeable to all. Joy incorporated Jim Kulp's job control, and added auto reboot, a 1K block file system, and support for the latest VAX machine, the VAX-11/750. By October 1980, a polished distribution that also included the Pascal compiler, the Franz Lisp system, and an enhanced mail handling system was released as 4BSD. During its nine-month lifetime, nearly 150 copies were shipped. The license arrangement was on a per-institution basis rather than a per machine basis; thus the distribution ran on about 500 machines.
With the increasingly wide distribution and visibility of Berkeley Unix, several critics began to emerge. David Kashtan at Stanford Research Institute wrote a paper describing the results of benchmarks he had run on both VMS and Berkeley Unix. These benchmarks showed severe performance problems with the Unix system for the VAX. Setting his future plans aside for several months, Joy systematically began tuning up the kernel. Within weeks he had a rebuttal paper written showing that Kashtan's benchmarks could be made to run as well on Unix as they could on VMS.
Rather than continue shipping 4BSD, the tuned-up system, with the addition of Robert Elz's auto configuration code, was released as 4.1BSD in June, 1981. Over its two- year lifetime about 400 distributions were shipped. The original intent had been to call it the 5BSD release; however, there were objections from AT&T that there would be customer confusion between their commercial Unix release, System V, and a Berkeley release named 5BSD. So, to resolve the issue, Berkeley agreed to change the naming scheme for future releases to stay at 4BSD and just increment the minor number.
With the release of 4.1BSD, much of the furor over performance died down. DARPA was sufficiently satisfied with the results of the first contract that a new two-year contract was granted to Berkeley with funding almost five times that of the original. Half of the money went to the Unix project, the rest to other researchers in the Computer Science department. The contract called for major work to be done on the system so the DARPA research community could better do their work.
Based on the needs of the DARPA community, goals were set and work begun to define the modifications to the system. In particular, the new system was expected to include a faster file system that would raise throughput to the speed of available disk technology, support processes with multi-gigabyte address space requirements, provide flexible interprocess communication facilities that allow researchers to do work in distributed systems, and would integrate networking support so that machines running the new system could easily participate in the ARPAnet.
To assist in defining the new system, Duane Adams, Berkeley's contract monitor at DARPA, formed a group known as the "steering committee" to help guide the design work and ensure that the research community's needs were addressed. This committee met twice a year between April 1981 and June 1983. It included Bob Fabry, Bill Joy, and Sam Leffler of the University of California at Berkeley; Alan Nemeth and Rob Gurwitz of Bolt, Beranek, and Newman; Dennis Ritchie of Bell Laboratories; Keith Lantz of Stanford University; Rick Rashid of Carnegie-Mellon University; Bert Halstead of the Massachusetts Institute of Technology; Dan Lynch of The Information Sciences Institute; Duane Adams and Bob Baker of DARPA; and Jerry Popek of the University of California at Los Angeles. Beginning in 1984, these meetings were supplanted by workshops that were expanded to include many more people.
An initial document proposing facilities to be included in the new system was circulated to the steering committee and other people outside Berkeley in July, 1981, sparking many lengthy debates. In the summer of 1981, I became involved with the CSRG and took on the implementation of the new file system. During the summer, Joy concentrated on implementing a prototype version of the interprocess communication facilities. In the fall of 1981, Sam Leffler joined the CSRG as a full-time staff member to work with Bill Joy.
When Rob Gurwitz released an early implementation of the TCP/IP protocols to Berkeley, Joy integrated it into the system and tuned its performance. During this work, it became clear to Joy and Leffler that the new system would need to provide support for more than just the DARPA standard network protocols. Thus, they redesigned the internal structuring of the software, refining the interfaces so that multiple network protocols could be used simultaneously.
With the internal restructuring completed and the TCP/IP
protocols integrated with the prototype IPC facilities, several simple
applications were created to provide local users access to remote resources.
rwho were intended to be temporary tools that would
eventually be replaced by more reasonable facilities (hence the use of the
distinguishing "r" prefix). This system, called
4.1a, was first distributed in April 1982 for local use; it was never intended
that it would have wide circulation, though bootleg copies of the system
proliferated as sites grew impatient waiting for the 4.2 release.
The 4.1a system was obsolete long before it was complete. However, feedback from its users provided valuable information that was used to create a revised proposal for the new system called the "4.2BSD System Manual." This document was circulated in February 1982 and contained a concise description of the proposed user interfaces to the system facilities that were to be implemented in 4.2BSD.
Concurrent with the 4.1a development, I completed the implementation of the new file system, and by June 1982, had fully integrated it into the 4.1a kernel. The resulting system was called 4.1b and ran on only a few select development machines at Berkeley. Joy felt that with significant impending changes to the system, it was best to avoid even a local distribution, particularly since it required every machine's file systems to be dumped and restored to convert from 4.1a to 4.1b. Once the file system proved to be stable, Leffler proceeded to add the new file system related system calls, while Joy worked on revising the interprocess communication facilities.
In the late spring of 1982, Joy announced he was joining Sun Microsystems. Over the summer, he split his time between Sun and Berkeley, spending most of his time polishing his revisions to the interprocess communication facilities and reorganizing the Unix kernel sources to isolate machine dependencies. With Joy's departure, Leffler took over responsibility for completing the project. Certain deadlines had already been established and the release had been promised to the DARPA community for the spring of 1983. Given the time constraints, the work remaining to complete the release was evaluated and priorities were set. In particular, the virtual memory enhancements and the most sophisticated parts of the interprocess communication design were relegated to low priority (and later shelved completely). Also, with the implementation more than a year old and the Unix community's expectations heightened, it was decided an intermediate release should be put together to hold people until the final system could be completed. This system, called 4.1c, was distributed in April 1983; many vendors used this release to prepare for ports of 4.2 to their hardware. Pauline Schwartz was hired to take over the distribution duties starting with the 4.1c release.
In June 1983, Bob Fabry turned over administrative control of the CSRG to Professors Domenico Ferrari and Susan Graham to begin a sabbatical free from the frantic pace of the previous four years. Leffler continued the completion of the system, implementing the new signal facilities, adding to the networking support, redoing the standalone I/O system to simplify the installation process, integrating the disc quota facilities from Robert Elz, updating all the documentation, and tracking the bugs from the 4.1c release. In August 1983, the system was released as 4.2BSD.
When Leffler left Berkeley for Lucasfilm following the completion of 4.2, he was replaced by Mike Karels. Karels' previous experience with the 2.9BSD PDP-11 software distribution provided an ideal background for his new job. After completing my Ph.D. in December 1984, I joined Mike Karels full-time at the CSRG.
The popularity of 4.2BSD was impressive; within eighteen months more than 1,000 site licenses had been issued. Thus, more copies of 4.2BSD had been shipped than of all the previous Berkeley software distributions combined. Most of the Unix vendors shipped a 4.2BSD system rather than the commercial System V from AT&T. The reason was that System V had neither networking nor the Berkley Fast filesystem. The BSD release of Unix only held its dominant commercial position for a few years before returning to its roots. As networking and other 4.2BSD improvements were integrated into the system V release, the vendors usually switched back to it. However, later BSD developments continued to be incorporated into System V.