What can you tell us about the ports system and the ports collection?
Mark Linimon: 7.0 will ship with nearly 18,000 different ports. (The version of the ports tree that will ship with 7.0 is frozen except for security updates; the current version of the tree already has more than 18,000).
Binary packages are being built for the following build environments: amd64-5, amd64-6, amd64-7, amd64-8, i386-5, i386-6, i386-7, i386-8, sparc64-6, and sparc64-7. (We are limited in how many sparc packages we can build by the amount of hardware we have available). By "-8" we mean "freebsd-current" here (e.g., what at some point will become 8.0). Within a few months, however, FreeBSD 5.X will no longer be supported by the security team, and package builds will stop at that time. By that time we will be recommending that everyone not using FreeBSD in some kind of embedded application should have already moved to 6.2, 6.3, or 7.0.
Ongoing work is being done to monitor the quality of the package builds and provide feedback to maintainers and committers to try to get problems resolved as quickly as possible.
This release includes gcc 4.2. Why did you choose to upgrade? What changes should users expect to see?
Mark Linimon: The 7.X branch of FreeBSD will have a support lifetime of several years. Towards the end of that, the gcc developers will have almost assuredly dropped support for the older versions. This seemed the best version for us to be using during that timespan.
gcc 4.2 is more strict about the code that it accepts. Because of this, we have had to modify a number of ports, and send the patches upstream. In a few cases (e.g., where the port is not longer being actively developed) we chose to instate a dependency on an older gcc version also in ports. We prefer to avoid this whenever possible.
Which X Window systems do you support? And which Window Manager (Gnome? KDE 3/4)? What about 3D desktops and 3D acceleration?
Mark Linimon: At the moment we have XFree86 and xorg; however, due to lack of interest, we intend to drop XFree86 once 7.0 is released. There are a variety of window managers, the most well-known being Gnome and KDE. At the moment we only have KDE 3 in the ports collection; active work is being done by the KDE team to do all the necessary upgrades for KDE 4.
There are a few popular applications that people could be interested in using, for example Skype and Flash. Is this possible at the moment?
Alexander Leidinger: Both depend upon the linuxulator. Currently the default linux kernel emulation is 2.4 based. In 7.0 there are a lot of improvements to the linuxulator that we are able to emulate parts of a linux 2.6 kernel, but there are known bugs and some missing features, so it is not enabled by default. There may even be a bug which results in some programs (some games and maybe some other programs) not being able to run in the default 2.4 emulation mode, you need to check the errata page of 7.0 when it is out if the bug in the linuxulator is there (when we were to late to get it into 7.0 in time for the release) or not. That being said: there are a lot of people using Skype even with the bug still present in the kernel. Personally I use the 2.6 emulation with acroread just fine.
Flash is a different kind of a beast. It's possible to use Flash 7 (install nspluginwrapper and follow the instructions in the message which is displayed after the installation, works fine for me). For Flash 9 we need the 2.6 emulation, but unlike acroread, Flash 9 seems to be demanding some 2.6 features which are not stable enough yet (apart from that, Flash 9 itself also doesn't seem really stable in linux, so the problems accumulate). Bottom line: Flash 7 works, but is not used that much on high profile websites anymore, Flash 8/9 is used on more and more high profile websites, but doesn't work stable yet.
Some readers might not know that FreeBSD can run Linux apps using a Linux ABI compatibility layer, called Linuxulator. What is the situation in 7.0?
Alexander Leidinger: The default is 2.4.2 emulation. The target is 2.6.16 emulation. There are some known problems with 2.6.16, so it is not the default yet. A lot of compatibility problems are fixed (bugfixes and new stuff), even in 2.4.2 emulation. Several of the bugfixes will also be in 6.3, but not the 2.6.16 parts.
We didn't do performance tests, so I don't know about performance improvements specific to the linuxulator, but the performance improvements for FreeBSD itself surely will improve the corresponding linuxulator parts.
There is a wiki with development info, but this is mainly for the 2.6.16 part. Some bugs listed there are also in 2.4.2, but they are there more or less since 3.x. The big subpage with the colored OK/failed test results doesn't show the severity of the failed tests, so just because there's a red marker, it doesn't mean it is a big problem (or a problem at all) in FreeBSD. A lot of tests...
What limits does FreeBSD 7.0 has when dealing with storage?
Pawel Jakub Dawidek: FreeBSD 7.0 is really good at working with large file systems. UFS2 is 64 bit file system, so should be enough for anyone. The only problem is fsck, which can take many hours to complete for really large UFS file systems. Here of course comes gjournal. FreeBSD 7.0 also has support for Sun's revolutionary file system called ZFS, which makes FreeBSD a great choice as a file server. I can talk how cool ZFS is for hours, so I'll just stop here:) I'm not sure about FAT32 file system, I use it rarely and only for small file systems. I also suggest not to go back to UFS1.
Craig Rodrigues: I am not an MSDOS-FS expert, but I have committed some fixes in this area. In FreeBSD 7, it should be possible to mount a 500 GB disk by doing "mount -t msdosfs -o large" when mounting the disk.
Is this the first FreeBSD release that includes a journaling facility (gjournal)?
Pawel Jakub Dawidek: Yes, FreeBSD 7.0 will be the first FreeBSD release with gjournal support. I was hoping to include gjournal in FreeBSD 6.3, but unfortunately I run out of time.
gjournal is not a separate filesystem, and it actually works below the file system layer, so I am wondering what performance does it provide and in which context should it be used?
Pawel Jakub Dawidek: You are right, gjournal offers block level journaling and is file system independent. You can use gjournal without any file system on top of it and with a really small amount of work you can use it with any file system FreeBSD has. Currently only UFS support is implemented. The gjournal is just another GEOM class, which allows to write data in transactions. In case of UFS, we start transaction, modify file system, and close the transactions by synchronizing the file system. This happens every few seconds (5 seconds by default) gjournal closes the transaction and starts to copy changes in the background to the destination provider from the journal. In the meantime new transaction is in progress. This allows to recover really fast from a power failure or a system crash.
Because gjournal operates below file system layer it cannot recognize if the given write request consist data or metadata, so it just journals everything. This of course introduces impact on performance, so I did some optimizations to mitigate it. gjournal does some work to optimize written data, for example it tries to combine smaller requests into larger ones to minimalize number of I/O requests send to disk and also it sorts the requests to avoid heads seeking as much as possible.
All this makes gjournal performance really interesting. Single streams of writes work twice as slow as UFS without gjournal, because there is not much to optimize. On the other hand, many processes running in parallel and operating on small writes can work even twice as fast as UFS without gjournal (my test was to untar FreeBSD source tree in eight processes in parallel).
unionfs has been fixed. What was the problem?
Daichi GOTO: There were several known problems in unionfs implementation of FreeBSD up until 6.2-RELEASE. The specification is ambiguous and its locking implementation was buggy. Because of these issues, mounting unionfs cd9660 file system as a lower layer had caused problems.
Unionfs makes it possible to mount one file system on top of another. For example, you can mount a memory file system on top of a CD-ROM. As a result, it looks as if you could write to files on the CD-ROM.
Changes are only made to the upper file system layer and no changes are made to the lower one. Therefore, you can use it to keep modifications without changing the lower layers. For a more detailed explanation have a look at Section 6.7 on page 256 of "The Design and Implementation of the FreeBSD Operating System" by Marshall Kirk McKusick and George V. Neville-Neil.
We made a new unionfs for FreeBSD. The most valuable codes of our new implementation has already been merged up until FreeBSD 8/FreeBSD 7.0/FreeBSD 6.3.
Solving "Ambiguous Specification Problems" involves discussions about what the appropriate behaviour is. Because the specification of unionfs has ambiguity of its behavior, so it is difficult to implement appropriately. Therefore, I have proposed different options for different situations. New implementation includes an option that allows unionfs to change its behavior on three ways: [traditional mode], [transparent mode] and [masquerade mode]. [transparent mode] seems to be the most reasonable default behavior. It fixes most of the problems in the original implementation.
Why did we rewrite from scratch? The original (old) unionfs implementation of FreeBSD until 6.2-RELEASE had many dead-lock scenes easily. Fixing it was harder than rewriting it ;)
What's new in FreeBSD 7.0 from a security standpoint?
Robert Watson: While security auditing was available as an experimental feature in FreeBSD 6.2, it is significantly enhanced in FreeBSD 7.0. The most important change is that it is now available out-of-the-box without a kernel recompile. Administrators can turn it on with a simple rc.conf entry and start the audit daemon (or reboot). There are also a number of other improvements, such as XML printing mode, which allow praudit(8) to generate an XML version of a trail and improved support for auditing Linux-emulated processes, which make Audit a more accessible and usable service. Some of these improvements, including XML printing, will also appear in FreeBSD 6.3.
The priv(9) work is quite exciting, but for most users won't make an immediate difference in system behavior in 7.0. This work classified all kernel privilege checks into a set of specific privileges (around 200 of them), and introduced new kernel interfaces to check for them. While the base system doesn't yet make use of this, third party TrustedBSD MAC Framework security modules, such as SEBSD and mac_privs, can now modify the operating system privilege policy, granting extra privileges or restricting them. This work is also the foundation for a great deal of future work, such as the ability to grant specific privileges to specific users, or limit or expand the set of privileges available in a Jail. I hope to see features like this begin to appear in 7.1, and really take flight in the 8.x release series.
How did you change the accounting file format?
Diomidis Spinellis: The accounting facility of FreeBSD stores a record for each process that terminates. This record includes the name of the command, its user, system, and elapsed time, as well as the user and group id under which it was executed. I revised the accounting record format to store time values with microsecond precision. Historically, the time values were stored in a bespoke 13 bit fraction floating point format. The smallest time that could be stored in that format was fifteen milliseconds. With modern GHz processors the vast majority of processes execute in less than a millisecond, and therefore their accounted time values were recorded as zero.
For the new file format I adopted the IEEE 754 "float" format for storing time and usage values. For performance reasons, we don't use any floating point arithmetic in the kernel. Therefore, I wrote bit twiddling code in C that compresses the time values stored in the kernel structures into floating point numbers. Adopting the IEEE floating point format greatly increases the range and precision of the numbers, and also simplifies the processing of accounting records by third party tools. In the past, processing the accounting records meant decoding those strange 13-bit floating point numbers. Now you can just read the data into a plain C floating point variable and work with that.
Despite the many changes, the new record format and the tools for examining the last commands and for summarizing the accounting data (lastcomm and sa) maintain backwards compatibility with the original accounting format. The new records are also versioned, which means that future improvements can be gracefully integrated.