diff --git a/en/news/status/report-2007-10-2007-12.xml b/en/news/status/report-2007-10-2007-12.xml
index 12924c93bc..687ee2449f 100644
--- a/en/news/status/report-2007-10-2007-12.xml
+++ b/en/news/status/report-2007-10-2007-12.xml
@@ -1,1561 +1,1561 @@
-
+
This report covers FreeBSD related projects between October and
December 2007.
AsiaBSDCon 2008
is approaching and will be held at the Tokyo University of Science in
Tokyo, Japan on the 27th - 30th of March 2008. The FreeBSD Foundation
has released a
Newsletter
detailing their activities over the past few months. FreeBSD 7.0 is nearing release and the 2nd Release Canidate is
+ FreeBSD 7.0 is nearing release and the 2nd Release Candidate is
ready for testing and is available for
download now. Thanks to all the reporters for the excellent work! We hope you
enjoy reading. As a result of a posting on freebsd-current@ complaining about a
communication gap between users and developers, there has been a
great deal of new interest in working on bugbusting -- in particular,
we brainstormed on ideas on how non-committers can help. The two main
ideas that are being discussed are incoming bug triage (classifying,
rating, and so forth), and working with users (helping users to work
through problems that aren't classical Problem Reports. As a result of this, we held our first Bugathon in quite some time
(on #freebsd-bugbusters on EFNet). Over 30 people participated. As a
result of this, over 120 PRs were closed, and dozens more were put
into the 'feedback' state. Most of these PRs were in the kern/ and
bin/ categories, which are the two that need the most work. (The new
arrival rate was over 40/day during this time, including ports, so
there was a significant net decrease.) Several new wiki pages were created to support this effort, and
finally capture a lot of the previous discussions from both the
mailing list and the IRC channel. There are even more good ideas
which Mark Linimon has promised to work up and investigate,
including: Note: at this time we are not yet looking to replace GNATS. The
idea right now is to see what we can learn about how our workflow
does (and ought to) work, and experiment with some low-cost changes
to get various people's reactions. Linimon's feeling is that any of
these kinds of changes would carry over to a new system, if we were
to change over. rwatson also created a wiki page to put down some thoughts about
how to work on the various kernel problems that are reported.
Although preliminary, this captures some expertise and puts it into a
place where prospective volunteers can more easily find it. The overall PR count is back up to just under 5300. Although this
is net increase from the previous report, there were long periods of
src and ports freeze during this time, which creates a spike in the
overall count. (src and ports both remain in slush during that time).
The peak number was approaching 5500. Overall, we seem to have some momentum and new volunteers
interested in working on user-reported problems. bugmeister is
hopeful that we can capitalize on this and make some good progress in
the rest of 2008. A large number of bugs have been fixed in the FreeBSD "coda"
kernel module over the past six months, and a man page has been
added to describe the module. Many of these bugs were the result of
the coda module failing to keep up with the many enhancements to
FreeBSD VFS over the last few years. As a result of these fixes, it
is now possible to use Coda with FreeBSD 7.x and 8.x without
immediate panics, and possibly for an extended period. The new man
page does clarify that Coda is an experimental distributed file
system and not yet appropriate for production use on FreeBSD, but
things are looking a lot better than they were. Since the last status report we have made a nice progress about
the website translation. The structure of the translated sites is
polished and we have brought a significant set of pages up-to-date.
New pages with important content have also been translated. Apart
from the good progress, there is a still a lot to do. Some pages
are still seriously outdated and some important parts are
missing. At the same time, we have added one new article translation and
one is still awaiting review before being committed. We have added the translation of the FreeBSD Flyer and
maintained the existing translations. A huge progress is being made
to provide a Hungarian translation of the FreeBSD Handbook. Also,
there is an ongoing effort to provide Hungarian release notes for
the upcoming FreeBSD releases. Thanks to support from Cisco Systems, Inc, the port of the
DTrace dynamic tracing framework from OpenSolaris to FreeBSD is
active again. A solution to the integration issues surrounding the
CDDL and BSD licenses has been found. There is an entirely BSD
licensed set of hooks/shims which are optionally compiled into the
kernel. This option can be included in the GENERIC kernel and
shipped without any CDDL patent encumberance. The CTF (Compact C
Type Format) tools now work across all architectures enabled in a
'make universe'. A BSD licensed DWARF library has been developed.
The kernel DTrace support is limited to amd64 and i386 at the
moment. It currently passes 822 of the tests in the DTrace Test
Suite. It is expected that the initial commit to FreeBSD-CURRENT
will occur within the next month after review. Refer to the change
summary page for details of the proposed changes. The FreeBSD Installer project (FIN) is yet another attempt to
replace the aging sysinstall(8). I am attempting to keep the best
parts of sysinstall(8) and combine them with the framework provided
by the BSDInstaller (bsdinstaller.org) to create an installation
program for FreeBSD that is multi-lingual, supports multiple
installation media, supports remote installation, and is easily
extensible to other installation types (gui, cgi, etc). The current
implementation will slice disks, install your choice of base
distributions, and set hostname and root password. glvm is a geom class which reads the metadata from a LVM2 (Linux
volume manager) disk and creates a geom provider for each logical
volume. An example is the logs lv on a volume group called vg0
appearing as /dev/lvm/vg0-logs, this can be mounted as a disk. The code is working and will be posted for testing soon. malloc(3) has been enhanced in several ways to reduce lock
contention when multi-threaded programs concurrently use the
malloc(3) functions. The primary enhancements are lazy deallocation
and dynamic arena load balancing. Lazy deallocation is designed to reduce contention for programs
that use the producer-consumer model, where a thread produces
(allocates) objects, and a pool of worker threads consumes
(deallocates) those objects. As a side benefit, lazy deallocation
also substantially reduces lock contention if multiple unrelated
threads are using the same arena. Allocation activity patterns can change throughout the lifetime
of a program. Dynamic arena load balancing monitors arena lock
- contention and re-assigns threads to other arenas as neccesary,
+ contention and re-assigns threads to other arenas as necessary,
thus smoothing out allocator performance. In order to monitor lock contention in support of arena load
balancing, I had to switch to using pthreads mutexes. This all by
itself smoothed out allocator performance under high load, since
the internal libc "spinlocks" aren't really spinlocks, whereas
malloc now spins for a bit before blocking. I plan to MFC these changes to RELENG_7, hopefully in time for
the FreeBSD 7.1 release. FreeBSD/mips boots to multiuser using gxemul on the MALTA board
with a 4Kc based CPU. The port is targeting MIPS32 and MIPS64
release 1 and release 2 based systems. Work is underway to support
multicore systems. Preliminary ports to adm 5120, the IDT RC32434, the Sentry 5,
- and a few other targets have started. These ports are in vairous
+ and a few other targets have started. These ports are in various
stages of stability. Juniper Networks has donated a generic MIPS FreeBSD port. This
port doesn't run on any real hardware, but contains the necessary
parts to run on idealized MIPS hardware. The FreeBSD/mips workers
have been merging the current base and the Juniper code into a
unified base. In addition, Cavium Networks has donated code
supporting their multicore mips64r2 platform. This code is also
being merged into the tree and cleaned up as well. The merged code
base presently is making it to the first (or maybe second) call to
cpu_switch before dying. Active work is underway in this area. There are several websites already with overview of the FreeBSD
FTP mirrror sites, but they all seem to have one problem: They are
not manually updated with the list of sites. For example,
http://mirrorlist.freebsd.org/FBSDsites.php, despite being hosted
by an Australia, doesn't have the Australian mirrors on it, while
http://people.freebsd.org/~kuriyama/mirrors/ doesn't tell you which
files are available from there. The data on my page shows the
availability of the ISO images on all FTP mirror sites. The list of
FTP mirror sites is obtained from DNS by either doing a
zone-transfer or by just trying the standard names. The first data
block shows a quick overview of the availability of the ISO image
directories per server, architecture and mirror site. The second
data block shows a verbose availability of the contents of the ISO
image directories per server. The multi-IPv4/v6 jails project was resumed in early January
after previous work had been abandoned in 2006. As an alternate solution to full network stack virtualization,
this work shall provide a lightweight solution for multi-IP
virtualization. The changes are even more important because of the
emerging demand for IPv6. The current status includes updated user space utilities. Kernel
side has grown support for multiple IP addresses for both address
families in jails, while the old kernel internal lookup/checking
- functions were kept and can be compiled in during the transistion
+ functions were kept and can be compiled in during the transition
period limiting jails to one IP address. Additionally a show jails
DDB command was added to ease debugging. As an auxiliary project the last suser(9) checks were replaced
in netinet6/ to support optional raw IPv6 sockets with jails. The
new priv(9) checks were committed to HEAD. The if_nxge driver, contributed by Neterion, has been merged
into FreeBSD 8-CURRENT and FreeBSD 7-STABLE, and will appear in
FreeBSD 7.0. The nxge driver provides support for Neterion Xframe-I and
Xframe-II adapters. The driver supports TCP Segmentation Offload
- (TSO/LSO), Large Receive Offlaod (LRO), Jumbo Frames (5 buffer
+ (TSO/LSO), Large Receive Offload (LRO), Jumbo Frames (5 buffer
mode), Header Separation (Rx 2 buffer mode), VLAN, and Promiscuous
mode. For general information and support, please visit the Neterion
support page http://www.neterion.com/support/support.html. The nxge driver supports Neterion Xframe 10 Gigabit Ethernet
adapters listed in http://www.neterion.com/how/pricing.html. As part of my thesis, I've been working on a framework to
monitor the performance of CURRENT over time. The project is now in
a state where a server and a slave are producing benchmark results
and publishing the results to a web page for testing. Already, the
setup has detected regressions. Lots of improvements can be made,
but it is already quite useful. Over the next month I'll be adding
a few features, fixing bugs and writing documentation. Completed initial requirements gathering. Selection of
development tools complete. General internal design complete. Ports 2.0 goals are: The ports count continues to accelerate and is now over 18,000.
The PR count, which had dipped to around 750 before the 6.3/7.0
freeze, is now back up to about 1000, due to the fact that we
remain in ports slush. Because of the freeze/slush, no experimental ports runs have
been committed since the last report. Although 2 more -exp runs
have been completed, we are waiting for 7.0R to commit them. Once 7.0R happens, a lot of chaos is going to happen in the
Ports Collection. This has built up during the long release cycle.
Get ready for the following changes, among others: Most of the portmgr activity was related to the QA process for
the releases. In addition, linimon spent quite some time to try to
get the sparc64 ports into better shape, and sent out a request for
more people to help test sparc64 ports. Some people have responded
- with offers for let committers get account on their machines.
Unfortunately during this time period, we became unable to build packages for ia64-7. As a result, we are not currently building packages for ia64 any more. If any one wants to step up to work on this architecture, let portmgr know.
We are currently building packages for amd64-5, amd64-6, amd64-7, amd64-8, i386-5, i386-6, i386-7, i386-8, sparc64-6, and sparc64-7. Note, however, that RELENG_5 will reach end of its supported life, and package builds for those 2 buildenvs will stop as of that date. (8 buildenvs * 18,000 ports should be enough to keep us busy.)
Other than that, the packages are in the best shape that they have been in for some time. linimon continues to work on package analysis tools for portsmon.
We have added 2 new committers since the last report.
A new command line tool, procstat(1), allows detailed inspection and printing of process properties, including file descriptors, threads, kernel thread stacks, credentials, and virtual memory mappings of processes. Several new sysctls have been added to the kernel in order to export this information cleanly, and the stack(9) facility has been enhanced to allow the capture of kernel stacks from threads other than curthread. None of these features depends on procfs, continuing the effort to remove a requirement for procfs in order to print process information, as well as adding new types of information not available with procfs. Kernel stack printing is particularly useful as it provides much more detailed information on why a thread is blocked in kernel beyond the useful but limited wmesg context provided to date. This is helpful in debugging both user process problems and kernel problems. procstat has been merged into FreeBSD 8-CURRENT, and will be merged to 7-STABLE after FreeBSD 7.0 is released.
The Greek doc translation team has grown significantly since we started the translations. Most of the Handbook has already been translated to Greek (and committed to the CVS tree), as a collaborative effort of Manolis Kiagias, Nikos Kokkalis, Panagiotis Kritikakos, Vaggelis Typaldos, Stylianos Sideridis and others. Manolis has started translating the FAQ too, and we also tagged most of the Greek documents with their original, English revision ID. There are also plans for a translation of www/en , but these may have to be deferred until we find the time to complete the Handbook, which is our primary target right now.
There are a variety of on-going projects relating to improving SMP scalability of the FreeBSD network stack post-7.0. These include:
Detailed profiling of application workloads such as BIND9, MySQL, PgSQL and Apache have been used to identify performance bottlenecks and to guide changes to the source code.
rwlock(9) use for pcbinfo and inpcb locking , allowing the acquisition of only read locks for pcbinfo and inpcb during UDP receive and transmit--this is highly desirable in order to improve BIND9 performance, which sends and receives from many threads at a time on a single UDP socket.
Breaking out pcbinfo into a series of parallel data structures , where the particular pcbinfo instance is selected using a hash of the connection tuple (and where ambiguous cases are present in all instances). This would allow greatly reducing pcbinfo contention for parallel input cases, which are increasingly likely with multiple input queue network devices, such as the Chelsio cxgb 10gbps driver.
Investigation of use opportunities for rmlock(9) -- rmlocks provide very lightweight acquisition for read, but expensive acquisition for write, and may be an appropriate replacement for rwlocks where significantly more reads than writes take place -- such as for firewall rule list protection, pf hook registration, address lists, etc.
Weak connection affinity , in which the effective affinity of a connection, determined by its hash/rss work assignment to a particular input queue by the network stack or network card, is tracked and exposed to user space so that work associated with that connection can be performed on or close to the CPU where the kernel will be processing input for the connection. Software work placement has been done using the netisr2 implementation, which creates per-CPU netisr threads and assigns work based on connection properties.
There are also many other pieces of related work going on, especially relating to 10gbps network drivers, and workloads of particular interest include BIND9, MySQL, pgsql, Apache, and general TCP parallelism.
In the time since the last status report, four security advisories have been issued concerning problems in the base system of FreeBSD; one of these problems was in "contributed" code maintained outside of FreeBSD. The FreeBSD Vulnerabilities and Exposures Markup Language (VuXML) document has continued to be updated; since the last status report, 61 new entries have been added, bringing the total up to 1023. Many of these new VuXML entries were made by members of the "ports-security" team.
The "ports-security" team is still looking for more committers who can periodically help with fixing ports security issues and documenting them in the FreeBSD VuXML document. Committers who wish to help with this effort can contact simon@ for details.
The following FreeBSD releases are supported by the FreeBSD Security Team: FreeBSD 5.5, FreeBSD 6.1, FreeBSD 6.2, and FreeBSD 6.3. The respective End of Life dates of supported releases are listed on the web site; it is expected that the upcoming FreeBSD 7.0 release will be supported for one year after its release.
Like every year for the past few years is held what in France is mostly called "Solutions Linux" in Paris La Défense. The exhibition will take place the 29, 30 and 31st of January in the CNIT.
The interesting thing about this event is that 80% of the floor is taken by companies (IBM, Novell, Oracle), and the remaining 20% is given freely to associations and non-profit organizations, where you'll find many (if not most) french LUGs, *BSDs, most Linux distributions, Mozilla, OOo...
This year, FreeBSD will once again have a booth, and we'll be showing what FreeBSD is, why it's the damn best OS out there. We'll also be distributing flyers and CD's for the whole three days
Admission to the exhibitions is free, so if you ever happen to pass by, come and see us, we'll be at booth A39.
Completed and tested. Awaiting review from other committers.
The FreeBSD TCP reassembly queue system has reached its limits with todays high speed links over long distances and large socket buffers. The old code is almost unchanged compared to 4.4BSD and gets quite inefficient with large mbuf chains.
The new code aggregates consecutive segments into blocks and inserts the blocks into a tail queue. The insertion points for a newly arrived segment are checked in order of their probability. This prevents full chain traversals and is very efficient.
To prevent easy resource exhaustion attacks the effective mbuf usage is accounted for and limited by the size of socket buffer. This way the reassembly queue can't be abused with many holes among small segments.
-A further addition is the combination received SACK block +
A further addition is the combination of received SACK block tracking with the reassembly queue. The reassembly queue now tracks all blocks of segments. This makes tracking it again for SACK unnecessary. Additionally the limitation to six SACK blocks is lifted and the size of the inpcb structure is reduced quite a bit.
The new code is stable and in testing correctly handles the download of a full set of FreeBSD CDROM images and 180 ports distfiles from widely distributed sites around the world at 2% packet loss.
The FreeBSD TCP code has evolved a lot over time and many new features were added. However over time it got crufty, complex and hard to read and track. In some places functionality was moved away - but the corresponding code in the main tcp functions was not or not + but the corresponding code in the main TCP functions was not or not fully removed.
The main purpose of of the TCP code cleanup and rewrite is to make the code:
Quite a bit of code is already (re)written but a lot still remains to be done.
The kernel DDB facility has been enhanced to add several new features:
DDB scripting - allows the user to define a set of simply scripts from within the + allows the user to define a set of simple scripts from within the debugger or userspace using the new ddb(8) tool to automate debugging steps. Scripts can be automatically executed when the debugger is entered ("kdb.enter.panic", "kdb.enter.break", ...) or manually using the DDB "run" command.
DDB output capture allows the user to request that the output of DDB be captured into a buffer for access from user space or to be written out in a textdump.
DDB textdumps, a new dump format that writes out a tarball of text-based debugging information, such as the kernel message buffer, panic message, kernel configuration, kernel version, and DDB capture buffer to the swap partition, to be extracted via savecore(8). This provides a compact, portable, and kernel compile independent debugging package.
Various interesting formulas for use are described in ddb(4) and textdump(4); the facilities are separable, so you can, for example, run a few DDB commands and capture their output, then write a regular dump and extract that output using kgdb, or you can do the same and write it out as a textdump. Likewise, scripts can be used to automate manual debugging, or implement textdumps by enabling output capture, running a series of commands, and forcing a textdump to be written before rebooting.
Support for these facilities has been merged into 8-CURRENT, and will be merged to 7-STABLE after the release of FreeBSD 7.0.
The TrustedBSD Project was proud to release OpenBSM 1.0, the first production release of OpenBSM, which is shipped with FreeBSD 6.3 and will ship with FreeBSD 7.0. This release represents largely polishing, bug fixing, and cleanup over the previous alpha release, but for FreeBSD 6.x introduced features such as XML audit trail printing, new token types, and new event identifiers.
A variety of development work continues on audit, including initial work on OpenBSM 1.1 alpha, work on improving the performance and semantics of audit pipes, and the experimental bsmtrace host intrusion detection package.
The patch to account the possibly required swap space and limit it by total amount of configured swap or per-uid limit is revived, ported to the 8-CURRENT. Now it is intensively tested by Peter Holm. Please, give it a run in the diverse workloads. Your comments are welcome!
The port will only run as a guest (ie. domU) right now, on +
The port will only run as a guest (i.e. domU) right now, on i386/PAE platforms. Status: