This bi-monthly report covers development activities on the FreeBSD Project for December 2001 and January 2002. A variety of accomplishments have been made over the last couple of months, including strong progress relating to the KSE project, which brings Scheduler Activations to the FreeBSD kernel, as well as less visible infrastructure projects such as improvements to the mount interface, PAM integration work, and translation efforts. Shortly following the deadline for this status report, the BSD Conference and FreeBSD Developer Summit were held, and will be covered in the next bi-monthly report at the end of March. Plans are already under way for the USENIX Annual Technical Conference in Monterey, CA, later this year, and all and sundry are encouraged to attend to get further insight in FreeBSD development.
Robert Watson
I've been working to integrate recent improvements in the NetBSD usb stack to FreeBSD -current. Both NetBSD and OpenBSD currently share the same source, as FreeBSD did too at once point before it diverged. The goal is to get back to that state, but there are many improvements on both sides that need to be merged before this is complete.
I'm currently looking for someone to help maintain usb in -stable. Please let me know if you're interested.
Patches for cp(1), ls(1), and mv(1) to bring in POSIX.1e-compliant Access Control List support have been updated to patch against builds of -CURRENT. Other system utilities are currently being evaluated for ACL support including install(1) (patch available) and mtree(8). Work is in progress to verify the native getfacl(1), setfacl(1), and other utilities build and work correctly on other ACL-enabled systems (e.g. Linux w/ACL patches) and to help verify POSIX-compliance of the continuing TrustedBSD work along with other systems. Finally, experimental Perl and PHP modules are available allowing limited access to native ACLs for languages other than C.
The project is making progress. The goal is to design and implement Host Controller Interface (HCI) and Link Layer Control and Adaptation Protocol (L2CAP) layers using Netgraph framework. More distant goal is to write support for Service Discovery Protocol (SDP) and RFCOMM protocol (Serial port emulation over Bluetooth link) . All information was obtained from Bluetooth Specification Book v1.1.
Project status: In progress. 1) Design: mostly complete, there are some minor issues to be resolved. 2) Implementation: Kernel - HCI and L2CAP Netgraph nodes have been implemented; 3) User space (API, library, utilities) - in progress. 4) Testing: In progress. I do not have real Bluetooth hardware at this point, so i wrote some tools that allow me to test the code. Some of them will be used as foundation for future user space utilities.
Issues: 1) Bluetooth hardware; I do not have real Bluetooth hardware, so if people can donate hardware/specs it would be great. I promise to write all required drivers and make them available. I also promise to return hardware/specs on first request. 2) Project name; I would like to see the name that reflects the following: it is a Bluetooth stack, implementation is for FreeBSD and implementation is based on Netgraph framework
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness. Basic MBR and BSD methods have been written and device attach/taste/dettach algorithms been implemented and validated.
I've update OS of buildboxes to the latest FreeBSD 5-current and 4-stable. Everything goes fine. From January 2002, I've started a webzine, SNAPSHOTS Notes (only Japanese version is available). SNAPSHOTs Notes pickups tips and information especially for the people living with FreeBSD 5-current/4-stable. Article or idea for SNAPSHOTs notes are always welcome (you don't need to write in Japanese :-).
Robert Watson created the TrustedBSD audit perforce tree, which is a branch from the TrustedBSD base tree, in order to start pushing development efforts towards using a revision control system. Andrew Reiter started to merge in some framework related code for generation of audit records, enqueueing writes, and handling data writing. There is a great deal of work to be done with updates and discussion on the trustedbsd-discuss@TrustedBSD.org mailing list.
The KSE project (an attempt to support scalable thread in FreeBSD using kernel support), has reached What I call "milestone 3". At this milestone it is possible to run a multithreaded program on a single CPU but with full concurrency of threads on that CPU. In other words the kernel supports the fact that one thread can block by allowing another thread to run in its place. A test program that demonstrates this is available at the above website.
Milestone 4 will be to allow threads from the same program to run on multiple CPUs but may require more input from the SMPng project. I am at the moment (Feb 6) getting ready to commit a first set of changes for milestone 3, that have no real effect but serve to drastically reduce the complexity of the remaining diff so that others can read it more easily. After changes to libkvm to support this diff have been added it should be possible to run 'ps' and look at multiple threads in a treaded process. I will be demonstrating KSE/M3 at BSDcon.
The Netgraph ATM package has been split into a number of smaller packages: bsnmp is a general-purpose SNMP daemon with support for loadable modules. Two modules come with it: one implementing the standard network-interface and IP related parts of MIB-2 and one for interfacing other modules to the NetGraph sub-system. ngatmbase contains the drivers for the ATM hardware, the ng_atm netgraph type and a few test tools. This package allows one to use ATM PVCs. It should be possible, for example, to do PPP over ATM with this package. Both bsnmp and ngatmbase are available in version 1.0 under the link above. Two other modules will be released in February: ngatmsig containing the UNI-4.0 signalling stack as netgraph nodes and ngatmip containing CLIP and LANE-2.0.
A significant amount of progress was made in December and January, particularly in the area of utility conformance. Several utilities were updated to conform to SUSv3, they include: at(1), mailx(1), pwd(1), split(1), and uudecode(1). Several patches have been submitted to increase conformance in other utilities, they include: fold(1), patch(1), m4(1), nice(1), pr(1), renice(1), wc(1), and xargs(1). These are in the process of being reviewed and committed. Two new utilities have been written, specifically pathchk(1) and tabs(1). These are also being reviewed and will be committed shortly.
A patch which implements most of the requirements of scanf(3) is being reviewed and is expected to be committed shortly. This will allow us to MFC a number of new functions and headers. Additionally, work has started on wide string and complex number support.
For 4.5-RELEASE, port ja-man-doc-4.5.tgz is in sync with base system except for OpenSSH pages (OpenSSH 2.3 based instead of 2.9) and perl5 pages (jpman project do not maintain). Section 3 updating has 55% finished.
OKAZAKI Tetsurou has incorporated changes on base system's groff into port japanese/groff. MORI Kouji has fixed two bugs of port japanese/man.
The KAME project is currently focusing on the scoped addressing architecture, the advanced API implementation, NATPT and the mobile ipv6 implementation. Though these stuffs are not stable enough to be merge into the FreeBSD tree, you can get and try them from the above URL.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
A guide for using FreeBSD with Bulgarian settings has been put up on the project's website. The CVS repository will be made public shortly, linked to on the URL's above.
An independent project for making FreeBSD easier to use by
- Bulgarians has appeared,
The past two months have been an exciting time in the FreeBSD Java Project with the signing of a license between the FreeBSD Foundation and Sun allowing us access to updated JDK source code and the Java Compatibility Kit (JCK). This license will also allow the project to release a binary version of both the JDK and JRE once JCK testing is complete. Work on this testing is under way with the project hopeful of being able to make a binary release in the not too distant future.
In lieu of the binary release which was hoped for with FreeBSD 4.5 the project will release an updated source patchset this weekend. This patchset will feature further work on the FreeBSD "native" threads subsystem from Bill Huey. Also, thanks to hard work by Joe Kelsey and Fuyuhiko Maruyama, the patchset will for the first time feature a working Java browser plugin!
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 90% complete.
OpenPAM, a new library intended to replace Linux-PAM in FreeBSD, has been written and is undergoing integration testing. It is available for download from the URL listed above.
In addition to this, a couple of new modules have been written (pam_lastlog(8), pam_login_access(8)), and the pam_unix(8) module has been extended to perform most of the tasks normally performed by login(1), which is now fully PAMified.
The PAM FDP article has been put on hold until OpenPAM replaces Linux-PAM in CVS, to avoid wasting effort on soon-to-be obsolete documentation.
Substantial progress has been made towards a working MAC implementation. The focus over the last two months has been moving from a hard-coded series of MAC policies to a more flexible implementation. A pluggable policy framework has been created (and is still under development), supporting Biba, MLS, TE, a "BSD Extended" model, and a sample mac_none module. Some modules must be compiled in or loaded prior to boot; others may be introduced at run-time. Support for networking has improved, with improved handling of IP fragmentation in IPv4, support for various pseudo-interfaces such as if_tun and if_tap, improved integration into userland, NFS-related fixes, moving the VFS enforcement out of individual filesystems, support for a 'multilevel' mount flag, support for explicit labeling in procfs and devfs, addition of an 'extattrctl lsattr' argument to list EAs on a filesystem, support for label ranges in the Biba and MAC policies, and much more.
Targets for the next two months include more universal enforcement of VFS-related calls, improved support for alternative ABIs, improved flexibility of in-kernel subject and object labels, support for IPv6 and IPsec, and improved support for NFS serving.
Development continues in the FreeBSD Perforce repository, which may be accessed using cvsup.
Now that the patch has been mailed to the freebsd-arch@FreeBSD.org mailing list, and that there were no objections, the commit will happen soon. Poul is currently testing it in his own tree. After it has been committed, it will be time to modify the filesystems in the tree to use VFS_NMOUNT instead of VFS_MOUNT. Mount(8) will also need some modifications. Some new manpages -- nmount(2) and kernel_vmount(9) -- are being created in the meantime.
Alfred Perlstein committed file descriptor locking code which was definitely a good push towards trying to lock down some important pieces of global data. Peter Wemm has made progress on pmap cleanups for x86 SMP TLB shootdowns. Matt Dillon and John Baldwin have made progress on getting patches done for moving accesses to ucred's out from under Giant's protection. John Baldwin has also made some commits in order to get the alpha port's SMP working. Matt Dillon has plans for hunting down fileops locking issues in order to continue his previous Giant pushdown work.
This report covers FreeBSD development activities from February, 2002 through April, 2002. It's been a busy few months -- BSDCon in San Francisco, the FreeBSD Developer Summit, a first development preview of 5.0-CURRENT, not to mention lots of progress on the 5.0 feature set (SMPng, sparc64, GEOM, ... the list goes on).
In the next two months, the USENIX ATC occurs (highly recommended event for both developers and users), and a number of new software components will hit the tree, including UFS2 and the TrustedBSD MAC framework. We'll also complete the elections for the FreeBSD Core Team, and should have the next Core Team online by the time the next report rolls around. Stay tuned for more!
Robert Watson
Packages are built from the FreeBSD Ports Collection on a cluster of i386 and alpha machines using scripts available in /usr/ports/Tools/portbuild/. Over the past few months I have been cleaning up and extending these scripts to improve efficiency and allow for greater flexibility in how package builds are performed. Major improvements so far have been: cleaning up and modularizing the scripts to avoid code duplication and reduce the need for ongoing maintenance; optimizing the build process and making it much more robust against client machine failure; and allowing package builds to be restarted if they are interrupted. The i386 package cluster is currently running FreeBSD 5.0-CURRENT, and it has proven to be a useful testing ground for exposing kernel bugs, especially those which only manifest under system load.
Future plans include the ability to perform incremental package rebuilds which only build packages that have changed since the last run. This will allow packages to be made available on the FTP site within an hour or two of the CVS commit to the ports collection. We also hope to set up a sparc64 package cluster in the near future, but this is contingent on suitable hardware.
FreeBSD's new kernel memory allocator has been committed to 5.0. UMA is a slabs derived allocator that supports memory reclaiming, object caching, type stable storage, and per CPU free lists for optimal SMP performance. It has both a malloc(9) interface and a zone style interface for specific object types. uma(9) will be available shortly.
Read-only support for UDF filesystems was checked into the 5-CURRENT branch in April. Backporting for 4-STABLE is being conducted by Jeroen. The next phase is to write a newfs_udf, then move on to adding write support to the filesystem. I'm still looking for a volunteer to handle read and write support for write-once media (e.g. CD-R).
I have released a new zero copy sockets snapshot, the first since November, 2000. The code has been ported up to the latest -current, and the jumbo code now has mutex protection. Also, zero copy send and receive can be selectively turned on and off via sysctl to make it easier to compare performance with and without zero copy. Reviews and comments are welcome.
I'm slowly making progress. The second engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020506.tar.gz
This release includes support for H4 UART transport layer, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices.
I'm currently working on RFCOMM protocol implementation (Serial port emulation over Bluetooth link). My next goal is to port Service Discovery Protocol (SDP) implementation from BlueZ (http://bluez.sf.net). I'm also thinking about adding USB device support (as soon as i find/buy hardware).
Issues: 1) Bluetooth hardware; I have couple PC-CARDs that i use for development and testing purposes, but i'd love to have more. 2) Time; My regular day job kicked in, so i will be spending more time doing stuff i'm getting paid for.
Since the last status report, two developers working on utility conformance were given commit access to the FreeBSD CVS repository to help expedite development. As a result, the following utilities have been brought up to conformance, they include: csplit(1), env(1), expr(1), fold(1), join(1), m4(1), mesg(1), paste(1), patch(1), pr(1), uuencode(1), uuexpand(1), and xargs(1). The printf(1) utility was brought up to conformance with the 1992 edition of POSIX.2, with further development planned.
On the header front, much progress has been made. Specifically, infrastructure to control visibility of components of a header, based on the standard requested by an application, has been added to <sys/cdefs.h>. Some work has been completed on renovating the way types are defined. This has lead to the creation of <sys/_types.h>. Further improvements such as the merger of <machine/ansi.h> and <machine/types.h> are planned. Additionally, the headers: <strings.h>, <string.h>, and <sys/un.h> have been made to conform to POSIX.1-2001.
On the API front, scanf(3) has received support for 5 new length modifiers (hh, j, ll, t, and z). A patch to implement two additional conversion specifiers (j and z) has been developed for printf(9) and is expected to be committed soon.
In other news, the project's web site has been moved to the main FreeBSD site. It is now available at the URL at the top of this status report. Please update your bookmarks.
Version 1.1 for FreeBSD-current is now available. It includes the SNMP-daemon package bsnmp, the driver package ngatmbase, the UNI4.0 signaling package ngatmsig and the network emulation package ngatmnet. NgAtm allows both to build applications running directly on top of ATM and to use ATM-Forum LAN emulation to use IP over ATM. Currently we are working on a simple switch module, that implements the network side signaling and ILMI as well as simple routing and call admission control.
The GNOME project has seen quite a few changes lately. For one, the author of this update has recently been given "The Bit." Joe Marcus Clarke now has CVS access, and is working primarily on the GNOME project. Joe has been closing a good deal of GNOME PRs, as well as patching some of the existing GNOME 1.4 components.
The GNOME 2 porting effort continues on. We have completed porting of the GNOME 2.0 API, and are 75% complete on porting the full GNOME 2.0 desktop. When complete, GNOME 1.4 and GNOME 2.0 will be co-resident in the ports tree. Both APIs can be installed concurrently in the same PREFIX, but the respective desktops will remain mutually independent. Maxim Sobolev is working on adapting bsd.gnome.mk to handle both versions of the desktop in an elegant fashion.
Not to be left out, the existing GNOME 1.4 components have received numerous updates to keep them in sync with the stable distfiles on gnome.org. We have seen many "1.0" milestone releases including the most recent AbiWord 1.0.0. In the next few weeks, we will be making sure all the GNOME 1.4 components build correct packages on bento so that GNOME 1.4 will be on the 4.6-RELEASE CD.
FreeBSD/KGI started last year after the port of GGI to VGL. KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with access to hardware graphic resources (dma, irqs, mmio). KGI is already available under Linux as a separate project. The FreeBSD/KGI project aims at integrating KGI in the FreeBSD kernel. Mostly a port for now, but optimized for FreeBSD in the future. Currently FreeBSD/KGI is under development and the code is only available for reading, compiling but not running. More interesting are design hints found at the project URL.
We now have a loadable mfsroot floppy. It contains just the diskeditor (which is really a disk partitioner) which has been enhanced and is probably in its final form. It's been geared towards making the newfs(1) and mount(1) steps separate dialogs, so it reduceed its complexity. A basic fstab class has been implemented to manipulate /etc/fstab and mountpoint. This might find a use outside libh, by the way. Libh package format is still incomplete and somehow buggy, so it's my next target.
There is a API documentation effort underway with the help of doxygen(1), so there's now more documentation for people that want to get started with libh.
All this lead me to prepare the release of another alpha preview of libh that will shortly be available in the ports collection (0.2.2). Also, a new committer (okumoto) has joined the project (as well as I) and he is currently working on cleaning up the build system. It's been a few months without news, so this probably seemed a bit long, but don't worry, we still need your help to really get this going!
There are several new topics, including: Source Code Tour is now separated into kernel part and userland part, yet another snapshots from RELENG_4_x branch (currently 4.5-RELEASE-p4), add several packages including XFree86 4.x to installation CD-ROM, new cdboot-only ISO image, fix breakage of duplex.iso, etc. See also the project webpage for more detail. Also, I have a plan to add FreeBSD/alpha distribution to this project -- stay tuned.
KAME Project has been extended until March 2004, and we decided the project roadmap for these two years. The first one year is for implementation, and the remaining year is for feedback of our results into other BSD projects (please refer to the above URL for further detail). Great change is lack of NAT-PT support due to a lack of human resource, although KAME snap still contains it as it is.
SUZUKI Shinsuke (suz@kame.net) has begun working for KAME and FreeBSD merge task in cooperation with Umemoto-san (ume@FreeBSD.org). Some of KAME stuff (critical bug fix, newest ports for pim6sd and racoon, etc) has been merged into 4-stable in this April.
Over the past couple of months, progress has pretty much stopped until very recently. The past few changes to the audit code were update the usage of zones to UMA zones, cleanup some old cruft, and start toying with the idea of having an audit write thread implemented as an ithd. The next step is to decide two realistic approaches to the where the records will be dumped -- whether that is to a local disk or fed up to userland and then dealt with. After that, the goal will be to expand the number of events that are being audited, while also working in some performance testing procedures. I will be posting to trustedbsd-audit about the recent changes shortly.
Over the last three months, there has been a lot of activity in the TrustedBSD MAC tree. An initial commit of the SEBSD code (NSA FLASK and SELinux implementation) was made; many MAC policies previously linked directly to the kernel via kernel options were moved to kernel modules; the flexibility of the framework was improved relating to the life cycle of object labels; additional labeling and access control hooks were introduced; new policies were introduced to demonstrate the flexibility of the framework (including a cleanup of inter-process authorization, additional VFS hooks, improved support for multilabel filesystems, network booting, IPv6, IPsec, support for "peer" labels on stream sockets). Current modules include Biba integrity policy, MLS confidentiality policy, Type Enforcement, "BSD Extended" (permitting firewall-like rulesets for filesystem protection), "ifoff" (limit interface communication by policy), mac_seeotheruids (limit visibility of processes/etc of other users), "babyaudit" (a simple audit implementation), and SEBSD (FLASK/SELinux port).
Over the next month, a final move to completely dynamic labeling will be made, permitting policies to introduce new state relating to process credentials, vnodes, sockets, mounts, interfaces, and mbufs at run-time, allowing a broad range of flexible label-driven policies to be developed. In addition, application APIs will be re-designed and re-implemented so as to better support a fully dynamic policy framework. We plan to make an initial prototype patchset available for review in June, with the intent of committing that patchset in mid-June.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
The painful parts are now completed, with all authentication- related utilities converted to PAM (except for those cases where it doesn't make sense, like Kerberos- or OPIE-specific commands). OpenPAM is complete (except for a few missing man pages) and seems to work well.
For more details, see the activity reports linked to above.
OpenSSH has been upgraded to 3.1, and the kinks seem to have been worked out by now. OpenSSH will now use PAM for both ssh1 and ssh2 authentication.
The KSE project had floundered due to lack of development time for awhile, but has been picked up recently by Jonathan Mini. Currently, the main focus is to prepare the "milestone 3" code for inclusion into -CURRENT.
The project is still working towards "milestone 4" (allowing threads from the same process to run on multiple CPUs), which should be significantly easier now due to work done by the SMPng project over the past several months.
Help could be used in several areas of the project, especially with porting the libc_r (pthreads) library to KSE's threading model.
NEWCARD support tried to merge CardBus functions with PCI functions, but that failed to properly route interrupts. A branch for the merge was created and will be merged into the main line at a later date. Too many other things going on in my life to make much progress.
Work on the host access point support for the Prism2 and Prism2.5 based wireless cards has been integrated into the kernel. This work is largely based on Thomas Skibo's initial implementation.
Continued bug fixing and hardening for this last few months.
Future work will include making target mode work correctly and fast.
The LSI-Logic chipset's MPT Fusion driver is also being evaluated.
The FreeBSD MTRR code has been made more robust against unexpected values sometimes found in the Athlon's Memory Type Range Registers. Problems with these values had prevented XFree 4.2 running on some motherboards. Experimentation indicates that these undocumented values may control the mapping of BIOS/ROMs or have something to do with SMM. If anyone can provide details of what these values mean, can they please let me know, so the MTRR code can be completed.
IPMI Tools for FreeBSD is a collection of C and Python applications and modules for exploring the information available via the Intelligent Platform Management Interface (IPMI), as implemented on server motherboards by Intel and HP. IPMI is an open standard with patent protection for adopters which defines standard interfaces to on-board management hardware. The management hardware consists of a CPU, sensors such as temperature probes and fan speeds, and repositories such as the System Event Log and Field-Replaceable Unit (FRU) inventory, and other system information.
A basic set of tools was recently made available which uses the KCS and SMIC system interfaces to retrieve the System Event Log, FRU repository, and system sensors. Additional features are currently under research. Suggestions for additional features and programs are greatly appreciated.
The PowerPC port is moving ahead. It can now mount a root file system and exec init, but fails when trying to map init's text segment in. I'm hoping to have it starting my fake "Hello, world!" init soon, after which I plan to try and get some libc bits in place so that I can build /bin and /sbin and try to get to actual single-user.
4.5-RELEASE Japanese manpage package, ja-man-doc-4.5.tgz, once published with OpenSSH 2.3 (as reported by previous status report) on January 31, is replaced with new package with OpenSSH 2.9 based manpages on March 3. Since then, we have been updating Japanese manpages for 4.6-RELEASE. For new translation and massive update, we have been making a lot of effort.
Continuing section 3 updating has 73% finished.
The GEOM code has gotten so far that it beats our current code in some areas while still lacking in others. Work continues on a generalized interface for "magic data" (boot blocks, disklabels MBR's etc) manipulation from userland.
With GEOM enabled in the kernel any FreeBSD platform will now recognize PC style MBR's, i386 disklabels, alpha disklabels, PC98 extended MBRs and SUN/Solaris style disklabels.
Since the last progress report, the initialization code was much cleaned (thanks to NetBSD's acort32 port) and partial DDB support as been added. I'm now struggling to put the pmap module into a working state. The latest patch set only includes the initialization changes. I did some tries to get what I had so far working on my iPAQ without much successes (downloading a kernel over a serial link is way too painful). If anyone has had success in getting any iPAQ to work as a USB storage device under *BSD please contact me.
I've been mentoring someone on locking up the protocol control blocks in the networking stack. She has already finished TCP and UDP and I'm currently reviewing the patch with her and going over some networking lock order issues. Locking up raw protocol interface control blocks follows next.
Support for stf(4), faith(4), and loopback interfaces has been committed to current. The stf and faith support has been MFC'd. In current the API has changed to move unit allocation into the generic cloning code reducing the amount of support code required in each driver. Code improvements to increase our API compatibility with NetBSD will be committed soon along with cloning support for discard interfaces and ppp(4) interfaces.
Thanks to
IA64 has had a busy few months. Aside from gcc, we are now fully self hosting on IA64. Doug Rabson has performed his magic and implemented the execution of 32 bit i386 application binaries although more work remains to be done to make ld-elf.so.1 happy with the different underlying page size. We have been using the i386 perforce binary to do actual development work and submit from the ia64 systems themselves. Marcel Moolenaar has been working on SMP and machine-check support. We have been running SMP kernels amazingly reliably on our development boxes for quite some time now. syscons is now functional. We have produced a self-booting run-root-on-cdrom ISO image (idea taken from the sparc64 folks) that has been used to manually self install an IA64 system from a blank disk. Aside from a few minor loose ends we now have complete 'make world' functionality. sysinstall works on ia64. We plan on producing a semi-respectable boot/install cdrom image shortly.
As of Thur May 9th, 2002 FreeBSD 5-CURRENT is now using a GCC 3.1 prerelease snapshot as the system C compiler. At this time of cutting over, the compiler is working well on i386, Alpha, Sparc64, and IA-64 for building world. There is a known problem with our atomic ops on Alpha that prevents a GCC 3.1 built kernel from booting.
Currently the C++ support libraries (libstdc++, et.al.) does not build and thus prevents the system C++ compiler from being used.
The release engineering team released FreeBSD
The release engineering team released FreeBSD 5.0-DP1 on 8 April 2002. This Developer Preview gives developers and other interested parties a chance to help test some of the new features to appear in 5.0-RELEASE. This distribution has known bugs and areas of instability, and should only be used for (non-production) testing and development.
The next releases of FreeBSD will be 4.6-RELEASE (scheduled for 1 June 2002) and 5.0-DP2 (scheduled for 25 June 2002). Information on the release schedules and more can be found on the team's new area on the FreeBSD Web site (see the URL above).
Finally, the team has gained two new members: Brian Somers and Bruce A. Mah.
libradius now supports RADIUS vendor attribute extensions and user-ppp is now capable of doing MS-CHAP authentication via a RADIUS server. A new net/freeradius port has been created for support of MS-CHAP in a RADIUS server.
MS-CHAPv2 support will be added soon.
The work is sponsored by Monzoon.
Mike Makonnen has done quite a bit of excellent work on porting the scripts from FreeBSD into the NetBSD framework. The next step seems to be to try to reduce the amount of diffs between our implementation and the original set from NetBSD.
The SMPng project has been picking up steam in the last few months thankfully. In February, Seigo Tanimura-san committed the first round of process group and session locking. Alfred Perlstein also added locking to most of the pipe implementation. In March, Alfred fixed several problems with the locking for select() and pushed down Giant some in several system calls. Andrew Reiter added locking for kernel module metadata, and Jeff Roberson wrote a new SMP-friendly slab allocator to replace both the zone allocator and the in-kernel malloc(). The use of the critical section API was cleaned up to not be abused as replacements for disabling and enabling interrupts. Also, Matt Dillon optimized the MD portion of the critical section code on the i386 architecture. Several other subsystems were also locked in April as well. See the SMPng website and todo list for more details.
Some of the current works in progress include locking for the kernel linker by Andrew Reiter and light-weight interrupt threads for the i386 by Bosko Milekic. Seigo Tanimura-san, Alfred Perlstein, and Jeffrey Hsu are also working on locking down various pieces of the networking stack. Alan Cox has started working on fixing the existing locking in the VM subsystem and moving bits of it out from under Giant. John Baldwin has written an implementation of turnstiles as well as adaptive mutexes in the jhb_lock Perforce branch. The adaptive mutexes appear to be stable on i386, alpha, and sparc64, but the turnstile code still contains several tricky lock order reversals. John also plans to commit the p_canfoo() API change to use td_ucred in the very near future and then finish the task of making ktrace(4) use a worker thread.
The patch for the new mount API has now been committed to the tree. Several filesystems also have been converted to this new mount API, namely procfs, linprocfs, fdescfs and devfs. I'm working on converting more filesystems to nmount, and actually already have UFS done. It has not been committed yet to avoid conflicting with the UFS2 work, but it should hit the tree soon. Manpages are still missing at the moment because I had to modify the API slightly. I hope to have them done soon now.
The second FreeBSD Developer Summit, held following the BSD Conference in San Francisco in February, was a great success. Around 40 developers attended in person, another five by phone, and many others by webcast. During a marathon-esque eight hour session, a variety of development topics were discussed, including adding inheritance to the KOBJ system, ports to new architectures, adaptations of the toolchain for new architectures, the GEOM extensible storage device framework, upcoming changes to the network stack, TrustedBSD features, KSE, SMPng, and the release engineering schedule. This event was sponsored by DARPA and NAI Labs, with webcasting provided by Joe Karthauser, bandwidth provided by Yahoo!. Planning for future such events is now underway; a summary/transcript of discussion may be found at the URL above.
Another busy pair of months at the FreeBSD Project have brought substantial maturity and feature completeness to the fledgeling 5.0-CURRENT branch. And just in time too, because by the time you read the next status report, we hope that you'll have FreeBSD 5.0 running on your desktop! Over the past two months, we've seen an upgrade of sparc64 to Tier 1 (Fully Supported) status, integration of a high quality storage encryption module, the commit of hardware-accelerated IPsec support, the addition of a general-purpose "Device Daemon" to process hardware attach/detach events to replace earlier single-purpose and bus-specific daemons, the commit of RAIDFrame, and the improved maturity of the TrustedBSD work. We've also seen another successful release of the 4.x branch, 4.7-RELEASE, which will continue to be the production supported platform as 5.X is brought in for landing.
Over the next two months, the FreeBSD Project will be focused almost entirely on making 5.0 a success: improving system stability and performance, as well as increasing the pool of applications that build and run on 5.0. The Release Engineering team will have announced the 5.0 code freeze, and released DP2 by the time you read this. Following DP2 will be a series of Release Candidates (RC's), and then the release itself. If you're interested in getting involved in the testing process, please lend a hand -- a spare box and a copy of the DP and RC ISOs burnt onto CD will make a difference. The normal caveats associated with pre-release versions of operating systems apply! You may also be interested in reading the Early Adopter's guide produced by the Release Engineering team to help determine when a transition from the 4.x branch to the 5.x branch will be appropriate for you and your organization.
Thanks,
Robert Watson, Scott Long
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20021104.tar.gz
This release features minor bug fixes and new OpenOBEX library port. The snapshot includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) port has been updated to version 0.8. (ported from BlueZ-sdp-0.8). Most of the RFCOMM issues have been resolved and now rfcommd works with Windows (3COM, Xircom and Widcomm) and Linux stacks.
New supported USB device - EPoX BT-DG02 dongle. Also I have received successful report about Mitsumi USB dongle and C413S Bluetooth enabled cell phone (L2CAP and SDP works, waiting on RFCOMM report).
I'm currently working on OBEX server (Push and File Transfer profiles) which will be based on OpenOBEX library (included in the snapshot).
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
October 10, 2002 marked the one year anniversary of our project. During that time we have made significant advances in FreeBSD's standards conformance. FreeBSD 5.0-RELEASE will be the showcase for most of our hard work. We hope that our tireless effort has had a positive effect on FreeBSD and software vendors that maintain or are considering porting their software to FreeBSD.
On the API front, _Exit(3) (an alias for _exit(2)) was added, sysconf(3) was update for POSIX.1-2001, and some of the glob(3) additions were MFC'd. The insque(), lsearch(), and remque() family of functions were reimplemented and moved to libc from libcompat. Several wide character functions were implemented, including all printf() and scanf() variants. Finally, support for wide character format types (%C, %S, %lc, %ls) were added to printf(3).
Work on utility conformance continued as getconf(1)'s compliance was updated, c99(1) (a new version of c89(1)) was implemented, and cd(1) and command(1) changes were MFC'd.
Almost 20 headers were brought up to conformance with applicable standards. Not much work remains to fix conformance issues in the remaining standard headers. Work in this area, as well as others, has slowed down in preparation for 5.0-RELEASE.
DEVD has been integrated into FreeBSD current. It was integrated in an incomplete state. However, it is useful in the state that it is in for doing simple things like running camcontrol rescan when a SCSI pcmcia card is inserted, or running /etc/pccard_ether with an ethernet card is inserted. The more sophisticated regular expression matching is not yet complete. Devd only does actions on device arrival and departure, but does not yet do anything with unknown devices. In addition to listening for device events, there is some desire to have /dev/devctl also allow for some direct control of the device tree.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work was committed to -current. To configure it for use specify options FAST_IPSEC in your system configuration file. At present support is limited to IPv4.
GBDE has been committed to -current.
The "Geom Based Disk Encryption" module provides a mechanism for very strong encryption of a GEOM "disk". The algorithm has passed informal review by a couple of seasoned crypto heavy-weights. Any GEOM device can be protected with GBDE, entire physical disks, MBR slices, BSD partitions etc etc. Booting from an encrypted partition is not possible, however.
The focus of GBDE is to protect a "cold" disk media. (FreeBSD is not equipped well for protecting key material on a running system from being compromised.) For a cold media, the only feasible attack on a GBDE protected media is guessing the pass-phrase.
Summary of the GBDE multilevel protection scheme: Up to four separate pass-phrases can unlock their own separate copies of the 2048 bit masterkey. The master-keys are protected using AES/256/CBC keyed with a SHA-2 hash derived from the pass-phrase. A salted MD5 hash over the sectoroffset "cherry-picks" which masterkey bytes participate in the MD5 hash which generates the "kkey" for each particular sector. The kkey AES/128/CBC encrypts the PRNG produced single-use key which AES/128/CBC encrypts the actual sector data.
GBDE has features for master-key destruction and pass-phrase invalidation.
See gbde(4) and gbde(8) for more details.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
The GEOM code is now the default on most (if not all ?) architectures and the few remaining issues in libdisk/sysinstall is being hashed out.
Although we are far from finished developing GEOM, its current feature set is a significant step forward for FreeBSD, providing not only immediate relief for new architectures (sparc64, ia64 etc) but also because it is designed as SMPng code from the start.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
These last two months have seen quite a lot of GNOME activity.
GNOME has started releasing development snapshots of the upcoming
GNOME 2.2 desktop. FreeBSD porting has begun outside of the
main ports tree in the
-
Evolution 1.2 is also close at hand. Ximian has posted its first release candidate, 1.1.90, which has been ported to FreeBSD, and is available from the MarcusCom CVS repo listed above. As soon as Ximian officially releases Evolution 1.2, it will be placed in the FreeBSD ports tree.
The Mozilla ports have received numerous updates. We are now tracking all three released Mozilla versions. The mozilla-vendor port is tracking the 1.0.x branch, mozilla is tracking 1.1.x, and mozilla-devel is tracking 1.2.x. The mozilla-devel port now has support for anti-aliased fonts as well as a GTK+-2 interface
Finally, the GNOME team would like to welcome its newest team member, Adam Weinberger. Adam has been submitting patches for both GNOME ports as well as documentation. Currently, he has been active in the GNOME 2.2 porting effort. We are happy to have him.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work was committed to the -current tree. To configure it for use specify device crypto in your system configuration file or you can load the crypto module. The /dev/crypto device support is brought in with device cryptodev or by loading the cryptodev module. Two crypto device drivers exist: ubsec for Broadcom-based PCI hardware and hifn for Hifn-based PCI hardware.
Integration of this work into the -stable source tree should be completed by the time this report is published.
Since the last status report the BSD Java Porting Team has continued to make steady progress. The most exciting news we have is courtesy of our newest team member, Alexey Zelkin of FreeBSD committer fame.
For 4.7-RELEASE, we privately published package ja-man-doc-4.7.tgz which consists of man[1256789] entries 10 days after the 4.7-RELEASE release date. Man3 update god no progress, as updating other sections busied us. We decided to suspend man3 update officially, as we need to spend most of our time to catch up with the forthcoming 5.0-RELEASE.
The KDE/FreeBSD team has been working on two major goals during the last two months, Maintenance of the KDE 3.0.x ports and Preparing the upcoming KDE 3.1 Release.
Maintenance KDE 3.0 conducted by Alan Eldrige: September started with the Removal of the KDE 2.x Ports from the FreeBSD-Repository. Later Packages of KDE 3.0.4 were released and the FreeBSD Ports were updated.
Preparing for KDE 3.1 conducted by Will Andrews: A lot of effort was spent on Improving the Fruitsalad-Build-System. We are now able to create packages directly from the KDE CVS.
The KSE code has now all the basic kernel functionality to start being used by the userland. There are still things to be done for testing and familiarization.
General system utilities have not yet been changed. e.g. ps and top etc. need to know about threads.
There is quite a lot of code in the kernel that still assumes that there is one thread in a process. Signals are not yet handled in the final manner (though they are delivered to a random thread in the process :-/ ).
The system calls and datastructures are now however in place. The test program successfully starts several threads that can be scheduled on different processors, and closes them down again. The userland is probably going to be able to do simple scheduling of pthread threads using KSE by the time that this report is published.
I still need someone to take over the "official" web page since jason left. LaTeX sure isn't my thing.
Not much since the last status report, except that we now have the repo and development web page back online, thanks to the services of John De Boskey who freely provided the necessary hardware and bandwidth to host the project. We have also ported LibH to GCC 3.x, so that it can compile on -CURRENT correctly. This, however, broke tvision, which doesn't compile under GCC 3.x, so we moved to rhtvision but this caused linking problems so we're stuck with no console front end, for now.
Work on a Hui rewrite and SWIG bindings stalled. Alex was able to come up with a simple patch to make the ports system use LibH's pkg_create script to build libh packages, so we're getting closer to a real pkg_create(1) drop-in replacement. I rewrote the milestone list to show a bit more relevant and encouraging tasks that will be dealt with in order to really push LibH forward.
A mailing list was created, freebsd-mips, and a Perforce branch was created in //depot/projects/mips. Changes which will be necessary to allow multiple MIPS (and PowerPC) metaports to exist under one architecture port were made, and are being pushed back into the main FreeBSD tree. Some preliminary header work has been done, and porting the ARCBIOS interfaces to the kernel has begun. The toolchain in tree was updated and modified in places to support a FreeBSD/MIPS (Big Endian) target, in the Perforce branch. Some early boot code has proven the GDB MIPS simulator to work, for at least R3000 code, though whether R3000 will be supported has been under discussion. Some initial architectural decisions were also made, to steer current work.
Work on newcard continues. A number of bugs have been fixed in the last few months. You are now able to load and unload drivers (including the bridge) to test changes to pccard and/or cardbus bus code. It is now possible to load a driver that has a pccard attachment and have a previously inserted card probe and attach. This is also true for CardBus. A number of issues remain to be solved before 5.0. However, with the integration of devd into the tree nearly all of old functionality of OLDCARD is now present in NEWCARD (the biggest remaining parts are power control for the sockets, as well as pccardc dumpcis).
The PowerPC port has been running diskless on NewWorld G3/G4 machines for a while now. A GEOM module to support Apple Partition Maps is being written. There should be an installable ISO image available in the near future.
RAIDFrame was imported into FreeBSD-current in late October, a major milestone after 18 months. It is still very experimental and not suitable for production environments. The website contains a lengthy TODO list which I hope to start attending to soon. Still, I encourage everyone to try it out and report bugs back to me.
The Release Engineering (RE) team completed and released FreeBSD 4.7 on 10 October 2002. This release features updates for a number of contributed software programs in the base system, as well as all of the security and bug fixes from FreeBSD 4.6.2. The next release in the 4.X series will be FreeBSD 4.8, which has a scheduled release date of 1 February 2003.
Before that time, however, will be the release of FreeBSD 5.0. Thus far, we have not been able to release the 5.0-DP2 developer snapshot due to various stability issues. Thanks to much effort from many of our fellow developers, we believe that most of these have been resolved. The RE team wishes to emphasize that FreeBSD 5.0 will involve new code and features that have not seen widespread testing, and that more conservative users may wish to continue to track the 4.X series for the near-term future. To provide more information on these issues, we have added an Early Adopter's Guide to the release documentation for 5.0.
Brian Somers has resigned from the RE team due to increased time pressures. We thank him for all of his help with FreeBSD 4.5, 4.6, 4.6.2, and 4.7, and we hope to continue working with him as a fellow developer.
Scott Long has graciously offered to help improve the communication between the RE team and the rest of the developer community. We greatly appreciate his assistance.
Recent 5-current release procedure troubles prevent the project from releasing a new snapshots. But 5-current FreeBSD/i386 release is back again in late Oct/2002! I have a plan to build daily FreeBSD/sparc64 snapshots for 5-current. Stay tuned...
A lot has happened recently for the sparc64 port. Sysinstall and make release work and can be used to build installable snapshots. The gdb5.3 port now works, and, thanks to Thomas Moestl, kernel crash dumps are supported which can be analyzed by gdb. These 2 items are the last things considered necessary by the Core team for FreeBSD/sparc64 to be a Tier 1 architecture, which means that 5.0-RELEASE for sparc64 will be officially supported by the release engineering team and by the security officer team.
Recently Jake Burkholder has been working on alternate installation methods other than bootable iso, including a mini-root filesystem which can be written to the swap partition of an existing machine. Thomas Moestl has been putting some finishing touches on the release process, ensuring that the release documentation can be built properly, and that the port readme files can be generated by the release process.
An experimental iso built with make release is now available on the freebsd ftp site and mirrors in /pub/FreeBSD/development/sparc64/5.0-20021031-SNAP. It is expected that by the middle of November new 5.0-SNAP releases will be available every few days for download and for ftp install, cpu power and bandwidth permitting.
Most progress on TrustedBSD over the last two months related to improving the maturity of the ACL and MAC implementations, and merging new aspects of those features into the primary FreeBSD CVS Repository for inclusion in FreeBSD 5.0. This included fixes to run better on sparc64, improved tuning of what system objects are mediated, locking fixes and optimizations especially relating to the vnode and pipe implementations, improved support for MAC labeling on symlinks, support for asynchronous process label changes as required in some locking situations, remove use of "temporary labels" and prefer use of object type specific labels reducing redundant and/or confusing label management code in policies, improve avoidance of memory allocation in M_NOWAIT scenarios for socket allocation in the syncache, mediation of link operations, race condition fixes for devfs involving label creation, improve handling of VM events such as mmaping, improve mediation of socket send/receive events (as distinguished from socket transmit/deliver events), support for manipulating EAs on symlinks using new system calls, support for MNT_ACLS and MNT_MULTILABEL flags at mount time, as well as FS_ACLS and FS_MULTILABEL superblock flags to key useful defaults using tunefs, correction of a memory leak in the UFS ACL code, enable UFS ACL support by default in GENERIC, mediation points for file creation, deletion, and rename, support for a mac_execve() execution interface in the style of SELinux's execve_secure() permitting a label transition request as part of the exec operation for policies that support it, more consistent handling of NFS lookups, support for labeling of multicast encapsulated packets, ATM packet labeling, FDDI packet labeling, STF packet labeling, revised label interface that avoids userland parsing of per-policy elements, reducing us to a single instance of parsing and printing for each policy (and further abstracting policy implementation details from the library code).
Also, change to single-level sockets for Biba and MLS policies, support for partial label updates for Biba and MLS, addition of mac.9 man page, revised user API system calls, implementation of mac_get_pid(), and various other related bits, creation of mac.conf(5) to specify label defaults, checks for various system operations including swapon(), settime(), and sysctl(), reboot(), acct(), introduction of command line utilities for maintaining file and process labels, support for user labels tied to login class, su support for label changes, ifconfig support for interface labels, ps support for process labels, ls support for file labels, ftpd support for login labels, development of the Biba and MLS notions of privilege, and a move to C99 sparse structure initialization, restoring full type checking for policy entry points.
Universally Unique Identifiers (UUIDs) are 128 bit values that may be generated independently on separate nodes (hosts), which result in globally unique strings. UUIDs are also known as Globally Unique Identifiers (GUIDs). The UUID support for FreeBSD (libc) conforms to the DCE 1.1 RPC specification.
UUID support has been added to FreeBSD -CURRENT, and will be available in version 5.0. It is being extensively used in GPT partition handling for IA-64 platform. For now, a simple manual page has been provided, which outlines information about the provided uuid routines. Many documentation additions and enhancements to uuidgen(1) are in the pipeline.
The goal of this project is to improve the wireless networking support in the system. The initial work will incorporate the 802.11 link layer done by Atsushi Onoe for NetBSD. This core support code implements the basic 802.11 protocols required for Station and AP operation in BSS, IBSS, and Ad Hoc modes of operation. Wireless device drivers will then be revised to use this common code instead of their private implementations.
Following this initial stage the wireless networking support will be extended to support functionality needed for workgroup, enterprise, and metropolitan (e.g. mesh) networking environments. This will include full power management support, the 802.1D spanning tree protocol for running multiple AP's in a bridged configuration, QoS support, and enhanced security protocols (LEAP, AES, EAP). Support for new hardware devices is also planned.
At long last, FreeBSD 5.0 is here. Along with putting the final polish on the tree, FreeBSD developers somehow found the time to work on other things too. IA64 took some major steps towards working on the Itanium2 platform, an effort was started to convert all drivers to use busdma and ban vtophys(), hardware crypto support and DEVD hit the tree, NewReno was fixed and effort began on locking down the network layer of the kernel. Also high performance, modular scheduler started taking shape and will be a welcome addition to the kernel soon.
Looking forward, the focus will be on stabilizing and improving the performance of 5.0. The RELENG_5 (aka 5-STABLE) branch will be created once we've reached our goals in this area, so hopefully we will get there quickly. Meanwhile, preparations for the next release from the 4.x series, 4.8, will begin soon. Of course, the best way to get 5.x to stabilize os to install and run it!
Thanks,
Scott Long, Robert Watson
I'm very pleased to announce that all kernel modules and few userland tools made it to the FreeBSD source tree. Many thanks to Julian Elischer.
Unfortunately no big changes since the last report. Some minor problems have been discovered and patches are available on request. I will prepare all the patches and submit them to Julian for review.
OBEX server and client (based on OpenOBEX library) is almost complete. I'm currently doing interoperability testing. If anyone has hardware and time please contact me. The HCI security daemon has been implemented and tested with Sony Ericsson T68i cell phone and Windows stack. It is now possible to setup secure Bluetooth connections.
A few people have complained about RFCOMM daemon. These individuals want to use GPRS and Bluetooth enabled cell phone to access Internet. If you have this problem please contact me for possible workaround. My next goal is to get robust RFCOMM implementation to address all these issues.
Largely bug-fixing and userland application tweaks; new interfaces were added to manipulate ACLs on extended attributes; bugs were fixed in ls relating to ACL flagging. Patches to teach cp, mv, gzip, bzip, and other apps about ACL preservation are in testing and review. tunefs flags were added to ease configuration of ACLs, especially on UFS2 file systems.
Possible changes to make use of Linux/Solaris umask semantics are under consideration: right now we implement verbatim POSIX.1e/IRIX merging of the umask, ACL mask, and requested creation mode during file, device, fifo, and directory creation. Solaris and the most recent Linux patches ignore the umask in the context of a default ACL; this requires some rearrangement of umask handling in our VFS, although the results would be quite useful. We're exploring how to do this in a low impact way.
Framework changes:
Instrument KLD system calls (module and kld load, unload, stat) Instrument NFSd system call. Instrument swapoff(2). Instrument per-architecture privileged parts of sysarch(). Make use of condition variables to allow callers to wait for the framework to "unbusy" when loading/unloading policies, rather than returning EBUSY. Store mount pointer in devfs_mount structure for use by policies. Improve handling of labels in loopback interface "re-align" packet copy case. Provide full paths on devfs object creations to help policies label them properly (not merged). Experimentation with moving MAC labels into m_tags (not merged). NFS server now uses real ucreds, not hacked up ucreds, meaning we can start laying the groundwork for enforcement on NFS operations. (not merged)
Policy changes
LOMAC: mac_lomac replaces lomac (LOMAC now uses the MAC Framework), SEBSD: Improved support for devfs labeling based on SELinux genfs. Handling of hard link checks. Support export of process transition information for login and others using sysctl. Login now prompts for roles. Allow policy reload. TTY labeling. Locking adaptation from Linux. Many, many policy adaptations and fixes. We can now boot in enforcing mode! mac_bsdextended: fix a bug in which VAPPEND wasn't mapped to VWRITE, so opens with the O_APPEND bug failed improperly.
Userland changes
setfmac(8) now supports a setfsmac(8) execution mode, which accepts initial labeling specification files. Supports an SELinux compatibility mode so it can accept SELinux label specfiles using the SEBSD module. sendmail(8) now sets user labels as part of the context switch for mail delivery.
Documentation changes
Man page updates for MAC command line tools, modules, admin hints, etc. Updates to the FreeBSD Developer's Handbook chapter on MAC policies and entry points. MAC section in FreeBSD Handbook.
This project has been coming along pretty well. The amd(4) and xl(4) drivers have now been converted to use the busdma API, sparc64 got the bus_dmamap_load_mbuf() and bus_dmamap_load_uio() functions, and the gem(4) and hme(4) drivers have been updated to use bus_dmamap_load_mbuf() instead of bus_dmamap_load().
A lot more still needs to be done, as shown on the project's page. A fair number of conversions are on their way though, and we can expect a fair number of drivers to be converted soon, thanks to all the developers who are working on this project.
The POSIX Utility Conformance in FreeBSD list (link above) has been updated to reflect current reality. Not much work remains to complete base utility conformance.
On the API front, grantpt(), posix_openpt(), unlockpt(), wordexp(), and wordfree() were implemented. The header <wordexp.h> was added.
There are currently about 40 unassigned tasks on our project's status board ranging from documentation, utilities, to kernel hacking. We would encourage any developers looking for something to work on to check out the status board and see if anything interests them.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work will be part of the 5.0 release and has been committed to the -stable source tree for inclusion in the 4.8 release.
Recent work has focused on improving performance. System statistics are now maintained and an optional profiling facility was added for analyzing performance. Using this facility the overhead for using the crypto API has been significantly reduced.
The ubsec (Broadcom) driver was changed to significantly improve performance under load. In addition several memory leaks were fixed in the driver and the public key support was enabled for use.
Upcoming work will focus on load-balancing requests across multiple crypto devices and integrating OpenSSL 0.9.7 which will automatically enable application use of crypto hardware.
Devd has been integrated into FreeBSD 5.0-RELEASE. The integrated code supports a range of configuration options. The config files are fully parsed now and their actions are performed.
Future work in this area is likely to be limited to improving the devctl interface. /dev/devctl likely will be a cloneable device in future versions. Individual device control via devctl is also planned.
The Donations project expedited several dozen donations during 2002, and was able to place most of what was offered. We still are in dire need of SMP and Sparc systems. You can see information on our needs and donations that have been handled by the team on the donations web page.
We are relying increasingly upon the developer wantlist to place items offered to the Project, and using the commit statistics to help place items. As such, active committers who ask for what they want beforehand have a decent chance of getting it. Less active committers, and committers who do not ask for what they want, will be lower in our priorities but will not be excluded.
We are in the process of streamlining the tax deduction process for donations, and hope to have news on that shortly. We are also always working to accelerate and reduce our internal processes, to get the most equipment in the hands of the most people as quickly as possible.
I especially want to thank David O'Brien and Tom Rhodes for stepping up and making the team far more successful. Also, the FreeBSD Foundation has been quite helpful in handling tax-deductible contributions.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work will be part of the 5.0 release. Performance has been improved due to work on the crypto subsystem.
The goal of the project is to use a small amount of space in the FFS superblock to store a volume label of the user's choice. A GEOM module will then expose the volume labels into a namespace in devfs. The idea is to make it easier to manage filesystems across disk swaps and movement from system to system.
At this point, everything pretty much works. I've submitted parts of the patch to respective subsystem maintainers for review. There are some issues with namespace collision that I haven't addressed yet, but the basic functionality is there
Most of the articles are translated too. Marc is still translating the handbook, 60% is currently translated. Stéphane has began the integration of our French localization web site in the US CVS Tree. Sébastien is still maintaining the Release Notes.
We launched a new site, www.FreeBSD-fr.info, consisting in a French Daemon News like site. Netasq have donated our new server; we will install it in a new hosting provider in the few next weeks. One of the big job now is the translation of the FAQ, and the big project will be the manual pages.
Since the ports tree has been frozen for most of this reporting period,
there have not been too many GNOME updates going into the official CVS
tree. However, development has not stopped. GNOME 2.2 is nearing
completion, and quite a few FreeBSD users have stepped up to test the
GNOME 2.1 port sources from the
-
The upcoming FreeBSD 5.0-RELEASE will be the first release to have the GNOME 2.0 desktop as the default GNOME desktop choice. During the previously mentioned ports freeze, all the GNOME 2 ports were fixed up so that they build and package on both i386 and Alpha platforms. Alas, the one port that will not make the cut for Alpha is Mozilla. There are still problems with the xpcom code, but work is ongoing to get a working Alpha port.
Finally, the FreeBSD Mono (an OpenSource C# runtime) port has also received some new life. Mono has been updated to 0.17 (the latest released version), and Juli Mallett has ported gtk-sharp (GTK+ bindings for C#).
The ia64 port is up and running on the new Itanium2 based hp machines thanks to a lot of hard work by Marcel Moolenaar. So far we are running on the hp rx2600 as these were the machines graciously donated by Hewlett-Packard and Intel. We had a prototype Intel Tiger4 system for a while, but we had to return the machine and we do not know if it currently runs. Most of the changes necessary to run these are sitting in the perforce tree and are not in the -current or RELENG_5 cvs tree. As a result, the cvs derived builds (-current and the 5.0-RC series and presumably 5.0-RELEASE) are only usable on obsolete Itanium1 systems.
Lots of other stability and functionality fixes have been made over the last few months, including initial libc_r support. The OS appears to be stable enough for sustained workloads - it is building packages now, for example. We still do not have gdb support, even for reading core files.
We have been updating our Japanese translated manual pages to RELENG_5 based. All existing entries have been updated, but 15 exceptions are not, most of which require massive update. We will also need to add translations which did not exist on RELENG_4.
KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with means to access hardware graphic resources (dma, irqs, mmio). KGI is already available under Linux as a separate standalone project. The KGI/FreeBSD project aims at integrating KGI in the FreeBSD kernel.
KGI/FreeBSD has been recently donated 2 PCI graphic cards (Matrox Millenium II and a coming Mach64) and other have been proposed. Please see the FreeBSD web pages for details. Thanks to donation@ for organizing and promoting donations. Thanks to the donators for their contribution to KGI/FreeBSD.
KGI/FreeBSD progressed fine the last months. Most of the VM issues for mapping HW resources in user space have been addressed and a first attempt of coding was made. This prototyping raised some API compatibility problems with the current Linux implementation and was discussed heavily on the kgi devel lists. Ask if you're interested in such issues, I'll be pleased to share them.
Most of coding is now done. Let's start debugging!
Work is ongoing to continue to lock up the network stack. Recently, the focus has been on the IP stack. The plan there involves a series of inter-related pieces to lock up the ifaddr ref count, the inet list, the ifaddr uses, the ARP code, the routing tree, and the routing entries. We are over 3/5 of the way done down this path.
In addition to TCP and UDP, the other networking protocols such as raw IP, IPv6, AppleTalk, and XNS need to be locked up. Around 1/4 these remaining protocols have been locked and will be committed after the IP stack is locked.
The protocol independent socket layer needs to be locked and operating correctly with the protocol dependent locks. This part is mostly done save for much needed testing and code cleanup.
Finally, a pass will be need to be made to lock up the devices drivers and various statistics counters.
This effort fixes some outstanding problems in our TCP stack with regard to congestion control. The first item is to fix our NewReno implementation. Following that, the next urgent correction is to fix a problem involving window updates and dupack counts. When that stabilizes, we will then change the recovery code to make use of SACK information. Eventually, this project will update the BSD stack to add Limited Transmit and other new internet standards and standards-track improvements.
The 3 FreeBSD package clusters (i386, alpha, sparc64) have been unified to run from the same master machine, instead of using 3 separate masters. This has freed up some machine resources to use as additional client machine, as well as simplifying administrative overheads. Build logs for all 3 architectures can now be found on the http://bento.FreeBSD.org webpage. The sparc64 package cluster now has 3 build machines (an u5 and two u10s), and an ia64 cluster is about to be created.
Package builds now keep track of how many sequential times a port has failed to build (html summaries are available on the bento website). This allows tracking of ports which have suddenly become broken (e.g. due to a bad upgrade, or due to changes in the FreeBSD source tree), and in the future will be used to send out notifications to port maintainers when their port fails to build 5 times in a row. This feature is currently experimental, and further code changes will be needed to stabilize it.
The goal of this project is to improve the wireless networking support in the system. By the time of this report the 802.11 link layer code should be committed. A version of the wi driver that uses this code should be committed shortly. Conversion of other drivers is planned as are drivers for new devices.
Support for 802.1x/EAP is the next planned milestone (both as a supplicant and authenticator).
November and December were especially busy for the release engineering team. Scott Long joined the team to help with secretary and communications tasks while Brian Somers bowed out to focus on other projects.
FreeBSD 5.0-DP2 was released in November after much delay and anticipation, and marked the final milestone needed for 5.0 to become a reality. Shortly after that, we imposed a code freeze on the HEAD branch of CVS and released 5.0-RC1. Creation of the RELENG_5_0 branch came next, followed by the release of 5.0-RC2 from this branch. At this point, enough critical problems still existed that we scheduled an RC3 release for the new year, and pushed the final 5.0-RELEASE date to mid-January. By the time this is published, FreeBSD 5.0-RELEASE should be a reality.
For the time being, there will not be a RELENG_5 (aka 5-STABLE) branch. FreeBSD 4.x releases will continue, with 4.8 being scheduled for March 2003. Release in the 4.x series will be lead by Murray Stokely, and releases in the 5.x series will be lead by Scott Long. Once HEAD has reached acceptable performance and stability goals, the RELENG_5 branch will be created and HEAD will move towards 6.0 development. We hope to reach this with the 5.1 release this spring.
A new scheduler will be available as an optional component along side the current scheduler in the 5.1 release. It has been designed to work well with KSE and SMP. Some ideas have been borrowed from solaris and linux along with many novel approaches. It has O(1) performance with regard to the number of processes in the system. It also has cpu affinity which should provide a speed boost for many applications.
The scheduler has a few loose ends and lots of tuning before it is production quality although it is quite stable. Please see the post to arch and subsequent discussion for more details.
The FreeBSD Bi-monthly status reports are back! In this edition, we catch up on seven highly productive months and look forward to the end of 2003.
As always, the FreeBSD development crew has been hard at work. Support for the AMD64 platform quickly sprang up and is nearly complete. KSE has improved greatly since the 5.1 release and will soon become the default threading package in FreeBSD. Many other projects are in the works to improve performance, enhance the user experience, and expand FreeBSD into new areas. Take a look below at the impressive summary of work!
Scott Long, Robert Watson
Still in the planning stage. Working on creating an extensible interface that is usable for both userland and kernel implementations for device drivers. Deciding on how to interface userland implemented device drivers with applications.
KSE seems to be working well on x86, amd64, and ia64. The alpha userland bits are done, but a couple of functions are unimplemented in the kernel. For sparc64, the necessary functions are implemented in the kernel, but the userland context switching functions need more attention.
Since 5.1, efficient scope system threads (no upcalls when they block) have been implemented, and KSE based pthread library can have both POSIX scope process threads and scope system threads. It is also possible that KSE based pthread library can implement pthread both in 1:1 and M:N mode, I know Dan has such Makefile file patch for libkse not yet committed.
KSE program now can work under ULE scheduler, its efficient should be improved under the new scheduler in future. BSD scheduler is still the best scheduler for current KSE implement.
Much has happened since the last bi-monthly report, which was more than half a year ago. FreeBSD 5.0 and FreeBSD 5.1 have been released for example. With FreeBSD 5.2 approaching quickly, we're not going to look back too far when it comes to our achievements. There's too much ahead of us...
Two milestones have been reached after FreeBSD 5.1. The first is the ability to support both Intel and HP machines with sources in CVS. This due to a whole new driver for serial ports, or UARTs. Unfortunately this still implies that syscons is not configured. That's another task for another time, but keep an eye on KGI/FreeBSD... The second milestone is the completion of KSE support. Both M:N and 1:1 threading is functional on ia64 and the old libc_r library has been obsoleted. Testing has shown that KSE (i.e. M:N) may well become the default threading model. It's looking good.
The ABI hasn't changed after 5.1 and the expectation is that it won't change much. This means that we can think about becoming a tier 1 platform. This also means we need gdb(1) support. Work on it has been started but the road is bumpy and long. Kernel stability also has improved significantly and we typically have one kernel panic remaining: VM fault on no fault entry. This will be addressed with the long awaited PMAP overhaul (see below).
Most work for FreeBSD 5.2 will be "sharpening the saw". Get those loose ends tied. This is a slight change of plan made possible by a slip in the release schedule. The 5.2 release is not going to be the start of the -stable branch; it has been moved to 5.3. So, we use the extra time to prepare the ground for 5.3.
The planned PMAP overhaul will probably be finished after 5.2. This should address all known issues with SMP and fix those last panics. As a side-effect, major performance improvements can be expected. More news about this in the next status reports.
The following items are in progress in the Disk I/O area: Turn scsi_cd.c into a GEOM driver. (Patch out for review). Turn atapi-cd.c into a GEOM driver. Turn fd.c into a GEOM driver. Move softupdates and snapshot processing from SPECFS to UFS/FFS. Move userland access to device drivers out of vnodes.
Once these preliminaries are dealt with, scatter/gather and mapped/unmapped support will be added to struct bio/GEOM.
FreeBSD Update is a system for tracking the FreeBSD release (security) branches. In addition to being faster and more convenient than source updates, FreeBSD Update also requires less bandwidth and is more secure than source updates via CVSup. However, FreeBSD Update is limited; it can only update files which were installed from an official RELEASE image and not recompiled locally. Right now I'm publishing binary updates for 4.7-RELEASE and 4.8-RELEASE; since my only available box takes 3.5 hours to buildworld, I don't have enough resources to do any more than that.
In the near future, I'd like to: Find someone who is willing to donate a faster buildbox; start building updates for other releases (at a minimum, for all "supported" FreeBSD releases); add warnings if a file would have been updated but can't be updated because it was recompiled locally; add code to compare the local system against a list of "valid" MD5 hashes for intrusion detection purposes; and add support for cross-signing, whereby several machines could build updates independently to protect against buildbox compromise.
The project started this spring and released version 1.0 with a port installation (security/pf) in may 2003. Version 2.0 is on the doorstep as OpenBSD 3.4 will be released. Due to the porting efforts we were able to reveal some bugs in the OpenBSD code and provided locking for the PFIL_HOOKS, which we utilize. Tarball installation of a loadable kernel module for testing can be found on the project homepage, a patchset is in the making.
PF was started at OpenBSD as a substitute for ipfilter and provides the same function set. However, in the two years it exists now, it has gained many superior features that no other packet filter has. For a impression take a look at the pf FAQ.
We hope to be eventually integrated into the base system. Before that we have to resolve some issues with tcpdump and kame.
I'm very pleased to announce that another release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20030908.tar.gz. I have also prepared patch for the FreeBSD source tree. The patch was submitted for review to the committers.
Fixed few bugs in kernel modules. The ng_hci(4) and ng_l2cap(4) modules were changed to fix issue with Netgraph timeouts. The ng_ubt(4) module was changed to fix compilation issue on -current.
Improved user-space utilities. Implemented new libsdp(3). Added new sdpcontrol(8) utility. The rfcomm_sppd(1), rfcomm_pppd(8) and obexapp(1) were changed and now can obtain RFCOMM channel via SDP from the server. The hccontorol(8) utility now has four new commands. The hcsecd(8) daemon now saves link keys on the disk.
I've been recently contacted by few individuals who whould like to port current FreeBSD Bluetooth code to other BSD systems (OpenBSD and NetBSD). The work is slowly progressing towards un-Netgraph'ing current code. In the mean time Netgraph version will be the primary supported version of the code.
The rescue build infrastructure has been committed. There is one known issue with make using both the '-s' and '-j' flags that appears to be a bug in make. Anyone interested in tracking down should contact us.
Support for a dynamically linked /bin and /sbin has been committed, although it is not turned on by default. Adventurous users can try it out by building /bin and /sbin using the WITH_DYNAMICROOT make flag. More testing is needed to determine if this is going to be default for 5.2-RELEASE. If anyone would like to benchmark worldstones with and without dynamically linked /bin and /sbin, please feel free to do so and submit the results.
Work is continuing on updating ACPI with new features as well as bugfixing. A new embedded controller driver was written in July with support for the ACPI 2.0 ECDT as well as more robust polling support. Also, a buffer overflow in the ACPICA resource list handling that caused panics for some users was fixed. Marcel helped get acpidump(8) tested and basically working on ia64.
Upcoming work includes integrating ACPI notifies with devd(8), committing user-submitted drivers for ASUS and Toshiba hotkeys, Cx processor sleep states (so my laptop doesn't burn my lap), and power resource support for intelligently powering down unused or idle devices.
Users who have problems with ACPI are encouraged to submit a PR and email its number to acpi-jp@jp.FreeBSD.org. Bug reports of panics or crashes have first priority and non-working features or missing devices (except suspend/resume problems) second. Reports of failed suspend/resume should NOT be submitted as PRs at this time due to most of them being a result of incomplete device support that is being addressed. However, feel free to mail them to the list as any information is helpful.
The uart(4) project was born out of the need to have a working serial interface (i.e. an RS-232-C interface) in a legacy-free configuration and after an unsuccessful attempt to convert sio(4). The biggest problem with sio(4) is that it has been intertwined in many ugly ways into the kernel's core. Conversion could not happen without breaking something that invariably affects some group of people negatively. With sio(4) as a good bad example and a strong desire to solve multiple problems at once, the idea of an UART (Universal Asynchronuous Receiver/Transmitter) device that, given its generic name, could handle different flavors of UART hardware started to settle firmly in the authors mind.
The biggest challenge was of course solving the problem of the low-level console access prior to the initialization of the bus infrastructure and still have a driver that uses the bus access exclusively. Along the way the problem of having an UART function as the keyboard on sparc64 was solved with the introduction of system devices, which also encapsulated the console as a system device.
The uart(4) driver can be enhanced to support the various UART hardware on pc98 and this is currently being worked on. Keyboard support on sparc64 is underway as well. Plans exist for a rewrite of the remote gdb support that uses a generic interface to allow various drivers, including uart(4), to register itself as a communications channel. And since uart(4) does not support multi- port cards by itself, we likely need to either enhance puc(4) or otherwise introduce other umbrella drivers
Since I ported icc to FreeBSD I wanted to build FreeBSD with icc. Now with icc 7.1 (and some patches) it is possible. There are still some bugs, e.g. NFS doesn't work with an icc compiled kernel, IP seems to be fragile, and some advanced optimizations trigger an ICE (Intel is working on it). At the moment I'm waiting for our admins to install icc on the FreeBSD cluster (we got a commercial license from Intel, so we are allowed to distribute binaries which are compiled with icc), after that I will try to convince some people with more knowledge of the IP and NFS parts of the kernel to debug the remaining problems. When the icc compiled kernel seems to work mostly bugfree the userland will get the porting focus. Interested people may try to do a build of the ports tree with icc independently from the status of the porting of the userland... if this happens at the FreeBSD cluster, we would also be allowed to distribute the binaries.
Benefits include: another set of compiler errors (debugging help), more portable source, and code which is better optimized for a P4 (gcc has some drawbacks in this area)
The FreeBSD ports were updated to KDE 3.1.4, another bug- and security-fixes release. With this update, the QT port was updated to version 3.2. Both will be included in FreeBSD 4.9. Significant work was spent to fix KDE on FreeBSD-CURRENT after the removal of the gcc -pthread Option. Automatic package builds from KDE CVS continued to ensure and improve the quality of the upcoming KDE 3.2 release.
Future: Work is in progress to setup a new server for hosting the KDE-FreeBSD Website, Repository and another KDE CVS mirror. With help from Marcel Moolenaar the project will try to make KDE compile and working on the Intel IA64. And last but not least efforts are being made to fix the currently broken kdesu program.
WifiBSD is a miniture version of FreeBSD for wireless applications. Originally for the Soekris Net45xx line of main-boards, but is now capable of being targeted to any hardware/architecture FreeBSD itself supports. Although not feature complete, WifiBSD is expected to be ready for 5.2-RELEASE. The design goal is to meet, or exceed, the functionality of commercial/consumer 802.11 wireless gear. Features that need attention (to name just a few) are: http interface, consol menu interface, and installation. Volunters are welcome.
Work has restarted after a hiatus. Current focus is on getting loadable modules working, NEWBUSing the NetBSD dbdma code, and completing the BMAC ethernet driver.
There is a huge amount of work to do. Volunteers more than welcome!
The last known bug that prevented AMD64 machines completing a full release has been fixed - one single character error that caused ghostscript to crash during rendering diagrams. SMP work is nearing completion and should be committed within the next few days. The SMP code uses the ACPI MADT table based on John Baldwin's work-in-progress there for i386. We need to spend some time on low level optimization because there are several suboptimal places that have been ignored for simplicity, context switching in particular. MTRR support has been committed and XFree86 can use it. cvsup now works but the ezm3 port has not been updated yet. The default data segment size limit is 8GB instead of 512M, and the (primitive) i386 binary emulation support knows how to lower the rlimits for executing 32 bit binaries.
Notable things missing still: Hardware debug register support needs to be written; gdb is still being done as an external set of patches relative to the not-yet-released FSF gdb tree; DDB does not disassemble properly; DDB cannot do stack traces without -fno-omit-frame-pointer - a stack unwinder is needed; i386 and amd64 linux binary emulation is needed, and the i386 FreeBSD binary emulation still needs work - removing the stackgap code in particular.
The platform in general is very reliable although a couple of problems have been reported over the last week. One appears to be a stuck interrupt, but all that code has been redone for SMP support.
The FreeBSD Java community has started an effort to improve the current framework for Java-based ports. The main objective is the automation of JDK/JRE build and run dependency checking.
The original version was aimed to ease the life of porters. Although it has proved to be useful and reliable to a great extend, we are currently working on a new version. We intend to reach a high degree of flexibility to cope with the recent increase of available JDK/JRE flavors. Furthermore, the new version will be easier to maintain, which means improved reliability, and hopefully more frequent updates.
The BSD Java Porting Team has recently reached an exciting milestone with the release of the first "Diablo" JDK and JRE courtesy of the FreeBSD Foundation. The release of Diablo Caffe and Diablo Latte 1.3.1 was the first binary release of a native FreeBSD JDK since 1.1.8 and marks an important step forward in FreeBSD Java support.
The team is continuing development work, with a focus on achieving a compliant JDK 1.4 release in the near future.
With the introduction of ATAng, some users of ATAPI/CAM have experienced various problems. These have been mostly tracked down to issues in the new ATA code, as well as two long-standing problems in portions of the CAM layer that are rarely exercised with "real" SCSI SIMs. This has also been an occasion to cleanup ATAPI/CAM to make it more robust, and to enable DMA for devices accessed through it, resulting in improved performances.
We have released Japanese translation of 5.1-RELEASE online manual pages on June 10.
Several months ago, I took it upon myself to to try present the
- information contained on
The initial intent of this was to make life easier for ports
maintainers; however, the "general" reports are also useful to anyone who
just wants to, e.g., find out if a particular port is working on their
particular architecture and OS combination before downloading it. Those
with that general interest should start with the
-
A lot of work done since last report: site reworked completly (see new URL), console design with console message in text or graphic modes implemented, implementation of a compatibility layer to compile Linux fbdev drivers with more or less changes in the original driver (experimental).
Except some memory allocation bugs, X (XGGI based on XFree 3.3.6) is now working with the same driver as the console. A basic terminal has now to be implemented.
Volonteers are welcome to the project...
A number of races have been identified in locking device_t. Most of the races have been identified in making device_t have to do with how drivers are written. Efforts are underway to identify all the races, and to contact the authors of subsystems that can help the drivers. Of special concern is the need for the driver to ensure that all threads are completely out of the driver code before detach() finishes. Of additional concern is making sure that all sleepers are woken up before certain routines are called so that other subsystems can ensure the last condition and leave no dangling references. Locking device_t is relatively straight forward apart from these issues. Towards the end of proper locking, sample strawmen drivers are being used to work out what, exactly proper is. Once these issues are all known and documented in the code, efforts will be made to update relevant documentation in the tree. There are many problems with driver locking that has been done to date, but until we nail down how to write a driver in current, it will be premature to contact specific driver writers with specific concerns.
Support for several new crypto devices was added. The SafeNet 1141 is a medium performance part that is not yet available on retail products. The Hifn 7955 and 7956 parts are starting to appear on retail products that should be available by the end of the year. Both devices support AES encryption. Support for public key operations for the SafeNet devices was recently done for OpenBSD and will be backported. Public key support for the Hifn parts is planned.
A paper about the performance work done on the cryptographic subsystem was presented at the Usenix BSDCon 2003 conference and received the best paper award.
NetBSD recently imported the cryptographic subsystem.
The release of 4.9 is just around the corner and offers Physical Address Extensions (PAE) for x86 along with the same world-class stability and performance that is expected from the 4-STABLE series. As always, don't forget to purchase a copy of the CD set from your favorite FreeBSD vendor.
FreeBSD 5.1 was released in June and offered vastly improved stability over 5.0 along with a working implementation of Kernel Scheduled Entities, allowing for true multithreading of applications across multiple CPUs. FreeBSD 5.2 will be released by the end of 2003 and will focus on improved network and overall performance.
Numerous bugs have been fixed since the last status report (and of course a few new ones added). Progress on improved security has been slowed by other work. But new features and fixes are coming in from other groups that are now sharing the code. In particular NetBSD recently imported the revised 802.11 layer and the Linux-based MADWIFI project is using it too (albeit in an older form). The MADWIFI users have already contributed features such as fragmentation reassembly of 802.11 frames and improved signal monitoring. Power save polling and an improved rate control algorothm are expected to come in from the NetBSD folks. WPA support is still in the plans; the best estimate is that work on that will start in January.
The purpose of this project is to improve performance of the network subsystem. A major part of this work is to complete the locking of the networking subsystem so that it no longer depends on the "Giant lock" for proper operation. Removing the use of Giant will improve performance and permit multiple instances of the network stack to operate concurrently on multiprocessor systems.
This project started in August. The emphasis has been on locking the "lower half" of the networking code so that packet forwarding through the IPv4 path can operate without the Giant lock as part of the 5.2 release. To this end locking was added to several network interface drivers and much of the "middleware" code in the network was locked (e.g. ipfw, dummynet, then routing table, multicast routing support, etc). Work towards this goal is still ongoing but should be ready for 5.2. A variety of test systems have been running for several months without the Giant lock in the network drivers and IP layer.
Past the 5.2 release Giant will be removed from the "upper half" of the network subsystem and the socket layer. Once this is done the plan is to measure and improve performance (though some work of this sort is always happening). The ultimate goal is a system that performs at least as well as 4.x for normal use on uniprocessor systems. On multiprocessor systems we expect to see significantly better performance than 4.x due to greater concurrency and reduced latency.
2004 started with another exciting two months for the project. FreeBSD 5.2 was released in early January and then quickly followed in February with the 5.2.1 bug-fix release. Looking forward, we are expecting a late-April release date for FreeBSD 4.10, and mid-summer date for FreeBSD 5.3. And don't forget to support the FreeBSD vendors and developers by buying a copy of the latest CD or DVD sets.
Thanks,
Scott Long
In the overall area of disk and device I/O, a significant milestone was reached with the implementation of proper reference counting on dev_t. We are now able to properly allocate and free dev_t. Cloning device drivers also had the job made easier for them with the addition of the unit number management routines.
It is not quite decided which will be the next step in the quest for a truly SMPng I/O subsystem, but a leading candidate is to implement the device-access vnode bypass to get more concurrency in the system: Instead of taking the tour through the vnodes for each i/o operation on a device we will go directly from the file descriptor layer to DEVFS/SPECFS. In addition to Giant-less disk I/O, this should enable us to pull the entire tty subsystem and the PTY driver out from under Giant and we expect that to improve the "snappiness" of the system measurably.
The Dutch Documentation Project is a ongoing project in translating the handbook and other documentation to the dutch language. Currently there is 1 active person (me) translating the documentation. I am currently working on the handbook/basics section. But i can use some more hands, please drop me an email if you wish to help out so that the dutch translation will speed up and be ready in some time. Contact remko@elvandar.org for information.
I have been producing weekly summaries of commits and the surrounding discussions as reported on the cvs-src mailing list. These summaries are posted to -current on Sunday evenings and archived on the Web. The reception has been overwhelmingly good. As of the end of February, Polish translations are being produced by Lukasz Dudek and Szymon Roczniak; they are also planning to translate the older summaries.
libarchive, with complete documentation, has been committed to -CURRENT. bsdtar should follow soon. For a few months, gtar and bsdtar will both be available in the base system. Once bsdtar is in the tree, I hope to resume work on libpkg and my pkg_add rewrite.
Note that bsdtar is not an exact replacement for gtar: it does some things better (reads/writes standard formats, archive ACLs and file flags, detects format and compression automatically), some things worse (does not handle multi-volume archives or sparse files) and a few things just different (writes POSIX-format archives by default, not GNU-format). The command lines are sufficiently similar that most users should have no problems with the transition. However, people who rely on peculiar options or capabilities of gtar may have to look to ports.
The first actual feature related to the if_xname conversion was committed in early February. Network interfaces can now be renamed with "ifconfig <if> name <newname>".
Work is slowly progressing on a new network interface cloning API to enable interesting cloners like auto-configurating vlans. This work is taking place in the perforce repository under: //depot/user/brooks/xname/...
After a slow time at the end of last year due to a disk crash, the project is moving along rapidly. The loader is fully functional with Forth support. Syscons has been integrated. New Powerbook models are supported. Work is starting on a G5 port.
There's still lots to do, so as usual volunteers are most welcome.
The project is a joint effort of volunteers, which focus in the internationalization and localization of the FreeBSD Operating System and applications running on FreeBSD. All of the work resulted in this project will be contributed back to the FreeBSD project.
Thanks to many volunteers' help, by this time of writing, we have finished more than 60% of the translation of the FreeBSD Handbook. We plan to submit a preliminary translation of the FreeBSD website as well as the FreeBSD Handbook when most part of them were finished, which is expected to happen in a couple of months. The snapshot of the documentation translation effort could be accessed through the URL listed above.
The project also supported individual efforts on porting applications (especially software that supports Simplified and/or Traditional Chinese) to FreeBSD. We are also doing some research on making FreeBSD kernel and base system more i18n-aware.
The verify source reachability option for ipfw2 checks if the source IP address of a packet entering the machine is reachable at all. Thus if we can't send a packet back because we don't have a route back we don't have to forward it because two way communication isn't possible anyway. It is more than likely that such a packet is spoofed. This option is almost the same as what is known on Cisco IOS as "ip verify unicast source reachable-via [any|ifn]". Using this option only makes sense when you don't have a default route which naturally always matches. So this is useful for machines acting as routers with a default-free view of the entire Internet as common when running a BGP daemon (Zebra/Quagga or OpenBSD bgpd).
One useful way of enabling it globally on a router looks like this: ipfw add xxxx deny ip from any to any not versrcreach or for an individual interface only: ipfw add xxxx deny ip from any to any not versrcreach recv fxp0
The ARP IP address to MAC address mapping does not belong into the routing table (FIB) as it is currently done. This will move it to its own hash based structure which will be instantiated per each 802.1 broadcast domain. With this change it is possible to have more than one interface in the same IP subnet and layer 2 broadcast domain. The ARP handling and the routing table will be quite a bit simplified afterwards. As an additional benefit full MAC address based accosting will be provided. Work on this project is already in progress.
The current TCP send and receive buffers are static and set to a conservative value to preserve kernel memory. This is sub-optimal for connections with a high bandwidth*delay product because the size of the TCP send buffer determines how big the send window can get. For high bandwidth trans-continental links this seriously limits the maximum transfer speed per TCP connection. For example a 170ms RTT and a 32kB send buffer limit the speed to approximately 1.5Mbit per second even thought you might have a 10Mbit pipe.
This project makes the TCP send buffer to automatically adapt to the optimal buffer size for maximal link usage. In the case above this would be a buffer of approximately 220kB. The main challenge is to have a stable and reliable measurement of the link parameters and manage the kernel memory properly and in a fair way. We don't want to have a few connections to monopolize all available socket buffer space and many edge cases have to be considered. The first implementation will be tuned conservatively but even that will provide significantly better performance than the static buffers currently. Work on this project is already in progress.
The TCP performance test and qualification testbed is an automated environment that simulates various common and uncommon end-to-end network and link characteristics such as delay, bandwidth limitations, congestion, packet drops, packet corruption and out of order arrival. The testbed automatically steps through all link types and tests various TCP optimizations and parameter adjustments. In the end all data is graphically arranged and compared against standard behaviour and each other to judge the positive or negative effects of the modifications. Work on this project has just started and is based on FreeBSDs dummynet.
Thanks to the loan of a box by Will Andrews, the system has
been moved into production. The previous installation
at lonesome.com now refers you to the new system. As part of
the installation, a preliminary
-
The database is updated once per hour.
New reports available include ones about ports marked DEPRECATED, since that function has now been incorporated into bsd.port.mk. (The author hopes that this will allow the port deprecation process to be much more visible to the general FreeBSD user community.) In addition, a report for ports marked FORBIDDEN was added (the code was essentially the same).
The next topic of interest is to try to identify ports which are slave ports because the status of these ports is not currently being updated automatically. This problem also affects FreshPorts. PR ports/63683 is an attempt to address this problem. Also, preliminary work has been done on creating some graphs and charts for various statistics, and in creating a tool to browse port dependencies for the entire ports tree.
Some general observations about the trends in ports PRs can be made:
The FreeSBIE Project aims to develop a set of scripts that allow anyone to create their own FreeBSD Bootable Cdrom, with their own set of installed packages. The Project releases an ISO builded with FreeSBIE scripts, to show what they can do. On Sunday 29 February 2004, FreeSBIE 1.0 was released and it had a great success, as there were post on Slashdot.org, OSnews, DaemonNews and BSDForums. Thanks to the huge amount of feedback they got, FreeSBIE Developers are now developing new features such as support for archs different from i386. Website redesign is on the way too.
Move to Perforce is done. I spent some time on building a common compilation tree with Linux: until now drivers were build in a FreeBSD makefile tree, not compatible with Linux.
The next priorities are ANSI support and keymaps in the KGC Kernel Graphic Console system.
Work on the PMAP overhaul has been put into gear. A lot of issues will be addressed, including support for sparse physical memory and of course SMP. Performance will be addressed to the extend possible, but functionality has priority. The redesign will lay the foundation for NUMA support where possible. An example of this is limiting TLB shootdowns to processors that actually have or had TLBs belonging to the PMAP loaded. Of course, without NUMA hardware the implementation of NUMA support is quite limited.
Distributed package builds are currently done using a set of home-grown shell scripts for managing, scheduling and dispatching of package builds on the client machines. This has been sufficient for our needs in the past, but has a number of significant shortcomings that limit future growth. I am rewriting the package build scripts to work on top of Sun GridEngine (ports/sysutils/sge), as a client application of a "FreeBSD package grid". Some of the design goals for the new system are:
The "geomification" of vinum has made some progress. I now have all basic setups working (concatenated plexes, striped plexes, RAID5 plexes, and RAID1), but I still have to implement correct error handling and status change handling.
Still missing is a userland tool, so currently you still have to use "old-style" vinum to configure your setup.
NanoBSD, src/tools/tools/nanobsd, is a tool for stuffing FreeBSD onto small disk media (like CompactFlash) for embedded applications. The disk image is built with three partitions, two for software images and one for configuration files. Having two software partitions means that new software can be uploaded to the non-active partition while running off the active partition.
The first really public version has been committed and many suggestions and offers of patches have started pouring in.
The sources were imported from OpenBSD 3.4R and patched with diffs obtained from the port. Since March the 8th it is linked to the build and install. There is some more work to be done in order make pf a home inside the tree, but the biggest hunk of work was lifted during the past two month.
OpenBSD 3.5 is scheduled for early May, so we might see an update before 5.3R. Work towards integration of the - often requested - ALTQ framework is in progress also, though it is not yet clear how well it goes along with the ongoing work towards a giant free net stack.
Development goes reasonably fast, right now it boots single user. It is still very simics-centric, and it deserves a huge cleanup and a few bug fixes, but there's already a decent amount of code to work with, mostly taken from NetBSD. I now plan to work on real hardware support (as soon as I can get some), to get the missing userland bits (mainly rtld and the pthread libs) so that I can build a full world.
Not much has changed since last report was submitted. The read-only access XFS volumes is quite stable now. The work is underway to rewrite xfs_buf layer to minimize local changes intrusiveness. Initial attempt to make XFS code to compile and run on amd64 is in progress too.
We really need a care-taker for our userland tools.
If nothing bad happened, the icc patches got committed around the date of the deadline for submissions of this report. Please search the archives of -current and/or cvs-all for more information.
The next steps in this project are to
Not much to report. Bluetooth Service Discovery Procotol daemon sdpd was integrated with existing Bluetooth utilities. From now on users should not use GNU sdpd (Linux BlueZ port).
Bluetooth HID profile implementation is almost complete. Thanks to Matt Peterson < matt at peterson dot org > for giving me Bluetooth keyboard and mouse for development.
It has been a year since our last status report, but we haven't slowed down. Since the last report, Alexander Nedotsukov (bland) and Pav Lucistnik (pav) have joined the FreeBSD GNOME team. GNOME 2.4 was released back in September 2003, followed by 2.4.1 and 2.4.2. We are actively working on getting GNOME 2.6.0 out the door at the end of March. GNOME 2.6 Beta releases can be obtained via the project URL above.
To help make GNOME 2.6.0 our best release to date, we have
created a script to automate the upgrade from GNOME 2.4. We
also have a new GNOME
-
Included in the release of GNOME 2.6 is GTK+ 2.4, the next installment in the GTK+ 2 series. Because GTK+ 2 has become very stable over the past few years, the FreeBSD GNOME Team is pushing for GTK+ 2 support to be included by default in all applications that support it. This has already been done with Mozilla, Firefox, and Thunderbird. A complete GNOME Desktop and application environment can already be built using only GTK+ 2. The ultimate goal is to phase GTK+ 1 out of the ports tree.
This project is aimed at converting the FreeBSD network stack from running under the single Giant kernel lock to permitting it to run in a fully parallel manner on multiple CPUs (i.e., a fully threaded network stack). This will improve performance/latency through reentrancy and preemption on single-processor machines, and also on multi-processor machines by permitting real parallelism in the processing of network traffic. As of FreeBSD 5.2, it was possible to run low level network functions, as well as the IP filtering and forwarding plane, without the Giant lock, as well as "process to completion" in the interrupt handler.
Work continues to improve the maturity and completeness of the locking (and performance) of the network stack for 5.3. The network stack locking development branch has been updated cothe latest CVS HEAD, tracking a variety of FreeBSD changes, including tracking and driving changes in the interface and device cloning APIs, push-down and fixes to locking in the Berkeley Packet Filter, consistency improvements in allocation flags for network objects, diagnosis of excessive acquisition of Giant in various system callouts and timeouts, removal of Giant from several system callouts, "const"-ification of a number of global variables in the network stack (IPv4, IPv6, elsewhere) as part of ananalysis of locking requirements, fine-grain locking of a number of pseudo-interfaces (disc, loopback, faith, stf, gif, tap, tun), IP encapsulation and tunneling, initial review and locking of parts of PPP and SLIP, experimentation with PCB assertions on IPv6, additional socket locking assertions, graphing of the FreeBSD sockets layer to support locking analysis, merging of theMT_TAG to m_tag conversion to improve the ability to queue packets, moving of the debug.mpsafenet tunable to controlling Giant over the forwarding plane to Giant over the entire stack("dual-mode" to support non-MPSAFE protocols), adaption of existing network lock assertions to also assert Giant when running non-MPSAFE, analysis of high cost of select() locking, improved locking and synchronization annotations, TCP callouts run MPSAFE, logtimeout() runs MPSAFE, uma_timeout() runs MPSAFE, callout sampling instrumentation, loadav() runs MPSAFE, AppleTalk locking begun: AARP locked down and DDP analysis, rawcb list locked, locking analysis of mrouter and IP ID code, IGMP locked, IPv6 analysis begun, IPX/SPX analysis begun, PPP timeouts converted to callouts, Netgraph analysis begun. Many of these changes have not yet been merged to the main FreeBSDtree, but this is a work in progress.
In related work on Pipe IPC (not quite network stack locking), substantial time was invested in diagnosing an increase in the cost of pipe allocation since FreeBSD 4.x, as well as coalescing the several allocations needed to create a pipe, as well as moving to slab allocation so as to amortize the cost of pipe initialization. Future work here will include caching the VM structures supporting pipe buffers.
Recent contributors include Robert Watson, Sam Leffler, MaxLaier, Maurycy Pawlowski-Wieronski, Brooks Davis, and many others who are omitted here only by accident.
This bi-monthly report covers development activities on the FreeBSD Project for December 2001 and January 2002. A variety of accomplishments have been made over the last couple of months, including strong progress relating to the KSE project, which brings Scheduler Activations to the FreeBSD kernel, as well as less visible infrastructure projects such as improvements to the mount interface, PAM integration work, and translation efforts. Shortly following the deadline for this status report, the BSD Conference and FreeBSD Developer Summit were held, and will be covered in the next bi-monthly report at the end of March. Plans are already under way for the USENIX Annual Technical Conference in Monterey, CA, later this year, and all and sundry are encouraged to attend to get further insight in FreeBSD development.
Robert Watson
I've been working to integrate recent improvements in the NetBSD usb stack to FreeBSD -current. Both NetBSD and OpenBSD currently share the same source, as FreeBSD did too at once point before it diverged. The goal is to get back to that state, but there are many improvements on both sides that need to be merged before this is complete.
I'm currently looking for someone to help maintain usb in -stable. Please let me know if you're interested.
Patches for cp(1), ls(1), and mv(1) to bring in POSIX.1e-compliant Access Control List support have been updated to patch against builds of -CURRENT. Other system utilities are currently being evaluated for ACL support including install(1) (patch available) and mtree(8). Work is in progress to verify the native getfacl(1), setfacl(1), and other utilities build and work correctly on other ACL-enabled systems (e.g. Linux w/ACL patches) and to help verify POSIX-compliance of the continuing TrustedBSD work along with other systems. Finally, experimental Perl and PHP modules are available allowing limited access to native ACLs for languages other than C.
The project is making progress. The goal is to design and implement Host Controller Interface (HCI) and Link Layer Control and Adaptation Protocol (L2CAP) layers using Netgraph framework. More distant goal is to write support for Service Discovery Protocol (SDP) and RFCOMM protocol (Serial port emulation over Bluetooth link) . All information was obtained from Bluetooth Specification Book v1.1.
Project status: In progress. 1) Design: mostly complete, there are some minor issues to be resolved. 2) Implementation: Kernel - HCI and L2CAP Netgraph nodes have been implemented; 3) User space (API, library, utilities) - in progress. 4) Testing: In progress. I do not have real Bluetooth hardware at this point, so i wrote some tools that allow me to test the code. Some of them will be used as foundation for future user space utilities.
Issues: 1) Bluetooth hardware; I do not have real Bluetooth hardware, so if people can donate hardware/specs it would be great. I promise to write all required drivers and make them available. I also promise to return hardware/specs on first request. 2) Project name; I would like to see the name that reflects the following: it is a Bluetooth stack, implementation is for FreeBSD and implementation is based on Netgraph framework
This project is now finally underway, thanks to DARPA and NAI getting a sponsorship lined up. The infrastructure code and data structures are currently taking form inside a userland simulation harness. Basic MBR and BSD methods have been written and device attach/taste/dettach algorithms been implemented and validated.
I've update OS of buildboxes to the latest FreeBSD 5-current and 4-stable. Everything goes fine. From January 2002, I've started a webzine, SNAPSHOTS Notes (only Japanese version is available). SNAPSHOTs Notes pickups tips and information especially for the people living with FreeBSD 5-current/4-stable. Article or idea for SNAPSHOTs notes are always welcome (you don't need to write in Japanese :-).
Robert Watson created the TrustedBSD audit perforce tree, which is a branch from the TrustedBSD base tree, in order to start pushing development efforts towards using a revision control system. Andrew Reiter started to merge in some framework related code for generation of audit records, enqueueing writes, and handling data writing. There is a great deal of work to be done with updates and discussion on the trustedbsd-discuss@TrustedBSD.org mailing list.
The KSE project (an attempt to support scalable thread in FreeBSD using kernel support), has reached What I call "milestone 3". At this milestone it is possible to run a multithreaded program on a single CPU but with full concurrency of threads on that CPU. In other words the kernel supports the fact that one thread can block by allowing another thread to run in its place. A test program that demonstrates this is available at the above website.
Milestone 4 will be to allow threads from the same program to run on multiple CPUs but may require more input from the SMPng project. I am at the moment (Feb 6) getting ready to commit a first set of changes for milestone 3, that have no real effect but serve to drastically reduce the complexity of the remaining diff so that others can read it more easily. After changes to libkvm to support this diff have been added it should be possible to run 'ps' and look at multiple threads in a treaded process. I will be demonstrating KSE/M3 at BSDcon.
The Netgraph ATM package has been split into a number of smaller packages: bsnmp is a general-purpose SNMP daemon with support for loadable modules. Two modules come with it: one implementing the standard network-interface and IP related parts of MIB-2 and one for interfacing other modules to the NetGraph sub-system. ngatmbase contains the drivers for the ATM hardware, the ng_atm netgraph type and a few test tools. This package allows one to use ATM PVCs. It should be possible, for example, to do PPP over ATM with this package. Both bsnmp and ngatmbase are available in version 1.0 under the link above. Two other modules will be released in February: ngatmsig containing the UNI-4.0 signalling stack as netgraph nodes and ngatmip containing CLIP and LANE-2.0.
A significant amount of progress was made in December and January, particularly in the area of utility conformance. Several utilities were updated to conform to SUSv3, they include: at(1), mailx(1), pwd(1), split(1), and uudecode(1). Several patches have been submitted to increase conformance in other utilities, they include: fold(1), patch(1), m4(1), nice(1), pr(1), renice(1), wc(1), and xargs(1). These are in the process of being reviewed and committed. Two new utilities have been written, specifically pathchk(1) and tabs(1). These are also being reviewed and will be committed shortly.
A patch which implements most of the requirements of scanf(3) is being reviewed and is expected to be committed shortly. This will allow us to MFC a number of new functions and headers. Additionally, work has started on wide string and complex number support.
For 4.5-RELEASE, port ja-man-doc-4.5.tgz is in sync with base system except for OpenSSH pages (OpenSSH 2.3 based instead of 2.9) and perl5 pages (jpman project do not maintain). Section 3 updating has 55% finished.
OKAZAKI Tetsurou has incorporated changes on base system's groff into port japanese/groff. MORI Kouji has fixed two bugs of port japanese/man.
The KAME project is currently focusing on the scoped addressing architecture, the advanced API implementation, NATPT and the mobile ipv6 implementation. Though these stuffs are not stable enough to be merge into the FreeBSD tree, you can get and try them from the above URL.
The FreeBSD in Bulgarian project aims to bring a more comfortable working environment to Bulgarian users of the FreeBSD OS. This includes, but is not limited to, font, keymap and locale support, translation of the FreeBSD documentation into Bulgarian, local user groups and various forms of on-line help channels and discussion forums to help Bulgarians adopt and use FreeBSD.
A guide for using FreeBSD with Bulgarian settings has been put up on the project's website. The CVS repository will be made public shortly, linked to on the URL's above.
An independent project for making FreeBSD easier to use by
- Bulgarians has appeared,
The past two months have been an exciting time in the FreeBSD Java Project with the signing of a license between the FreeBSD Foundation and Sun allowing us access to updated JDK source code and the Java Compatibility Kit (JCK). This license will also allow the project to release a binary version of both the JDK and JRE once JCK testing is complete. Work on this testing is under way with the project hopeful of being able to make a binary release in the not too distant future.
In lieu of the binary release which was hoped for with FreeBSD 4.5 the project will release an updated source patchset this weekend. This patchset will feature further work on the FreeBSD "native" threads subsystem from Bill Huey. Also, thanks to hard work by Joe Kelsey and Fuyuhiko Maruyama, the patchset will for the first time feature a working Java browser plugin!
Extending camcontrol's page definition file format to include both modepage and logpage definitions; adding support to camcontrol to query and reset log page parameters. Consideration is being made to possibly include support for diagnostic and vital product data pages, but that is outside the current project scope. New page definition file format includes capability to conditionally include page definitions based on SCSI INQUIRY results allowing vendor-specific pages to be described also. Approximately 90% complete.
OpenPAM, a new library intended to replace Linux-PAM in FreeBSD, has been written and is undergoing integration testing. It is available for download from the URL listed above.
In addition to this, a couple of new modules have been written (pam_lastlog(8), pam_login_access(8)), and the pam_unix(8) module has been extended to perform most of the tasks normally performed by login(1), which is now fully PAMified.
The PAM FDP article has been put on hold until OpenPAM replaces Linux-PAM in CVS, to avoid wasting effort on soon-to-be obsolete documentation.
Substantial progress has been made towards a working MAC implementation. The focus over the last two months has been moving from a hard-coded series of MAC policies to a more flexible implementation. A pluggable policy framework has been created (and is still under development), supporting Biba, MLS, TE, a "BSD Extended" model, and a sample mac_none module. Some modules must be compiled in or loaded prior to boot; others may be introduced at run-time. Support for networking has improved, with improved handling of IP fragmentation in IPv4, support for various pseudo-interfaces such as if_tun and if_tap, improved integration into userland, NFS-related fixes, moving the VFS enforcement out of individual filesystems, support for a 'multilevel' mount flag, support for explicit labeling in procfs and devfs, addition of an 'extattrctl lsattr' argument to list EAs on a filesystem, support for label ranges in the Biba and MAC policies, and much more.
Targets for the next two months include more universal enforcement of VFS-related calls, improved support for alternative ABIs, improved flexibility of in-kernel subject and object labels, support for IPv6 and IPsec, and improved support for NFS serving.
Development continues in the FreeBSD Perforce repository, which may be accessed using cvsup.
Now that the patch has been mailed to the freebsd-arch@FreeBSD.org mailing list, and that there were no objections, the commit will happen soon. Poul is currently testing it in his own tree. After it has been committed, it will be time to modify the filesystems in the tree to use VFS_NMOUNT instead of VFS_MOUNT. Mount(8) will also need some modifications. Some new manpages -- nmount(2) and kernel_vmount(9) -- are being created in the meantime.
Alfred Perlstein committed file descriptor locking code which was definitely a good push towards trying to lock down some important pieces of global data. Peter Wemm has made progress on pmap cleanups for x86 SMP TLB shootdowns. Matt Dillon and John Baldwin have made progress on getting patches done for moving accesses to ucred's out from under Giant's protection. John Baldwin has also made some commits in order to get the alpha port's SMP working. Matt Dillon has plans for hunting down fileops locking issues in order to continue his previous Giant pushdown work.
This report covers FreeBSD development activities from February, 2002 through April, 2002. It's been a busy few months -- BSDCon in San Francisco, the FreeBSD Developer Summit, a first development preview of 5.0-CURRENT, not to mention lots of progress on the 5.0 feature set (SMPng, sparc64, GEOM, ... the list goes on).
In the next two months, the USENIX ATC occurs (highly recommended event for both developers and users), and a number of new software components will hit the tree, including UFS2 and the TrustedBSD MAC framework. We'll also complete the elections for the FreeBSD Core Team, and should have the next Core Team online by the time the next report rolls around. Stay tuned for more!
Robert Watson
Packages are built from the FreeBSD Ports Collection on a cluster of i386 and alpha machines using scripts available in /usr/ports/Tools/portbuild/. Over the past few months I have been cleaning up and extending these scripts to improve efficiency and allow for greater flexibility in how package builds are performed. Major improvements so far have been: cleaning up and modularizing the scripts to avoid code duplication and reduce the need for ongoing maintenance; optimizing the build process and making it much more robust against client machine failure; and allowing package builds to be restarted if they are interrupted. The i386 package cluster is currently running FreeBSD 5.0-CURRENT, and it has proven to be a useful testing ground for exposing kernel bugs, especially those which only manifest under system load.
Future plans include the ability to perform incremental package rebuilds which only build packages that have changed since the last run. This will allow packages to be made available on the FTP site within an hour or two of the CVS commit to the ports collection. We also hope to set up a sparc64 package cluster in the near future, but this is contingent on suitable hardware.
FreeBSD's new kernel memory allocator has been committed to 5.0. UMA is a slabs derived allocator that supports memory reclaiming, object caching, type stable storage, and per CPU free lists for optimal SMP performance. It has both a malloc(9) interface and a zone style interface for specific object types. uma(9) will be available shortly.
Read-only support for UDF filesystems was checked into the 5-CURRENT branch in April. Backporting for 4-STABLE is being conducted by Jeroen. The next phase is to write a newfs_udf, then move on to adding write support to the filesystem. I'm still looking for a volunteer to handle read and write support for write-once media (e.g. CD-R).
I have released a new zero copy sockets snapshot, the first since November, 2000. The code has been ported up to the latest -current, and the jumbo code now has mutex protection. Also, zero copy send and receive can be selectively turned on and off via sysctl to make it easier to compare performance with and without zero copy. Reviews and comments are welcome.
I'm slowly making progress. The second engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20020506.tar.gz
This release includes support for H4 UART transport layer, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices.
I'm currently working on RFCOMM protocol implementation (Serial port emulation over Bluetooth link). My next goal is to port Service Discovery Protocol (SDP) implementation from BlueZ (http://bluez.sf.net). I'm also thinking about adding USB device support (as soon as i find/buy hardware).
Issues: 1) Bluetooth hardware; I have couple PC-CARDs that i use for development and testing purposes, but i'd love to have more. 2) Time; My regular day job kicked in, so i will be spending more time doing stuff i'm getting paid for.
Since the last status report, two developers working on utility conformance were given commit access to the FreeBSD CVS repository to help expedite development. As a result, the following utilities have been brought up to conformance, they include: csplit(1), env(1), expr(1), fold(1), join(1), m4(1), mesg(1), paste(1), patch(1), pr(1), uuencode(1), uuexpand(1), and xargs(1). The printf(1) utility was brought up to conformance with the 1992 edition of POSIX.2, with further development planned.
On the header front, much progress has been made. Specifically, infrastructure to control visibility of components of a header, based on the standard requested by an application, has been added to <sys/cdefs.h>. Some work has been completed on renovating the way types are defined. This has lead to the creation of <sys/_types.h>. Further improvements such as the merger of <machine/ansi.h> and <machine/types.h> are planned. Additionally, the headers: <strings.h>, <string.h>, and <sys/un.h> have been made to conform to POSIX.1-2001.
On the API front, scanf(3) has received support for 5 new length modifiers (hh, j, ll, t, and z). A patch to implement two additional conversion specifiers (j and z) has been developed for printf(9) and is expected to be committed soon.
In other news, the project's web site has been moved to the main FreeBSD site. It is now available at the URL at the top of this status report. Please update your bookmarks.
Version 1.1 for FreeBSD-current is now available. It includes the SNMP-daemon package bsnmp, the driver package ngatmbase, the UNI4.0 signaling package ngatmsig and the network emulation package ngatmnet. NgAtm allows both to build applications running directly on top of ATM and to use ATM-Forum LAN emulation to use IP over ATM. Currently we are working on a simple switch module, that implements the network side signaling and ILMI as well as simple routing and call admission control.
The GNOME project has seen quite a few changes lately. For one, the author of this update has recently been given "The Bit." Joe Marcus Clarke now has CVS access, and is working primarily on the GNOME project. Joe has been closing a good deal of GNOME PRs, as well as patching some of the existing GNOME 1.4 components.
The GNOME 2 porting effort continues on. We have completed porting of the GNOME 2.0 API, and are 75% complete on porting the full GNOME 2.0 desktop. When complete, GNOME 1.4 and GNOME 2.0 will be co-resident in the ports tree. Both APIs can be installed concurrently in the same PREFIX, but the respective desktops will remain mutually independent. Maxim Sobolev is working on adapting bsd.gnome.mk to handle both versions of the desktop in an elegant fashion.
Not to be left out, the existing GNOME 1.4 components have received numerous updates to keep them in sync with the stable distfiles on gnome.org. We have seen many "1.0" milestone releases including the most recent AbiWord 1.0.0. In the next few weeks, we will be making sure all the GNOME 1.4 components build correct packages on bento so that GNOME 1.4 will be on the 4.6-RELEASE CD.
FreeBSD/KGI started last year after the port of GGI to VGL. KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with access to hardware graphic resources (dma, irqs, mmio). KGI is already available under Linux as a separate project. The FreeBSD/KGI project aims at integrating KGI in the FreeBSD kernel. Mostly a port for now, but optimized for FreeBSD in the future. Currently FreeBSD/KGI is under development and the code is only available for reading, compiling but not running. More interesting are design hints found at the project URL.
We now have a loadable mfsroot floppy. It contains just the diskeditor (which is really a disk partitioner) which has been enhanced and is probably in its final form. It's been geared towards making the newfs(1) and mount(1) steps separate dialogs, so it reduceed its complexity. A basic fstab class has been implemented to manipulate /etc/fstab and mountpoint. This might find a use outside libh, by the way. Libh package format is still incomplete and somehow buggy, so it's my next target.
There is a API documentation effort underway with the help of doxygen(1), so there's now more documentation for people that want to get started with libh.
All this lead me to prepare the release of another alpha preview of libh that will shortly be available in the ports collection (0.2.2). Also, a new committer (okumoto) has joined the project (as well as I) and he is currently working on cleaning up the build system. It's been a few months without news, so this probably seemed a bit long, but don't worry, we still need your help to really get this going!
There are several new topics, including: Source Code Tour is now separated into kernel part and userland part, yet another snapshots from RELENG_4_x branch (currently 4.5-RELEASE-p4), add several packages including XFree86 4.x to installation CD-ROM, new cdboot-only ISO image, fix breakage of duplex.iso, etc. See also the project webpage for more detail. Also, I have a plan to add FreeBSD/alpha distribution to this project -- stay tuned.
KAME Project has been extended until March 2004, and we decided the project roadmap for these two years. The first one year is for implementation, and the remaining year is for feedback of our results into other BSD projects (please refer to the above URL for further detail). Great change is lack of NAT-PT support due to a lack of human resource, although KAME snap still contains it as it is.
SUZUKI Shinsuke (suz@kame.net) has begun working for KAME and FreeBSD merge task in cooperation with Umemoto-san (ume@FreeBSD.org). Some of KAME stuff (critical bug fix, newest ports for pim6sd and racoon, etc) has been merged into 4-stable in this April.
Over the past couple of months, progress has pretty much stopped until very recently. The past few changes to the audit code were update the usage of zones to UMA zones, cleanup some old cruft, and start toying with the idea of having an audit write thread implemented as an ithd. The next step is to decide two realistic approaches to the where the records will be dumped -- whether that is to a local disk or fed up to userland and then dealt with. After that, the goal will be to expand the number of events that are being audited, while also working in some performance testing procedures. I will be posting to trustedbsd-audit about the recent changes shortly.
Over the last three months, there has been a lot of activity in the TrustedBSD MAC tree. An initial commit of the SEBSD code (NSA FLASK and SELinux implementation) was made; many MAC policies previously linked directly to the kernel via kernel options were moved to kernel modules; the flexibility of the framework was improved relating to the life cycle of object labels; additional labeling and access control hooks were introduced; new policies were introduced to demonstrate the flexibility of the framework (including a cleanup of inter-process authorization, additional VFS hooks, improved support for multilabel filesystems, network booting, IPv6, IPsec, support for "peer" labels on stream sockets). Current modules include Biba integrity policy, MLS confidentiality policy, Type Enforcement, "BSD Extended" (permitting firewall-like rulesets for filesystem protection), "ifoff" (limit interface communication by policy), mac_seeotheruids (limit visibility of processes/etc of other users), "babyaudit" (a simple audit implementation), and SEBSD (FLASK/SELinux port).
Over the next month, a final move to completely dynamic labeling will be made, permitting policies to introduce new state relating to process credentials, vnodes, sockets, mounts, interfaces, and mbufs at run-time, allowing a broad range of flexible label-driven policies to be developed. In addition, application APIs will be re-designed and re-implemented so as to better support a fully dynamic policy framework. We plan to make an initial prototype patchset available for review in June, with the intent of committing that patchset in mid-June.
Updated prototype code may be retrieved from the TrustedBSD CVS trees on cvsup10.FreeBSD.org.
The painful parts are now completed, with all authentication- related utilities converted to PAM (except for those cases where it doesn't make sense, like Kerberos- or OPIE-specific commands). OpenPAM is complete (except for a few missing man pages) and seems to work well.
For more details, see the activity reports linked to above.
OpenSSH has been upgraded to 3.1, and the kinks seem to have been worked out by now. OpenSSH will now use PAM for both ssh1 and ssh2 authentication.
The KSE project had floundered due to lack of development time for awhile, but has been picked up recently by Jonathan Mini. Currently, the main focus is to prepare the "milestone 3" code for inclusion into -CURRENT.
The project is still working towards "milestone 4" (allowing threads from the same process to run on multiple CPUs), which should be significantly easier now due to work done by the SMPng project over the past several months.
Help could be used in several areas of the project, especially with porting the libc_r (pthreads) library to KSE's threading model.
NEWCARD support tried to merge CardBus functions with PCI functions, but that failed to properly route interrupts. A branch for the merge was created and will be merged into the main line at a later date. Too many other things going on in my life to make much progress.
Work on the host access point support for the Prism2 and Prism2.5 based wireless cards has been integrated into the kernel. This work is largely based on Thomas Skibo's initial implementation.
Continued bug fixing and hardening for this last few months.
Future work will include making target mode work correctly and fast.
The LSI-Logic chipset's MPT Fusion driver is also being evaluated.
The FreeBSD MTRR code has been made more robust against unexpected values sometimes found in the Athlon's Memory Type Range Registers. Problems with these values had prevented XFree 4.2 running on some motherboards. Experimentation indicates that these undocumented values may control the mapping of BIOS/ROMs or have something to do with SMM. If anyone can provide details of what these values mean, can they please let me know, so the MTRR code can be completed.
IPMI Tools for FreeBSD is a collection of C and Python applications and modules for exploring the information available via the Intelligent Platform Management Interface (IPMI), as implemented on server motherboards by Intel and HP. IPMI is an open standard with patent protection for adopters which defines standard interfaces to on-board management hardware. The management hardware consists of a CPU, sensors such as temperature probes and fan speeds, and repositories such as the System Event Log and Field-Replaceable Unit (FRU) inventory, and other system information.
A basic set of tools was recently made available which uses the KCS and SMIC system interfaces to retrieve the System Event Log, FRU repository, and system sensors. Additional features are currently under research. Suggestions for additional features and programs are greatly appreciated.
The PowerPC port is moving ahead. It can now mount a root file system and exec init, but fails when trying to map init's text segment in. I'm hoping to have it starting my fake "Hello, world!" init soon, after which I plan to try and get some libc bits in place so that I can build /bin and /sbin and try to get to actual single-user.
4.5-RELEASE Japanese manpage package, ja-man-doc-4.5.tgz, once published with OpenSSH 2.3 (as reported by previous status report) on January 31, is replaced with new package with OpenSSH 2.9 based manpages on March 3. Since then, we have been updating Japanese manpages for 4.6-RELEASE. For new translation and massive update, we have been making a lot of effort.
Continuing section 3 updating has 73% finished.
The GEOM code has gotten so far that it beats our current code in some areas while still lacking in others. Work continues on a generalized interface for "magic data" (boot blocks, disklabels MBR's etc) manipulation from userland.
With GEOM enabled in the kernel any FreeBSD platform will now recognize PC style MBR's, i386 disklabels, alpha disklabels, PC98 extended MBRs and SUN/Solaris style disklabels.
Since the last progress report, the initialization code was much cleaned (thanks to NetBSD's acort32 port) and partial DDB support as been added. I'm now struggling to put the pmap module into a working state. The latest patch set only includes the initialization changes. I did some tries to get what I had so far working on my iPAQ without much successes (downloading a kernel over a serial link is way too painful). If anyone has had success in getting any iPAQ to work as a USB storage device under *BSD please contact me.
I've been mentoring someone on locking up the protocol control blocks in the networking stack. She has already finished TCP and UDP and I'm currently reviewing the patch with her and going over some networking lock order issues. Locking up raw protocol interface control blocks follows next.
Support for stf(4), faith(4), and loopback interfaces has been committed to current. The stf and faith support has been MFC'd. In current the API has changed to move unit allocation into the generic cloning code reducing the amount of support code required in each driver. Code improvements to increase our API compatibility with NetBSD will be committed soon along with cloning support for discard interfaces and ppp(4) interfaces.
Thanks to
IA64 has had a busy few months. Aside from gcc, we are now fully self hosting on IA64. Doug Rabson has performed his magic and implemented the execution of 32 bit i386 application binaries although more work remains to be done to make ld-elf.so.1 happy with the different underlying page size. We have been using the i386 perforce binary to do actual development work and submit from the ia64 systems themselves. Marcel Moolenaar has been working on SMP and machine-check support. We have been running SMP kernels amazingly reliably on our development boxes for quite some time now. syscons is now functional. We have produced a self-booting run-root-on-cdrom ISO image (idea taken from the sparc64 folks) that has been used to manually self install an IA64 system from a blank disk. Aside from a few minor loose ends we now have complete 'make world' functionality. sysinstall works on ia64. We plan on producing a semi-respectable boot/install cdrom image shortly.
As of Thur May 9th, 2002 FreeBSD 5-CURRENT is now using a GCC 3.1 prerelease snapshot as the system C compiler. At this time of cutting over, the compiler is working well on i386, Alpha, Sparc64, and IA-64 for building world. There is a known problem with our atomic ops on Alpha that prevents a GCC 3.1 built kernel from booting.
Currently the C++ support libraries (libstdc++, et.al.) does not build and thus prevents the system C++ compiler from being used.
The release engineering team released FreeBSD
The release engineering team released FreeBSD 5.0-DP1 on 8 April 2002. This Developer Preview gives developers and other interested parties a chance to help test some of the new features to appear in 5.0-RELEASE. This distribution has known bugs and areas of instability, and should only be used for (non-production) testing and development.
The next releases of FreeBSD will be 4.6-RELEASE (scheduled for 1 June 2002) and 5.0-DP2 (scheduled for 25 June 2002). Information on the release schedules and more can be found on the team's new area on the FreeBSD Web site (see the URL above).
Finally, the team has gained two new members: Brian Somers and Bruce A. Mah.
libradius now supports RADIUS vendor attribute extensions and user-ppp is now capable of doing MS-CHAP authentication via a RADIUS server. A new net/freeradius port has been created for support of MS-CHAP in a RADIUS server.
MS-CHAPv2 support will be added soon.
The work is sponsored by Monzoon.
Mike Makonnen has done quite a bit of excellent work on porting the scripts from FreeBSD into the NetBSD framework. The next step seems to be to try to reduce the amount of diffs between our implementation and the original set from NetBSD.
The SMPng project has been picking up steam in the last few months thankfully. In February, Seigo Tanimura-san committed the first round of process group and session locking. Alfred Perlstein also added locking to most of the pipe implementation. In March, Alfred fixed several problems with the locking for select() and pushed down Giant some in several system calls. Andrew Reiter added locking for kernel module metadata, and Jeff Roberson wrote a new SMP-friendly slab allocator to replace both the zone allocator and the in-kernel malloc(). The use of the critical section API was cleaned up to not be abused as replacements for disabling and enabling interrupts. Also, Matt Dillon optimized the MD portion of the critical section code on the i386 architecture. Several other subsystems were also locked in April as well. See the SMPng website and todo list for more details.
Some of the current works in progress include locking for the kernel linker by Andrew Reiter and light-weight interrupt threads for the i386 by Bosko Milekic. Seigo Tanimura-san, Alfred Perlstein, and Jeffrey Hsu are also working on locking down various pieces of the networking stack. Alan Cox has started working on fixing the existing locking in the VM subsystem and moving bits of it out from under Giant. John Baldwin has written an implementation of turnstiles as well as adaptive mutexes in the jhb_lock Perforce branch. The adaptive mutexes appear to be stable on i386, alpha, and sparc64, but the turnstile code still contains several tricky lock order reversals. John also plans to commit the p_canfoo() API change to use td_ucred in the very near future and then finish the task of making ktrace(4) use a worker thread.
The patch for the new mount API has now been committed to the tree. Several filesystems also have been converted to this new mount API, namely procfs, linprocfs, fdescfs and devfs. I'm working on converting more filesystems to nmount, and actually already have UFS done. It has not been committed yet to avoid conflicting with the UFS2 work, but it should hit the tree soon. Manpages are still missing at the moment because I had to modify the API slightly. I hope to have them done soon now.
The second FreeBSD Developer Summit, held following the BSD Conference in San Francisco in February, was a great success. Around 40 developers attended in person, another five by phone, and many others by webcast. During a marathon-esque eight hour session, a variety of development topics were discussed, including adding inheritance to the KOBJ system, ports to new architectures, adaptations of the toolchain for new architectures, the GEOM extensible storage device framework, upcoming changes to the network stack, TrustedBSD features, KSE, SMPng, and the release engineering schedule. This event was sponsored by DARPA and NAI Labs, with webcasting provided by Joe Karthauser, bandwidth provided by Yahoo!. Planning for future such events is now underway; a summary/transcript of discussion may be found at the URL above.
2004 started with another exciting two months for the project. FreeBSD 5.2 was released in early January and then quickly followed in February with the 5.2.1 bug-fix release. Looking forward, we are expecting a late-April release date for FreeBSD 4.10, and mid-summer date for FreeBSD 5.3. And don't forget to support the FreeBSD vendors and developers by buying a copy of the latest CD or DVD sets.
Thanks,
Scott Long
In the overall area of disk and device I/O, a significant milestone was reached with the implementation of proper reference counting on dev_t. We are now able to properly allocate and free dev_t. Cloning device drivers also had the job made easier for them with the addition of the unit number management routines.
It is not quite decided which will be the next step in the quest for a truly SMPng I/O subsystem, but a leading candidate is to implement the device-access vnode bypass to get more concurrency in the system: Instead of taking the tour through the vnodes for each i/o operation on a device we will go directly from the file descriptor layer to DEVFS/SPECFS. In addition to Giant-less disk I/O, this should enable us to pull the entire tty subsystem and the PTY driver out from under Giant and we expect that to improve the "snappiness" of the system measurably.
The Dutch Documentation Project is a ongoing project in translating the handbook and other documentation to the dutch language. Currently there is 1 active person (me) translating the documentation. I am currently working on the handbook/basics section. But i can use some more hands, please drop me an email if you wish to help out so that the dutch translation will speed up and be ready in some time. Contact remko@elvandar.org for information.
I have been producing weekly summaries of commits and the surrounding discussions as reported on the cvs-src mailing list. These summaries are posted to -current on Sunday evenings and archived on the Web. The reception has been overwhelmingly good. As of the end of February, Polish translations are being produced by Lukasz Dudek and Szymon Roczniak; they are also planning to translate the older summaries.
libarchive, with complete documentation, has been committed to -CURRENT. bsdtar should follow soon. For a few months, gtar and bsdtar will both be available in the base system. Once bsdtar is in the tree, I hope to resume work on libpkg and my pkg_add rewrite.
Note that bsdtar is not an exact replacement for gtar: it does some things better (reads/writes standard formats, archive ACLs and file flags, detects format and compression automatically), some things worse (does not handle multi-volume archives or sparse files) and a few things just different (writes POSIX-format archives by default, not GNU-format). The command lines are sufficiently similar that most users should have no problems with the transition. However, people who rely on peculiar options or capabilities of gtar may have to look to ports.
The first actual feature related to the if_xname conversion was committed in early February. Network interfaces can now be renamed with "ifconfig <if> name <newname>".
Work is slowly progressing on a new network interface cloning API to enable interesting cloners like auto-configurating vlans. This work is taking place in the perforce repository under: //depot/user/brooks/xname/...
After a slow time at the end of last year due to a disk crash, the project is moving along rapidly. The loader is fully functional with Forth support. Syscons has been integrated. New Powerbook models are supported. Work is starting on a G5 port.
There's still lots to do, so as usual volunteers are most welcome.
The project is a joint effort of volunteers, which focus in the internationalization and localization of the FreeBSD Operating System and applications running on FreeBSD. All of the work resulted in this project will be contributed back to the FreeBSD project.
Thanks to many volunteers' help, by this time of writing, we have finished more than 60% of the translation of the FreeBSD Handbook. We plan to submit a preliminary translation of the FreeBSD website as well as the FreeBSD Handbook when most part of them were finished, which is expected to happen in a couple of months. The snapshot of the documentation translation effort could be accessed through the URL listed above.
The project also supported individual efforts on porting applications (especially software that supports Simplified and/or Traditional Chinese) to FreeBSD. We are also doing some research on making FreeBSD kernel and base system more i18n-aware.
The verify source reachability option for ipfw2 checks if the source IP address of a packet entering the machine is reachable at all. Thus if we can't send a packet back because we don't have a route back we don't have to forward it because two way communication isn't possible anyway. It is more than likely that such a packet is spoofed. This option is almost the same as what is known on Cisco IOS as "ip verify unicast source reachable-via [any|ifn]". Using this option only makes sense when you don't have a default route which naturally always matches. So this is useful for machines acting as routers with a default-free view of the entire Internet as common when running a BGP daemon (Zebra/Quagga or OpenBSD bgpd).
One useful way of enabling it globally on a router looks like this: ipfw add xxxx deny ip from any to any not versrcreach or for an individual interface only: ipfw add xxxx deny ip from any to any not versrcreach recv fxp0
The ARP IP address to MAC address mapping does not belong into the routing table (FIB) as it is currently done. This will move it to its own hash based structure which will be instantiated per each 802.1 broadcast domain. With this change it is possible to have more than one interface in the same IP subnet and layer 2 broadcast domain. The ARP handling and the routing table will be quite a bit simplified afterwards. As an additional benefit full MAC address based accosting will be provided. Work on this project is already in progress.
The current TCP send and receive buffers are static and set to a conservative value to preserve kernel memory. This is sub-optimal for connections with a high bandwidth*delay product because the size of the TCP send buffer determines how big the send window can get. For high bandwidth trans-continental links this seriously limits the maximum transfer speed per TCP connection. For example a 170ms RTT and a 32kB send buffer limit the speed to approximately 1.5Mbit per second even thought you might have a 10Mbit pipe.
This project makes the TCP send buffer to automatically adapt to the optimal buffer size for maximal link usage. In the case above this would be a buffer of approximately 220kB. The main challenge is to have a stable and reliable measurement of the link parameters and manage the kernel memory properly and in a fair way. We don't want to have a few connections to monopolize all available socket buffer space and many edge cases have to be considered. The first implementation will be tuned conservatively but even that will provide significantly better performance than the static buffers currently. Work on this project is already in progress.
The TCP performance test and qualification testbed is an automated environment that simulates various common and uncommon end-to-end network and link characteristics such as delay, bandwidth limitations, congestion, packet drops, packet corruption and out of order arrival. The testbed automatically steps through all link types and tests various TCP optimizations and parameter adjustments. In the end all data is graphically arranged and compared against standard behaviour and each other to judge the positive or negative effects of the modifications. Work on this project has just started and is based on FreeBSDs dummynet.
Thanks to the loan of a box by Will Andrews, the system has
been moved into production. The previous installation
at lonesome.com now refers you to the new system. As part of
the installation, a preliminary
-
The database is updated once per hour.
New reports available include ones about ports marked DEPRECATED, since that function has now been incorporated into bsd.port.mk. (The author hopes that this will allow the port deprecation process to be much more visible to the general FreeBSD user community.) In addition, a report for ports marked FORBIDDEN was added (the code was essentially the same).
The next topic of interest is to try to identify ports which are slave ports because the status of these ports is not currently being updated automatically. This problem also affects FreshPorts. PR ports/63683 is an attempt to address this problem. Also, preliminary work has been done on creating some graphs and charts for various statistics, and in creating a tool to browse port dependencies for the entire ports tree.
Some general observations about the trends in ports PRs can be made:
The FreeSBIE Project aims to develop a set of scripts that allow anyone to create their own FreeBSD Bootable Cdrom, with their own set of installed packages. The Project releases an ISO builded with FreeSBIE scripts, to show what they can do. On Sunday 29 February 2004, FreeSBIE 1.0 was released and it had a great success, as there were post on Slashdot.org, OSnews, DaemonNews and BSDForums. Thanks to the huge amount of feedback they got, FreeSBIE Developers are now developing new features such as support for archs different from i386. Website redesign is on the way too.
Move to Perforce is done. I spent some time on building a common compilation tree with Linux: until now drivers were build in a FreeBSD makefile tree, not compatible with Linux.
The next priorities are ANSI support and keymaps in the KGC Kernel Graphic Console system.
Work on the PMAP overhaul has been put into gear. A lot of issues will be addressed, including support for sparse physical memory and of course SMP. Performance will be addressed to the extend possible, but functionality has priority. The redesign will lay the foundation for NUMA support where possible. An example of this is limiting TLB shootdowns to processors that actually have or had TLBs belonging to the PMAP loaded. Of course, without NUMA hardware the implementation of NUMA support is quite limited.
Distributed package builds are currently done using a set of home-grown shell scripts for managing, scheduling and dispatching of package builds on the client machines. This has been sufficient for our needs in the past, but has a number of significant shortcomings that limit future growth. I am rewriting the package build scripts to work on top of Sun GridEngine (ports/sysutils/sge), as a client application of a "FreeBSD package grid". Some of the design goals for the new system are:
The "geomification" of vinum has made some progress. I now have all basic setups working (concatenated plexes, striped plexes, RAID5 plexes, and RAID1), but I still have to implement correct error handling and status change handling.
Still missing is a userland tool, so currently you still have to use "old-style" vinum to configure your setup.
NanoBSD, src/tools/tools/nanobsd, is a tool for stuffing FreeBSD onto small disk media (like CompactFlash) for embedded applications. The disk image is built with three partitions, two for software images and one for configuration files. Having two software partitions means that new software can be uploaded to the non-active partition while running off the active partition.
The first really public version has been committed and many suggestions and offers of patches have started pouring in.
The sources were imported from OpenBSD 3.4R and patched with diffs obtained from the port. Since March the 8th it is linked to the build and install. There is some more work to be done in order make pf a home inside the tree, but the biggest hunk of work was lifted during the past two month.
OpenBSD 3.5 is scheduled for early May, so we might see an update before 5.3R. Work towards integration of the - often requested - ALTQ framework is in progress also, though it is not yet clear how well it goes along with the ongoing work towards a giant free net stack.
Development goes reasonably fast, right now it boots single user. It is still very simics-centric, and it deserves a huge cleanup and a few bug fixes, but there's already a decent amount of code to work with, mostly taken from NetBSD. I now plan to work on real hardware support (as soon as I can get some), to get the missing userland bits (mainly rtld and the pthread libs) so that I can build a full world.
Not much has changed since last report was submitted. The read-only access XFS volumes is quite stable now. The work is underway to rewrite xfs_buf layer to minimize local changes intrusiveness. Initial attempt to make XFS code to compile and run on amd64 is in progress too.
We really need a care-taker for our userland tools.
If nothing bad happened, the icc patches got committed around the date of the deadline for submissions of this report. Please search the archives of -current and/or cvs-all for more information.
The next steps in this project are to
Not much to report. Bluetooth Service Discovery Procotol daemon sdpd was integrated with existing Bluetooth utilities. From now on users should not use GNU sdpd (Linux BlueZ port).
Bluetooth HID profile implementation is almost complete. Thanks to Matt Peterson < matt at peterson dot org > for giving me Bluetooth keyboard and mouse for development.
It has been a year since our last status report, but we haven't slowed down. Since the last report, Alexander Nedotsukov (bland) and Pav Lucistnik (pav) have joined the FreeBSD GNOME team. GNOME 2.4 was released back in September 2003, followed by 2.4.1 and 2.4.2. We are actively working on getting GNOME 2.6.0 out the door at the end of March. GNOME 2.6 Beta releases can be obtained via the project URL above.
To help make GNOME 2.6.0 our best release to date, we have
created a script to automate the upgrade from GNOME 2.4. We
also have a new GNOME
-
Included in the release of GNOME 2.6 is GTK+ 2.4, the next installment in the GTK+ 2 series. Because GTK+ 2 has become very stable over the past few years, the FreeBSD GNOME Team is pushing for GTK+ 2 support to be included by default in all applications that support it. This has already been done with Mozilla, Firefox, and Thunderbird. A complete GNOME Desktop and application environment can already be built using only GTK+ 2. The ultimate goal is to phase GTK+ 1 out of the ports tree.
This project is aimed at converting the FreeBSD network stack from running under the single Giant kernel lock to permitting it to run in a fully parallel manner on multiple CPUs (i.e., a fully threaded network stack). This will improve performance/latency through reentrancy and preemption on single-processor machines, and also on multi-processor machines by permitting real parallelism in the processing of network traffic. As of FreeBSD 5.2, it was possible to run low level network functions, as well as the IP filtering and forwarding plane, without the Giant lock, as well as "process to completion" in the interrupt handler.
Work continues to improve the maturity and completeness of the locking (and performance) of the network stack for 5.3. The network stack locking development branch has been updated cothe latest CVS HEAD, tracking a variety of FreeBSD changes, including tracking and driving changes in the interface and device cloning APIs, push-down and fixes to locking in the Berkeley Packet Filter, consistency improvements in allocation flags for network objects, diagnosis of excessive acquisition of Giant in various system callouts and timeouts, removal of Giant from several system callouts, "const"-ification of a number of global variables in the network stack (IPv4, IPv6, elsewhere) as part of ananalysis of locking requirements, fine-grain locking of a number of pseudo-interfaces (disc, loopback, faith, stf, gif, tap, tun), IP encapsulation and tunneling, initial review and locking of parts of PPP and SLIP, experimentation with PCB assertions on IPv6, additional socket locking assertions, graphing of the FreeBSD sockets layer to support locking analysis, merging of theMT_TAG to m_tag conversion to improve the ability to queue packets, moving of the debug.mpsafenet tunable to controlling Giant over the forwarding plane to Giant over the entire stack("dual-mode" to support non-MPSAFE protocols), adaption of existing network lock assertions to also assert Giant when running non-MPSAFE, analysis of high cost of select() locking, improved locking and synchronization annotations, TCP callouts run MPSAFE, logtimeout() runs MPSAFE, uma_timeout() runs MPSAFE, callout sampling instrumentation, loadav() runs MPSAFE, AppleTalk locking begun: AARP locked down and DDP analysis, rawcb list locked, locking analysis of mrouter and IP ID code, IGMP locked, IPv6 analysis begun, IPX/SPX analysis begun, PPP timeouts converted to callouts, Netgraph analysis begun. Many of these changes have not yet been merged to the main FreeBSDtree, but this is a work in progress.
In related work on Pipe IPC (not quite network stack locking), substantial time was invested in diagnosing an increase in the cost of pipe allocation since FreeBSD 4.x, as well as coalescing the several allocations needed to create a pipe, as well as moving to slab allocation so as to amortize the cost of pipe initialization. Future work here will include caching the VM structures supporting pipe buffers.
Recent contributors include Robert Watson, Sam Leffler, MaxLaier, Maurycy Pawlowski-Wieronski, Brooks Davis, and many others who are omitted here only by accident.
The FreeBSD Bi-monthly status reports are back! In this edition, we catch up on seven highly productive months and look forward to the end of 2003.
As always, the FreeBSD development crew has been hard at work. Support for the AMD64 platform quickly sprang up and is nearly complete. KSE has improved greatly since the 5.1 release and will soon become the default threading package in FreeBSD. Many other projects are in the works to improve performance, enhance the user experience, and expand FreeBSD into new areas. Take a look below at the impressive summary of work!
Scott Long, Robert Watson
Still in the planning stage. Working on creating an extensible interface that is usable for both userland and kernel implementations for device drivers. Deciding on how to interface userland implemented device drivers with applications.
KSE seems to be working well on x86, amd64, and ia64. The alpha userland bits are done, but a couple of functions are unimplemented in the kernel. For sparc64, the necessary functions are implemented in the kernel, but the userland context switching functions need more attention.
Since 5.1, efficient scope system threads (no upcalls when they block) have been implemented, and KSE based pthread library can have both POSIX scope process threads and scope system threads. It is also possible that KSE based pthread library can implement pthread both in 1:1 and M:N mode, I know Dan has such Makefile file patch for libkse not yet committed.
KSE program now can work under ULE scheduler, its efficient should be improved under the new scheduler in future. BSD scheduler is still the best scheduler for current KSE implement.
Much has happened since the last bi-monthly report, which was more than half a year ago. FreeBSD 5.0 and FreeBSD 5.1 have been released for example. With FreeBSD 5.2 approaching quickly, we're not going to look back too far when it comes to our achievements. There's too much ahead of us...
Two milestones have been reached after FreeBSD 5.1. The first is the ability to support both Intel and HP machines with sources in CVS. This due to a whole new driver for serial ports, or UARTs. Unfortunately this still implies that syscons is not configured. That's another task for another time, but keep an eye on KGI/FreeBSD... The second milestone is the completion of KSE support. Both M:N and 1:1 threading is functional on ia64 and the old libc_r library has been obsoleted. Testing has shown that KSE (i.e. M:N) may well become the default threading model. It's looking good.
The ABI hasn't changed after 5.1 and the expectation is that it won't change much. This means that we can think about becoming a tier 1 platform. This also means we need gdb(1) support. Work on it has been started but the road is bumpy and long. Kernel stability also has improved significantly and we typically have one kernel panic remaining: VM fault on no fault entry. This will be addressed with the long awaited PMAP overhaul (see below).
Most work for FreeBSD 5.2 will be "sharpening the saw". Get those loose ends tied. This is a slight change of plan made possible by a slip in the release schedule. The 5.2 release is not going to be the start of the -stable branch; it has been moved to 5.3. So, we use the extra time to prepare the ground for 5.3.
The planned PMAP overhaul will probably be finished after 5.2. This should address all known issues with SMP and fix those last panics. As a side-effect, major performance improvements can be expected. More news about this in the next status reports.
The following items are in progress in the Disk I/O area: Turn scsi_cd.c into a GEOM driver. (Patch out for review). Turn atapi-cd.c into a GEOM driver. Turn fd.c into a GEOM driver. Move softupdates and snapshot processing from SPECFS to UFS/FFS. Move userland access to device drivers out of vnodes.
Once these preliminaries are dealt with, scatter/gather and mapped/unmapped support will be added to struct bio/GEOM.
FreeBSD Update is a system for tracking the FreeBSD release (security) branches. In addition to being faster and more convenient than source updates, FreeBSD Update also requires less bandwidth and is more secure than source updates via CVSup. However, FreeBSD Update is limited; it can only update files which were installed from an official RELEASE image and not recompiled locally. Right now I'm publishing binary updates for 4.7-RELEASE and 4.8-RELEASE; since my only available box takes 3.5 hours to buildworld, I don't have enough resources to do any more than that.
In the near future, I'd like to: Find someone who is willing to donate a faster buildbox; start building updates for other releases (at a minimum, for all "supported" FreeBSD releases); add warnings if a file would have been updated but can't be updated because it was recompiled locally; add code to compare the local system against a list of "valid" MD5 hashes for intrusion detection purposes; and add support for cross-signing, whereby several machines could build updates independently to protect against buildbox compromise.
The project started this spring and released version 1.0 with a port installation (security/pf) in may 2003. Version 2.0 is on the doorstep as OpenBSD 3.4 will be released. Due to the porting efforts we were able to reveal some bugs in the OpenBSD code and provided locking for the PFIL_HOOKS, which we utilize. Tarball installation of a loadable kernel module for testing can be found on the project homepage, a patchset is in the making.
PF was started at OpenBSD as a substitute for ipfilter and provides the same function set. However, in the two years it exists now, it has gained many superior features that no other packet filter has. For a impression take a look at the pf FAQ.
We hope to be eventually integrated into the base system. Before that we have to resolve some issues with tcpdump and kame.
I'm very pleased to announce that another release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20030908.tar.gz. I have also prepared patch for the FreeBSD source tree. The patch was submitted for review to the committers.
Fixed few bugs in kernel modules. The ng_hci(4) and ng_l2cap(4) modules were changed to fix issue with Netgraph timeouts. The ng_ubt(4) module was changed to fix compilation issue on -current.
Improved user-space utilities. Implemented new libsdp(3). Added new sdpcontrol(8) utility. The rfcomm_sppd(1), rfcomm_pppd(8) and obexapp(1) were changed and now can obtain RFCOMM channel via SDP from the server. The hccontorol(8) utility now has four new commands. The hcsecd(8) daemon now saves link keys on the disk.
I've been recently contacted by few individuals who whould like to port current FreeBSD Bluetooth code to other BSD systems (OpenBSD and NetBSD). The work is slowly progressing towards un-Netgraph'ing current code. In the mean time Netgraph version will be the primary supported version of the code.
The rescue build infrastructure has been committed. There is one known issue with make using both the '-s' and '-j' flags that appears to be a bug in make. Anyone interested in tracking down should contact us.
Support for a dynamically linked /bin and /sbin has been committed, although it is not turned on by default. Adventurous users can try it out by building /bin and /sbin using the WITH_DYNAMICROOT make flag. More testing is needed to determine if this is going to be default for 5.2-RELEASE. If anyone would like to benchmark worldstones with and without dynamically linked /bin and /sbin, please feel free to do so and submit the results.
Work is continuing on updating ACPI with new features as well as bugfixing. A new embedded controller driver was written in July with support for the ACPI 2.0 ECDT as well as more robust polling support. Also, a buffer overflow in the ACPICA resource list handling that caused panics for some users was fixed. Marcel helped get acpidump(8) tested and basically working on ia64.
Upcoming work includes integrating ACPI notifies with devd(8), committing user-submitted drivers for ASUS and Toshiba hotkeys, Cx processor sleep states (so my laptop doesn't burn my lap), and power resource support for intelligently powering down unused or idle devices.
Users who have problems with ACPI are encouraged to submit a PR and email its number to acpi-jp@jp.FreeBSD.org. Bug reports of panics or crashes have first priority and non-working features or missing devices (except suspend/resume problems) second. Reports of failed suspend/resume should NOT be submitted as PRs at this time due to most of them being a result of incomplete device support that is being addressed. However, feel free to mail them to the list as any information is helpful.
The uart(4) project was born out of the need to have a working serial interface (i.e. an RS-232-C interface) in a legacy-free configuration and after an unsuccessful attempt to convert sio(4). The biggest problem with sio(4) is that it has been intertwined in many ugly ways into the kernel's core. Conversion could not happen without breaking something that invariably affects some group of people negatively. With sio(4) as a good bad example and a strong desire to solve multiple problems at once, the idea of an UART (Universal Asynchronuous Receiver/Transmitter) device that, given its generic name, could handle different flavors of UART hardware started to settle firmly in the authors mind.
The biggest challenge was of course solving the problem of the low-level console access prior to the initialization of the bus infrastructure and still have a driver that uses the bus access exclusively. Along the way the problem of having an UART function as the keyboard on sparc64 was solved with the introduction of system devices, which also encapsulated the console as a system device.
The uart(4) driver can be enhanced to support the various UART hardware on pc98 and this is currently being worked on. Keyboard support on sparc64 is underway as well. Plans exist for a rewrite of the remote gdb support that uses a generic interface to allow various drivers, including uart(4), to register itself as a communications channel. And since uart(4) does not support multi- port cards by itself, we likely need to either enhance puc(4) or otherwise introduce other umbrella drivers
Since I ported icc to FreeBSD I wanted to build FreeBSD with icc. Now with icc 7.1 (and some patches) it is possible. There are still some bugs, e.g. NFS doesn't work with an icc compiled kernel, IP seems to be fragile, and some advanced optimizations trigger an ICE (Intel is working on it). At the moment I'm waiting for our admins to install icc on the FreeBSD cluster (we got a commercial license from Intel, so we are allowed to distribute binaries which are compiled with icc), after that I will try to convince some people with more knowledge of the IP and NFS parts of the kernel to debug the remaining problems. When the icc compiled kernel seems to work mostly bugfree the userland will get the porting focus. Interested people may try to do a build of the ports tree with icc independently from the status of the porting of the userland... if this happens at the FreeBSD cluster, we would also be allowed to distribute the binaries.
Benefits include: another set of compiler errors (debugging help), more portable source, and code which is better optimized for a P4 (gcc has some drawbacks in this area)
The FreeBSD ports were updated to KDE 3.1.4, another bug- and security-fixes release. With this update, the QT port was updated to version 3.2. Both will be included in FreeBSD 4.9. Significant work was spent to fix KDE on FreeBSD-CURRENT after the removal of the gcc -pthread Option. Automatic package builds from KDE CVS continued to ensure and improve the quality of the upcoming KDE 3.2 release.
Future: Work is in progress to setup a new server for hosting the KDE-FreeBSD Website, Repository and another KDE CVS mirror. With help from Marcel Moolenaar the project will try to make KDE compile and working on the Intel IA64. And last but not least efforts are being made to fix the currently broken kdesu program.
WifiBSD is a miniture version of FreeBSD for wireless applications. Originally for the Soekris Net45xx line of main-boards, but is now capable of being targeted to any hardware/architecture FreeBSD itself supports. Although not feature complete, WifiBSD is expected to be ready for 5.2-RELEASE. The design goal is to meet, or exceed, the functionality of commercial/consumer 802.11 wireless gear. Features that need attention (to name just a few) are: http interface, consol menu interface, and installation. Volunters are welcome.
Work has restarted after a hiatus. Current focus is on getting loadable modules working, NEWBUSing the NetBSD dbdma code, and completing the BMAC ethernet driver.
There is a huge amount of work to do. Volunteers more than welcome!
The last known bug that prevented AMD64 machines completing a full release has been fixed - one single character error that caused ghostscript to crash during rendering diagrams. SMP work is nearing completion and should be committed within the next few days. The SMP code uses the ACPI MADT table based on John Baldwin's work-in-progress there for i386. We need to spend some time on low level optimization because there are several suboptimal places that have been ignored for simplicity, context switching in particular. MTRR support has been committed and XFree86 can use it. cvsup now works but the ezm3 port has not been updated yet. The default data segment size limit is 8GB instead of 512M, and the (primitive) i386 binary emulation support knows how to lower the rlimits for executing 32 bit binaries.
Notable things missing still: Hardware debug register support needs to be written; gdb is still being done as an external set of patches relative to the not-yet-released FSF gdb tree; DDB does not disassemble properly; DDB cannot do stack traces without -fno-omit-frame-pointer - a stack unwinder is needed; i386 and amd64 linux binary emulation is needed, and the i386 FreeBSD binary emulation still needs work - removing the stackgap code in particular.
The platform in general is very reliable although a couple of problems have been reported over the last week. One appears to be a stuck interrupt, but all that code has been redone for SMP support.
The FreeBSD Java community has started an effort to improve the current framework for Java-based ports. The main objective is the automation of JDK/JRE build and run dependency checking.
The original version was aimed to ease the life of porters. Although it has proved to be useful and reliable to a great extend, we are currently working on a new version. We intend to reach a high degree of flexibility to cope with the recent increase of available JDK/JRE flavors. Furthermore, the new version will be easier to maintain, which means improved reliability, and hopefully more frequent updates.
The BSD Java Porting Team has recently reached an exciting milestone with the release of the first "Diablo" JDK and JRE courtesy of the FreeBSD Foundation. The release of Diablo Caffe and Diablo Latte 1.3.1 was the first binary release of a native FreeBSD JDK since 1.1.8 and marks an important step forward in FreeBSD Java support.
The team is continuing development work, with a focus on achieving a compliant JDK 1.4 release in the near future.
With the introduction of ATAng, some users of ATAPI/CAM have experienced various problems. These have been mostly tracked down to issues in the new ATA code, as well as two long-standing problems in portions of the CAM layer that are rarely exercised with "real" SCSI SIMs. This has also been an occasion to cleanup ATAPI/CAM to make it more robust, and to enable DMA for devices accessed through it, resulting in improved performances.
We have released Japanese translation of 5.1-RELEASE online manual pages on June 10.
Several months ago, I took it upon myself to to try present the
- information contained on
The initial intent of this was to make life easier for ports
maintainers; however, the "general" reports are also useful to anyone who
just wants to, e.g., find out if a particular port is working on their
particular architecture and OS combination before downloading it. Those
with that general interest should start with the
-
A lot of work done since last report: site reworked completly (see new URL), console design with console message in text or graphic modes implemented, implementation of a compatibility layer to compile Linux fbdev drivers with more or less changes in the original driver (experimental).
Except some memory allocation bugs, X (XGGI based on XFree 3.3.6) is now working with the same driver as the console. A basic terminal has now to be implemented.
Volonteers are welcome to the project...
A number of races have been identified in locking device_t. Most of the races have been identified in making device_t have to do with how drivers are written. Efforts are underway to identify all the races, and to contact the authors of subsystems that can help the drivers. Of special concern is the need for the driver to ensure that all threads are completely out of the driver code before detach() finishes. Of additional concern is making sure that all sleepers are woken up before certain routines are called so that other subsystems can ensure the last condition and leave no dangling references. Locking device_t is relatively straight forward apart from these issues. Towards the end of proper locking, sample strawmen drivers are being used to work out what, exactly proper is. Once these issues are all known and documented in the code, efforts will be made to update relevant documentation in the tree. There are many problems with driver locking that has been done to date, but until we nail down how to write a driver in current, it will be premature to contact specific driver writers with specific concerns.
Support for several new crypto devices was added. The SafeNet 1141 is a medium performance part that is not yet available on retail products. The Hifn 7955 and 7956 parts are starting to appear on retail products that should be available by the end of the year. Both devices support AES encryption. Support for public key operations for the SafeNet devices was recently done for OpenBSD and will be backported. Public key support for the Hifn parts is planned.
A paper about the performance work done on the cryptographic subsystem was presented at the Usenix BSDCon 2003 conference and received the best paper award.
NetBSD recently imported the cryptographic subsystem.
The release of 4.9 is just around the corner and offers Physical Address Extensions (PAE) for x86 along with the same world-class stability and performance that is expected from the 4-STABLE series. As always, don't forget to purchase a copy of the CD set from your favorite FreeBSD vendor.
FreeBSD 5.1 was released in June and offered vastly improved stability over 5.0 along with a working implementation of Kernel Scheduled Entities, allowing for true multithreading of applications across multiple CPUs. FreeBSD 5.2 will be released by the end of 2003 and will focus on improved network and overall performance.
Numerous bugs have been fixed since the last status report (and of course a few new ones added). Progress on improved security has been slowed by other work. But new features and fixes are coming in from other groups that are now sharing the code. In particular NetBSD recently imported the revised 802.11 layer and the Linux-based MADWIFI project is using it too (albeit in an older form). The MADWIFI users have already contributed features such as fragmentation reassembly of 802.11 frames and improved signal monitoring. Power save polling and an improved rate control algorothm are expected to come in from the NetBSD folks. WPA support is still in the plans; the best estimate is that work on that will start in January.
The purpose of this project is to improve performance of the network subsystem. A major part of this work is to complete the locking of the networking subsystem so that it no longer depends on the "Giant lock" for proper operation. Removing the use of Giant will improve performance and permit multiple instances of the network stack to operate concurrently on multiprocessor systems.
This project started in August. The emphasis has been on locking the "lower half" of the networking code so that packet forwarding through the IPv4 path can operate without the Giant lock as part of the 5.2 release. To this end locking was added to several network interface drivers and much of the "middleware" code in the network was locked (e.g. ipfw, dummynet, then routing table, multicast routing support, etc). Work towards this goal is still ongoing but should be ready for 5.2. A variety of test systems have been running for several months without the Giant lock in the network drivers and IP layer.
Past the 5.2 release Giant will be removed from the "upper half" of the network subsystem and the socket layer. Once this is done the plan is to measure and improve performance (though some work of this sort is always happening). The ultimate goal is a system that performs at least as well as 4.x for normal use on uniprocessor systems. On multiprocessor systems we expect to see significantly better performance than 4.x due to greater concurrency and reduced latency.
At long last, FreeBSD 5.0 is here. Along with putting the final polish on the tree, FreeBSD developers somehow found the time to work on other things too. IA64 took some major steps towards working on the Itanium2 platform, an effort was started to convert all drivers to use busdma and ban vtophys(), hardware crypto support and DEVD hit the tree, NewReno was fixed and effort began on locking down the network layer of the kernel. Also high performance, modular scheduler started taking shape and will be a welcome addition to the kernel soon.
Looking forward, the focus will be on stabilizing and improving the performance of 5.0. The RELENG_5 (aka 5-STABLE) branch will be created once we've reached our goals in this area, so hopefully we will get there quickly. Meanwhile, preparations for the next release from the 4.x series, 4.8, will begin soon. Of course, the best way to get 5.x to stabilize os to install and run it!
Thanks,
Scott Long, Robert Watson
I'm very pleased to announce that all kernel modules and few userland tools made it to the FreeBSD source tree. Many thanks to Julian Elischer.
Unfortunately no big changes since the last report. Some minor problems have been discovered and patches are available on request. I will prepare all the patches and submit them to Julian for review.
OBEX server and client (based on OpenOBEX library) is almost complete. I'm currently doing interoperability testing. If anyone has hardware and time please contact me. The HCI security daemon has been implemented and tested with Sony Ericsson T68i cell phone and Windows stack. It is now possible to setup secure Bluetooth connections.
A few people have complained about RFCOMM daemon. These individuals want to use GPRS and Bluetooth enabled cell phone to access Internet. If you have this problem please contact me for possible workaround. My next goal is to get robust RFCOMM implementation to address all these issues.
Largely bug-fixing and userland application tweaks; new interfaces were added to manipulate ACLs on extended attributes; bugs were fixed in ls relating to ACL flagging. Patches to teach cp, mv, gzip, bzip, and other apps about ACL preservation are in testing and review. tunefs flags were added to ease configuration of ACLs, especially on UFS2 file systems.
Possible changes to make use of Linux/Solaris umask semantics are under consideration: right now we implement verbatim POSIX.1e/IRIX merging of the umask, ACL mask, and requested creation mode during file, device, fifo, and directory creation. Solaris and the most recent Linux patches ignore the umask in the context of a default ACL; this requires some rearrangement of umask handling in our VFS, although the results would be quite useful. We're exploring how to do this in a low impact way.
Framework changes:
Instrument KLD system calls (module and kld load, unload, stat) Instrument NFSd system call. Instrument swapoff(2). Instrument per-architecture privileged parts of sysarch(). Make use of condition variables to allow callers to wait for the framework to "unbusy" when loading/unloading policies, rather than returning EBUSY. Store mount pointer in devfs_mount structure for use by policies. Improve handling of labels in loopback interface "re-align" packet copy case. Provide full paths on devfs object creations to help policies label them properly (not merged). Experimentation with moving MAC labels into m_tags (not merged). NFS server now uses real ucreds, not hacked up ucreds, meaning we can start laying the groundwork for enforcement on NFS operations. (not merged)
Policy changes
LOMAC: mac_lomac replaces lomac (LOMAC now uses the MAC Framework), SEBSD: Improved support for devfs labeling based on SELinux genfs. Handling of hard link checks. Support export of process transition information for login and others using sysctl. Login now prompts for roles. Allow policy reload. TTY labeling. Locking adaptation from Linux. Many, many policy adaptations and fixes. We can now boot in enforcing mode! mac_bsdextended: fix a bug in which VAPPEND wasn't mapped to VWRITE, so opens with the O_APPEND bug failed improperly.
Userland changes
setfmac(8) now supports a setfsmac(8) execution mode, which accepts initial labeling specification files. Supports an SELinux compatibility mode so it can accept SELinux label specfiles using the SEBSD module. sendmail(8) now sets user labels as part of the context switch for mail delivery.
Documentation changes
Man page updates for MAC command line tools, modules, admin hints, etc. Updates to the FreeBSD Developer's Handbook chapter on MAC policies and entry points. MAC section in FreeBSD Handbook.
This project has been coming along pretty well. The amd(4) and xl(4) drivers have now been converted to use the busdma API, sparc64 got the bus_dmamap_load_mbuf() and bus_dmamap_load_uio() functions, and the gem(4) and hme(4) drivers have been updated to use bus_dmamap_load_mbuf() instead of bus_dmamap_load().
A lot more still needs to be done, as shown on the project's page. A fair number of conversions are on their way though, and we can expect a fair number of drivers to be converted soon, thanks to all the developers who are working on this project.
The POSIX Utility Conformance in FreeBSD list (link above) has been updated to reflect current reality. Not much work remains to complete base utility conformance.
On the API front, grantpt(), posix_openpt(), unlockpt(), wordexp(), and wordfree() were implemented. The header <wordexp.h> was added.
There are currently about 40 unassigned tasks on our project's status board ranging from documentation, utilities, to kernel hacking. We would encourage any developers looking for something to work on to check out the status board and see if anything interests them.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work will be part of the 5.0 release and has been committed to the -stable source tree for inclusion in the 4.8 release.
Recent work has focused on improving performance. System statistics are now maintained and an optional profiling facility was added for analyzing performance. Using this facility the overhead for using the crypto API has been significantly reduced.
The ubsec (Broadcom) driver was changed to significantly improve performance under load. In addition several memory leaks were fixed in the driver and the public key support was enabled for use.
Upcoming work will focus on load-balancing requests across multiple crypto devices and integrating OpenSSL 0.9.7 which will automatically enable application use of crypto hardware.
Devd has been integrated into FreeBSD 5.0-RELEASE. The integrated code supports a range of configuration options. The config files are fully parsed now and their actions are performed.
Future work in this area is likely to be limited to improving the devctl interface. /dev/devctl likely will be a cloneable device in future versions. Individual device control via devctl is also planned.
The Donations project expedited several dozen donations during 2002, and was able to place most of what was offered. We still are in dire need of SMP and Sparc systems. You can see information on our needs and donations that have been handled by the team on the donations web page.
We are relying increasingly upon the developer wantlist to place items offered to the Project, and using the commit statistics to help place items. As such, active committers who ask for what they want beforehand have a decent chance of getting it. Less active committers, and committers who do not ask for what they want, will be lower in our priorities but will not be excluded.
We are in the process of streamlining the tax deduction process for donations, and hope to have news on that shortly. We are also always working to accelerate and reduce our internal processes, to get the most equipment in the hands of the most people as quickly as possible.
I especially want to thank David O'Brien and Tom Rhodes for stepping up and making the team far more successful. Also, the FreeBSD Foundation has been quite helpful in handling tax-deductible contributions.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work will be part of the 5.0 release. Performance has been improved due to work on the crypto subsystem.
The goal of the project is to use a small amount of space in the FFS superblock to store a volume label of the user's choice. A GEOM module will then expose the volume labels into a namespace in devfs. The idea is to make it easier to manage filesystems across disk swaps and movement from system to system.
At this point, everything pretty much works. I've submitted parts of the patch to respective subsystem maintainers for review. There are some issues with namespace collision that I haven't addressed yet, but the basic functionality is there
Most of the articles are translated too. Marc is still translating the handbook, 60% is currently translated. Stéphane has began the integration of our French localization web site in the US CVS Tree. Sébastien is still maintaining the Release Notes.
We launched a new site, www.FreeBSD-fr.info, consisting in a French Daemon News like site. Netasq have donated our new server; we will install it in a new hosting provider in the few next weeks. One of the big job now is the translation of the FAQ, and the big project will be the manual pages.
Since the ports tree has been frozen for most of this reporting period,
there have not been too many GNOME updates going into the official CVS
tree. However, development has not stopped. GNOME 2.2 is nearing
completion, and quite a few FreeBSD users have stepped up to test the
GNOME 2.1 port sources from the
-
The upcoming FreeBSD 5.0-RELEASE will be the first release to have the GNOME 2.0 desktop as the default GNOME desktop choice. During the previously mentioned ports freeze, all the GNOME 2 ports were fixed up so that they build and package on both i386 and Alpha platforms. Alas, the one port that will not make the cut for Alpha is Mozilla. There are still problems with the xpcom code, but work is ongoing to get a working Alpha port.
Finally, the FreeBSD Mono (an OpenSource C# runtime) port has also received some new life. Mono has been updated to 0.17 (the latest released version), and Juli Mallett has ported gtk-sharp (GTK+ bindings for C#).
The ia64 port is up and running on the new Itanium2 based hp machines thanks to a lot of hard work by Marcel Moolenaar. So far we are running on the hp rx2600 as these were the machines graciously donated by Hewlett-Packard and Intel. We had a prototype Intel Tiger4 system for a while, but we had to return the machine and we do not know if it currently runs. Most of the changes necessary to run these are sitting in the perforce tree and are not in the -current or RELENG_5 cvs tree. As a result, the cvs derived builds (-current and the 5.0-RC series and presumably 5.0-RELEASE) are only usable on obsolete Itanium1 systems.
Lots of other stability and functionality fixes have been made over the last few months, including initial libc_r support. The OS appears to be stable enough for sustained workloads - it is building packages now, for example. We still do not have gdb support, even for reading core files.
We have been updating our Japanese translated manual pages to RELENG_5 based. All existing entries have been updated, but 15 exceptions are not, most of which require massive update. We will also need to add translations which did not exist on RELENG_4.
KGI (Kernel Graphic Interface) is a kernel infrastructure providing user applications with means to access hardware graphic resources (dma, irqs, mmio). KGI is already available under Linux as a separate standalone project. The KGI/FreeBSD project aims at integrating KGI in the FreeBSD kernel.
KGI/FreeBSD has been recently donated 2 PCI graphic cards (Matrox Millenium II and a coming Mach64) and other have been proposed. Please see the FreeBSD web pages for details. Thanks to donation@ for organizing and promoting donations. Thanks to the donators for their contribution to KGI/FreeBSD.
KGI/FreeBSD progressed fine the last months. Most of the VM issues for mapping HW resources in user space have been addressed and a first attempt of coding was made. This prototyping raised some API compatibility problems with the current Linux implementation and was discussed heavily on the kgi devel lists. Ask if you're interested in such issues, I'll be pleased to share them.
Most of coding is now done. Let's start debugging!
Work is ongoing to continue to lock up the network stack. Recently, the focus has been on the IP stack. The plan there involves a series of inter-related pieces to lock up the ifaddr ref count, the inet list, the ifaddr uses, the ARP code, the routing tree, and the routing entries. We are over 3/5 of the way done down this path.
In addition to TCP and UDP, the other networking protocols such as raw IP, IPv6, AppleTalk, and XNS need to be locked up. Around 1/4 these remaining protocols have been locked and will be committed after the IP stack is locked.
The protocol independent socket layer needs to be locked and operating correctly with the protocol dependent locks. This part is mostly done save for much needed testing and code cleanup.
Finally, a pass will be need to be made to lock up the devices drivers and various statistics counters.
This effort fixes some outstanding problems in our TCP stack with regard to congestion control. The first item is to fix our NewReno implementation. Following that, the next urgent correction is to fix a problem involving window updates and dupack counts. When that stabilizes, we will then change the recovery code to make use of SACK information. Eventually, this project will update the BSD stack to add Limited Transmit and other new internet standards and standards-track improvements.
The 3 FreeBSD package clusters (i386, alpha, sparc64) have been unified to run from the same master machine, instead of using 3 separate masters. This has freed up some machine resources to use as additional client machine, as well as simplifying administrative overheads. Build logs for all 3 architectures can now be found on the http://bento.FreeBSD.org webpage. The sparc64 package cluster now has 3 build machines (an u5 and two u10s), and an ia64 cluster is about to be created.
Package builds now keep track of how many sequential times a port has failed to build (html summaries are available on the bento website). This allows tracking of ports which have suddenly become broken (e.g. due to a bad upgrade, or due to changes in the FreeBSD source tree), and in the future will be used to send out notifications to port maintainers when their port fails to build 5 times in a row. This feature is currently experimental, and further code changes will be needed to stabilize it.
The goal of this project is to improve the wireless networking support in the system. By the time of this report the 802.11 link layer code should be committed. A version of the wi driver that uses this code should be committed shortly. Conversion of other drivers is planned as are drivers for new devices.
Support for 802.1x/EAP is the next planned milestone (both as a supplicant and authenticator).
November and December were especially busy for the release engineering team. Scott Long joined the team to help with secretary and communications tasks while Brian Somers bowed out to focus on other projects.
FreeBSD 5.0-DP2 was released in November after much delay and anticipation, and marked the final milestone needed for 5.0 to become a reality. Shortly after that, we imposed a code freeze on the HEAD branch of CVS and released 5.0-RC1. Creation of the RELENG_5_0 branch came next, followed by the release of 5.0-RC2 from this branch. At this point, enough critical problems still existed that we scheduled an RC3 release for the new year, and pushed the final 5.0-RELEASE date to mid-January. By the time this is published, FreeBSD 5.0-RELEASE should be a reality.
For the time being, there will not be a RELENG_5 (aka 5-STABLE) branch. FreeBSD 4.x releases will continue, with 4.8 being scheduled for March 2003. Release in the 4.x series will be lead by Murray Stokely, and releases in the 5.x series will be lead by Scott Long. Once HEAD has reached acceptable performance and stability goals, the RELENG_5 branch will be created and HEAD will move towards 6.0 development. We hope to reach this with the 5.1 release this spring.
A new scheduler will be available as an optional component along side the current scheduler in the 5.1 release. It has been designed to work well with KSE and SMP. Some ideas have been borrowed from solaris and linux along with many novel approaches. It has O(1) performance with regard to the number of processes in the system. It also has cpu affinity which should provide a speed boost for many applications.
The scheduler has a few loose ends and lots of tuning before it is production quality although it is quite stable. Please see the post to arch and subsequent discussion for more details.
Another busy pair of months at the FreeBSD Project have brought substantial maturity and feature completeness to the fledgeling 5.0-CURRENT branch. And just in time too, because by the time you read the next status report, we hope that you'll have FreeBSD 5.0 running on your desktop! Over the past two months, we've seen an upgrade of sparc64 to Tier 1 (Fully Supported) status, integration of a high quality storage encryption module, the commit of hardware-accelerated IPsec support, the addition of a general-purpose "Device Daemon" to process hardware attach/detach events to replace earlier single-purpose and bus-specific daemons, the commit of RAIDFrame, and the improved maturity of the TrustedBSD work. We've also seen another successful release of the 4.x branch, 4.7-RELEASE, which will continue to be the production supported platform as 5.X is brought in for landing.
Over the next two months, the FreeBSD Project will be focused almost entirely on making 5.0 a success: improving system stability and performance, as well as increasing the pool of applications that build and run on 5.0. The Release Engineering team will have announced the 5.0 code freeze, and released DP2 by the time you read this. Following DP2 will be a series of Release Candidates (RC's), and then the release itself. If you're interested in getting involved in the testing process, please lend a hand -- a spare box and a copy of the DP and RC ISOs burnt onto CD will make a difference. The normal caveats associated with pre-release versions of operating systems apply! You may also be interested in reading the Early Adopter's guide produced by the Release Engineering team to help determine when a transition from the 4.x branch to the 5.x branch will be appropriate for you and your organization.
Thanks,
Robert Watson, Scott Long
I'm very pleased to announce that another engineering release is available for download at http://www.geocities.com/m_evmenkin/ngbt-fbsd-20021104.tar.gz
This release features minor bug fixes and new OpenOBEX library port. The snapshot includes support for H4 UART and H2 USB transport layers, Host Controller Interface (HCI), Link Layer Control and Adaptation Protocol (L2CAP) and Bluetooth sockets layer. It also comes with several user space utilities that can be used to configure and test Bluetooth devices. Also there are several man pages.
Service Discovery Protocol (SDP) port has been updated to version 0.8. (ported from BlueZ-sdp-0.8). Most of the RFCOMM issues have been resolved and now rfcommd works with Windows (3COM, Xircom and Widcomm) and Linux stacks.
New supported USB device - EPoX BT-DG02 dongle. Also I have received successful report about Mitsumi USB dongle and C413S Bluetooth enabled cell phone (L2CAP and SDP works, waiting on RFCOMM report).
I'm currently working on OBEX server (Push and File Transfer profiles) which will be based on OpenOBEX library (included in the snapshot).
The BSDCon 2003 Program Committee invites you to contribute original and innovative papers on topics related to BSD-derived systems and the Open Source world. Topics of interest include but are not limited to:
Submissions in the form of extended abstracts are due by April 1, 2003. Be sure to review the extended abstract expectations before submitting. Selection will be based on the quality of the written submission and whether the work is of interest to the community.
We look forward to receiving your submissions!
October 10, 2002 marked the one year anniversary of our project. During that time we have made significant advances in FreeBSD's standards conformance. FreeBSD 5.0-RELEASE will be the showcase for most of our hard work. We hope that our tireless effort has had a positive effect on FreeBSD and software vendors that maintain or are considering porting their software to FreeBSD.
On the API front, _Exit(3) (an alias for _exit(2)) was added, sysconf(3) was update for POSIX.1-2001, and some of the glob(3) additions were MFC'd. The insque(), lsearch(), and remque() family of functions were reimplemented and moved to libc from libcompat. Several wide character functions were implemented, including all printf() and scanf() variants. Finally, support for wide character format types (%C, %S, %lc, %ls) were added to printf(3).
Work on utility conformance continued as getconf(1)'s compliance was updated, c99(1) (a new version of c89(1)) was implemented, and cd(1) and command(1) changes were MFC'd.
Almost 20 headers were brought up to conformance with applicable standards. Not much work remains to fix conformance issues in the remaining standard headers. Work in this area, as well as others, has slowed down in preparation for 5.0-RELEASE.
DEVD has been integrated into FreeBSD current. It was integrated in an incomplete state. However, it is useful in the state that it is in for doing simple things like running camcontrol rescan when a SCSI pcmcia card is inserted, or running /etc/pccard_ether with an ethernet card is inserted. The more sophisticated regular expression matching is not yet complete. Devd only does actions on device arrival and departure, but does not yet do anything with unknown devices. In addition to listening for device events, there is some desire to have /dev/devctl also allow for some direct control of the device tree.
The main goal of this project is to modify the IPsec protocols to use the kernel-level crypto subsystem imported from OpenBSD (see elsewhere). A secondary goal is to do general performance tuning of the IPsec protocols.
This work was committed to -current. To configure it for use specify options FAST_IPSEC in your system configuration file. At present support is limited to IPv4.
GBDE has been committed to -current.
The "Geom Based Disk Encryption" module provides a mechanism for very strong encryption of a GEOM "disk". The algorithm has passed informal review by a couple of seasoned crypto heavy-weights. Any GEOM device can be protected with GBDE, entire physical disks, MBR slices, BSD partitions etc etc. Booting from an encrypted partition is not possible, however.
The focus of GBDE is to protect a "cold" disk media. (FreeBSD is not equipped well for protecting key material on a running system from being compromised.) For a cold media, the only feasible attack on a GBDE protected media is guessing the pass-phrase.
Summary of the GBDE multilevel protection scheme: Up to four separate pass-phrases can unlock their own separate copies of the 2048 bit masterkey. The master-keys are protected using AES/256/CBC keyed with a SHA-2 hash derived from the pass-phrase. A salted MD5 hash over the sectoroffset "cherry-picks" which masterkey bytes participate in the MD5 hash which generates the "kkey" for each particular sector. The kkey AES/128/CBC encrypts the PRNG produced single-use key which AES/128/CBC encrypts the actual sector data.
GBDE has features for master-key destruction and pass-phrase invalidation.
See gbde(4) and gbde(8) for more details.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
The GEOM code is now the default on most (if not all ?) architectures and the few remaining issues in libdisk/sysinstall is being hashed out.
Although we are far from finished developing GEOM, its current feature set is a significant step forward for FreeBSD, providing not only immediate relief for new architectures (sparc64, ia64 etc) but also because it is designed as SMPng code from the start.
This software was developed for the FreeBSD Project by Poul-Henning Kamp and NAI Labs, the Security Research Division of Network Associates, Inc. under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS research program.
These last two months have seen quite a lot of GNOME activity.
GNOME has started releasing development snapshots of the upcoming
GNOME 2.2 desktop. FreeBSD porting has begun outside of the
main ports tree in the
-
Evolution 1.2 is also close at hand. Ximian has posted its first release candidate, 1.1.90, which has been ported to FreeBSD, and is available from the MarcusCom CVS repo listed above. As soon as Ximian officially releases Evolution 1.2, it will be placed in the FreeBSD ports tree.
The Mozilla ports have received numerous updates. We are now tracking all three released Mozilla versions. The mozilla-vendor port is tracking the 1.0.x branch, mozilla is tracking 1.1.x, and mozilla-devel is tracking 1.2.x. The mozilla-devel port now has support for anti-aliased fonts as well as a GTK+-2 interface
Finally, the GNOME team would like to welcome its newest team member, Adam Weinberger. Adam has been submitting patches for both GNOME ports as well as documentation. Currently, he has been active in the GNOME 2.2 porting effort. We are happy to have him.
The goal of this project is to import the OpenBSD kernel-level crypto subsystem. This facility provides kernel- and user-level access to hardware crypto devices for the calculation of cryptographic hashes, ciphers, and public key operations. The main clients of this facility are the kernel RNG (/dev/random), network protocols (e.g. IPsec), and OpenSSL (through the /dev/crypto device).
This work was committed to the -current tree. To configure it for use specify device crypto in your system configuration file or you can load the crypto module. The /dev/crypto device support is brought in with device cryptodev or by loading the cryptodev module. Two crypto device drivers exist: ubsec for Broadcom-based PCI hardware and hifn for Hifn-based PCI hardware.
Integration of this work into the -stable source tree should be completed by the time this report is published.
Since the last status report the BSD Java Porting Team has continued to make steady progress. The most exciting news we have is courtesy of our newest team member, Alexey Zelkin of FreeBSD committer fame.
For 4.7-RELEASE, we privately published package ja-man-doc-4.7.tgz which consists of man[1256789] entries 10 days after the 4.7-RELEASE release date. Man3 update god no progress, as updating other sections busied us. We decided to suspend man3 update officially, as we need to spend most of our time to catch up with the forthcoming 5.0-RELEASE.
The KDE/FreeBSD team has been working on two major goals during the last two months, Maintenance of the KDE 3.0.x ports and Preparing the upcoming KDE 3.1 Release.
Maintenance KDE 3.0 conducted by Alan Eldrige: September started with the Removal of the KDE 2.x Ports from the FreeBSD-Repository. Later Packages of KDE 3.0.4 were released and the FreeBSD Ports were updated.
Preparing for KDE 3.1 conducted by Will Andrews: A lot of effort was spent on Improving the Fruitsalad-Build-System. We are now able to create packages directly from the KDE CVS.
The KSE code has now all the basic kernel functionality to start being used by the userland. There are still things to be done for testing and familiarization.
General system utilities have not yet been changed. e.g. ps and top etc. need to know about threads.
There is quite a lot of code in the kernel that still assumes that there is one thread in a process. Signals are not yet handled in the final manner (though they are delivered to a random thread in the process :-/ ).
The system calls and datastructures are now however in place. The test program successfully starts several threads that can be scheduled on different processors, and closes them down again. The userland is probably going to be able to do simple scheduling of pthread threads using KSE by the time that this report is published.
I still need someone to take over the "official" web page since jason left. LaTeX sure isn't my thing.
Not much since the last status report, except that we now have the repo and development web page back online, thanks to the services of John De Boskey who freely provided the necessary hardware and bandwidth to host the project. We have also ported LibH to GCC 3.x, so that it can compile on -CURRENT correctly. This, however, broke tvision, which doesn't compile under GCC 3.x, so we moved to rhtvision but this caused linking problems so we're stuck with no console front end, for now.
Work on a Hui rewrite and SWIG bindings stalled. Alex was able to come up with a simple patch to make the ports system use LibH's pkg_create script to build libh packages, so we're getting closer to a real pkg_create(1) drop-in replacement. I rewrote the milestone list to show a bit more relevant and encouraging tasks that will be dealt with in order to really push LibH forward.
A mailing list was created, freebsd-mips, and a Perforce branch was created in //depot/projects/mips. Changes which will be necessary to allow multiple MIPS (and PowerPC) metaports to exist under one architecture port were made, and are being pushed back into the main FreeBSD tree. Some preliminary header work has been done, and porting the ARCBIOS interfaces to the kernel has begun. The toolchain in tree was updated and modified in places to support a FreeBSD/MIPS (Big Endian) target, in the Perforce branch. Some early boot code has proven the GDB MIPS simulator to work, for at least R3000 code, though whether R3000 will be supported has been under discussion. Some initial architectural decisions were also made, to steer current work.
Work on newcard continues. A number of bugs have been fixed in the last few months. You are now able to load and unload drivers (including the bridge) to test changes to pccard and/or cardbus bus code. It is now possible to load a driver that has a pccard attachment and have a previously inserted card probe and attach. This is also true for CardBus. A number of issues remain to be solved before 5.0. However, with the integration of devd into the tree nearly all of old functionality of OLDCARD is now present in NEWCARD (the biggest remaining parts are power control for the sockets, as well as pccardc dumpcis).
The PowerPC port has been running diskless on NewWorld G3/G4 machines for a while now. A GEOM module to support Apple Partition Maps is being written. There should be an installable ISO image available in the near future.
RAIDFrame was imported into FreeBSD-current in late October, a major milestone after 18 months. It is still very experimental and not suitable for production environments. The website contains a lengthy TODO list which I hope to start attending to soon. Still, I encourage everyone to try it out and report bugs back to me.
The Release Engineering (RE) team completed and released FreeBSD 4.7 on 10 October 2002. This release features updates for a number of contributed software programs in the base system, as well as all of the security and bug fixes from FreeBSD 4.6.2. The next release in the 4.X series will be FreeBSD 4.8, which has a scheduled release date of 1 February 2003.
Before that time, however, will be the release of FreeBSD 5.0. Thus far, we have not been able to release the 5.0-DP2 developer snapshot due to various stability issues. Thanks to much effort from many of our fellow developers, we believe that most of these have been resolved. The RE team wishes to emphasize that FreeBSD 5.0 will involve new code and features that have not seen widespread testing, and that more conservative users may wish to continue to track the 4.X series for the near-term future. To provide more information on these issues, we have added an Early Adopter's Guide to the release documentation for 5.0.
Brian Somers has resigned from the RE team due to increased time pressures. We thank him for all of his help with FreeBSD 4.5, 4.6, 4.6.2, and 4.7, and we hope to continue working with him as a fellow developer.
Scott Long has graciously offered to help improve the communication between the RE team and the rest of the developer community. We greatly appreciate his assistance.
Recent 5-current release procedure troubles prevent the project from releasing a new snapshots. But 5-current FreeBSD/i386 release is back again in late Oct/2002! I have a plan to build daily FreeBSD/sparc64 snapshots for 5-current. Stay tuned...
A lot has happened recently for the sparc64 port. Sysinstall and make release work and can be used to build installable snapshots. The gdb5.3 port now works, and, thanks to Thomas Moestl, kernel crash dumps are supported which can be analyzed by gdb. These 2 items are the last things considered necessary by the Core team for FreeBSD/sparc64 to be a Tier 1 architecture, which means that 5.0-RELEASE for sparc64 will be officially supported by the release engineering team and by the security officer team.
Recently Jake Burkholder has been working on alternate installation methods other than bootable iso, including a mini-root filesystem which can be written to the swap partition of an existing machine. Thomas Moestl has been putting some finishing touches on the release process, ensuring that the release documentation can be built properly, and that the port readme files can be generated by the release process.
An experimental iso built with make release is now available on the freebsd ftp site and mirrors in /pub/FreeBSD/development/sparc64/5.0-20021031-SNAP. It is expected that by the middle of November new 5.0-SNAP releases will be available every few days for download and for ftp install, cpu power and bandwidth permitting.
Most progress on TrustedBSD over the last two months related to improving the maturity of the ACL and MAC implementations, and merging new aspects of those features into the primary FreeBSD CVS Repository for inclusion in FreeBSD 5.0. This included fixes to run better on sparc64, improved tuning of what system objects are mediated, locking fixes and optimizations especially relating to the vnode and pipe implementations, improved support for MAC labeling on symlinks, support for asynchronous process label changes as required in some locking situations, remove use of "temporary labels" and prefer use of object type specific labels reducing redundant and/or confusing label management code in policies, improve avoidance of memory allocation in M_NOWAIT scenarios for socket allocation in the syncache, mediation of link operations, race condition fixes for devfs involving label creation, improve handling of VM events such as mmaping, improve mediation of socket send/receive events (as distinguished from socket transmit/deliver events), support for manipulating EAs on symlinks using new system calls, support for MNT_ACLS and MNT_MULTILABEL flags at mount time, as well as FS_ACLS and FS_MULTILABEL superblock flags to key useful defaults using tunefs, correction of a memory leak in the UFS ACL code, enable UFS ACL support by default in GENERIC, mediation points for file creation, deletion, and rename, support for a mac_execve() execution interface in the style of SELinux's execve_secure() permitting a label transition request as part of the exec operation for policies that support it, more consistent handling of NFS lookups, support for labeling of multicast encapsulated packets, ATM packet labeling, FDDI packet labeling, STF packet labeling, revised label interface that avoids userland parsing of per-policy elements, reducing us to a single instance of parsing and printing for each policy (and further abstracting policy implementation details from the library code).
Also, change to single-level sockets for Biba and MLS policies, support for partial label updates for Biba and MLS, addition of mac.9 man page, revised user API system calls, implementation of mac_get_pid(), and various other related bits, creation of mac.conf(5) to specify label defaults, checks for various system operations including swapon(), settime(), and sysctl(), reboot(), acct(), introduction of command line utilities for maintaining file and process labels, support for user labels tied to login class, su support for label changes, ifconfig support for interface labels, ps support for process labels, ls support for file labels, ftpd support for login labels, development of the Biba and MLS notions of privilege, and a move to C99 sparse structure initialization, restoring full type checking for policy entry points.
Universally Unique Identifiers (UUIDs) are 128 bit values that may be generated independently on separate nodes (hosts), which result in globally unique strings. UUIDs are also known as Globally Unique Identifiers (GUIDs). The UUID support for FreeBSD (libc) conforms to the DCE 1.1 RPC specification.
UUID support has been added to FreeBSD -CURRENT, and will be available in version 5.0. It is being extensively used in GPT partition handling for IA-64 platform. For now, a simple manual page has been provided, which outlines information about the provided uuid routines. Many documentation additions and enhancements to uuidgen(1) are in the pipeline.
The goal of this project is to improve the wireless networking support in the system. The initial work will incorporate the 802.11 link layer done by Atsushi Onoe for NetBSD. This core support code implements the basic 802.11 protocols required for Station and AP operation in BSS, IBSS, and Ad Hoc modes of operation. Wireless device drivers will then be revised to use this common code instead of their private implementations.
Following this initial stage the wireless networking support will be extended to support functionality needed for workgroup, enterprise, and metropolitan (e.g. mesh) networking environments. This will include full power management support, the 802.1D spanning tree protocol for running multiple AP's in a bridged configuration, QoS support, and enhanced security protocols (LEAP, AES, EAP). Support for new hardware devices is also planned.