diff --git a/documentation/content/en/articles/committers-guide/_index.adoc b/documentation/content/en/articles/committers-guide/_index.adoc index 557630327b..e52393a258 100644 --- a/documentation/content/en/articles/committers-guide/_index.adoc +++ b/documentation/content/en/articles/committers-guide/_index.adoc @@ -1,4027 +1,4027 @@ --- title: Committer's Guide authors: - author: The FreeBSD Documentation Project copyright: 1999-2021 The FreeBSD Documentation Project description: Introductory information for FreeBSD committers trademarks: ["freebsd", "coverity", "ibm", "intel", "general"] weight: 25 tags: ["FreeBSD Committer's Guide", "Guide", "Community"] --- = Committer's Guide :doctype: article :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :source-highlighter: rouge :experimental: :images-path: articles/committers-guide/ ifdef::env-beastie[] ifdef::backend-html5[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] :imagesdir: ../../../images/{images-path} endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [.abstract-title] Abstract This document provides information for the FreeBSD committer community. All new committers should read this document before they start, and existing committers are strongly encouraged to review it from time to time. Almost all FreeBSD developers have commit rights to one or more repositories. However, a few developers do not, and some of the information here applies to them as well. (For instance, some people only have rights to work with the Problem Report database). Please see <> for more information. This document may also be of interest to members of the FreeBSD community who want to learn more about how the project works. ''' toc::[] [[admin]] == Administrative Details [.informaltable] [cols="1,1", frame="none"] |=== |_Login Methods_ |man:ssh[1], protocol 2 only |_Main Shell Host_ |`freefall.FreeBSD.org` |_Reference Machines_ |`ref*.FreeBSD.org`, `universe*.freeBSD.org` (see also link:https://www.FreeBSD.org/internal/machines/[FreeBSD Project Hosts]) |_SMTP Host_ |`smtp.FreeBSD.org:587` (see also <>). |`_src/_` Git Repository |`ssh://git@gitrepo.FreeBSD.org/src.git` (see also <>). |`_doc/_` Git Repository |`ssh://git@gitrepo.FreeBSD.org/doc.git` (see also <>). |`_ports/_` Git Repository |`ssh://git@gitrepo.FreeBSD.org/ports.git` (see also <>). |_Internal Mailing Lists_ |developers (technically called all-developers), doc-developers, doc-committers, ports-developers, ports-committers, src-developers, src-committers. (Each project repository has its own -developers and -committers mailing lists. Archives for these lists can be found in the files [.filename]#/local/mail/repository-name-developers-archive# and [.filename]#/local/mail/repository-name-committers-archive# on the `FreeBSD.org` cluster.) |_Core Team monthly reports_ |[.filename]#/home/core/public/monthly-reports# on the `FreeBSD.org` cluster. |_Ports Management Team monthly reports_ |[.filename]#/home/portmgr/public/monthly-reports# on the `FreeBSD.org` cluster. |_Noteworthy `src/` Git Branches:_ |`stable/n` (`n`-STABLE), `main` (-CURRENT) |=== man:ssh[1] is required to connect to the project hosts. For more information, see <>. Useful links: * link:https://www.FreeBSD.org/internal/[FreeBSD Project Internal Pages] * link:https://www.FreeBSD.org/internal/machines/[FreeBSD Project Hosts] * link:https://www.FreeBSD.org/administration/[FreeBSD Project Administrative Groups] [[pgpkeys]] == OpenPGP Keys for FreeBSD Cryptographic keys conforming to the OpenPGP (__Pretty Good Privacy__) standard are used by the FreeBSD project to authenticate committers. Messages carrying important information like public SSH keys can be signed with the OpenPGP key to prove that they are really from the committer. See http://www.nostarch.com/pgp_ml.htm[PGP & GPG: Email for the Practical Paranoid by Michael Lucas] and http://en.wikipedia.org/wiki/Pretty_Good_Privacy[] for more information. [[pgpkeys-creating]] === Creating a Key Existing keys can be used, but should be checked with [.filename]#documentation/tools/checkkey.sh# first. In this case, make sure the key has a FreeBSD user ID. For those who do not yet have an OpenPGP key, or need a new key to meet FreeBSD security requirements, here we show how to generate one. [[pgpkeys-create-steps]] [.procedure] ==== . Install [.filename]#security/gnupg#. Enter these lines in [.filename]#~/.gnupg/gpg.conf# to set minimum acceptable defaults: + [.programlisting] .... fixed-list-mode keyid-format 0xlong personal-digest-preferences SHA512 SHA384 SHA256 SHA224 default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 BZIP2 ZLIB ZIP Uncompressed verify-options show-uid-validity list-options show-uid-validity sig-notation issuer-fpr@notations.openpgp.fifthhorseman.net=%g cert-digest-algo SHA512 .... . Generate a key: + [source,shell] .... % gpg --full-gen-key gpg (GnuPG) 2.1.8; Copyright (C) 2015 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Warning: using insecure memory! Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 2048 <.> Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) 3y <.> Key expires at Wed Nov 4 17:20:20 2015 MST Is this correct? (y/N) y GnuPG needs to construct a user ID to identify your key. Real name: Chucky Daemon <.> Email address: notreal@example.com Comment: You selected this USER-ID: "Chucky Daemon " Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o You need a Passphrase to protect your secret key. .... <.> 2048-bit keys with a three-year expiration provide adequate protection at present (2013-12). http://danielpocock.com/rsa-key-sizes-2048-or-4096-bits[] describes the situation in more detail. <.> A three year key lifespan is short enough to obsolete keys weakened by advancing computer power, but long enough to reduce key management problems. <.> Use your real name here, preferably matching that shown on government-issued ID to make it easier for others to verify your identity. Text that may help others identify you can be entered in the `Comment` section. + After the email address is entered, a passphrase is requested. Methods of creating a secure passphrase are contentious. Rather than suggest a single way, here are some links to sites that describe various methods: http://world.std.com/~reinhold/diceware.html[], http://www.iusmentis.com/security/passphrasefaq/[], http://xkcd.com/936/[], http://en.wikipedia.org/wiki/Passphrase[]. ==== Protect the private key and passphrase. If either the private key or passphrase may have been compromised or disclosed, immediately notify mailto:accounts@FreeBSD.org[accounts@FreeBSD.org] and revoke the key. Committing the new key is shown in <>. [[kerberos-ldap]] == Kerberos and LDAP web Password for FreeBSD Cluster The FreeBSD cluster requires a Kerberos password to access certain services. The Kerberos password also serves as the LDAP web password, since LDAP is proxying to Kerberos in the cluster. Some of the services which require this include: * https://bugs.freebsd.org/bugzilla[Bugzilla] * https://ci.freebsd.org[Jenkins] To create a new Kerberos account in the FreeBSD cluster, or to reset a Kerberos password for an existing account using a random password generator: [source,shell] .... % ssh kpasswd.freebsd.org .... [NOTE] ==== This must be done from a machine outside of the FreeBSD.org cluster. ==== A Kerberos password can also be set manually by logging into `freefall.FreeBSD.org` and running: [source,shell] .... % kpasswd .... [NOTE] ==== Unless the Kerberos-authenticated services of the FreeBSD.org cluster have been used previously, `Client unknown` will be shown. This error means that the `ssh kpasswd.freebsd.org` method shown above must be used first to initialize the Kerberos account. ==== [[committer.types]] == Commit Bit Types The FreeBSD repository has a number of components which, when combined, support the basic operating system source, documentation, third party application ports infrastructure, and various maintained utilities. When FreeBSD commit bits are allocated, the areas of the tree where the bit may be used are specified. Generally, the areas associated with a bit reflect who authorized the allocation of the commit bit. Additional areas of authority may be added at a later date: when this occurs, the committer should follow normal commit bit allocation procedures for that area of the tree, seeking approval from the appropriate entity and possibly getting a mentor for that area for some period of time. [.informaltable] [cols="1,1,1", frame="none"] |=== |__Committer Type__ |__Responsible__ |__Tree Components__ |src |core@ |src/ |doc |doceng@ |doc/, ports/, src/ documentation |ports |portmgr@ |ports/ |=== Commit bits allocated prior to the development of the notion of areas of authority may be appropriate for use in many parts of the tree. However, common sense dictates that a committer who has not previously worked in an area of the tree seek review prior to committing, seek approval from the appropriate responsible party, and/or work with a mentor. Since the rules regarding code maintenance differ by area of the tree, this is as much for the benefit of the committer working in an area of less familiarity as it is for others working on the tree. Committers are encouraged to seek review for their work as part of the normal development process, regardless of the area of the tree where the work is occurring. === Policy for Committer Activity in Other Trees * All committers may modify [.filename]#src/share/misc/committers-*.dot#, [.filename]#src/usr.bin/calendar/calendars/calendar.freebsd#, and [.filename]#ports/astro/xearth/files#. * doc committers may commit documentation changes to [.filename]#src# files, such as man pages, READMEs, fortune databases, calendar files, and comment fixes without approval from a src committer, subject to the normal care and tending of commits. * Any committer may make changes to any other tree with an "Approved by" from a non-mentored committer with the appropriate bit. Mentored committers can provide a "Reviewed by" but not an "Approved by". * Committers can acquire an additional bit by the usual process of finding a mentor who will propose them to core, doceng, or portmgr, as appropriate. When approved, they will be added to 'access' and the normal mentoring period will ensue, which will involve a continuing of "Approved by" for some period. [[doc-blanket-approval]] ==== Documentation Implicit (Blanket) Approval Some types of fixes have "blanket approval" from the {doceng}, allowing any committer to fix those categories of problems on any part of the doc tree. These fixes do not need approval or review from a doc committer if the author doesn't have a doc commit bit. Blanket approval applies to these types of fixes: * Typos * Trivial fixes + Punctuation, URLs, dates, paths and file names with outdated or incorrect information, and other common mistakes that may confound the readers. Over the years, some implicit approvals were granted in the doc tree. This list shows the most common cases: * Changes in [.filename]#documentation/content/en/books/porters-handbook/versions/_index.adoc# + extref:{porters-handbook}versions/[__FreeBSD_version Values (Porter's Handbook)], mainly used for src committers. * Changes in [.filename]#doc/shared/contrib-additional.adoc# + extref:{contributors}[Additional FreeBSD Contributors, contrib-additional] maintenance. * All link:#commit-steps[Steps for New Committers], doc related * Security advisories; Errata Notices; Releases; + Used by {security-officer} and {re}. * Changes in [.filename]#website/content/en/donations/donors.adoc# + Used by {donations}. Before any commit, a build test is necessary; see the 'Overview' and 'The FreeBSD Documentation Build Process' sections of the extref:{fdp-primer}[FreeBSD Documentation Project Primer for New Contributors] for more details. [[git-primer]] == Git Primer [[git-basics]] === Git basics When one searches for "Git Primer" a number of good ones come up. Daniel Miessler's link:https://danielmiessler.com/study/git/[A git primer] and Willie Willus' link:https://gist.github.com/williewillus/068e9a8543de3a7ef80adb2938657b6b[Git - Quick Primer] are both good overviews. The Git book is also complete, but much longer https://git-scm.com/book/en/v2. There is also this website https://dangitgit.com/ for common traps and pitfalls of Git, in case you need guidance to fix things up. Finally, an introduction link:https://eagain.net/articles/git-for-computer-scientists/[targeted at computer scientists] has proven helpful to some at explaining the Git world view. This document will assume that you've read through it and will try not to belabor the basics (though it will cover them briefly). [[git-mini-primer]] === Git Mini Primer This primer is less ambitiously scoped than the old Subversion Primer, but should cover the basics. ==== Scope If you want to download FreeBSD, compile it from sources, and generally keep up to date that way, this primer is for you. It covers getting the sources, updating the sources, bisecting and touches briefly on how to cope with a few local changes. It covers the basics, and tries to give good pointers to more in-depth treatment for when the reader finds the basics insufficient. Other sections of this guide cover more advanced topics related to contributing to the project. The goal of this section is to highlight those bits of Git needed to track sources. They assume a basic understanding of Git. There are many primers for Git on the web, but the https://git-scm.com/book/en/v2[Git Book] provides one of the better treatments. [[git-mini-primer-getting-started]] ==== Getting Started For Developers This section describes the read-write access for committers to push the commits from developers or contributors. ===== Daily use * Clone the repository: + [source,shell] .... % git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/${repo}.git .... + Then you should have the official mirrors as your remote: + [source,shell] .... % git remote -v freebsd https://git.freebsd.org/${repo}.git (fetch) freebsd https://git.freebsd.org/${repo}.git (push) .... * Configure the FreeBSD committer data: + The commit hook in repo.freebsd.org checks the "Commit" field matches the committer's information in FreeBSD.org. The easiest way to get the suggested config is by executing `/usr/local/bin/gen-gitconfig.sh` script on freefall: + [source,shell] .... % gen-gitconfig.sh [...] % git config user.name (your name in gecos) % git config user.email (your login)@FreeBSD.org .... * Set the push URL: + [source,shell] .... % git remote set-url --push freebsd git@gitrepo.freebsd.org:${repo}.git .... + Then you should have separated fetch and push URLs as the most efficient setup: + [source,shell] .... % git remote -v freebsd https://git.freebsd.org/${repo}.git (fetch) freebsd git@gitrepo.freebsd.org:${repo}.git (push) .... + Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. * Install commit message template hook: + [source,shell] .... % fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks % chmod 755 .git/hooks/prepare-commit-msg .... [[admin-branch]] ===== "admin" branch The `access` and `mentors` files are stored in an orphan branch, `internal/admin`, in each repository. Following example is how to check out the `internal/admin` branch to a local branch named `admin`: [source,shell] .... % git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*' % git fetch % git checkout -b admin internal/admin .... Alternatively, you can add a worktree for the `admin` branch: [source,shell] .... git worktree add -b admin ../${repo}-admin internal/admin .... For browsing `internal/admin` branch on web: https://cgit.freebsd.org/${repo}/log/?h=internal/admin For pushing, either specify the full refspec: [source,shell] .... git push freebsd HEAD:refs/internal/admin .... Or set `push.default` to `freebsd` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow: [source,shell] .... git config push.default freebsd .... ==== Keeping Current With The FreeBSD src Tree [[keeping_current]] First step: cloning a tree. This downloads the entire tree. There are two ways to download. Most people will want to do a deep clone of the repository. However, there are times when you may wish to do a shallow clone. ===== Branch names The branch names in the new Git repository are similar to the old names. For the stable branches, they are stable/X where X is the major release (like 11 or 12). The main branch in the new repository is 'main'. The main branch in the old GitHub mirror was 'master', but is now 'main'. Both reflect the defaults of Git at the time they were created. The 'main' branch is the default branch if you omit the '-b branch' or '--branch branch' options below. ===== Repositories Please see the <> for the latest information on where to get FreeBSD sources. $URL below can be obtained from that page. Note: The project doesn't use submodules as they are a poor fit for our workflows and development model. How we track changes in third-party applications is discussed elsewhere and generally of little concern to the casual user. ===== Deep Clone A deep clone pulls in the entire tree, as well as all the history and branches. It is the easiest to do. It also allows you to use Git's worktree feature to have all your active branches checked out into separate directories but with only one copy of the repository. [source,shell] .... % git clone -o freebsd $URL -b branch [dir] .... is how you make a deep clone. 'branch' should be one of the branches listed in the previous section. It is optional if it is the main branch. 'dir' is an optional directory to place it in (the default will be the name of the repo you are cloning (src, doc, etc)). You will want a deep clone if you are interested in the history, plan on making local changes, or plan on working on more than one branch. It is the easiest to keep up to date as well. If you are interested in the history, but are working with only one branch and are short on space, you can also use --single-branch to only download the one branch (though some merge commits will not reference the merged-from branch which may be important for some users who are interested in detailed versions of history). ===== Shallow Clone A shallow clone copies just the most current code, but none or little of the history. This can be useful when you need to build a specific revision of FreeBSD, or when you are just starting out and plan to track the tree more fully. You can also use it to limit history to only so many revisions. However, see below for a significant limitation of this approach. [source,shell] .... % git clone -o freebsd -b branch --depth 1 $URL [dir] .... This clones the repository, but only has the most recent version in the repository. The rest of the history is not downloaded. Should you change your mind later, you can do 'git fetch --unshallow' to get the old history. [WARNING] ==== When you make a shallow clone, you will lose the commit count in your uname output. This can make it more difficult to determine if your system needs to be updated when a security advisory is issued. ==== ===== Building Once you've downloaded, building is done as described in the handbook, e.g.: [source,shell] .... % cd src % make buildworld % make buildkernel % make installkernel % make installworld .... so that won't be covered in depth here. If you want to build a custom kernel, extref:{handbook}[the kernel config section, kernelconfig] of the FreeBSD Handbook recommends creating a file MYKERNEL under sys/${ARCH}/conf with your changes against GENERIC. To have MYKERNEL disregarded by Git, it can be added to .git/info/exclude. ===== Updating To update both types of trees uses the same commands. This pulls in all the revisions since your last update. [source,shell] .... % git pull --ff-only .... will update the tree. In Git, a 'fast forward' merge is one that only needs to set a new branch pointer and doesn't need to re-create the commits. By always doing a 'fast forward' merge/pull, you'll ensure that you have an exact copy of the FreeBSD tree. This will be important if you want to maintain local patches. See below for how to manage local changes. The simplest is to use --autostash on the 'git pull' command, but more sophisticated options are available. ==== Selecting a Specific Version In Git, the 'git checkout' checks out both branches and specific versions. Git's versions are the long hashes rather than a sequential number. When you checkout a specific version, just specify the hash you want on the command line (the git log command can help you decide which hash you might want): [source,shell] .... % git checkout 08b8197a74 .... and you have that checked out. You will be greeted with a message similar to the following: [source,shell] .... Note: checking out '08b8197a742a96964d2924391bf9fdfeb788865d'. You are in a 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b HEAD is now at 08b8197a742a hook gpiokeys.4 to the build .... where the last line is generated from the hash you are checking out and the first line of the commit message from that revision. The hash can be abbreviated to the shortest unique length. Git itself is inconsistent about how many digits it displays. ==== Bisecting Sometimes, things go wrong. The last version worked, but the one you just updated to does not. A developer may ask you to bisect the problem to track down which commit caused the regression. Git makes bisecting changes easy with a powerful 'git bisect' command. Here's a brief outline of how to use it. For more information, you can view https://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination or https://git-scm.com/docs/git-bisect for more details. The man git-bisect page is good at describing what can go wrong, what to do when versions won't build, when you want to use terms other than 'good' and 'bad', etc, none of which will be covered here. `git bisect start --first-parent` will start the bisection process. Next, you need to tell a range to go through. 'git bisect good XXXXXX' will tell it the working version and 'git bisect bad XXXXX' will tell it the bad version. The bad version will almost always be HEAD (a special tag for what you have checked out). The good version will be the last one you checked out. The `--first-parent` argument is necessary so that subsequent `git bisect` commands do not try to check out a vendor branch which lacks the full FreeBSD source tree. [TIP] ==== If you want to know the last version you checked out, you should use 'git reflog': [source,shell] .... 5ef0bd68b515 (HEAD -> main, freebsd/main, freebsd/HEAD) HEAD@{0}: pull --ff-only: Fast-forward a8163e165c5b (upstream/main) HEAD@{1}: checkout: moving from b6fb97efb682994f59b21fe4efb3fcfc0e5b9eeb to main ... .... shows me moving the working tree to the main branch (a816...) and then updating from upstream (to 5ef0...). In this case, bad would be HEAD (or 5rf0bd68) and good would be a8163e165. As you can see from the output, HEAD@{1} also often works, but isn't foolproof if you have done other things to your Git tree after updating, but before you discover the need to bisect. ==== Set the 'good' version first, then set the bad (though the order doesn't matter). When you set the bad version, it will give you some statistics on the process: [source,shell] .... % git bisect start --first-parent % git bisect good a8163e165c5b % git bisect bad HEAD Bisecting: 1722 revisions left to test after this (roughly 11 steps) [c427b3158fd8225f6afc09e7e6f62326f9e4de7e] Fixup r361997 by balancing parens. Duh. .... You would then build/install that version. If it's good you'd type 'git bisect good' otherwise 'git bisect bad'. If the version doesn't compile, type 'git bisect skip'. You will get a similar message to the above after each step. When you are done, report the bad version to the developer (or fix the bug yourself and send a patch). 'git bisect reset' will end the process and return you back to where you started (usually tip of main). Again, the git-bisect manual (linked above) is a good resource for when things go wrong or for unusual cases. [[git-gpg-signing]] ==== Signing the commits, tags, and pushes, with GnuPG Git knows how to sign commits, tags, and pushes. When you sign a Git commit or a tag, you can prove that the code you submitted came from you and wasn't altered while you were transferring it. You also can prove that you submitted the code and not someone else. A more in-depth documentation on signing commits and tags can be found in the https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work[Git Tools - Signing Your Work] chapter of the Git's book. The rationale behind signing pushes can be found in the https://github.com/git/git/commit/a85b377d0419a9dfaca8af2320cc33b051cbed04[commit that introduced the feature]. The best way is to simply tell Git you always want to sign commits, tags, and pushes. You can do this by setting a few configuration variables: [source,shell] .... % git config --add user.signingKey=LONG-KEY-ID % git config --add commit.gpgSign=true % git config --add tag.gpgSign=true % git config --add push.gpgSign=if-asked .... // push.gpgSign should probably be set to `yes` once we enable it, or be set with --global, so that it is enabled for all repositories. [NOTE] ====== To avoid possible collisions, make sure you give a long key id to Git. You can get the long id with: `gpg --list-secret-keys --keyid-format LONG`. ====== [TIP] ====== To use specific subkeys, and not have GnuPG to resolve the subkey to a primary key, attach `!` to the key. For example, to encrypt for the subkey `DEADBEEF`, use `DEADBEEF!`. ====== ===== Verifying signatures Commit signatures can be verified by running either `git verify-commit `, or `git log --show-signature`. Tag signatures can be verified with `git verity-tag `, or `git tag -v `. //// Commented out for now until we decide what to do. Git pushes are a bit different, they live in a special ref in the repository. TODO: write how to verify them //// ==== Ports Considerations The ports tree operates the same way. The branch names are different and the repositories are in different locations. The cgit repository web interface for use with web browsers is at https://cgit.FreeBSD.org/ports/ . The production Git repository is at https://git.FreeBSD.org/ports.git and at ssh://anongit@git.FreeBSD.org/ports.git (or anongit@git.FreeBSD.org:ports.git). There is also a mirror on GitHub, see extref:{handbook}/mirrors[External mirrors, mirrors] for an overview. The 'current' branch is 'main' . The quarterly branches are named 'yyyyQn' for year 'yyyy' and quarter 'n'. ===== Commit message formats A hook is available in the ports repository to help you write up your commit messages in https://cgit.freebsd.org/ports/tree/.hooks/prepare-commit-msg[.hooks/prepare-commit-message]. It can be enabled by running ``git config --add core.hooksPath .hooks``. The main point being that a commit message should be formatted in the following way: .... category/port: Summary. Description of why the changes where made. PR: 12345 .... [IMPORTANT] ==== The first line is the subject of the commit, it contains what port was changed, and a summary of the commit. It should contain 50 characters or less. A blank line should separate it from the rest of the commit message. The rest of the commit message should be wrapped at the 72 characters boundary. Another blank line should be added if there are any metadata fields, so that they are easily distinguishable from the commit message. ==== ==== Managing Local Changes This section addresses tracking local changes. If you have no local changes, you can stop reading now (it is the last section and OK to skip). One item that is important for all of them: all changes are local until pushed. Unlike Subversion, Git uses a distributed model. For users, for most things, there is very little difference. However, if you have local changes, you can use the same tool to manage them as you use to pull in changes from FreeBSD. All changes that you have not pushed are local and can easily be modified (git rebase, discussed below does this). ===== Keeping local changes The simplest way to keep local changes (especially trivial ones) is to use 'git stash'. In its simplest form, you use 'git stash' to record the changes (which pushes them onto the stash stack). Most people use this to save changes before updating the tree as described above. They then use 'git stash apply' to re-apply them to the tree. The stash is a stack of changes that can be examined with 'git stash list'. The git-stash man page (https://git-scm.com/docs/git-stash) has all the details. This method is suitable when you have tiny tweaks to the tree. When you have anything non trivial, you'll likely be better off keeping a local branch and rebasing. Stashing is also integrated with the 'git pull' command: just add '--autostash' to the command line. ===== Keeping a local branch [[keeping_a_local_branch]] It is much easier to keep a local branch with Git than Subversion. In Subversion you need to merge the commit, and resolve the conflicts. This is manageable, but can lead to a convoluted history that's hard to upstream should that ever be necessary, or hard to replicate if you need to do so. Git also allows one to merge, along with the same problems. That's one way to manage the branch, but it's the least flexible. In addition to merging, Git supports the concept of 'rebasing' which avoids these issues. The 'git rebase' command replays all the commits of a branch at a newer location on the parent branch. We will cover the most common scenarios that arise using it. ====== Create a branch Let's say you want to make a change to FreeBSD's ls command to never, ever do color. There are many reasons to do this, but this example will use that as a baseline. The FreeBSD ls command changes from time to time, and you'll need to cope with those changes. Fortunately, with Git rebase it usually is automatic. [source,shell] .... % cd src % git checkout main % git checkout -b no-color-ls % cd bin/ls % vi ls.c # hack the changes in % git diff # check the changes diff --git a/bin/ls/ls.c b/bin/ls/ls.c index 7378268867ef..cfc3f4342531 100644 --- a/bin/ls/ls.c +++ b/bin/ls/ls.c @@ -66,6 +66,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#undef COLORLS #ifdef COLORLS #include #include % # these look good, make the commit... % git commit ls.c .... The commit will pop you into an editor to describe what you've done. Once you enter that, you have your own **local** branch in the Git repo. Build and install it like you normally would, following the directions in the handbook. Git differs from other version control systems in that you have to tell it explicitly which files to commit. I have opted to do it on the commit command line, but you can also do it with 'git add' which many of the more in depth tutorials cover. ====== Time to update When it is time to bring in a new version, it is almost the same as w/o the branches. You would update like you would above, but there is one extra command before you update, and one after. The following assumes you are starting with an unmodified tree. It is important to start rebasing operations with a clean tree (Git requires this). [source,shell] .... % git checkout main % git pull --ff-only % git rebase -i main no-color-ls .... This will bring up an editor that lists all the commits in it. For this example, do not change it at all. This is typically what you are doing while updating the baseline (though you also use the Git rebase command to curate the commits you have in the branch). Once you are done with the above, you have to move the commits to ls.c forward from the old version of FreeBSD to the newer one. Sometimes there are merge conflicts. That is OK. Do not panic. Instead, handle them the same as any other merge conflicts. To keep it simple, I will just describe a common issue that may arise. A pointer to a more complete treatment can be found at the end of this section. Let's say the includes changes upstream in a radical shift to terminfo as well as a name change for the option. When you updated, you might see something like this: [source,shell] .... Auto-merging bin/ls/ls.c CONFLICT (content): Merge conflict in bin/ls/ls.c error: could not apply 646e0f9cda11... no color ls Resolve all conflicts manually, mark them as resolved with "git add/rm ", then run "git rebase --continue". You can instead skip this commit: run "git rebase --skip". To abort and get back to the state before "git rebase", run "git rebase --abort". Could not apply 646e0f9cda11... no color ls .... which looks scary. If you bring up an editor, you will see it is a typical 3-way merge conflict resolution that you may be familiar with from other source code systems (the rest of ls.c has been omitted): [source,shell] .... <<<<<<< HEAD #ifdef COLORLS_NEW #include ======= #undef COLORLS #ifdef COLORLS #include >>>>>>> 646e0f9cda11... no color ls .... The new code is first, and your code is second. The right fix here is to just add a #undef COLORLS_NEW before #ifdef and then delete the old changes: [source,shell] .... #undef COLORLS_NEW #ifdef COLORLS_NEW #include .... save the file. The rebase was interrupted, so you have to complete it: [source,shell] .... % git add ls.c % git rebase --continue .... which tells Git that ls.c has been fixed and to continue the rebase operation. Since there was a conflict, you will get kicked into the editor to update the commit message if necessary. If the commit message is still accurate, just exit the editor. If you get stuck during the rebase, do not panic. git rebase --abort will take you back to a clean slate. It is important, though, to start with an unmodified tree. An aside: The above mentioned 'git reflog' comes in handy here, as it will have a list of all the (intermediate) commits that you can view or inspect or cherry-pick. For more on this topic, https://www.freecodecamp.org/news/the-ultimate-guide-to-git-merge-and-git-rebase/ provides a rather extensive treatment. It is a good resource for issues that arise occasionally but are too obscure for this guide. ===== Switching to a Different FreeBSD Branch If you wish to shift from stable/12 to the current branch. If you have a deep clone, the following will suffice: [source,shell] .... % git checkout main % # build and install here... .... If you have a local branch, though, there are one or two caveats. First, rebase will rewrite history, so you will likely want to do something to save it. Second, jumping branches tends to cause more conflicts. If we pretend the example above was relative to stable/12, then to move to main, I'd suggest the following: [source,shell] .... % git checkout no-color-ls % git checkout -b no-color-ls-stable-12 # create another name for this branch % git rebase -i stable/12 no-color-ls --onto main .... What the above does is checkout no-color-ls. Then create a new name for it (no-color-ls-stable-12) in case you need to get back to it. Then you rebase onto the main branch. This will find all the commits to the current no-color-ls branch (back to where it meets up with the stable/12 branch) and then it will replay them onto the main branch creating a new no-color-ls branch there (which is why I had you create a place holder name). ===== Migrating from an existing Git clone If you have work based on a previous Git conversion or a locally running git-svn conversion, migrating to new repository can encounter problems because Git has no knowledge about the connection between the two. When you have only a few local changes, the easiest way would be to cherry-pick those changes to the new base: [source,shell] .... % git checkout main % git cherry-pick old_branch..your_branch .... Or alternatively, do the same thing with rebase: [source,shell] .... % git rebase --onto main master your_branch .... If you do have a lot of changes, you would probably want to perform a merge instead. The idea is to create a merge point that consolidates the history of the old_branch, and the new FreeBSD repository (main). You can find out by looking up the same commit that are found on both parents: [source,shell] .... % git show old_branch .... You will see a commit message, now search for that in the new branch: [source,shell] .... % git log --grep="commit message on old_branch" freebsd/main .... You would help locate the commit hash on the new main branch, create a helper branch (in the example we call it 'stage') from that hash: [source,shell] .... % git checkout -b stage _hash_found_from_git_log_ .... Then perform a merge of the old branch: [source,shell] .... % git merge -s ours -m "Mark old branch as merged" old_branch .... With that, it's possible to merge your work branch or the main branch in any order without problem. Eventually, when you are ready to commit your work back to main, you can perform a rebase to main, or do a squash commit by combining everything into one commit. [[mfc-with-git]] === MFC (Merge From Current) Procedures ==== Summary MFC workflow can be summarized as `git cherry-pick -x` plus `git commit --amend` to adjust the commit message. For multiple commits, use `git rebase -i` to squash them together and edit the commit message. ==== Single commit MFC [source,shell] .... % git checkout stable/X % git cherry-pick -x $HASH --edit .... For MFC commits, for example a vendor import, you would need to specify one parent for cherry-pick purposes. Normally, that would be the "first parent" of the branch you are cherry-picking from, so: [source,shell] .... % git checkout stable/X % git cherry-pick -x $HASH -m 1 --edit .... If things go wrong, you'll either need to abort the cherry-pick with `git cherry-pick --abort` or fix it up and do a `git cherry-pick --continue`. Once the cherry-pick is finished, push with `git push`. If you get an error due to losing the commit race, use `git pull --rebase` and try to push again. ==== MFC to RELENG branch MFCs to branches that require approval require a bit more care. The process is the same for either a typical merge or an exceptional direct commit. * Merge or direct commit to the appropriate `stable/X` branch first before merging to the `releng/X.Y` branch. * Use the hash that's in the `stable/X` branch for the MFC to `releng/X.Y` branch. * Leave both "cherry picked from" lines in the commit message. * Be sure to add the `Approved by:` line when you are in the editor. [source,shell] .... % git checkout releng/13.0 % git cherry-pick -x $HASH --edit .... If you forget to to add the `Approved by:` line, you can do a `git commit --amend` to edit the commit message before you push the change. ==== Multiple commit MFC [source,shell] .... % git checkout -b tmp-branch stable/X % for h in $HASH_LIST; do git cherry-pick -x $h; done % git rebase -i stable/X # mark each of the commits after the first as 'squash' # Update the commit message to reflect all elements of commit, if necessary. # Be sure to retain the "cherry picked from" lines. % git push freebsd HEAD:stable/X .... If the push fails due to losing the commit race, rebase and try again: [source,shell] .... % git checkout stable/X % git pull % git checkout tmp-branch % git rebase stable/X % git push freebsd HEAD:stable/X .... Once the MFC is complete, you can delete the temporary branch: [source,shell] .... % git checkout stable/X % git branch -d tmp-branch .... ==== MFC a vendor import Vendor imports are the only thing in the tree that creates a merge commit in the main line. Cherry picking merge commits into stable/XX presents an additional difficulty because there are two parents for a merge commit. Generally, you'll want the first parent's diff since that's the diff to mainline (though there may be some exceptions). [source,shell] .... % git cherry-pick -x -m 1 $HASH .... is typically what you want. This will tell cherry-pick to apply the correct diff. There are some, hopefully, rare cases where it's possible that the mainline was merged backwards by the conversion script. Should that be the case (and we've not found any yet), you'd change the above to '-m 2' to pickup the proper parent. Just do [source,shell] .... % git cherry-pick --abort % git cherry-pick -x -m 2 $HASH .... to do that. The `--abort` will cleanup the failed first attempt. ==== Redoing a MFC If you do a MFC, and it goes horribly wrong and you want to start over, then the easiest way is to use `git reset --hard` like so: [source,shell] .... % git reset --hard freebsd/stable/12 .... though if you have some revs you want to keep, and others you don't, using 'git rebase -i' is better. ==== Considerations when MFCing When committing source commits to stable and releng branches, we have the following goals: * Clearly mark direct commits distinct from commits that land a change from another branch. * Avoid introducing known breakage into stable and releng branches. * Allow developers to determine which changes have or have not been landed from one branch to another. With Subversion, we used the following practices to achieve these goals: * Using 'MFC' and 'MFS' tags to mark commits that merged changes from another branch. * Squashing fixup commits into the main commit when merging a change. * Recording mergeinfo so that `svn mergeinfo --show-revs` worked. With Git, we will need to use different strategies to achieve the same goals. This document aims to define best practices when merging source commits using Git that achieve these goals. In general, we aim to use Git's native support to achieve these goals rather than enforcing practices built on Subversion's model. One general note: due to technical differences with Git, we will not be using Git "merge commits" (created via `git merge`) in stable or releng branches. Instead, when this document refers to "merge commits", it means a commit originally made to `main` that is replicated or "landed" to a stable branch, or a commit from a stable branch that is replicated to a releng branch with some variation of `git cherry-pick`. ==== Finding Eligible Hashes to MFC Git provides some built-in support for this via the `git cherry` and `git log --cherry` commands. These commands compare the raw diffs of commits (but not other metadata such as log messages) to determine if two commits are identical. This works well when each commit from head is landed as a single commit to a stable branch, but it falls over if multiple commits from main are squashed together as a single commit to a stable branch. There are a few options for resolving this: 1. We could ban squashing of commits and instead require that committers stage all of the fixup / follow-up commits to stable into a single push. This would still achieve the goal of stability in stable and releng branches since pushes are atomic and users doing a simple pull will never end up with a tree that has the main commit without the fixup(s). `git bisect` is also able to cope with this model via `git bisect skip`. 2. We could adopt a consistent style for describing MFCs and write our own tooling to wrap around `git cherry` to determine the list of eligible commits. A simple approach here might be to use the syntax from `git cherry-pick -x`, but require that a squashed commit list all of the hashes (one line per hash) at the end of the commit message. Developers could do this by using `git cherry-pick -x` of each individual commit into a branch and then use `git rebase` to squash the commits down into a single commit, but collecting the `-x` annotations at the end of the landed commit log. ==== Commit message standards ===== Marking MFCs The project has adopted the following practice for marking MFCs: * Use the `-x` flag with `git cherry-pick`. This adds a line to the commit message that includes the hash of the original commit when merging. Since it is added by Git directly, committers do not have to manually edit the commit log when merging. When merging multiple commits, keep all the "cherry picked from" lines. ===== Trim Metadata? One area that was not clearly documented with Subversion (or even CVS) is how to format metadata in log messages for MFC commits. Should it include the metadata from the original commit unchanged, or should it be altered to reflect information about the MFC commit itself? Historical practice has varied, though some of the variance is by field. For example, MFCs that are relevant to a PR generally include the PR field in the MFC so that MFC commits are included in the bug tracker's audit trail. Other fields are less clear. For example, Phabricator shows the diff of the last commit tagged to a review, so including Phabricator URLs replaces the `main` commit with the landed commits. The list of reviewers is also not clear. If a reviewer has approved a change to `main`, does that mean they have approved the MFC commit? Is that true if it's identical code only, or with merely trivial rework? It's clearly not true for more extensive reworks. Even for identical code what if the commit doesn't conflict but introduces an ABI change? A reviewer may have ok'd a commit for `main` due to the ABI breakage but may not approve of merging the same commit as-is. One will have to use one's best judgment until clear guidelines can be agreed upon. For MFCs regulated by re@, new metadata fields are added, such as the Approved by tag for approved commits. This new metadata will have to be added via `git commit --amend` or similar after the original commit has been reviewed and approved. We may also want to reserve some metadata fields in MFC commits such as Phabricator URLs for use by re@ in the future. Preserving existing metadata provides a very simple workflow. Developers can just use `git cherry-pick -x` without having to edit the log message. If instead we choose to adjust metadata in MFCs, developers will have to edit log messages explicitly via the use of `git cherry-pick --edit` or `git commit --amend`. However, as compared to svn, at least the existing commit message can be pre-populated and metadata fields can be added or removed without having to re-enter the entire commit message. The bottom line is that developers will likely need to curate their commit message for MFCs that are non-trivial. ==== Examples ===== Merging a Single Subversion Commit This walks through the process of merging a commit to stable/12 that was originally committed to head in Subversion. In this case, the original commit is r368685. The first step is to map the Subversion commit to a Git hash. Once you have fetched refs/notes/commits, you can pass the revision number to `git log --grep`: [source,shell] .... % git log main --grep 368685 commit ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81 Author: John Baldwin Date: Wed Dec 16 00:11:30 2020 +0000 Use the 't' modifier to print a ptrdiff_t. Reviewed by: imp Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27576 Notes: svn path=/head/; revision=368685 .... Next, MFC the commit to a `stable/12` checkout: [source,shell] .... git checkout stable/12 git cherry-pick -x ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81 --edit .... Git will invoke the editor. Use this to remove the metadata that only applied to the original commit (Phabricator URL and Reviewed by). After the editor saves the updated log message, Git completes the commit: [source,shell] .... [stable/12 3e3a548c4874] Use the 't' modifier to print a ptrdiff_t. Date: Wed Dec 16 00:11:30 2020 +0000 1 file changed, 1 insertion(+), 1 deletion(-) .... The contents of the MFCd commit can be examined via `git show`: [source,shell] .... % git show commit 3e3a548c487450825679e4bd63d8d1a67fd8bd2d (HEAD -> stable/12) Author: John Baldwin Date: Wed Dec 16 00:11:30 2020 +0000 Use the 't' modifier to print a ptrdiff_t. Obtained from: CheriBSD Sponsored by: DARPA (cherry picked from commit ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81) diff --git a/sys/compat/linuxkpi/common/include/linux/printk.h b/sys/compat/linuxkpi/common/include/linux/printk.h index 31802bdd2c99..e6510e9e9834 100644 --- a/sys/compat/linuxkpi/common/include/linux/printk.h +++ b/sys/compat/linuxkpi/common/include/linux/printk.h @@ -68,7 +68,7 @@ print_hex_dump(const char *level, const char *prefix_str, printf("[%p] ", buf); break; case DUMP_PREFIX_OFFSET: - printf("[%p] ", (const char *)((const char *)buf - + printf("[%#tx] ", ((const char *)buf - (const char *)buf_old)); break; default: .... The MFC commit can now be published via `git push` [source,shell] .... % git push freebsd Enumerating objects: 17, done. Counting objects: 100% (17/17), done. Delta compression using up to 4 threads Compressing objects: 100% (7/7), done. Writing objects: 100% (9/9), 817 bytes | 204.00 KiB/s, done. Total 9 (delta 5), reused 1 (delta 1), pack-reused 0 To gitrepo-dev.FreeBSD.org:src.git 525bd9c9dda7..3e3a548c4874 stable/12 -> stable/12 .... ===== Merging a Single Subversion Commit with a Conflict This example is similar to the previous example except that the commit in question encounters a merge conflict. In this case, the original commit is r368314. As above, the first step is to map the Subversion commit to a Git hash: [source,shell] .... % git log main --grep 368314 commit 99963f5343a017e934e4d8ea2371a86789a46ff9 Author: John Baldwin Date: Thu Dec 3 22:01:13 2020 +0000 Don't transmit mbufs that aren't yet ready on TOE sockets. This includes mbufs waiting for data from sendfile() I/O requests, or mbufs awaiting encryption for KTLS. Reviewed by: np MFC after: 2 weeks Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D27469 Notes: svn path=/head/; revision=368314 .... Next, MFC the commit to a `stable/12` checkout: [source,shell] .... % git checkout stable/12 % git cherry-pick -x 99963f5343a017e934e4d8ea2371a86789a46ff9 --edit Auto-merging sys/dev/cxgbe/tom/t4_cpl_io.c CONFLICT (content): Merge conflict in sys/dev/cxgbe/tom/t4_cpl_io.c warning: inexact rename detection was skipped due to too many files. warning: you may want to set your merge.renamelimit variable to at least 7123 and retry the command. error: could not apply 99963f5343a0... Don't transmit mbufs that aren't yet ready on TOE sockets. hint: after resolving the conflicts, mark the corrected paths hint: with 'git add ' or 'git rm ' hint: and commit the result with 'git commit' .... In this case, the commit encountered a merge conflict in sys/dev/cxge/tom/t4_cpl_io.c as kernel TLS is not present in stable/12. Note that Git does not invoke an editor to adjust the commit message due to the conflict. `git status` confirms that this file has merge conflicts: [source,shell] .... % git status On branch stable/12 Your branch is up to date with 'upstream/stable/12'. You are currently cherry-picking commit 99963f5343a0. (fix conflicts and run "git cherry-pick --continue") (use "git cherry-pick --skip" to skip this patch) (use "git cherry-pick --abort" to cancel the cherry-pick operation) Unmerged paths: (use "git add ..." to mark resolution) both modified: sys/dev/cxgbe/tom/t4_cpl_io.c no changes added to commit (use "git add" and/or "git commit -a") .... After editing the file to resolve the conflict, `git status` shows the conflict as resolved: [source,shell] .... % git status On branch stable/12 Your branch is up to date with 'upstream/stable/12'. You are currently cherry-picking commit 99963f5343a0. (all conflicts fixed: run "git cherry-pick --continue") (use "git cherry-pick --skip" to skip this patch) (use "git cherry-pick --abort" to cancel the cherry-pick operation) Changes to be committed: modified: sys/dev/cxgbe/tom/t4_cpl_io.c .... The cherry-pick can now be completed: [source,shell] .... % git cherry-pick --continue .... Since there was a merge conflict, Git invokes the editor to adjust the commit message. Trim the metadata fields from the commit log from the original commit to head and save the updated log message. The contents of the MFC commit can be examined via `git show`: [source,shell] .... % git show commit 525bd9c9dda7e7c7efad2d4570c7fd8e1a8ffabc (HEAD -> stable/12) Author: John Baldwin Date: Thu Dec 3 22:01:13 2020 +0000 Don't transmit mbufs that aren't yet ready on TOE sockets. This includes mbufs waiting for data from sendfile() I/O requests, or mbufs awaiting encryption for KTLS. Sponsored by: Chelsio Communications (cherry picked from commit 99963f5343a017e934e4d8ea2371a86789a46ff9) diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c index 8e8c2b8639e6..43861f10b689 100644 --- a/sys/dev/cxgbe/tom/t4_cpl_io.c +++ b/sys/dev/cxgbe/tom/t4_cpl_io.c @@ -746,6 +746,8 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) for (m = sndptr; m != NULL; m = m->m_next) { int n; + if ((m->m_flags & M_NOTAVAIL) != 0) + break; if (IS_AIOTX_MBUF(m)) n = sglist_count_vmpages(aiotx_mbuf_pages(m), aiotx_mbuf_pgoff(m), m->m_len); @@ -821,8 +823,9 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) /* nothing to send */ if (plen == 0) { - KASSERT(m == NULL, - ("%s: nothing to send, but m != NULL", __func__)); + KASSERT(m == NULL || (m->m_flags & M_NOTAVAIL) != 0, + ("%s: nothing to send, but m != NULL is ready", + __func__)); break; } @@ -910,7 +913,7 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) toep->txsd_avail--; t4_l2t_send(sc, wr, toep->l2te); - } while (m != NULL); + } while (m != NULL && (m->m_flags & M_NOTAVAIL) == 0); /* Send a FIN if requested, but only if there's no more data to send */ if (m == NULL && toep->flags & TPF_SEND_FIN) .... The MFC commit can now be published via `git push` [source,shell] .... git push freebsd Enumerating objects: 13, done. Counting objects: 100% (13/13), done. Delta compression using up to 4 threads Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 819 bytes | 117.00 KiB/s, done. Total 7 (delta 6), reused 0 (delta 0), pack-reused 0 To gitrepo.FreeBSD.org:src.git f4d0bc6aa6b9..525bd9c9dda7 stable/12 -> stable/12 .... [[vendor-import-git]] === Vendor Imports with Git This section describes the vendor import procedure with Git in detail. ==== Branch naming convention All vendor branches and tags start with `vendor/`. These branches and tags are visible by default. [NOTE] ==== This chapter follows the convention that the `freebsd` origin is the origin name for the official FreeBSD Git repository. If you use a different convention, replace `freebsd` with the name you use instead in the examples below. ==== We will explore an example for updating NetBSD's mtree that is in our tree. The vendor branch for this is `vendor/NetBSD/mtree`. ==== Updating an old vendor import The vendor trees usually have only the subset of the third-party software that is appropriate to FreeBSD. These trees are usually tiny in comparison to the FreeBSD tree. Git worktrees are thus quite small and fast and the preferred method to use. Make sure that whatever directory you choose below (the `../mtree`) does not currently exist. [source,shell] .... % git worktree add ../mtree vendor/NetBSD/mtree .... ==== Update the Sources in the Vendor Branch Prepare a full, clean tree of the vendor sources. Import everything but merge only what is needed. This example assumes the NetBSD source is checked out from their GitHub mirror in `~/git/NetBSD`. Note that "upstream" might have added or removed files, so we want to make sure deletions are propagated as well. package:net/rsync[] is commonly installed, so I'll use that. [source,shell] .... % cd ../mtree % rsync -va --del --exclude=".git" ~/git/NetBSD/usr.sbin/mtree/ . % git add -A % git status ... % git diff --staged ... % git commit -m "Vendor import of NetBSD's mtree at 2020-12-11" [vendor/NetBSD/mtree 8e7aa25fcf1] Vendor import of NetBSD's mtree at 2020-12-11 7 files changed, 114 insertions(+), 82 deletions(-) % git tag -a vendor/NetBSD/mtree/20201211 .... Note: I run the `git diff` and `git status` commands to make sure nothing weird was present. Also I used `-m` to illustrate, but you should compose a proper message in an editor (using a commit message template). It is also important to create an annotated tag using `git tag -a`, otherwise the push will be rejected. Only annotated tags are allowed to be pushed. The annotated tag gives you a chance to enter a commit message. Enter the version you are importing, along with any salient new features or fixes in that version. ==== Updating the FreeBSD Copy At this point you can push the import to `vendor` into our repo. [source,shell] .... % git push --follow-tags freebsd vendor/NetBSD/mtree .... `--follow-tags` tells `git push` to also push tags associated with the locally committed revision. ==== Updating the FreeBSD source tree Now you need to update the mtree in FreeBSD. The sources live in `contrib/mtree` since it is upstream software. [source,shell] .... % cd ../src % git subtree merge -P contrib/mtree vendor/NetBSD/mtree .... This would generate a subtree merge commit of `contrib/mtree` against the local `vendor/NetBSD/mtree` branch. If there were conflicts, you would need to fix them before committing. Include details about the changes being merged in the merge commit message. ==== Rebasing your change against latest FreeBSD source tree Because the current policy recommends against using merges, if the upstream FreeBSD `main` moved forward before you get a chance to push, you would have to redo the merge. Regular `git rebase` or `git pull --rebase` doesn't know how to rebase a merge commit **as a merge commit**, so instead of that you would have to recreate the commit. The easiest way to do this would be to create a side branch with the **contents** of the merged tree: [source,shell] .... % cd ../src % git fetch freebsd % git checkout -b merge_result % git merge freebsd/main .... Typically, there would be no merge conflicts here (because developers tend to work on different components). In the worst case scenario, you would still have to resolve merge conflicts, if there was any, but this should be really rare. Now, checkout `freebsd/main` again as `new_merge`, and redo the merge: [source,shell] .... % git checkout -b new_merge freebsd/main % git subtree merge -P contrib/mtree vendor/NetBSD/mtree .... Instead of resolving the conflicts, perform this instead: [source,shell] .... % git checkout merge_result . .... Which will overwrite the files with conflicts with the version found in `merge_result`. Examine the tree against `merge_result` to make sure that you haven't missed deleted files: [source,shell] .... % git diff merge_result .... ==== Pushing the changes Once you are sure that you have a set of deltas you think is good, you can push it to a fork off GitHub or GitLab for others to review. One nice thing about Git is that it allows you to publish rough drafts of your work for others to review. While Phabricator is good for content review, publishing the updated vendor branch and merge commits lets others check the details as they will eventually appear in the repository. After review, when you are sure it is a good change, you can push it to the FreeBSD repo: [source,shell] .... % git push freebsd main .... === Creating a new vendor branch There are a number of ways to create a new vendor branch. The recommended way is to create a new repository and then merge that with FreeBSD. If one is importing `glorbnitz` into the FreeBSD tree, release 3.1415. For the sake of simplicity, we will not trim this release. It is a simple user command that puts the nitz device into different magical glorb states and is small enough trimming will not save much. ==== Create the repo [source,shell] .... % cd /some/where % mkdir glorbnitz % cd glorbnitz % git init % git checkout -b vendor/glorbnitz .... At this point, you have a new repo, where all new commits will go on the `vendor/glorbnitz` branch. Git experts can also do this right in their FreeBSD clone, using `git checkout --orphan vendor/glorbnitz` if they are more comfortable with that. ==== Copy the sources in Since this is a new import, you can just cp the sources in, or use tar or even rsync as shown above. And we will add everything, assuming no dot files. [source,shell] .... % cp -r ~/glorbnitz/* . % git add * .... At this point, you should have a pristine copy of glorbnitz ready to commit. [source,shell] .... % git commit -m "Import GlorbNitz frobnosticator revision 3.1415" .... As above, I used `-m` for simplicity, but you should likely create a commit message that explains what a Glorb is and why you'd use a Nitz to get it. Not everybody will know so, for your actual commit, you should follow the <> section instead of emulating the brief style used here. ==== Now import it into our repository Now you need to import the branch into our repository. [source,shell] .... % cd /path/to/freebsd/repo/src % git remote add glorbnitz /some/where/glorbnitz % git fetch glorbnitz vendor/glorbnitz .... Note the vendor/glorbnitz branch is in the repo. At this point the `/some/where/glorbnitz` can be deleted, if you like. It was only a means to an end. // perhaps the real treasure was the friends it made along the way... ==== Tag and push Steps from here on out are much the same as they are in the case of updating a vendor branch, though without the updating the vendor branch step. [source,shell] .... % git worktree add ../glorbnitz vendor/glorbnitz % cd ../glorbnitz % git tag --annotate vendor/glorbnitz/3.1415 # Make sure the commit is good with "git show" % git push --follow-tags freebsd vendor/glorbnitz .... By 'good' we mean: . All the right files are present . None of the wrong files are present . The vendor branch points at something sensible . The tag looks good, and is annotated . The commit message for the tag has a quick summary of what's new since the last tag ==== Time to finally merge it into the base tree [source,shell] .... % cd ../src % git subtree add -P contrib/glorbnitz vendor/glorbnitz # Make sure the commit is good with "git show" % git commit --amend # one last sanity check on commit message % git push freebsd .... Here 'good' means: . All the right files, and none of the wrong ones, were merged into contrib/glorbnitz. . No other changes are in the tree. . The commit messages look <>. It should contain a summary of what's changed since the last merge to the FreeBSD main line and any caveats. . UPDATING should be updated if there is anything of note, such as user visible changes, important upgrade concerns, etc. [NOTE] ==== This hasn't connected `glorbnitz` to the build yet. How so do that is specific to the software being imported and is beyond the scope of this tutorial. ==== === FreeBSD Src Committer Transition Guide This section is designed to walk people through the conversion process from Subversion to Git, written from the source committer's point of view. ==== Migrating from a Subversion tree This section will cover a couple of common scenarios for migrating from using the FreeBSD Subversion repo to the FreeBSD source Git repo. The FreeBSD Git conversion is still in beta status, so some minor things may change between this and going into production. The first thing to do is install Git. Any version of Git will do, though the latest one in ports / packages generally will be good. Either build it from ports, or install it using pkg (though some folks might use `su` or `doas` instead of `sudo`): [source,shell] .... % sudo pkg install git .... ===== No staged changes migration If you have no changes pending, the migration is straightforward. In this, you abandon the Subversion tree and clone the Git repository. It's likely best to retain your Subversion tree, in case there's something you've forgotten about there. First, let's clone the repository: [source,shell] .... % git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/src.git freebsd-src .... will create a clone of the FreeBSD src repository into a subdirectory called `freebsd-src` and include the 'notes' about the revisions. We are currently mirroring the source repository to https://github.com/freebsd/freebsd-src.git as well. https://github.com/freebsd/freebsd-legacy.git has the old GitHub mirror with the old hashes should you need that for your migration. The GitHub `master` branch has been frozen. As the default in Git has changed, we've shifted from `master` to `main`; the new repository uses `main`. We also mirror the repository to GitLab at https://gitlab.com/FreeBSD/src.git . It's useful to have the old Subversion revisions available. This data is stored using Git notes, but Git doesn't fetch those by default. The --config and the argument above changed the default to fetch the notes. If you've cloned the repository without this, or wish to add notes to a previously cloned repository, use the following commands: [source,shell] .... % git config --add remote.freebsd.fetch "+refs/notes/*:refs/notes/*" % git fetch .... At this point you have the src checked out into a Git tree, ready to do other things. ===== But I have changes that I've not committed If you are migrating from a tree that has changes you've not yet committed to FreeBSD, you'll need to follow the steps from the previous section first, and then follow these. [source,shell] .... % cd path-to-svn-checkout-tree % svn diff > /tmp/src.diff % cd _mumble_/freebsd-src % git checkout -b working .... This will create a diff of your current changes. The last command creates a branch called `working` though you can call it whatever you want. [source,shell] .... % git apply /tmp/src.diff .... this will apply all your pending changes to the working tree. This doesn't commit the change, so you'll need to make this permanent: [source,shell] .... % git add _files_ % git commit .... The last command will commit these changes to the branch. The editor will prompt you for a commit message. Enter one as if you were committing to FreeBSD. At this point, your work is preserved, and in the Git repository. ===== Keeping current So, time passes. It's time now to update the tree for the latest changes upstream. When you checkout `main` make sure that you have no diffs. It's a lot easier to commit those to a branch (or use `git stash`) before doing the following. If you are used to `git pull`, we strongly recommend using the `--ff-only` option, and further setting it as the default option. Alternatively, `git pull --rebase` is useful if you have changes staged in the main branch. [source,shell] .... % git config --global pull.ff only .... You may need to omit the --global if you want this setting to apply to only this repository. [source,shell] .... % cd freebsd-src % git checkout main % git pull (--ff-only|--rebase) .... There is a common trap, that the combination command `git pull` will try to perform a merge, which would sometimes creates a merge commit that didn't exist before. This can be harder to recover from. The longer form is also recommended. [source,shell] .... % cd freebsd-src % git checkout main % git fetch freebsd % git merge --ff-only freebsd/main .... These commands reset your tree to the main branch, and then update it from where you pulled the tree from originally. It's important to switch to `main` before doing this so it moves forward. Now, it's time to move the changes forward: [source,shell] .... % git rebase -i main working .... This will bring up an interactive screen to change the defaults. For now, just exit the editor. Everything should just apply. If not, then you'll need to resolve the diffs. https://docs.github.com/en/free-pro-team@latest/github/using-git/resolving-merge-conflicts-after-a-git-rebase[This github document] can help you navigate this process. [[git-push-upstream]] ===== Time to push changes upstream First, ensure that the push URL is properly configured for the upstream repository. [source,shell] .... % git remote set-url --push freebsd ssh://git@gitrepo.freebsd.org/src.git .... Then, verify that user name and email are configured right. We require that they exactly match the passwd entry in FreeBSD cluster. Use [source,shell] .... freefall% gen-gitconfig.sh .... on freefall.freebsd.org to get a recipe that you can use directly, assuming /usr/local/bin is in the PATH. The below command merges the `working` branch into the upstream main line. It's important that you curate your changes to be just like you want them in the FreeBSD source repo before doing this. This syntax pushes the `working` branch to main, moving the `main` branch forward. You will only be able to do this if this results in a linear change to `main` (e.g. no merges). [source,shell] .... % git push freebsd working:main .... If your push is rejected due to losing a commit race, rebase your branch before trying again: [source,shell] .... % git checkout working % git fetch freebsd % git rebase freebsd/main % git push freebsd working:main .... [[git-push-upstream-alt]] ===== Time to push changes upstream (alternative) Some people find it easier to merge their changes to their local `main` before pushing to the remote repository. Also, `git arc stage` moves changes from a branch to the local `main` when you need to do a subset of a branch. The instructions are similar to the prior section: [source,shell] .... % git checkout main % git merge --ff-only `working` % git push freebsd .... If you lose the race, then try again with [source,shell] .... % git pull --rebase % git push freebsd .... These commands will fetch the most recent `freebsd/main` and then rebase the local `main` changes on top of that, which is what you want when you lose the commit race. Note: merging vendor branch commits will not work with this technique. ===== Finding the Subversion Revision You'll need to make sure that you've fetched the notes (see the `No staged changes migration` section above for details. Once you have these, notes will show up in the git log command like so: [source,shell] .... % git log .... If you have a specific version in mind, you can use this construct: [source,shell] .... % git log --grep revision=XXXX .... to find the specific revision. The hex number after 'commit' is the hash you can use to refer to this commit. ==== Migrating from GitHub fork Note: as of this writing, https://github.com/freebsd/freebsd-src is mirroring all official branches, along with a `master` branch which is the legacy svn2git result. The `master` branch will not be updated anymore, and the link:https://github.com/freebsd/freebsd-src/commit/de1aa3dab23c06fec962a14da3e7b4755c5880cf[last commit] contains the instructions for migrating to the new `main` branch. We'll retain the `master` branch for a certain time, but in the future it will only be kept in the link:https://github.com/freebsd/freebsd-legacy[freebsd-legacy] repository. In addition, link:https://github.com/freebsd/git_conv/wiki/Migrating-merge-based-project-from-legacy-git-tree[this article] has an earlier version of the last commit instructions that may be helpful. When migrating branches from a GitHub fork from the old GitHub mirror to the official repo, the process is straight forward. This assumes that you have a `freebsd` upstream pointing to GitHub, adjust if necessary. This also assumes a clean tree before starting... ===== Add the new `freebsd` upstream repository: [source,shell] .... % git remote add freebsd https://git.freebsd.org/src.git % git fetch freebsd % git checkout --track freebsd/main .... ===== Rebase all your WIP branches. For each branch FOO, do the following after fetching the `freebsd` sources and creating a local `main` branch with the above checkout: [source,shell] .... % git rebase -i freebsd/master FOO --onto main .... And you'll now be tracking the official repository. You can then follow the `Keeping Current` section above to stay up to date. If you need to then commit work to FreeBSD, you can do so following the `Time to push changes upstream` instructions. You'll need to do the following once to update the push URL if you are a FreeBSD committer: [source,shell] .... % git remote set-url --push freebsd ssh://git@gitrepo.freebsd.org/src.git .... (note that gitrepo.freebsd.org will be change to repo.freebsd.org in the future.) You will also need to add `freebsd` as the location to push to. The author recommends that your upstream GitHub repository remain the default push location so that you only push things into FreeBSD you intend to by making it explicit. [[git-faq]] === Git FAQ This section provides a number of targeted answers to questions that are likely to come up often for users and developers. [NOTE] ==== We use the common convention of having the origin for the FreeBSD repository being 'freebsd' rather than the default 'origin' to allow people to use that for their own development and to minimize "whoops" pushes to the wrong repository. ==== ==== Users ===== How do I track -current and -stable with only one copy of the repository? **Q:** Although disk space is not a huge issue, it's more efficient to use only one copy of the repository. With SVN mirroring, I could checkout multiple trees from the same repository. How do I do this with Git? **A:** You can use Git worktrees. There's a number of ways to do this, but the simplest way is to use a clone to track -current, and a worktree to track stable releases. While using a 'bare repository' has been put forward as a way to cope, it's more complicated and will not be documented here. First, you need to clone the FreeBSD repository, shown here cloning into `freebsd-current` to reduce confusion. $URL is whatever mirror works best for you: [source,shell] .... % git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' $URL freebsd-current .... then once that's cloned, you can simply create a worktree from it: [source,shell] .... % cd freebsd-current % git worktree add ../freebsd-stable-12 stable/12 .... this will checkout `stable/12` into a directory named `freebsd-stable-12` that's a peer to the `freebsd-current` directory. Once created, it's updated very similarly to how you might expect: [source,shell] .... % cd freebsd-current % git checkout main % git pull --ff-only # changes from upstream now local and current tree updated % cd ../freebsd-stable-12 % git merge --ff-only freebsd/stable/12 # now your stable/12 is up to date too .... I recommend using `--ff-only` because it's safer and you avoid accidentally getting into a 'merge nightmare' where you have an extra change in your tree, forcing a complicated merge rather than a simple one. Here's https://adventurist.me/posts/00296[a good writeup] that goes into more detail. ==== Developers ===== Ooops! I committed to `main` instead of a branch. **Q:** From time to time, I goof up and commit to main instead of to a branch. What do I do? **A:** First, don't panic. Second, don't push. In fact, you can fix almost anything if you haven't pushed. All the answers in this section assume no push has happened. The following answer assumes you committed to `main` and want to create a branch called `issue`: [source,shell] .... % git branch issue # Create the 'issue' branch % git reset --hard freebsd/main # Reset 'main' back to the official tip % git checkout issue # Back to where you were .... ===== Ooops! I committed something to the wrong branch! **Q:** I was working on feature on the `wilma` branch, but accidentally committed a change relevant to the `fred` branch in 'wilma'. What do I do? **A:** The answer is similar to the previous one, but with cherry picking. This assumes there's only one commit on wilma, but will generalize to more complicated situations. It also assumes that it's the last commit on wilma (hence using wilma in the `git cherry-pick` command), but that too can be generalized. [source,shell] .... # We're on branch wilma % git checkout fred # move to fred branch % git cherry-pick wilma # copy the misplaced commit % git checkout wilma # go back to wilma branch % git reset --hard HEAD^ # move what wilma refers to back 1 commit .... Git experts would first rewind the wilma branch by 1 commit, switch over to fred and then use `git reflog` to see what that 1 deleted commit was and cherry-pick it over. **Q:** But what if I want to commit a few changes to `main`, but keep the rest in `wilma` for some reason? **A:** The same technique above also works if you are wanting to 'land' parts of the branch you are working on into `main` before the rest of the branch is ready (say you noticed an unrelated typo, or fixed an incidental bug). You can cherry pick those changes into main, then push to the parent repository. Once you've done that, cleanup couldn't be simpler: just `git rebase -i`. Git will notice you've done this and skip the common changes automatically (even if you had to change the commit message or tweak the commit slightly). There's no need to switch back to wilma to adjust it: just rebase! **Q:** I want to split off some changes from branch `wilma` into branch `fred` **A:** The more general answer would be the same as the previous. You'd checkout/create the `fred` branch, cherry pick the changes you want from `wilma` one at a time, then rebase `wilma` to remove those changes you cherry picked. `git rebase -i main wilma` will toss you into an editor, and remove the `pick` lines that correspond to the commits you copied to `fred`. If all goes well, and there are no conflicts, you're done. If not, you'll need to resolve the conflicts as you go. The other way to do this would be to checkout `wilma` and then create the branch `fred` to point to the same point in the tree. You can then `git rebase -i` both these branches, selecting the changes you want in `fred` or `wilma` by retaining the pick likes, and deleting the rest from the editor. Some people would create a tag/branch called `pre-split` before starting in case something goes wrong in the split. You can undo it with the following sequence: [source,shell] .... % git checkout pre-split # Go back % git branch -D fred # delete the fred branch % git checkout -B wilma # reset the wilma branch % git branch -d pre-split # Pretend it didn't happen .... The last step is optional. If you are going to try again to split, you'd omit it. **Q:** But I did things as I read along and didn't see your advice at the end to create a branch, and now `fred` and `wilma` are all screwed up. How do I find what `wilma` was before I started. I don't know how many times I moved things around. **A:** All is not lost. You can figure out it, so long as it hasn't been too long, or too many commits (hundreds). So I created a wilma branch and committed a couple of things to it, then decided I wanted to split it into fred and wilma. Nothing weird happened when I did that, but let's say it did. The way to look at what you've done is with the `git reflog`: [source,shell] .... % git reflog 6ff9c25 (HEAD -> wilma) HEAD@{0}: rebase -i (finish): returning to refs/heads/wilma 6ff9c25 (HEAD -> wilma) HEAD@{1}: rebase -i (start): checkout main 869cbd3 HEAD@{2}: rebase -i (start): checkout wilma a6a5094 (fred) HEAD@{3}: rebase -i (finish): returning to refs/heads/fred a6a5094 (fred) HEAD@{4}: rebase -i (pick): Encourage contributions 1ccd109 (freebsd/main, main) HEAD@{5}: rebase -i (start): checkout main 869cbd3 HEAD@{6}: rebase -i (start): checkout fred 869cbd3 HEAD@{7}: checkout: moving from wilma to fred 869cbd3 HEAD@{8}: commit: Encourage contributions ... % .... Here we see the changes I've made. You can use it to figure out where things went wrong. I'll just point out a few things here. The first one is that HEAD@{X} is a 'commitish' thing, so you can use that as an argument to a command. Although if that command commits anything to the repository, the X numbers change. You can also use the hash (first column). Next, 'Encourage contributions' was the last commit I made to `wilma` before I decided to split things up. You can also see the same hash is there when I created the `fred` branch to do that. I started by rebasing `fred` and you see the 'start', each step, and the 'finish' for that process. While we don't need it here, you can figure out exactly what happened. Fortunately, to fix this, you can follow the prior answer's steps, but with the hash `869cbd3` instead of `pre-split`. While that seems a bit verbose, it's easy to remember since you're doing one thing at a time. You can also stack: [source,shell] .... % git checkout -B wilma 869cbd3 % git branch -D fred .... and you are ready to try again. The 'checkout -B' with the hash combines checking out and creating a branch for it. The -B instead of -b forces the movement of a pre-existing branch. Either way works, which is what's great (and awful) about Git. One reason I tend to use `git checkout -B xxxx hash` instead of checking out the hash, and then creating / moving the branch is purely to avoid the slightly distressing message about detached heads: [source,shell] .... % git checkout 869cbd3 M faq.md Note: checking out '869cbd3'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b HEAD is now at 869cbd3 Encourage contributions % git checkout -B wilma .... this produces the same effect, but I have to read a lot more and severed heads aren't an image I like to contemplate. ===== Ooops! I did a `git pull` and it created a merge commit, what do I do? **Q:** I was on autopilot and did a `git pull` for my development tree and that created a merge commit on the mainline. How do I recover? **A:** This can happen when you invoke the pull with your development branch checked out. Right after the pull, you will have the new merge commit checked out. Git supports a `HEAD^#` syntax to examine the parents of a merge commit: [source,shell] .... git log --oneline HEAD^1 # Look at the first parent's commits git log --oneline HEAD^2 # Look at the second parent's commits .... From those logs, you can easily identify which commit is your development work. Then you simply reset your branch to the corresponding `HEAD^#`: [source,shell] .... git reset --hard HEAD^2 .... **Q:** But I also need to fix my `main` branch. How do I do that? **A:** Git keeps track of the remote repository branches in a `freebsd/` namespace. To fix your `main` branch, just make it point to the remote's `main`: [source,shell] .... git branch -f main freebsd/main .... There's nothing magical about branches in Git: they are just labels on a graph that are automatically moved forward by making commits. So the above works because you're just moving a label. There's no metadata about the branch that needs to be preserved due to this. ===== Mixing and matching branches **Q:** So I have two branches `worker` and `async` that I'd like to combine into one branch called `feature` while maintaining the commits in both. **A:** This is a job for cherry pick. [source,shell] .... % git checkout worker % git checkout -b feature # create a new branch % git cherry-pick main..async # bring in the changes .... You now have a new branch called `feature`. This branch combines commits from both branches. You can further curate it with `git rebase`. **Q:** I have a branch called `driver` and I'd like to break it up into `kernel` and `userland` so I can evolve them separately and commit each branch as it becomes ready. **A:** This takes a little bit of prep work, but `git rebase` will do the heavy lifting here. [source,shell] .... % git checkout driver # Checkout the driver % git checkout -b kernel # Create kernel branch % git checkout -b userland # Create userland branch .... Now you have two identical branches. So, it's time to separate out the commits. We'll assume first that all the commits in `driver` go into either the `kernel` or the `userland` branch, but not both. [source,shell] .... % git rebase -i main kernel .... and just include the changes you want (with a 'p' or 'pick' line) and just delete the commits you don't (this sounds scary, but if worse comes to worse, you can throw this all away and start over with the `driver` branch since you've not yet moved it). [source,shell] .... % git rebase -i main userland .... and do the same thing you did with the `kernel` branch. **Q:** Oh great! I followed the above and forgot a commit in the `kernel` branch. How do I recover? **A:** You can use the `driver` branch to find the hash of the commit is missing and cherry pick it. [source,shell] .... % git checkout kernel % git log driver % git cherry-pick $HASH .... **Q:** OK. I have the same situation as the above, but my commits are all mixed up. I need parts of one commit to go to one branch and the rest to go to the other. In fact, I have several. Your rebase method to select sounds tricky. **A:** In this situation, you'd be better off to curate the original branch to separate out the commits, and then use the above method to split the branch. So let's assume that there's just one commit with a clean tree. You can either use `git rebase` with an `edit` line, or you can use this with the commit on the tip. The steps are the same either way. The first thing we need to do is to back up one commit while leaving the changes uncommitted in the tree: [source,shell] .... % git reset HEAD^ .... Note: Do not, repeat do not, add `--hard` here since that also removes the changes from your tree. Now, if you are lucky, the change needing to be split up falls entirely along file lines. In that case you can just do the usual `git add` for the files in each group than do a `git commit`. Note: when you do this, you'll lose the commit message when you do the reset, so if you need it for some reason, you should save a copy (though `git log $HASH` can recover it). If you are not lucky, you'll need to split apart files. There's another tool to do that which you can apply one file at a time. [source,shell] .... git add -i foo/bar.c .... will step through the diffs, prompting you, one at time, whether to include or exclude the hunk. Once you're done, `git commit` and you'll have the remainder in your tree. You can run it multiple times as well, and even over multiple files (though I find it easier to do one file at a time and use the `git rebase -i` to fold the related commits together). ==== Cloning and Mirroring **Q:** I'd like to mirror the entire Git repository, how do I do that? **A:** If all you want to do is mirror, then [source,shell] .... % git clone --mirror $URL .... will do the trick. However, there are two disadvantages to this if you want to use it for anything other than a mirror you'll reclone. First, this is a 'bare repository' which has the repository database, but no checked out worktree. This is great for mirroring, but terrible for day to day work. There's a number of ways around this with 'git worktree': [source,shell] .... % git clone --mirror https://git.freebsd.org/ports.git ports.git % cd ports.git % git worktree add ../ports main % git worktree add ../quarterly branches/2020Q4 % cd ../ports .... But if you aren't using your mirror for further local clones, then it's a poor match. The second disadvantage is that Git normally rewrites the refs (branch name, tags, etc) from upstream so that your local refs can evolve independently of upstream. This means that you'll lose changes if you are committing to this repository on anything other than private project branches. **Q:** So what can I do instead? **A:** Well, you can stuff all of the upstream repository's refs into a private namespace in your local repository. Git clones everything via a 'refspec' and the default refspec is: [source,shell] .... fetch = +refs/heads/*:refs/remotes/freebsd/* .... which says just fetch the branch refs. However, the FreeBSD repository has a number of other things in it. To see those, you can add explicit refspecs for each ref namespace, or you can fetch everything. To setup your repository to do that: [source,shell] .... git config --add remote.freebsd.fetch '+refs/*:refs/freebsd/*' .... which will put everything in the upstream repository into your local repository's 'refs/freebsd/' namespace. Please note, that this also grabs all the unconverted vendor branches and the number of refs associated with them is quite large. You'll need to refer to these 'refs' with their full name because they aren't in and of Git's regular namespaces. [source,shell] .... git log refs/freebsd/vendor/zlib/1.2.10 .... would look at the log for the vendor branch for zlib starting at 1.2.10. === Collaborating with others One of the keys to good software development on a project as large as FreeBSD is the ability to collaborate with others before you push your changes to the tree. The FreeBSD project's Git repositories do not, yet, allow user-created branches to be pushed to the repository, and therefore if you wish to share your changes with others you must use another mechanism, such as a hosted GitLab or GitHub, in order to share changes in a user-generated branch. The following instructions show how to set up a user-generated branch, based on the FreeBSD main branch, and push it to GitHub. Before you begin, make sure that your local Git repo is up to date and has the correct origins set <> [source,shell] ```` % git remote -v freebsd https://git.freebsd.org/src.git (fetch) freebsd ssh://git@gitrepo.freebsd.org/src.git (push) ```` The first step is to create a fork of https://github.com/freebsd/freebsd-src[FreeBSD] on GitHub following these https://docs.github.com/en/github/getting-started-with-github/fork-a-repo[guidelines]. The destination of the fork should be your own, personal, GitHub account (gvnn3 in my case). Now add a remote on your local system that points to your fork: [source,shell] .... % git remote add github git@github.com:gvnn3/freebsd-src.git % git remote -v github git@github.com:gvnn3/freebsd-src.git (fetch) github git@github.com:gvnn3/freebsd-src.git (push) freebsd https://git.freebsd.org/src.git (fetch) freebsd ssh://git@gitrepo.freebsd.org/src.git (push) .... With this in place you can create a branch <> [source,shell] .... % git checkout -b gnn-pr2001-fix .... Make whatever modifications you wish in your branch. Build, test, and once you're ready to collaborate with others it's time to push your changes into your hosted branch. Before you can push you'll have to set the appropriate upstream, as Git will tell you the first time you try to push to your +github+ remote: [source,shell] .... % git push github fatal: The current branch gnn-pr2001-fix has no upstream branch. To push the current branch and set the remote as upstream, use git push --set-upstream github gnn-pr2001-fix .... Setting the push as +git+ advises allows it to succeed: [source,shell] .... % git push --set-upstream github gnn-feature Enumerating objects: 20486, done. Counting objects: 100% (20486/20486), done. Delta compression using up to 8 threads Compressing objects: 100% (12202/12202), done. Writing objects: 100% (20180/20180), 56.25 MiB | 13.15 MiB/s, done. Total 20180 (delta 11316), reused 12972 (delta 7770), pack-reused 0 remote: Resolving deltas: 100% (11316/11316), completed with 247 local objects. remote: remote: Create a pull request for 'gnn-feature' on GitHub by visiting: remote: https://github.com/gvnn3/freebsd-src/pull/new/gnn-feature remote: To github.com:gvnn3/freebsd-src.git * [new branch] gnn-feature -> gnn-feature Branch 'gnn-feature' set up to track remote branch 'gnn-feature' from 'github'. .... Subsequent changes to the same branch will push correctly by default: [source,shell] .... % git push Enumerating objects: 4, done. Counting objects: 100% (4/4), done. Delta compression using up to 8 threads Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 314 bytes | 1024 bytes/s, done. Total 3 (delta 1), reused 1 (delta 0), pack-reused 0 remote: Resolving deltas: 100% (1/1), completed with 1 local object. To github.com:gvnn3/freebsd-src.git 9e5243d7b659..cf6aeb8d7dda gnn-feature -> gnn-feature .... At this point your work is now in your branch on +GitHub+ and you can share the link with other collaborators. [[github-pull-land]] === Landing a github pull request This section documents how to land a GitHub pull request that's submitted against the FreeBSD Git mirrors at GitHub. While this is not an official way to submit patches at this time, sometimes good fixes come in this way and it is easiest just to bring them into a committer's tree and have them pushed into the FreeBSD's tree from there. Similar steps can be used to pull branches from other repositories and land those. When committing pull requests from others, one should take extra care to examine all the changes to ensure they are exactly as represented. Before beginning, make sure that the local Git repo is up to date and has the correct origins set <> In addition, make sure to have the following origins: [source,shell] .... % git remote -v freebsd https://git.freebsd.org/src.git (fetch) freebsd ssh://git@gitrepo.freebsd.org/src.git (push) github https://github.com/freebsd/freebsd-src (fetch) github https://github.com/freebsd/freebsd-src (fetch) .... Often pull requests are simple: requests that contain only a single commit. In this case, a streamlined approach may be used, though the approach in the prior section will also work. Here, a branch is created, the change is cherry picked, the commit message adjusted, and sanity-checked before being pushed. The branch `staging` is used in this example but it can be any name. This technique works for any number of commits in the pull request, especially when the changes apply cleanly to the FreeBSD tree. However, when there's multiple commits, especially when minor adjustments are needed, `git rebase -i` works better than `git cherry-pick`. Briefly, these commands create a branch; cherry-picks the changes from the pull request; tests it; adjusts the commit messages; and fast forward merges it back to `main`. The PR number is `$PR` below. When adjusting the message, add `Pull Request: https://github.com/freebsd-src/pull/$PR`. All pull requests committed to the FreeBSD repository should be reviewed by at least one person. This need not be the person committing it, but in that case the person committing it should trust the other reviewers competence to review the commit. Committers that do a code review of pull requests before pushing them into the repo should add a `Reviewed by:` line to the commit, because in this case it is not implicit. Add anybody that reviews and approves the commit on github to `Reviewed by:` as well. As always, care should be taken to ensure the change does what it is supposed to, and that no malicious code is present. [NOTE] ====== In addition, please check to make sure that the pull request author name is not anonymous. Github's web editing interface generates names like: [source,shell] .... Author: github-user <38923459+github-user@users.noreply.github.com> .... A polite request to the author for a better name and/or email should be made. Extra care should be taken to ensure no style issue or malicious code is introduced. ====== [source,shell] .... % git fetch github pull/$PR/head:staging % git rebase -i main staging # to move the staging branch forward, adjust commit message here % git checkout main % git pull --ff-only # to get the latest if time has passed % git checkout main % git merge --ff-only staging % git push freebsd --push-option=confirm-author .... [.procedure] ==== For complicated pull requests that have multiple commits with conflicts, follow the following outline. . checkout the pull request `git checkout github/pull/XXX` . create a branch to rebase `git checkout -b staging` . rebase the `staging` branch to the latest `main` with `git rebase -i main staging` . resolve conflicts and do whatever testing is needed . fast forward the `staging` branch into `main` as above . final sanity check of changes to make sure all is well . push to FreeBSD's Git repository. This will also work when bringing branches developed elsewhere into the local tree for committing. ==== Once finished with the pull request, close it using GitHub's web interface. It is worth noting that if your `github` origin uses `https://`, the only step you'll need a GitHub account for is closing the pull request. [[vcs-history]] == Version Control History The project has moved to <>. The FreeBSD source repository switched from CVS to Subversion on May 31st, 2008. The first real SVN commit is __r179447__. The source repository switched from Subversion to Git on December 23rd, 2020. The last real svn commit is __r368820__. The first real git commit hash is __5ef5f51d2bef80b0ede9b10ad5b0e9440b60518c__. The FreeBSD `doc/www` repository switched from CVS to Subversion on May 19th, 2012. The first real SVN commit is __r38821__. The documentation repository switched from Subversion to Git on December 8th, 2020. The last SVN commit is __r54737__. The first real git commit hash is __3be01a475855e7511ad755b2defd2e0da5d58bbe__. The FreeBSD `ports` repository switched from CVS to Subversion on July 14th, 2012. The first real SVN commit is __r300894__. The ports repository switched from Subversion to Git on April 6, 2021. The last SVN commit is __r569609__ The first real git commit hash is __ed8d3eda309dd863fb66e04bccaa513eee255cbf__. [[conventions]] == Setup, Conventions, and Traditions There are a number of things to do as a new developer. The first set of steps is specific to committers only. These steps must be done by a mentor for those who are not committers. [[conventions-committers]] === For New Committers Those who have been given commit rights to the FreeBSD repositories must follow these steps. * Get mentor approval before committing each of these changes! * All [.filename]#src# commits go to FreeBSD-CURRENT first before being merged to FreeBSD-STABLE. The FreeBSD-STABLE branch must maintain ABI and API compatibility with earlier versions of that branch. Do not merge changes that break this compatibility. [[commit-steps]] [.procedure] ==== *Procedure 1. Steps for New Committers* . Add an Author Entity + [.filename]#doc/shared/authors.adoc# - Add an author entity. Later steps depend on this entity, and missing this step will cause the [.filename]#doc/# build to fail. This is a relatively easy task, but remains a good first test of version control skills. . Update the List of Developers and Contributors + [.filename]#doc/shared/contrib-committers.adoc# - Add an entry, which will then appear in the "Developers" section of the extref:{contributors}[Contributors List, staff-committers]. Entries are sorted by last name. + [.filename]#doc/shared/contrib-additional.adoc# - _Remove_ the entry. Entries are sorted by first name. . Add a News Item + [.filename]#doc/website/data/en/news/news.toml# - Add an entry. Look for the other entries that announce new committers and follow the format. Use the date from the commit bit approval email from mailto:core@FreeBSD.org[core@FreeBSD.org]. . Add a PGP Key + `{des}` has written a shell script ([.filename]#doc/documentation/tools/addkey.sh#) to make this easier. See the https://cgit.freebsd.org/doc/plain/documentation/static/pgpkeys/README[README] file for more information. + Use [.filename]#doc/documentation/tools/checkkey.sh# to verify that keys meet minimal best-practices standards. + After adding and checking a key, add both updated files to source control and then commit them. Entries in this file are sorted by last name. + [NOTE] ====== It is very important to have a current PGP/GnuPG key in the repository. The key may be required for positive identification of a committer. For example, the `{admins}` might need it for account recovery. A complete keyring of `FreeBSD.org` users is available for download from link:https://docs.FreeBSD.org/pgpkeys/pgpkeys.txt[https://docs.FreeBSD.org/pgpkeys/pgpkeys.txt]. ====== . Update Mentor and Mentee Information + [.filename]#src/share/misc/committers-.dot# - Add an entry to the current committers section, where _repository_ is `doc`, `ports`, or `src`, depending on the commit privileges granted. + Add an entry for each additional mentor/mentee relationship in the bottom section. . Generate a Kerberos Password + See <> to generate or set a Kerberos for use with other FreeBSD services like the bug tracking database. . Optional: Enable Wiki Account + https://wiki.freebsd.org[FreeBSD Wiki] Account - A wiki account allows sharing projects and ideas. Those who do not yet have an account can follow instructions on the https://wiki.freebsd.org/AboutWiki[AboutWiki Page] to obtain one. Contact mailto:wiki-admin@FreeBSD.org[wiki-admin@FreeBSD.org] if you need help with your Wiki account. . Optional: Update Wiki Information + Wiki Information - After gaining access to the wiki, some people add entries to the https://wiki.freebsd.org/HowWeGotHere[How We Got Here], https://wiki.freebsd.org/IRC/Nicknames[IRC Nicks], and https://wiki.freebsd.org/Community/Dogs[Dogs of FreeBSD] pages. . Optional: Update Ports with Personal Information + [.filename]#ports/astro/xearth/files/freebsd.committers.markers# and [.filename]#src/usr.bin/calendar/calendars/calendar.freebsd# - Some people add entries for themselves to these files to show where they are located or the date of their birthday. . Optional: Prevent Duplicate Mailings + Subscribers to {dev-commits-doc-all}, {dev-commits-ports-all} or {dev-commits-src-all} might wish to unsubscribe to avoid receiving duplicate copies of commit messages and followups. ==== [[conventions-everyone]] === For Everyone [[conventions-everyone-steps]] [.procedure] ==== . Introduce yourself to the other developers, otherwise no one will have any idea who you are or what you are working on. The introduction need not be a comprehensive biography, just write a paragraph or two about who you are, what you plan to be working on as a developer in FreeBSD, and who will be your mentor. Email this to the {developers-name} and you will be on your way! . Log into `freefall.FreeBSD.org` and create a [.filename]#/var/forward/user# (where _user_ is your username) file containing the e-mail address where you want mail addressed to _yourusername_@FreeBSD.org to be forwarded. This includes all of the commit messages as well as any other mail addressed to the {committers-name} and the {developers-name}. Really large mailboxes which have taken up permanent residence on `freefall` may get truncated without warning if space needs to be freed, so forward it or save it elsewhere. + [NOTE] ====== If your e-mail system uses SPF with strict rules, you should exclude `mx2.FreeBSD.org` from SPF checks. ====== + Due to the severe load dealing with SPAM places on the central mail servers that do the mailing list processing, the front-end server does do some basic checks and will drop some messages based on these checks. At the moment proper DNS information for the connecting host is the only check in place but that may change. Some people blame these checks for bouncing valid email. To have these checks turned off for your email, create a file named [.filename]#~/.spam_lover# on `freefall.FreeBSD.org`. + [NOTE] ====== Those who are developers but not committers will not be subscribed to the committers or developers mailing lists. The subscriptions are derived from the access rights. ====== ==== [[smtp-setup]] ==== SMTP Access Setup For those willing to send e-mail messages through the FreeBSD.org infrastructure, follow the instructions below: [.procedure] ==== . Point your mail client at `smtp.FreeBSD.org:587`. . Enable STARTTLS. . Ensure your `From:` address is set to `_yourusername_@FreeBSD.org`. . For authentication, you can use your FreeBSD Kerberos username and password (see <>). The `_yourusername_/mail` principal is preferred, as it is only valid for authenticating to mail resources. + [NOTE] ====== Do not include `@FreeBSD.org` when entering in your username. ====== + .Additional Notes [NOTE] ====== * Will only accept mail from `_yourusername_@FreeBSD.org`. If you are authenticated as one user, you are not permitted to send mail from another. * A header will be appended with the SASL username: (`Authenticated sender: _username_`). * Host has various rate limits in place to cut down on brute force attempts. ====== ==== [[smtp-setup-local-mta]] ===== Using a Local MTA to Forward Emails to the FreeBSD.org SMTP Service It is also possible to use a local MTA to forward locally sent emails to the FreeBSD.org SMTP servers. [[smtp-setup-local-postfix]] .Using Postfix [example] ==== To tell a local Postfix instance that anything from `_yourusername_@FreeBSD.org` should be forwarded to the FreeBSD.org servers, add this to your [.filename]#main.cf#: [.programlisting] .... sender_dependent_relayhost_maps = hash:/usr/local/etc/postfix/relayhost_maps smtp_sasl_auth_enable = yes smtp_sasl_security_options = noanonymous smtp_sasl_password_maps = hash:/usr/local/etc/postfix/sasl_passwd smtp_use_tls = yes .... Create [.filename]#/usr/local/etc/postfix/relayhost_maps# with the following content: [.programlisting] .... yourusername@FreeBSD.org [smtp.freebsd.org]:587 .... Create [.filename]#/usr/local/etc/postfix/sasl_passwd# with the following content: [.programlisting] .... [smtp.freebsd.org]:587 yourusername:yourpassword .... If the email server is used by other people, you may want to prevent them from sending e-mails from your address. To achieve this, add this to your [.filename]#main.cf#: [.programlisting] .... smtpd_sender_login_maps = hash:/usr/local/etc/postfix/sender_login_maps smtpd_sender_restrictions = reject_known_sender_login_mismatch .... Create [.filename]#/usr/local/etc/postfix/sender_login_maps# with the following content: [.programlisting] .... yourusername@FreeBSD.org yourlocalusername .... Where _yourlocalusername_ is the SASL username used to connect to the local instance of Postfix. ==== [[smtp-setup-local-opensmtpd]] .Using OpenSMTPD [example] ==== To tell a local OpenSMTPD instance that anything from `_yourusername_@FreeBSD.org` should be forwarded to the FreeBSD.org servers, add this to your [.filename]#smtpd.conf#: [.programlisting] .... action "freebsd" relay host smtp+tls://freebsd@smtp.freebsd.org:587 auth match from any auth yourlocalusername mail-from "_yourusername_@freebsd.org" for any action "freebsd" .... Where _yourlocalusername_ is the SASL username used to connect to the local instance of OpenSMTPD. Create [.filename]#/usr/local/etc/mail/secrets# with the following content: [.programlisting] .... freebsd yourusername:yourpassword .... ==== [[mentors]] === Mentors All new developers have a mentor assigned to them for the first few months. A mentor is responsible for teaching the mentee the rules and conventions of the project and guiding their first steps in the developer community. The mentor is also personally responsible for the mentee's actions during this initial period. For committers: do not commit anything without first getting mentor approval. Document that approval with an `Approved by:` line in the commit message. When the mentor decides that a mentee has learned the ropes and is ready to commit on their own, the mentor announces it with a commit to [.filename]#mentors#. This file is in the [.filename]#admin# orphan branch of each repository. Detailed information on how to access these branches can be found in <>. [[pre-commit-review]] == Pre-Commit Review Code review is one way to increase the quality of software. The following guidelines apply to commits to the `head` (-CURRENT) branch of the `src` repository. Other branches and the `ports` and `docs` trees have their own review policies, but these guidelines generally apply to commits requiring review: * All non-trivial changes should be reviewed before they are committed to the repository. * Reviews may be conducted by email, in Bugzilla, in Phabricator, or by another mechanism. Where possible, reviews should be public. * The developer responsible for a code change is also responsible for making all necessary review-related changes. * Code review can be an iterative process, which continues until the patch is ready to be committed. Specifically, once a patch is sent out for review, it should receive an explicit "looks good" before it is committed. So long as it is explicit, this can take whatever form makes sense for the review method. * Timeouts are not a substitute for review. Sometimes code reviews will take longer than you would hope for, especially for larger features. Accepted ways to speed up review times for your patches are: * Review other people's patches. If you help out, everybody will be more willing to do the same for you; goodwill is our currency. * Ping the patch. If it is urgent, provide reasons why it is important to you to get this patch landed and ping it every couple of days. If it is not urgent, the common courtesy ping rate is one week. Remember that you are asking for valuable time from other professional developers. * Ask for help on mailing lists, IRC, etc. Others may be able to either help you directly, or suggest a reviewer. * Split your patch into multiple smaller patches that build on each other. The smaller your patch, the higher the probability that somebody will take a quick look at it. + When making large changes, it is helpful to keep this in mind from the beginning of the effort as breaking large changes into smaller ones is often difficult after the fact. Developers should participate in code reviews as both reviewers and reviewees. If someone is kind enough to review your code, you should return the favor for someone else. Note that while anyone is welcome to review and give feedback on a patch, only an appropriate subject-matter expert can approve a change. This will usually be a committer who works with the code in question on a regular basis. In some cases, no subject-matter expert may be available. In those cases, a review by an experienced developer is sufficient when coupled with appropriate testing. [[commit-log-message]] == Commit Log Messages This section contains some suggestions and traditions for how commit logs are formatted. === Why are commit messages important? When you commit a change in Git, Subversion, or another version control system (VCS), you're prompted to write some text describing the commit -- a commit message. How important is this commit message? Should you spend some significant effort writing it? Does it really matter if you write simply fixed a bug? Most projects have more than one developer and last for some length of time. Commit messages are a very important method of communicating with other developers, in the present and for the future. FreeBSD has hundreds of active developers and hundreds of thousands of commits spanning decades of history. Over that time the developer community has learned how valuable good commit messages are; sometimes these are hard-learned lessons. Commit messages serve at least three purposes: * Communicating with other developers + FreeBSD commits generate email to various mailing lists. These include the commit message along with a copy of the patch itself. Commit messages are also viewed through commands like git log. These serve to make other developers aware of changes that are ongoing; that other developer may want to test the change, may have an interest in the topic and will want to review in more detail, or may have their own projects underway that would benefit from interaction. * Making Changes Discoverable + In a large project with a long history it may be difficult to find changes of interest when investigating an issue or change in behaviour. Verbose, detailed commit messages allow searches for changes that might be relevant. For example, `git log --since 1year --grep 'USB timeout'`. * Providing historical documentation + Commit messages serve to document changes for future developers, perhaps years or decades later. This future developer may even be you, the original author. A change that seems obvious today may be decidedly not so much later on. The `git blame` command annotates each line of a source file with the change (hash and subject line) that brought it in. Having established the importance, here are elements of a good FreeBSD commit message: === Start with a subject line Commit messages should start with a single-line subject that briefly summarizes the change. The subject should, by itself, allow the reader to quickly determine if the change is of interest or not. === Keep subject lines short The subject line should be as short as possible while still retaining the required information. This is to make browsing Git log more efficient, and so that git log --oneline can display the short hash and subject on a single 80-column line. A good rule of thumb is to stay below 63 characters, and aim for about 50 or fewer if possible. === Prefix the subject line with a component, if applicable If the change relates to a specific component the subject line may be prefixed with that component name and a colon (:). ✓ `foo: Add -k option to keep temporary data` Include the prefix in the 63-character limit suggested above, so that `git log --oneline` avoids wrapping. === Capitalize the first letter of the subject Capitalize the first letter of the subject itself. The prefix, if any, is not capitalized unless necessary (e.g., `USB:` is capitalized). === Do not end the subject line with punctuation Do not end with a period or other punctuation. In this regard the subject line is like a newspaper headline. === Separate the subject and body with a blank line Separate the body from the subject with a blank line. Some trivial commits do not require a body, and will have only a subject. ✓ `ls: Fix typo in usage text` === Limit messages to 72 columns `git log` and `git format-patch` indent the commit message by four spaces. Wrapping at 72 columns provides a matching margin on the right edge. Limiting messages to 72 characters also keeps the commit message in formatted patches below RFC 2822's suggested email line length limit of 78 characters. This limit works well with a variety of tools that may render commit messages; line wrapping might be inconsistent with longer line length. === Use the present tense, imperative mood This facilitates short subject lines and provides consistency, including with automatically generated commit messages (e.g., as generated by git revert). This is important when reading a list of commit subjects. Think of the subject as finishing the sentence "when applied, this change will ...". ✓ `foo: Implement the -k (keep) option` + ✗ `foo: Implemented the -k option` + ✗ `This change implements the -k option in foo` + ✗ `-k option added` === Focus on what and why, not how Explain what the change accomplishes and why it is being done, rather than how. Do not assume that the reader is familiar with the issue. Explain the background and motivation for the change. Include benchmark data if you have it. If there are limitations or incomplete aspects of the change, describe them in the commit message. === Consider whether parts of the commit message could be code comments instead Sometimes while writing a commit message you may find yourself writing a sentence or two explaining some tricky or confusing aspect of the change. When this happens consider whether it would be valuable to have that explanation as a comment in the code itself. === Write commit messages for your future self While writing the commit message for a change you have all of the context in mind - what prompted the change, alternate approaches that were considered and rejected, limitations of the change, and so on. Imagine yourself revisiting the change a year or two in the future, and write the commit message in a way that would provide that necessary context. === Commit messages should stand alone You may include references to mailing list postings, benchmark result web sites, or code review links. However, the commit message should contain all of the relevant information in case these references are no longer available in the future. Similarly, a commit may refer to a previous commit, for example in the case of a bug fix or revert. In addition to the commit identifier (revision or hash), include the subject line from the referenced commit (or another suitable brief reference). With each VCS migration (from CVS to Subversion to Git) revision identifiers from previous systems may become difficult to follow. === Include appropriate metadata in a footer As well as including an informative message with each commit, some additional information may be needed. This information consists of one or more lines containing the key word or phrase, a colon, tabs for formatting, and then the additional information. The key words or phrases are: [.informaltable] [cols="20%,80%", frame="none"] |=== |`PR:` |The problem report (if any) which is affected (typically, by being closed) by this commit. Multiple PRs may be specified on one line, separated by commas or spaces. |`Reported by:` |The name and e-mail address of the person that reported the issue; for developers, just the username on the FreeBSD cluster. Typically used when there is no PR, for example if the issue was reported on a mailing list. |`Submitted by:` |This has been deprecated with git; submitted patches should have the author set by using `git commit --author` with a full name and valid email. |`Reviewed by:` a| The name and e-mail address of the person or people that reviewed the change; for developers, just the username on the FreeBSD cluster. If a patch was submitted to a mailing list for review, and the review was favorable, then just include the list name. If the reviewer is not a member of the project, provide the name, email, and if ports an external role like maintainer: Reviewed by a developer: [source,shell] .... Reviewed by: username .... Reviewed by a ports maintainer that is not a developer: [source,shell] .... Reviewed by: Full Name (maintainer) .... |`Tested by:` |The name and e-mail address of the person or people that tested the change; for developers, just the username on the FreeBSD cluster. |`Approved by:` a| The name and e-mail address of the person or people that approved the change; for developers, just the username on the FreeBSD cluster. There are several cases where approval is customary: * while a new committer is under mentorship * commits to an area of the tree covered by the LOCKS file (src) * during a release cycle * committing to a repo where you do not hold a commit bit (e.g. src committer committing to docs) While under mentorship, get mentor approval before the commit. Enter the mentor's username in this field, and note that they are a mentor: [source,shell] .... Approved by: username-of-mentor (mentor) .... If a team approved these commits then include the team name followed by the username of the approver in parentheses. For example: [source,shell] .... Approved by: re (username) .... |`Obtained from:` |The name of the project (if any) from which the code was obtained. Do not use this line for the name of an individual person. |`Fixes:` |The Git short hash and the title line of a commit that is fixed by this change as returned by `git log -n 1 --oneline GIT-COMMIT-HASH`. |`MFC after:` |To receive an e-mail reminder to MFC at a later date, specify the number of days, weeks, or months after which an MFC is planned. |`MFC to:` |If the commit should be merged to a subset of stable branches, specify the branch names. |`MFC with:` |If the commit should be merged together with a previous one in a single MFC commit (for example, where this commit corrects a bug in the previous change), specify the corresponding Git hash. |`MFH:` |If the commit is to be merged into a ports quarterly branch name, specify the quarterly branch. For example `2021Q2`. |`Relnotes:` |If the change is a candidate for inclusion in the release notes for the next release from the branch, set to `yes`. |`Security:` |If the change is related to a security vulnerability or security exposure, include one or more references or a description of the issue. If possible, include a VuXML URL or a CVE ID. |`Event:` |The description for the event where this commit was made. If this is a recurring event, add the year or even the month to it. For example, this could be `FooBSDcon 2019`. The idea behind this line is to put recognition to conferences, gatherings, and other types of meetups and to show that these are useful to have. Please do not use the `Sponsored by:` line for this as that is meant for organizations sponsoring certain features or developers working on them. |`Sponsored by:` |Sponsoring organizations for this change, if any. Separate multiple organizations with commas. If only a portion of the work was sponsored, or different amounts of sponsorship were provided to different authors, please give appropriate credit in parentheses after each sponsor name. For example, `Example.com (alice, code refactoring), Wormulon (bob), Momcorp (cindy)` shows that Alice was sponsored by Example.com to do code refactoring, while Wormulon sponsored Bob's work and Momcorp sponsored Cindy's work. Other authors were either not sponsored or chose not to list sponsorship. |`Pull Request:` |This change was submitted as a pull request or merge request against one of FreeBSD's public read-only Git repositories. It should include the entire URL to the pull request, as these often act as code reviews for the code. For example: `https://github.com/freebsd/freebsd-src/pull/745` |`Signed-off-by:` |ID certifies compliance with https://developercertificate.org/ |`Differential Revision:` |The full URL of the Phabricator review. This line __must be the last line__. For example: `https://reviews.freebsd.org/D1708`. |=== .Commit Log for a Commit Based on a PR [example] ==== The commit is based on a patch from a PR submitted by John Smith. The commit message "PR" field is filled. [.programlisting] .... ... PR: 12345 .... The committer sets the author of the patch with `git commit --author "John Smith "`. ==== .Commit Log for a Commit Needing Review [example] ==== The virtual memory system is being changed. After posting patches to the appropriate mailing list (in this case, `freebsd-arch`) and the changes have been approved. [.programlisting] .... ... Reviewed by: -arch .... ==== .Commit Log for a Commit Needing Approval [example] ==== Commit a port, after working with the listed MAINTAINER, who said to go ahead and commit. [.programlisting] .... ... Approved by: abc (maintainer) .... Where _abc_ is the account name of the person who approved. ==== .Commit Log for a Commit Bringing in Code from OpenBSD [example] ==== Committing some code based on work done in the OpenBSD project. [.programlisting] .... ... Obtained from: OpenBSD .... ==== .Commit Log for a Change to FreeBSD-CURRENT with a Planned Commit to FreeBSD-STABLE to Follow at a Later Date. [example] ==== Committing some code which will be merged from FreeBSD-CURRENT into the FreeBSD-STABLE branch after two weeks. [.programlisting] .... ... MFC after: 2 weeks .... Where _2_ is the number of days, weeks, or months after which an MFC is planned. The _weeks_ option may be `day`, `days`, `week`, `weeks`, `month`, `months`. ==== It is often necessary to combine these. Consider the situation where a user has submitted a PR containing code from the NetBSD project. Looking at the PR, the developer sees it is not an area of the tree they normally work in, so they have the change reviewed by the `arch` mailing list. Since the change is complex, the developer opts to MFC after one month to allow adequate testing. The extra information to include in the commit would look something like .Example Combined Commit Log [example] ==== [.programlisting] .... PR: 54321 Reviewed by: -arch Obtained from: NetBSD MFC after: 1 month Relnotes: yes .... ==== [[pref-license]] == Preferred License for New Files The FreeBSD Project's full license policy can be found at link:https://www.FreeBSD.org/internal/software-license/[https://www.FreeBSD.org/internal/software-license]. The rest of this section is intended to help you get started. As a rule, when in doubt, ask. It is much easier to give advice than to fix the source tree. The FreeBSD Project suggests and uses this text as the preferred license scheme: [.programlisting] .... /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * * Copyright (c) [year] [your name] * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * [id for your version control system, if any] */ .... The FreeBSD project strongly discourages the so-called "advertising clause" in new code. Due to the large number of contributors to the FreeBSD project, complying with this clause for many commercial vendors has become difficult. If you have code in the tree with the advertising clause, please consider removing it. In fact, please consider using the above license for your code. The FreeBSD project discourages completely new licenses and variations on the standard licenses. New licenses require the approval of the {core-email} to reside in the main repository. The more different licenses that are used in the tree, the more problems that this causes to those wishing to utilize this code, typically from unintended consequences from a poorly worded license. Project policy dictates that code under some non-BSD licenses must be placed only in specific sections of the repository, and in some cases, compilation must be conditional or even disabled by default. For example, the GENERIC kernel must be compiled under only licenses identical to or substantially similar to the BSD license. GPL, APSL, CDDL, etc, licensed software must not be compiled into GENERIC. Developers are reminded that in open source, getting "open" right is just as important as getting "source" right, as improper handling of intellectual property has serious consequences. Any questions or concerns should immediately be brought to the attention of the core team. [[tracking.license.grants]] == Keeping Track of Licenses Granted to the FreeBSD Project Various software or data exist in the repositories where the FreeBSD project has been granted a special license to be able to use them. A case in point are the Terminus fonts for use with man:vt[4]. Here the author Dimitar Zhekov has allowed us to use the "Terminus BSD Console" font under a 2-clause BSD license rather than the regular Open Font License he normally uses. It is clearly sensible to keep a record of any such license grants. To that end, the {core-email} has decided to keep an archive of them. Whenever the FreeBSD project is granted a special license we require the {core-email} to be notified. Any developers involved in arranging such a license grant, please send details to the {core-email} including: * Contact details for people or organizations granting the special license. * What files, directories etc. in the repositories are covered by the license grant including the revision numbers where any specially licensed material was committed. * The date the license comes into effect from. Unless otherwise agreed, this will be the date the license was issued by the authors of the software in question. * The license text. * A note of any restrictions, limitations or exceptions that apply specifically to FreeBSD's usage of the licensed material. * Any other relevant information. Once the {core-email} is satisfied that all the necessary details have been gathered and are correct, the secretary will send a PGP-signed acknowledgment of receipt including the license details. This receipt will be persistently archived and serve as our permanent record of the license grant. The license archive should contain only details of license grants; this is not the place for any discussions around licensing or other subjects. Access to data within the license archive will be available on request to the {core-email}. [[spdx.tags]] == SPDX Tags in the tree The project uses https://spdx.dev[SPDX] tags in our source base. At present, these tags are indented to help automated tools reconstruct license requirements mechanically. All _SPDX-License-Identifier_ tags in the tree should be considered to be informative. All files in the FreeBSD source tree with these tags also have a copy of the license which governs use of that file. In the event of a discrepancy, the verbatim license is controlling. The project tries to follow the https://spdx.github.io/spdx-spec/[SPDX Specification, Version 2.2]. How to mark source files and valid algebraic expressions are found in https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/[Appendix IV] and https://spdx.github.io/spdx-spec/appendix-V-using-SPDX-short-identifiers-in-source-files/[Appendix V]. The project draws identifiers from SPDX's list of valid https://spdx.org/licenses/[short license identifiers]. The project uses only the _SPDX-License-Identifier_ tag. As of March 2021, approximately 25,000 out of 90,000 files in the tree have been marked. [[developer.relations]] == Developer Relations When working directly on your own code or on code which is already well established as your responsibility, then there is probably little need to check with other committers before jumping in with a commit. Working on a bug in an area of the system which is clearly orphaned (and there are a few such areas, to our shame), the same applies. When modifying parts of the system which are maintained, formally, or informally, consider asking for review just as a developer would have before becoming a committer. For ports, contact the listed `MAINTAINER` in the [.filename]#Makefile#. To determine if an area of the tree is maintained, check the MAINTAINERS file at the root of the tree. If nobody is listed, scan the revision history to see who has committed changes in the past. An example script that lists each person who has committed to a given file along with the number of commits each person has made can be found at on `freefall` at [.filename]#~eadler/bin/whodid#. If queries go unanswered or the committer otherwise indicates a lack of interest in the area affected, go ahead and commit it. [IMPORTANT] ==== Avoid sending private emails to maintainers. Other people might be interested in the conversation, not just the final output. ==== If there is any doubt about a commit for any reason at all, have it reviewed before committing. Better to have it flamed then and there rather than when it is part of the repository. If a commit does results in controversy erupting, it may be advisable to consider backing the change out again until the matter is settled. Remember, with a version control system we can always change it back. Do not impugn the intentions of others. If they see a different solution to a problem, or even a different problem, it is probably not because they are stupid, because they have questionable parentage, or because they are trying to destroy hard work, personal image, or FreeBSD, but basically because they have a different outlook on the world. Different is good. Disagree honestly. Argue your position from its merits, be honest about any shortcomings it may have, and be open to seeing their solution, or even their vision of the problem, with an open mind. Accept correction. We are all fallible. When you have made a mistake, apologize and get on with life. Do not beat up yourself, and certainly do not beat up others for your mistake. Do not waste time on embarrassment or recrimination, just fix the problem and move on. Ask for help. Seek out (and give) peer reviews. One of the ways open source software is supposed to excel is in the number of eyeballs applied to it; this does not apply if nobody will review code. [[if-in-doubt]] == If in Doubt... When unsure about something, whether it be a technical issue or a project convention be sure to ask. If you stay silent you will never make progress. If it relates to a technical issue ask on the public mailing lists. Avoid the temptation to email the individual person that knows the answer. This way everyone will be able to learn from the question and the answer. For project specific or administrative questions ask, in order: * Your mentor or former mentor. * An experienced committer on IRC, email, etc. * Any team with a "hat", as they can give you a definitive answer. * If still not sure, ask on {developers-name}. Once your question is answered, if no one pointed you to documentation that spelled out the answer to your question, document it, as others will have the same question. [[bugzilla]] == Bugzilla The FreeBSD Project utilizes Bugzilla for tracking bugs and change requests. Be sure that if you commit a fix or suggestion found in the PR database to close it. It is also considered nice if you take time to close any PRs associated with your commits, if appropriate. Committers with non-``FreeBSD.org`` Bugzilla accounts can have the old account merged with the `FreeBSD.org` account by following these steps: [.procedure] ==== . Log in using your old account. . Open new bug. Choose `Services` as the Product, and `Bug Tracker` as the Component. In bug description list accounts you wish to be merged. . Log in using `FreeBSD.org` account and post comment to newly opened bug to confirm ownership. See <> for more details on how to generate or set a password for your `FreeBSD.org` account. . If there are more than two accounts to merge, post comments from each of them. ==== You can find out more about Bugzilla at: * extref:{pr-guidelines}[FreeBSD Problem Report Handling Guidelines] * link:https://www.FreeBSD.org/support/[https://www.FreeBSD.org/support] [[phabricator]] == Phabricator The FreeBSD Project utilizes https://reviews.freebsd.org[Phabricator] for code review requests. See the https://wiki.freebsd.org/CodeReview[CodeReview] wiki page for details. Committers with non-``FreeBSD.org`` Phabricator accounts can have the old account renamed to the ``FreeBSD.org`` account by following these steps: [.procedure] ==== . Change your Phabricator account email to your `FreeBSD.org` email. . Open new bug on our bug tracker using your `FreeBSD.org` account, see <> for more information. Choose `Services` as the Product, and `Code Review` as the Component. In bug description request that your Phabricator account be renamed, and provide a link to your Phabricator user. For example, `https://reviews.freebsd.org/p/bob_example.com/` ==== [IMPORTANT] ==== Phabricator accounts cannot be merged, please do not open a new account. ==== [[people]] == Who's Who Besides the repository meisters, there are other FreeBSD project members and teams whom you will probably get to know in your role as a committer. Briefly, and by no means all-inclusively, these are: `{doceng}`:: doceng is the group responsible for the documentation build infrastructure, approving new documentation committers, and ensuring that the FreeBSD website and documentation on the FTP site is up to date with respect to the Subversion tree. It is not a conflict resolution body. The vast majority of documentation related discussion takes place on the {freebsd-doc}. More details regarding the doceng team can be found in its https://www.FreeBSD.org/internal/doceng/[charter]. Committers interested in contributing to the documentation should familiarize themselves with the extref:{fdp-primer}[Documentation Project Primer]. `{re-members}`:: These are the members of the `{re}`. This team is responsible for setting release deadlines and controlling the release process. During code freezes, the release engineers have final authority on all changes to the system for whichever branch is pending release status. If there is something you want merged from FreeBSD-CURRENT to FreeBSD-STABLE (whatever values those may have at any given time), these are the people to talk to about it. `{so}`:: `{so-name}` is the link:https://www.FreeBSD.org/security/[FreeBSD Security Officer] and oversees the `{security-officer}`. `{wollman}`:: If you need advice on obscure network internals or are not sure of some potential change to the networking subsystem you have in mind, Garrett is someone to talk to. Garrett is also very knowledgeable on the various standards applicable to FreeBSD. {committers-name}:: {svn-src-all}, {svn-ports-all} and {svn-doc-all} are the mailing lists that the version control system uses to send commit messages to. _Never_ send email directly to these lists. Only send replies to this list when they are short and are directly related to a commit. {developers-name}:: All committers are subscribed to -developers. This list was created to be a forum for the committers "community" issues. Examples are Core voting, announcements, etc. + The {developers-name} is for the exclusive use of FreeBSD committers. To develop FreeBSD, committers must have the ability to openly discuss matters that will be resolved before they are publicly announced. Frank discussions of work in progress are not suitable for open publication and may harm FreeBSD. + All FreeBSD committers are expected not to not publish or forward messages from the {developers-name} outside the list membership without permission of all of the authors. Violators will be removed from the {developers-name}, resulting in a suspension of commit privileges. Repeated or flagrant violations may result in permanent revocation of commit privileges. + This list is _not_ intended as a place for code reviews or for any technical discussion. In fact using it as such hurts the FreeBSD Project as it gives a sense of a closed list where general decisions affecting all of the FreeBSD using community are made without being "open". Last, but not least __never, never ever, email the {developers-name} and CC:/BCC: another FreeBSD list__. Never, ever email another FreeBSD email list and CC:/BCC: the {developers-name}. Doing so can greatly diminish the benefits of this list. [[ssh.guide]] == SSH Quick-Start Guide [.procedure] ==== . If you do not wish to type your password in every time you use man:ssh[1], and you use keys to authenticate, man:ssh-agent[1] is there for your convenience. If you want to use man:ssh-agent[1], make sure that you run it before running other applications. X users, for example, usually do this from their [.filename]#.xsession# or [.filename]#.xinitrc#. See man:ssh-agent[1] for details. . Generate a key pair using man:ssh-keygen[1]. The key pair will wind up in your [.filename]#$HOME/.ssh/# directory. + [IMPORTANT] ====== Only ECDSA, Ed25519 or RSA keys are supported. ====== . Send your public key ([.filename]#$HOME/.ssh/id_ecdsa.pub#, [.filename]#$HOME/.ssh/id_ed25519.pub#, or [.filename]#$HOME/.ssh/id_rsa.pub#) to the person setting you up as a committer so it can be put into [.filename]#yourlogin# in [.filename]#/etc/ssh-keys/# on `freefall`. ==== Now man:ssh-add[1] can be used for authentication once per session. It prompts for the private key's pass phrase, and then stores it in the authentication agent (man:ssh-agent[1]). Use `ssh-add -d` to remove keys stored in the agent. Test with a simple remote command: `ssh freefall.FreeBSD.org ls /usr`. For more information, see package:security/openssh-portable[], man:ssh[1], man:ssh-add[1], man:ssh-agent[1], man:ssh-keygen[1], and man:scp[1]. For information on adding, changing, or removing man:ssh[1] keys, see https://wiki.freebsd.org/clusteradm/ssh-keys[this article]. [[coverity]] == Coverity(R) Availability for FreeBSD Committers All FreeBSD developers can obtain access to Coverity analysis results of all FreeBSD Project software. All who are interested in obtaining access to the analysis results of the automated Coverity runs, can sign up at http://scan.coverity.com/[Coverity Scan]. The FreeBSD wiki includes a mini-guide for developers who are interested in working with the Coverity(R) analysis reports: https://wiki.freebsd.org/CoverityPrevent[https://wiki.freebsd.org/CoverityPrevent]. Please note that this mini-guide is only readable by FreeBSD developers, so if you cannot access this page, you will have to ask someone to add you to the appropriate Wiki access list. Finally, all FreeBSD developers who are going to use Coverity(R) are always encouraged to ask for more details and usage information, by posting any questions to the mailing list of the FreeBSD developers. [[rules]] == The FreeBSD Committers' Big List of Rules Everyone involved with the FreeBSD project is expected to abide by the _Code of Conduct_ available from link:https://www.FreeBSD.org/internal/code-of-conduct/[https://www.FreeBSD.org/internal/code-of-conduct]. As committers, you form the public face of the project, and how you behave has a vital impact on the public perception of it. This guide expands on the parts of the _Code of Conduct_ specific to committers. . Respect other committers. . Respect other contributors. . Discuss any significant change _before_ committing. . Respect existing maintainers (if listed in the `MAINTAINER` field in [.filename]#Makefile# or in [.filename]#MAINTAINER# in the top-level directory). . Any disputed change must be backed out pending resolution of the dispute if requested by a maintainer. Security related changes may override a maintainer's wishes at the Security Officer's discretion. . Changes go to FreeBSD-CURRENT before FreeBSD-STABLE unless specifically permitted by the release engineer or unless they are not applicable to FreeBSD-CURRENT. Any non-trivial or non-urgent change which is applicable should also be allowed to sit in FreeBSD-CURRENT for at least 3 days before merging so that it can be given sufficient testing. The release engineer has the same authority over the FreeBSD-STABLE branch as outlined for the maintainer in rule #5. . Do not fight in public with other committers; it looks bad. . Respect all code freezes and read the `committers` and `developers` mailing lists in a timely manner so you know when a code freeze is in effect. . When in doubt on any procedure, ask first! . Test your changes before committing them. . Do not commit to contributed software without _explicit_ approval from the respective maintainers. As noted, breaking some of these rules can be grounds for suspension or, upon repeated offense, permanent removal of commit privileges. Individual members of core have the power to temporarily suspend commit privileges until core as a whole has the chance to review the issue. In case of an "emergency" (a committer doing damage to the repository), a temporary suspension may also be done by the repository meisters. Only a 2/3 majority of core has the authority to suspend commit privileges for longer than a week or to remove them permanently. This rule does not exist to set core up as a bunch of cruel dictators who can dispose of committers as casually as empty soda cans, but to give the project a kind of safety fuse. If someone is out of control, it is important to be able to deal with this immediately rather than be paralyzed by debate. In all cases, a committer whose privileges are suspended or revoked is entitled to a "hearing" by core, the total duration of the suspension being determined at that time. A committer whose privileges are suspended may also request a review of the decision after 30 days and every 30 days thereafter (unless the total suspension period is less than 30 days). A committer whose privileges have been revoked entirely may request a review after a period of 6 months has elapsed. This review policy is _strictly informal_ and, in all cases, core reserves the right to either act on or disregard requests for review if they feel their original decision to be the right one. In all other aspects of project operation, core is a subset of committers and is bound by the __same rules__. Just because someone is in core this does not mean that they have special dispensation to step outside any of the lines painted here; core's "special powers" only kick in when it acts as a group, not on an individual basis. As individuals, the core team members are all committers first and core second. === Details [[respect]] . Respect other committers. + This means that you need to treat other committers as the peer-group developers that they are. Despite our occasional attempts to prove the contrary, one does not get to be a committer by being stupid and nothing rankles more than being treated that way by one of your peers. Whether we always feel respect for one another or not (and everyone has off days), we still have to _treat_ other committers with respect at all times, on public forums and in private email. + Being able to work together long term is this project's greatest asset, one far more important than any set of changes to the code, and turning arguments about code into issues that affect our long-term ability to work harmoniously together is just not worth the trade-off by any conceivable stretch of the imagination. + To comply with this rule, do not send email when you are angry or otherwise behave in a manner which is likely to strike others as needlessly confrontational. First calm down, then think about how to communicate in the most effective fashion for convincing the other persons that your side of the argument is correct, do not just blow off some steam so you can feel better in the short term at the cost of a long-term flame war. Not only is this very bad "energy economics", but repeated displays of public aggression which impair our ability to work well together will be dealt with severely by the project leadership and may result in suspension or termination of your commit privileges. The project leadership will take into account both public and private communications brought before it. It will not seek the disclosure of private communications, but it will take it into account if it is volunteered by the committers involved in the complaint. + All of this is never an option which the project's leadership enjoys in the slightest, but unity comes first. No amount of code or good advice is worth trading that away. . Respect other contributors. + You were not always a committer. At one time you were a contributor. Remember that at all times. Remember what it was like trying to get help and attention. Do not forget that your work as a contributor was very important to you. Remember what it was like. Do not discourage, belittle, or demean contributors. Treat them with respect. They are our committers in waiting. They are every bit as important to the project as committers. Their contributions are as valid and as important as your own. After all, you made many contributions before you became a committer. Always remember that. + Consider the points raised under <> and apply them also to contributors. . Discuss any significant change _before_ committing. + The repository is not where changes are initially submitted for correctness or argued over, that happens first in the mailing lists or by use of the Phabricator service. The commit will only happen once something resembling consensus has been reached. This does not mean that permission is required before correcting every obvious syntax error or manual page misspelling, just that it is good to develop a feel for when a proposed change is not quite such a no-brainer and requires some feedback first. People really do not mind sweeping changes if the result is something clearly better than what they had before, they just do not like being _surprised_ by those changes. The very best way of making sure that things are on the right track is to have code reviewed by one or more other committers. + When in doubt, ask for review! . Respect existing maintainers if listed. + Many parts of FreeBSD are not "owned" in the sense that any specific individual will jump up and yell if you commit a change to "their" area, but it still pays to check first. One convention we use is to put a maintainer line in the [.filename]#Makefile# for any package or subtree which is being actively maintained by one or more people; see extref:{developers-handbook}[Source Tree Guidelines and Policies, policies] for documentation on this. Where sections of code have several maintainers, commits to affected areas by one maintainer need to be reviewed by at least one other maintainer. In cases where the "maintainer-ship" of something is not clear, look at the repository logs for the files in question and see if someone has been working recently or predominantly in that area. . Any disputed change must be backed out pending resolution of the dispute if requested by a maintainer. Security related changes may override a maintainer's wishes at the Security Officer's discretion. + This may be hard to swallow in times of conflict (when each side is convinced that they are in the right, of course) but a version control system makes it unnecessary to have an ongoing dispute raging when it is far easier to simply reverse the disputed change, get everyone calmed down again and then try to figure out what is the best way to proceed. If the change turns out to be the best thing after all, it can be easily brought back. If it turns out not to be, then the users did not have to live with the bogus change in the tree while everyone was busily debating its merits. People _very_ rarely call for back-outs in the repository since discussion generally exposes bad or controversial changes before the commit even happens, but on such rare occasions the back-out should be done without argument so that we can get immediately on to the topic of figuring out whether it was bogus or not. . Changes go to FreeBSD-CURRENT before FreeBSD-STABLE unless specifically permitted by the release engineer or unless they are not applicable to FreeBSD-CURRENT. Any non-trivial or non-urgent change which is applicable should also be allowed to sit in FreeBSD-CURRENT for at least 3 days before merging so that it can be given sufficient testing. The release engineer has the same authority over the FreeBSD-STABLE branch as outlined in rule #5. + This is another "do not argue about it" issue since it is the release engineer who is ultimately responsible (and gets beaten up) if a change turns out to be bad. Please respect this and give the release engineer your full cooperation when it comes to the FreeBSD-STABLE branch. The management of FreeBSD-STABLE may frequently seem to be overly conservative to the casual observer, but also bear in mind the fact that conservatism is supposed to be the hallmark of FreeBSD-STABLE and different rules apply there than in FreeBSD-CURRENT. There is also really no point in having FreeBSD-CURRENT be a testing ground if changes are merged over to FreeBSD-STABLE immediately. Changes need a chance to be tested by the FreeBSD-CURRENT developers, so allow some time to elapse before merging unless the FreeBSD-STABLE fix is critical, time sensitive or so obvious as to make further testing unnecessary (spelling fixes to manual pages, obvious bug/typo fixes, etc.) In other words, apply common sense. + Changes to the security branches (for example, `releng/9.3`) must be approved by a member of the `{security-officer}`, or in some cases, by a member of the `{re}`. . Do not fight in public with other committers; it looks bad. + This project has a public image to uphold and that image is very important to all of us, especially if we are to continue to attract new members. There will be occasions when, despite everyone's very best attempts at self-control, tempers are lost and angry words are exchanged. The best thing that can be done in such cases is to minimize the effects of this until everyone has cooled back down. Do not air angry words in public and do not forward private correspondence or other private communications to public mailing lists, mail aliases, instant messaging channels or social media sites. What people say one-to-one is often much less sugar-coated than what they would say in public, and such communications therefore have no place there - they only serve to inflame an already bad situation. If the person sending a flame-o-gram at least had the grace to send it privately, then have the grace to keep it private yourself. If you feel you are being unfairly treated by another developer, and it is causing you anguish, bring the matter up with core rather than taking it public. Core will do its best to play peace makers and get things back to sanity. In cases where the dispute involves a change to the codebase and the participants do not appear to be reaching an amicable agreement, core may appoint a mutually-agreeable third party to resolve the dispute. All parties involved must then agree to be bound by the decision reached by this third party. . Respect all code freezes and read the `committers` and `developers` mailing list on a timely basis so you know when a code freeze is in effect. + Committing unapproved changes during a code freeze is a really big mistake and committers are expected to keep up-to-date on what is going on before jumping in after a long absence and committing 10 megabytes worth of accumulated stuff. People who abuse this on a regular basis will have their commit privileges suspended until they get back from the FreeBSD Happy Reeducation Camp we run in Greenland. . When in doubt on any procedure, ask first! + Many mistakes are made because someone is in a hurry and just assumes they know the right way of doing something. If you have not done it before, chances are good that you do not actually know the way we do things and really need to ask first or you are going to completely embarrass yourself in public. There is no shame in asking "how in the heck do I do this?" We already know you are an intelligent person; otherwise, you would not be a committer. . Test your changes before committing them. + This may sound obvious, but if it really were so obvious then we probably would not see so many cases of people clearly not doing this. If your changes are to the kernel, make sure you can still compile both GENERIC and LINT. If your changes are anywhere else, make sure you can still make world. If your changes are to a branch, make sure your testing occurs with a machine which is running that code. If you have a change which also may break another architecture, be sure and test on all supported architectures. Please refer to the https://www.FreeBSD.org/internal/[FreeBSD Internal Page] for a list of available resources. As other architectures are added to the FreeBSD supported platforms list, the appropriate shared testing resources will be made available. . Do not commit to contributed software without _explicit_ approval from the respective maintainers. + Contributed software is anything under the [.filename]#src/contrib#, [.filename]#src/crypto#, or [.filename]#src/sys/contrib# trees. + The trees mentioned above are for contributed software usually imported onto a vendor branch. Committing something there may cause unnecessary headaches when importing newer versions of the software. As a general consider sending patches upstream to the vendor. Patches may be committed to FreeBSD first with permission of the maintainer. + Reasons for modifying upstream software range from wanting strict control over a tightly coupled dependency to lack of portability in the canonical repository's distribution of their code. Regardless of the reason, effort to minimize the maintenance burden of fork is helpful to fellow maintainers. Avoid committing trivial or cosmetic changes to files since it makes every merge thereafter more difficult: such patches need to be manually re-verified every import. + If a particular piece of software lacks a maintainer, you are encouraged to take up ownership. If you are unsure of the current maintainership email {freebsd-arch} and ask. === Policy on Multiple Architectures FreeBSD has added several new architecture ports during recent release cycles and is truly no longer an i386(TM) centric operating system. In an effort to make it easier to keep FreeBSD portable across the platforms we support, core has developed this mandate: [.blockquote] Our 32-bit reference platform is i386, and our 64-bit reference platform is amd64. Major design work (including major API and ABI changes) must prove itself on at least one 32-bit and at least one 64-bit platform, preferably the primary reference platforms, before it may be committed to the source tree. The i386 and amd64 platforms were chosen due to being more readily available to developers and as representatives of more diverse processor and system designs - big versus little endian, register file versus register stack, different DMA and cache implementations, hardware page tables versus software TLB management etc. We will continue to re-evaluate this policy as cost and availability of the 64-bit platforms change. Developers should also be aware of our Tier Policy for the long term support of hardware architectures. The rules here are intended to provide guidance during the development process, and are distinct from the requirements for features and architectures listed in that section. The Tier rules for feature support on architectures at release-time are more strict than the rules for changes during the development process. === Other Suggestions When committing documentation changes, use a spell checker before committing. For all XML docs, verify that the formatting directives are correct by running `make lint` and package:textproc/igor[]. For manual pages, run package:sysutils/manck[] and package:textproc/igor[] over the manual page to verify all of the cross references and file references are correct and that the man page has all of the appropriate `MLINKS` installed. Do not mix style fixes with new functionality. A style fix is any change which does not modify the functionality of the code. Mixing the changes obfuscates the functionality change when asking for differences between revisions, which can hide any new bugs. Do not include whitespace changes with content changes in commits to [.filename]#doc/#. The extra clutter in the diffs makes the translators' job much more difficult. Instead, make any style or whitespace changes in separate commits that are clearly labeled as such in the commit message. === Deprecating Features When it is necessary to remove functionality from software in the base system, follow these guidelines whenever possible: . Mention is made in the manual page and possibly the release notes that the option, utility, or interface is deprecated. Use of the deprecated feature generates a warning. . The option, utility, or interface is preserved until the next major (point zero) release. . The option, utility, or interface is removed and no longer documented. It is now obsolete. It is also generally a good idea to note its removal in the release notes. === Privacy and Confidentiality . Most FreeBSD business is done in public. + FreeBSD is an _open_ project. Which means that not only can anyone use the source code, but that most of the development process is open to public scrutiny. . Certain sensitive matters must remain private or held under embargo. + There unfortunately cannot be complete transparency. As a FreeBSD developer you will have a certain degree of privileged access to information. Consequently you are expected to respect certain requirements for confidentiality. Sometimes the need for confidentiality comes from external collaborators or has a specific time limit. Mostly though, it is a matter of not releasing private communications. . The Security Officer has sole control over the release of security advisories. + Where there are security problems that affect many different operating systems, FreeBSD frequently depends on early access to be able to prepare advisories for coordinated release. Unless FreeBSD developers can be trusted to maintain security, such early access will not be made available. The Security Officer is responsible for controlling pre-release access to information about vulnerabilities, and for timing the release of all advisories. He may request help under condition of confidentiality from any developer with relevant knowledge to prepare security fixes. . Communications with Core are kept confidential for as long as necessary. + Communications to core will initially be treated as confidential. Eventually however, most of Core's business will be summarized into the monthly or quarterly core reports. Care will be taken to avoid publicising any sensitive details. Records of some particularly sensitive subjects may not be reported on at all and will be retained only in Core's private archives. . Non-disclosure Agreements may be required for access to certain commercially sensitive data. + Access to certain commercially sensitive data may only be available under a Non-Disclosure Agreement. The FreeBSD Foundation legal staff must be consulted before any binding agreements are entered into. . Private communications must not be made public without permission. + Beyond the specific requirements above there is a general expectation not to publish private communications between developers without the consent of all parties involved. Ask permission before forwarding a message onto a public mailing list, or posting it to a forum or website that can be accessed by other than the original correspondents. . Communications on project-only or restricted access channels must be kept private. + Similarly to personal communications, certain internal communications channels, including FreeBSD Committer only mailing lists and restricted access IRC channels are considered private communications. Permission is required to publish material from these sources. . Core may approve publication. + Where it is impractical to obtain permission due to the number of correspondents or where permission to publish is unreasonably withheld, Core may approve release of such private matters that merit more general publication. [[archs]] == Support for Multiple Architectures FreeBSD is a highly portable operating system intended to function on many different types of hardware architectures. Maintaining clean separation of Machine Dependent (MD) and Machine Independent (MI) code, as well as minimizing MD code, is an important part of our strategy to remain agile with regards to current hardware trends. Each new hardware architecture supported by FreeBSD adds substantially to the cost of code maintenance, toolchain support, and release engineering. It also dramatically increases the cost of effective testing of kernel changes. As such, there is strong motivation to differentiate between classes of support for various architectures while remaining strong in a few key architectures that are seen as the FreeBSD "target audience". === Statement of General Intent The FreeBSD Project targets "production quality commercial off-the-shelf (COTS) workstation, server, and high-end embedded systems". By retaining a focus on a narrow set of architectures of interest in these environments, the FreeBSD Project is able to maintain high levels of quality, stability, and performance, as well as minimize the load on various support teams on the project, such as the ports team, documentation team, security officer, and release engineering teams. Diversity in hardware support broadens the options for FreeBSD consumers by offering new features and usage opportunities, but these benefits must always be carefully considered in terms of the real-world maintenance cost associated with additional platform support. The FreeBSD Project differentiates platform targets into four tiers. Each tier includes a list of guarantees consumers may rely on as well as obligations by the Project and developers to fulfill those guarantees. These lists define the minimum guarantees for each tier. The Project and developers may provide additional levels of support beyond the minimum guarantees for a given tier, but such additional support is not guaranteed. Each platform target is assigned to a specific tier for each stable branch. As a result, a platform target might be assigned to different tiers on concurrent stable branches. === Platform Targets Support for a hardware platform consists of two components: kernel support and userland Application Binary Interfaces (ABIs). Kernel platform support includes things needed to run a FreeBSD kernel on a hardware platform such as machine-dependent virtual memory management and device drivers. A userland ABI specifies an interface for user processes to interact with a FreeBSD kernel and base system libraries. A userland ABI includes system call interfaces, the layout and semantics of public data structures, and the layout and semantics of arguments passed to subroutines. Some components of an ABI may be defined by specifications such as the layout of C++ exception objects or calling conventions for C functions. A FreeBSD kernel also uses an ABI (sometimes referred to as the Kernel Binary Interface (KBI)) which includes the semantics and layouts of public data structures and the layout and semantics of arguments to public functions within the kernel itself. A FreeBSD kernel may support multiple userland ABIs. For example, FreeBSD's amd64 kernel supports FreeBSD amd64 and i386 userland ABIs as well as Linux x86_64 and i386 userland ABIs. A FreeBSD kernel should support a "native" ABI as the default ABI. The native "ABI" generally shares certain properties with the kernel ABI such as the C calling convention, sizes of basic types, etc. Tiers are defined for both kernels and userland ABIs. In the common case, a platform's kernel and FreeBSD ABIs are assigned to the same tier. === Tier 1: Fully-Supported Architectures Tier 1 platforms are the most mature FreeBSD platforms. -They are supported by the security officer, release engineering, and port management teams. +They are supported by the security officer, release engineering, and Ports Management Team. Tier 1 architectures are expected to be Production Quality with respect to all aspects of the FreeBSD operating system, including installation and development environments. The FreeBSD Project provides the following guarantees to consumers of Tier 1 platforms: * Official FreeBSD release images will be provided by the release engineering team. * Binary updates and source patches for Security Advisories and Errata Notices will be provided for supported releases. * Source patches for Security Advisories will be provided for supported branches. * Binary updates and source patches for cross-platform Security Advisories will typically be provided at the time of the announcement. * Changes to userland ABIs will generally include compatibility shims to ensure correct operation of binaries compiled against any stable branch where the platform is Tier 1. These shims might not be enabled in the default install. If compatibility shims are not provided for an ABI change, the lack of shims will be clearly documented in the release notes. * Changes to certain portions of the kernel ABI will include compatibility shims to ensure correct operation of kernel modules compiled against the oldest supported release on the branch. Note that not all parts of the kernel ABI are protected. * Official binary packages for third party software will be provided by the ports team. For embedded architectures, these packages may be cross-built from a different architecture. * Most relevant ports should either build or have the appropriate filters to prevent inappropriate ones from building. * New features which are not inherently platform-specific will be fully functional on all Tier 1 architectures. * Features and compatibility shims used by binaries compiled against older stable branches may be removed in newer major versions. Such removals will be clearly documented in the release notes. * Tier 1 platforms should be fully documented. Basic operations will be documented in the FreeBSD Handbook. * Tier 1 platforms will be included in the source tree. * Tier 1 platforms should be self-hosting either via the in-tree toolchain or an external toolchain. If an external toolchain is required, official binary packages for an external toolchain will be provided. To maintain maturity of Tier 1 platforms, the FreeBSD Project will maintain the following resources to support development: * Build and test automation support either in the FreeBSD.org cluster or some other location easily available for all developers. Embedded platforms may substitute an emulator available in the FreeBSD.org cluster for actual hardware. * Inclusion in the `make universe` and `make tinderbox` targets. * Dedicated hardware in one of the FreeBSD clusters for package building (either natively or via qemu-user). Collectively, developers are required to provide the following to maintain the Tier 1 status of a platform: * Changes to the source tree should not knowingly break the build of a Tier 1 platform. * Tier 1 architectures must have a mature, healthy ecosystem of users and active developers. * Developers should be able to build packages on commonly available, non-embedded Tier 1 systems. This can mean either native builds if non-embedded systems are commonly available for the platform in question, or it can mean cross-builds hosted on some other Tier 1 architecture. * Changes cannot break the userland ABI. If an ABI change is required, ABI compatibility for existing binaries should be provided via use of symbol versioning or shared library version bumps. * Changes merged to stable branches cannot break the protected portions of the kernel ABI. If a kernel ABI change is required, the change should be modified to preserve functionality of existing kernel modules. === Tier 2: Developmental and Niche Architectures Tier 2 platforms are functional, but less mature FreeBSD platforms. -They are not supported by the security officer, release engineering, and port management teams. +They are not supported by the security officer, release engineering, and Ports Management Team. Tier 2 platforms may be Tier 1 platform candidates that are still under active development. Architectures reaching end of life may also be moved from Tier 1 status to Tier 2 status as the availability of resources to continue to maintain the system in a Production Quality state diminishes. Well-supported niche architectures may also be Tier 2. The FreeBSD Project provides the following guarantees to consumers of Tier 2 platforms: * The ports infrastructure should include basic support for Tier 2 architectures sufficient to support building ports and packages. This includes support for basic packages such as ports-mgmt/pkg, but there is no guarantee that arbitrary ports will be buildable or functional. * New features which are not inherently platform-specific should be feasible on all Tier 2 architectures if not implemented. * Tier 2 platforms will be included in the source tree. * Tier 2 platforms should be self-hosting either via the in-tree toolchain or an external toolchain. If an external toolchain is required, official binary packages for an external toolchain will be provided. * Tier 2 platforms should provide functional kernels and userlands even if an official release distribution is not provided. To maintain maturity of Tier 2 platforms, the FreeBSD Project will maintain the following resources to support development: * Inclusion in the `make universe` and `make tinderbox` targets. Collectively, developers are required to provide the following to maintain the Tier 2 status of a platform: * Changes to the source tree should not knowingly break the build of a Tier 2 platform. * Tier 2 architectures must have an active ecosystem of users and developers. * While changes are permitted to break the userland ABI, the ABI should not be broken gratuitously. Significant userland ABI changes should be restricted to major versions. * New features that are not yet implemented on Tier 2 architectures should provide a means of disabling them on those architectures. === Tier 3: Experimental Architectures Tier 3 platforms have at least partial FreeBSD support. -They are _not_ supported by the security officer, release engineering, and port management teams. +They are _not_ supported by the security officer, release engineering, and Ports Management Team. Tier 3 platforms are architectures in the early stages of development, for non-mainstream hardware platforms, or which are considered legacy systems unlikely to see broad future use. Initial support for Tier 3 platforms may exist in a separate repository rather than the main source repository. The FreeBSD Project provides no guarantees to consumers of Tier 3 platforms and is not committed to maintaining resources to support development. Tier 3 platforms may not always be buildable, nor are any kernel or userland ABIs considered stable. === Unsupported Architectures Other platforms are not supported in any form by the project. The project previously described these as Tier 4 systems. After a platform transitions to unsupported, all support for the platform is removed from the source, ports and documentation trees. Note that ports support should remain as long as the platform is supported in a branch supported by ports. === Policy on Changing the Tier of an Architecture Systems may only be moved from one tier to another by approval of the FreeBSD Core Team, which shall make that decision in collaboration with the Security Officer, Release Engineering, and ports management teams. For a platform to be promoted to a higher tier, any missing support guarantees must be satisfied before the promotion is completed. [[ports]] == Ports Specific FAQ [[ports-qa-adding]] === Adding a New Port [[ports-qa-add-new]] ==== How do I add a new port? First, please read the section about repository copies. The easiest way to add a new port is the `addport` script located in the [.filename]#ports/Tools/scripts# directory. It adds a port from the directory specified, determining the category automatically from the port [.filename]#Makefile#. It also adds an entry to the port's category [.filename]#Makefile#. It was written by `{mharo}`, `{will}`, and `{garga}`. When sending questions about this script to the {freebsd-ports}, please also CC `{crees}`, the current maintainer. [[ports-qa-add-new-extra]] ==== Any other things I need to know when I add a new port? Check the port, preferably to make sure it compiles and packages correctly. The extref:{porters-handbook}testing[Porters Handbook's Testing Chapter] contains more detailed instructions. See the extref:{porters-handbook}testing[Portclippy / Portfmt, testing-portclippy] and the extref:{porters-handbook}testing[Poudriere, testing-poudriere] sections. You do not necessarily have to eliminate all warnings but make sure you have fixed the simple ones. If the port came from a submitter who has not contributed to the Project before, add that person's name to the extref:{contributors}[Additional Contributors, contrib-additional] section of the FreeBSD Contributors List. Close the PR if the port came in as a PR. To close a PR, change the state to `Issue Resolved` and the resolution as `Fixed`. [NOTE] ==== If for some reason using extref:{porters-handbook}testing[Poudriere, testing-poudriere] to test the new port is not possible, the bare minimum of testing includes this sequence: [source,shell] .... # make install # make package # make deinstall # pkg add package you built above # make deinstall # make reinstall # make package .... Note that poudriere is the reference for package building, it the port does not build in poudriere, it will be removed. ==== [[ports-qa-removing]] === Removing an Existing Port [[ports-qa-remove-one]] ==== How do I remove an existing port? First, please read the section about repository copies. Before you remove the port, you have to verify there are no other ports depending on it. * Make sure there is no dependency on the port in the ports collection: ** The port's PKGNAME appears in exactly one line in a recent INDEX file. ** No other ports contains any reference to the port's directory or PKGNAME in their Makefiles + [TIP] ==== When using Git, consider using man:git-grep[1], it is much faster than `grep -r`. ==== + * Then, remove the port: + [.procedure] ==== * Remove the port's files and directory with `git rm`. * Remove the `SUBDIR` listing of the port in the parent directory [.filename]#Makefile#. * Add an entry to [.filename]#ports/MOVED#. * Search for entries in [.filename]#ports/security/vuxml/vuln.xml# and adjust them accordingly. In particular, check for previous packages with the new name which version could include the new port. * Remove the port from [.filename]#ports/LEGAL# if it is there. ==== Alternatively, you can use the rmport script, from [.filename]#ports/Tools/scripts#. This script was written by {vd}. When sending questions about this script to the {freebsd-ports}, please also CC {crees}, the current maintainer. [[ports-qa-move-port]] === How do I move a port to a new location? [.procedure] ==== . Perform a thorough check of the ports collection for any dependencies on the old port location/name, and update them. Running `grep` on [.filename]#INDEX# is not enough because some ports have dependencies enabled by compile-time options. A full man:git-grep[1] of the ports collection is recommended. . Remove the `SUBDIR` entry from the old category Makefile and add a `SUBDIR` entry to the new category Makefile. . Add an entry to [.filename]#ports/MOVED#. . Move the port with `git mv`. . Commit the changes. ==== [[ports-qa-copy-port]] === How do I copy a port to a new location? [.procedure] ==== . Copy port with `cp -R old-cat/old-port new-cat/new-port`. . Add the new port to the [.filename]#new-cat/Makefile#. . Change stuff in [.filename]#new-cat/new-port#. . Commit the changes. ==== [[ports-qa-freeze]] === Ports Freeze [[ports-qa-freeze-what]] ==== What is a “ports freezeâ€? A “ports freeze†was a restricted state the ports tree was put in before a release. It was used to ensure a higher quality for the packages shipped with a release. It usually lasted a couple of weeks. During that time, build problems were fixed, and the release packages were built. This practice is no longer used, as the packages for the releases are built from the current stable, quarterly branch. For more information on how to merge commits to the quarterly branch, see <>. [[ports-qa-quarterly]] === Quarterly Branches [[ports-qa-misc-request-mfh]] ==== What is the procedure to request authorization for merging a commit to the quarterly branch? As of November 30, 2020, there is no need to seek explicit approval to commit to the quarterly branch. [[ports-qa-misc-commit-mfh]] ==== What is the procedure for merging commits to the quarterly branch? Merging commits to the quarterly branch (a process we call MFH for a historical reason) is very similar to MFC'ing a commit in the src repository, so basically: [source,shell] .... % git checkout 2021Q2 % git cherry-pick -x $HASH (verify everything is OK, for example by doing a build test) % git push .... where '$HASH' is the hash of the commit you want to copy over to the quarterly branch. The -x parameter ensures the hash '$HASH' of the main branch is included in the new commit message of the quarterly branch. [[ports-qa-new-category]] === Creating a New Category [[ports-qa-new-category-how]] ==== What is the procedure for creating a new category? Please see extref:{porters-handbook}[Proposing a New Category, proposing-categories] in the Porter's Handbook. Once that procedure has been followed and the PR has been assigned to the {portmgr}, it is their decision whether or not to approve it. If they do, it is their responsibility to: [.procedure] ==== . Perform any needed moves. (This only applies to physical categories.) . Update the `VALID_CATEGORIES` definition in [.filename]#ports/Mk/bsd.port.mk#. . Assign the PR back to you. ==== [[ports-qa-new-category-physical]] ==== What do I need to do to implement a new physical category? [.procedure] ==== . Upgrade each moved port's [.filename]#Makefile#. Do not connect the new category to the build yet. + To do this, you will need to: + [.procedure] ====== . Change the port's `CATEGORIES` (this was the point of the exercise, remember?) The new category is listed first. This will help to ensure that the PKGORIGIN is correct. . Run a `make describe`. Since the top-level `make index` that you will be running in a few steps is an iteration of `make describe` over the entire ports hierarchy, catching any errors here will save you having to re-run that step later on. . If you want to be really thorough, now might be a good time to run man:portlint[1]. ====== + . Check that the ``PKGORIGIN``s are correct. The ports system uses each port's `CATEGORIES` entry to create its `PKGORIGIN`, which is used to connect installed packages to the port directory they were built from. If this entry is wrong, common port tools like man:pkg_version[1] and man:portupgrade[1] fail. + To do this, use the [.filename]#chkorigin.sh# tool: `env PORTSDIR=/path/to/ports sh -e /path/to/ports/Tools/scripts/chkorigin.sh`. This will check every port in the ports tree, even those not connected to the build, so you can run it directly after the move operation. Hint: do not forget to look at the ``PKGORIGIN``s of any slave ports of the ports you just moved! . On your own local system, test the proposed changes: first, comment out the SUBDIR entries in the old ports' categories' [.filename]##Makefile##s; then enable building the new category in [.filename]#ports/Makefile#. Run make checksubdirs in the affected category directories to check the SUBDIR entries. Next, in the [.filename]#ports/# directory, run make index. This can take over 40 minutes on even modern systems; however, it is a necessary step to prevent problems for other people. . Once this is done, you can commit the updated [.filename]#ports/Makefile# to connect the new category to the build and also commit the [.filename]#Makefile# changes for the old category or categories. . Add appropriate entries to [.filename]#ports/MOVED#. . Update the documentation by modifying: ** the extref:{porters-handbook}[list of categories, PORTING-CATEGORIES] in the Porter's Handbook + . Only once all the above have been done, and no one is any longer reporting problems with the new ports, should the old ports be deleted from their previous locations in the repository. ==== ==== What do I need to do to implement a new virtual category? This is much simpler than a physical category. Only a few modifications are needed: * the extref:{porters-handbook}[list of categories, PORTING-CATEGORIES] in the Porter's Handbook [[ports-qa-misc-questions]] === Miscellaneous Questions [[ports-qa-misc-blanket-approval]] ==== Are there changes that can be committed without asking the maintainer for approval? Blanket approval for most ports applies to these types of fixes: * Most infrastructure changes to a port (that is, modernizing, but not changing the functionality). For example, the blanket covers converting to new `USES` macros, enabling verbose builds, and switching to new ports system syntaxes. * Trivial and _tested_ build and runtime fixes. * Documentations or metadata changes to ports, like [.filename]#pkg-descr# or `COMMENT`. [IMPORTANT] ==== Exceptions to this are anything maintained by the {portmgr}, or the {security-officer}. No unauthorized commits may ever be made to ports maintained by those groups. ==== [[ports-qa-misc-correctly-building]] ==== How do I know if my port is building correctly or not? The packages are built multiple times each week. If a port fails, the maintainer will receive an email from `pkg-fallout@FreeBSD.org`. Reports for all the package builds (official, experimental, and non-regression) are aggregated at link:pkg-status.FreeBSD.org[pkg-status.FreeBSD.org]. [[ports-qa-misc-INDEX]] ==== I added a new port. Do I need to add it to the [.filename]#INDEX#? No. The file can either be generated by running `make index`, or a pre-generated version can be downloaded with `make fetchindex`. [[ports-qa-misc-no-touch]] ==== Are there any other files I am not allowed to touch? Any file directly under [.filename]#ports/#, or any file under a subdirectory that starts with an uppercase letter ([.filename]#Mk/#, [.filename]#Tools/#, etc.). In particular, the {portmgr} is very protective of [.filename]#ports/Mk/bsd.port*.mk# so do not commit changes to those files unless you want to face their wrath. [[ports-qa-misc-updated-distfile]] ==== What is the proper procedure for updating the checksum for a port distfile when the file changes without a version change? When the checksum for a distribution file is updated due to the author updating the file without changing the port revision, the commit message includes a summary of the relevant diffs between the original and new distfile to ensure that the distfile has not been corrupted or maliciously altered. If the current version of the port has been in the ports tree for a while, a copy of the old distfile will usually be available on the ftp servers; otherwise the author or maintainer should be contacted to find out why the distfile has changed. [[ports-exp-run]] ==== How can an experimental test build of the ports tree (exp-run) be requested? An exp-run must be completed before patches with a significant ports impact are committed. The patch can be against the ports tree or the base system. Full package builds will be done with the patches provided by the submitter, and the submitter is required to fix detected problems _(fallout)_ before commit. [.procedure] ==== . Go to the link:https://bugs.freebsd.org/submit[Bugzilla new PR page]. . Select the product your patch is about. . Fill in the bug report as normal. Remember to attach the patch. . If at the top it says “Show Advanced Fields†click on it. It will now say “Hide Advanced Fieldsâ€. Many new fields will be available. If it already says “Hide Advanced Fieldsâ€, no need to do anything. . In the “Flags†section, set the “exp-run†one to `?`. As for all other fields, hovering the mouse over any field shows more details. . Submit. Wait for the build to run. . {portmgr} will reply with a possible fallout. . Depending on the fallout: ** If there is no fallout, the procedure stops here, and the change can be committed, pending any other approval required. ... If there is fallout, it _must_ be fixed, either by fixing the ports directly in the ports tree, or adding to the submitted patch. ... When this is done, go back to step 6 saying the fallout was fixed and wait for the exp-run to be run again. Repeat as long as there are broken ports. ==== [[non-committers]] == Issues Specific to Developers Who Are Not Committers A few people who have access to the FreeBSD machines do not have commit bits. Almost all of this document will apply to these developers as well (except things specific to commits and the mailing list memberships that go with them). In particular, we recommend that you read: * <> * <> + [NOTE] ==== Get your mentor to add you to the "Additional Contributors" ([.filename]#doc/shared/contrib-additional.adoc#), if you are not already listed there. ==== * <> * <> * <> [[google-analytics]] == Information About Google Analytics As of December 12, 2012, Google Analytics was enabled on the FreeBSD Project website to collect anonymized usage statistics regarding usage of the site. The information collected is valuable to the FreeBSD Documentation Project, to identify various problems on the FreeBSD website. [[google-analytics-policy]] === Google Analytics General Policy The FreeBSD Project takes visitor privacy very seriously. As such, the FreeBSD Project website honors the "Do Not Track" header _before_ fetching the tracking code from Google. For more information, please see the https://www.FreeBSD.org/privacy/[FreeBSD Privacy Policy]. Google Analytics access is _not_ arbitrarily allowed - access must be requested, voted on by the `{doceng}`, and explicitly granted. Requests for Google Analytics data must include a specific purpose. For example, a valid reason for requesting access would be "to see the most frequently used web browsers when viewing FreeBSD web pages to ensure page rendering speeds are acceptable." Conversely, "to see what web browsers are most frequently used" (without stating __why__) would be rejected. All requests must include the timeframe for which the data would be required. For example, it must be explicitly stated if the requested data would be needed for a timeframe covering a span of 3 weeks, or if the request would be one-time only. Any request for Google Analytics data without a clear, reasonable reason beneficial to the FreeBSD Project will be rejected. [[google-analytics-data]] === Data Available Through Google Analytics A few examples of the types of Google Analytics data available include: * Commonly used web browsers * Page load times * Site access by language [[misc]] == Miscellaneous Questions === Are there changes that can be committed without asking the maintainer for approval? Blanket approval for most ports applies to these types of fixes: * Most infrastructure changes to a port (that is, modernizing, but not changing the functionality). For example, the blanket covers converting to new `USES` macros, enabling verbose builds, and switching to new ports system syntaxes. * Trivial and _tested_ build and runtime fixes. * Documentations or metadata changes to ports, like [.filename]#pkg-descr# or `COMMENT`. === How do I access people.FreeBSD.org to put up personal or project information? `people.FreeBSD.org` is the same as `freefall.FreeBSD.org`. Just create a [.filename]#public_html# directory. Anything you place in that directory will automatically be visible under https://people.FreeBSD.org/[https://people.FreeBSD.org/]. === Where are the mailing list archives stored? The mailing lists are archived under [.filename]#/local/mail# on `freefall.FreeBSD.org`. === I would like to mentor a new committer. What process do I need to follow? See the https://www.freebsd.org/internal/new-account/[New Account Creation Procedure] document on the internal pages. [[benefits]] == Benefits and Perks for FreeBSD Committers [[benefits-recognition]] === Recognition Recognition as a competent software engineer is the longest lasting value. In addition, getting a chance to work with some of the best people that every engineer would dream of meeting is a great perk! [[benefits-freebsdmall]] === FreeBSD Mall FreeBSD committers can get a free 4-CD or DVD set at conferences from http://www.freebsdmall.com[FreeBSD Mall, Inc.]. [[benefits-gandi]] === `Gandi.net` https://gandi.net[Gandi] provides website hosting, cloud computing, domain registration, and X.509 certificate services. Gandi offers an E-rate discount to all FreeBSD developers. In order to streamline the process of getting the discount first set up a Gandi account, fill in the billing information and select the currency. Then send an mail to mailto:non-profit@gandi.net[non-profit@gandi.net] using your `@freebsd.org` mail address, and indicate your Gandi handle. [[benefits-rsync]] === `rsync.net` https://rsync.net[rsync.net] provides cloud storage for offsite backup that is optimized for UNIX users. Their service runs entirely on FreeBSD and ZFS. rsync.net offers a free-forever 500 GB account to FreeBSD developers. Simply sign up at https://www.rsync.net/freebsd.html[https://www.rsync.net/freebsd.html] using your `@freebsd.org` address to receive this free account. diff --git a/website/content/en/administration.adoc b/website/content/en/administration.adoc index 3eff03abb9..821af45704 100644 --- a/website/content/en/administration.adoc +++ b/website/content/en/administration.adoc @@ -1,356 +1,356 @@ --- title: "FreeBSD Project Administration and Management" sidenav: about --- include::shared/authors.adoc[] //// NOTE: If any alias listed on this page is modified in the /etc/aliases on the FreeBSD project's mail server, then this page must be updated. //// = FreeBSD Project Administration and Management == Introduction This page lists teams, groups and individuals within the FreeBSD project with designated project roles and areas of responsibility, along with brief descriptions and contact information. * Project Management ** <> ** <> -** <> +** <> ** <> * Release Engineering ** <> ** <> * Teams ** <> ** <> ** <> * Secretaries ** <> ** <> -** <> +** <> * Internal Administration ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ** <> ''' [[t-core]] == FreeBSD Core Team The FreeBSD Core Team constitutes the project's "Board of Directors", responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape. The Core Team is elected by the active developers in the project. * {bapt} (Clusteradm Liaison) * {emaste} (GIT Transition WG and Security Team Liason) * {gnn} (Foundation Liaison) * {hrs} (Code-of-Conduct Committee Liaison) * {kevans} (Portmgr Team Liaison) * {markj} (Bugmeister and CI Team Liaison) * {scottl} (Release Engineering Team Liaison) * {seanc} (Doceng Team Liaison) * {imp} (GIT Transition WG Liaison) [[t-doceng]] == FreeBSD Documentation Engineering Team The FreeBSD Documentation Engineering Team is responsible for defining and following up documentation goals for the committers in the Documentation project. The https://www.freebsd.org/internal/doceng/[doceng team charter] describes the duties and responsibilities of the Documentation Engineering Team in greater detail. * {gjb} * {carlavilla} * {blackend} * {dbaio} * {bcr} * {hrs} * {ryusuke} * {0mp} (Secretary) [[t-portmgr]] -== FreeBSD Port Management Team +== FreeBSD Ports Management Team -The primary responsibility of the FreeBSD Port Management Team is to ensure that the FreeBSD Ports Developer community provides a ports collection that is functional, stable, up-to-date and full-featured. +The primary responsibility of the FreeBSD Ports Management Team is to ensure that the FreeBSD Ports Developer community provides a ports collection that is functional, stable, up-to-date and full-featured. Its secondary responsibility is to coordinate among the committers and developers who work on it. -The https://www.freebsd.org/portmgr/charter/[portmgr team charter] describes the duties and responsibilities of the Port Management Team in greater detail. +The https://www.freebsd.org/portmgr/charter/[portmgr team charter] describes the duties and responsibilities of the Ports Management Team in greater detail. * {antoine} * {bapt} (Core Team Liaison) * {bdrewery} (Release Engineering Team Liaison) * {mat} (Cluster Administration Team Liaison) * {rene} (Secretary) * {swills} * {tcberner} ''' [[t-ports-secteam]] == FreeBSD Ports Security Team The primary responsibility of the FreeBSD Port Security Team is to provide rapid response to security incidents that affects the FreeBSD ports collection and protect the FreeBSD user community by keeping the community informed of bugs, exploits, popular attacks, and other risks. More details are available on the https://wiki.freebsd.org/ports-secteam[Wiki page]. * {fluffy} * {joneum} * {riggs} ''' [[t-re]] == Primary Release Engineering Team The Primary Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining `releng/*` branches, among other things. The https://www.freebsd.org/releng/charter/[release engineering team charter] describes the duties and responsibilities of the Primary Release Engineering Team in greater detail. * {gjb} (Lead) * {kib} * {blackend} * {delphij} * {cperciva} (Deputy Lead) [[t-re-builder]] == Builders Release Engineering Team The builders release engineering team is responsible for building and packaging FreeBSD releases on the various supported platforms. * {marcel} * {nyan} * {nwhitehorn} ''' [[t-donations]] == Donations Team The FreeBSD Donations Team is responsible for responding to donations offers, establishing donation guidelines and procedures, and coordinating donation offers with the FreeBSD developer community. A more detailed description of the duties of the Donations Team is available on the https://www.freebsd.org/donations/[FreeBSD Donations Liaison] page. * {gahr} * {bhughes} * {adridg} [[t-secteam]] == Security Team The FreeBSD Security Team (headed by the Security Officer) is responsible for keeping the community aware of bugs, exploits and security risks affecting the FreeBSD src and ports trees, and to promote and distribute information needed to safely run FreeBSD systems. Furthermore, it is responsible for resolving software bugs affecting the security of FreeBSD and issuing security advisories. The FreeBSD Security Officer Charter describes the duties and responsibilities of the Security Officer in greater detail. * {gordon} (Officer) * {delphij} (Officer Emeritus, Release Engineering Team Liaison) * {des} (Officer Emeritus) * {gjb} (Cluster Administrators Team Liaison) * {emaste} (Officer Deputy) * {bz} * {gnn} * {markj} * {philip} [[t-vendor]] == Vendor Relations Vendor Relations is responsible for handling email from hardware and software vendors. Email sent to Vendor Relations is forwarded to the FreeBSD Core Team in addition to the FreeBSD Foundation. ''' [[t-core-secretary]] == Core Team Secretary The FreeBSD Core Team Secretary is a non-voting member of the Core Team, responsible for documenting the work done by core, keeping track of the core agenda, contacting non-core members on behalf of core, sending mail to core, and interfacing with the admin team for committer/account approval. The Core Team Secretary is also responsible for writing and sending out monthly status reports to the FreeBSD Developer community, containing a summary of core's latest decisions and actions. * {bofh} [[t-doceng-secretary]] == Documentation Engineering Team Secretary The FreeBSD Documentation Engineering Team Secretary is a non-voting member of the Documentation Engineering Team, responsible for documenting the work done by doceng, keeping track of voting procedures, and to be an interface to the other teams, especially the admin and Core teams. * {0mp} [[t-portmgr-secretary]] -== Port Management Team Secretary +== Ports Management Team Secretary -The FreeBSD Port Management Team Secretary is a non-voting member of the Port Management Team, responsible for documenting the work done by portmgr, keeping track of voting procedures, and to be an interface to the other teams, especially the admin and Core teams. -The Port Management Team Secretary is also responsible for writing and sending out monthly status reports to the FreeBSD Developer community, containing a summary of portmgr's latest decisions and actions. +The FreeBSD Ports Management Team Secretary is a non-voting member of the Ports Management Team, responsible for documenting the work done by portmgr, keeping track of voting procedures, and to be an interface to the other teams, especially the admin and Core teams. +ThePorts Management Team Secretary is also responsible for writing and sending out monthly status reports to the FreeBSD Developer community, containing a summary of portmgr's latest decisions and actions. * {rene} ''' [[t-accounts]] == Accounts Team //// admins mail aliases intentionally left incomplete //// The Accounts Team is responsible for setting up accounts for new committers in the project. Requests for new accounts will not be acted upon without the proper approval from the appropriate entity. Email sent to the Accounts Team is currently forwarded to the Cluster Administrators. [[t-backups]] == Backups Administrators //// admins mail aliases intentionally left incomplete //// The Backups Administrators handle all backups on the FreeBSD cluster. Email sent to the Backups Team is currently forwarded to the Cluster Administrators. [[t-bugmeister]] == Bugmeisters The Bugmeisters are responsible for ensuring that the problem report software is in working order, that the entries are correctly categorised and that there are no invalid entries. * {eadler} * {mva} * {gavin} * {koobs} * {gonzo} [[t-clusteradm]] == Cluster Administrators //// admins mail aliases intentionally left incomplete //// The Cluster Administrators consists of the people responsible for maintaing the machines and services that the project relies on for its distributed work and communication. Issues concerning the projects infrastructure or setting up new machines should be directed to them. This team is led by the lead cluster administrator whose duties and responsbilities are described in the https://www.freebsd.org/internal/clusteradm/[cluster administration charter] in greater detail. * {allanjude} * {bapt} * {brd} * {dhw} * {gavin} * {gjb} * {lwhsu} * {peter} * {philip} * {sbruno} * {simon} * {zi} (Lead) [[t-dnsadm]] == DNS Administrators //// admins mail aliases intentionally left incomplete //// The DNS Administrators are responsible for managing DNS and related services. E-mail to the DNS Administrators is currently forwarded to the Cluster Administrators. [[t-forum-admins]] == FreeBSD Forum Administrators The Forum Administrators maintain the FreeBSD Project's Internet forum, located at https://forums.freebsd.org/ and lead the group of moderators who work to ensure the relevance and quality of the forum's content. * {brd} * {danger} * {dutchdaemon} * {lme} [[t-github-automation]] == Repository Automated Mirroring to GitHub Coordinators The GitHub Automation team oversees the export of FreeBSD source code repository content to the read-only repository instances on GitHub * {koobs} * {mva} * {robak} * {rodrigc} * {uqs} [[t-jenkins-admin]] == Jenkins Continuous Integration Testing Administrators The Jenkins Administrators maintain the Continuous Integration and testing infrastructure for The FreeBSD Project. This includes maintaining the Jenkins instance and the jobs that run builds and execute tests. * {bapt} * {brd} * {brooks} * {lwhsu} * {markj} (Core Team Liaison) * {seanc} * {swills} [[t-mirror-admin]] == FTP/WWW Mirror Site Coordinators The FTP/WWW Mirror Site Coordinators coordinate all the FTP/WWW mirror site administrators to ensure that they are distributing current versions of the software, that they have the capacity to update themselves when major updates are in progress, and making it easy for the general public to find their closest FTP/WWW mirror. E-mail to the Mirror Site Coordinators is currently forwarded to the Cluster Administrators. [[t-phabric-admin]] == Phabricator Code Review Application Administrators The Phabricator Administrators are responsible for maintaining the FreeBSD's instance of the Phabricator on-line code review tool located at https://reviews.freebsd.org/ * {allanjude} * {bofh} * {eadler} * {emaste} * {jtl} * {lwhsu} * {mat} * {miwi} * {pi} For any problems regarding Phabricator, please https://bugs.freebsd.org/submit/[open a bug report] and select "Services" and then "Code Review". [[t-postmaster]] == Postmaster Team The Postmaster Team is responsible for mail being correctly delivered to the committers' email address, ensuring that the mailing lists work, and should take measures against possible disruptions of project mail services, such as having troll-, spam- and virus-filters. * {krion} * {ler} * {philip} * {pi} * {rea} * {zi} [[t-subversion]] == Subversion Administrators The FreeBSD Subversion team is responsible for maintaining the health of the Subversion Repositories. Email to the Subversion Administration team is currently forwarded to the Cluster Administrators. [[t-webmaster]] == Webmaster Team The FreeBSD Webmaster Team is appointed by FreeBSD Documentation Engineering Team, and responsible for keeping the main FreeBSD web sites up and running. This means web server configuration, CGI scripts, fulltext and mailing list search. Anything web related, technical stuff belongs to the scope of the Webmaster Team, excluding bugs in the documentation. Email to the Webmaster Team is currently forwarded to the Documentation Engineering team with the addition of: * {wosch} [[t-wiki]] == Wiki Admin Team The FreeBSD Wiki Team is responsible for keeping the FreeBSD https://wiki.FreeBSD.org/[Wiki] site up and running. They also shape the overall design and content structure. * {linimon} * {koobs} diff --git a/website/content/en/internal/clusteradm.adoc b/website/content/en/internal/clusteradm.adoc index e62eabd512..e4e8feab3e 100644 --- a/website/content/en/internal/clusteradm.adoc +++ b/website/content/en/internal/clusteradm.adoc @@ -1,38 +1,38 @@ --- title: "Charter for the Cluster Administrators" sidenav: docs --- = Charter for the Cluster Administrators Lead cluster administrator is a delegated officer role (aka. "hat") that answers to the FreeBSD Core Team and ultimately the FreeBSD community at large. This person shall have the operational authority over the FreeBSD cluster infrastructure (to the extent that the Core Team can delegate this authority) and will be responsible for the following in general: * Ensure the reliable operation of the Project's equipment and network resources. * Ensure that the Project's resources are suitably and effectively used to serve the Project's interests. * Ensure that reasonable security precautions and mitigations are implemented within the constraints of the nature of a highly distributed project. * Delegate to and coordinating with both the site-specific admin teams and the admins at large. * Ensure that standard operating procedures, rules, guidelines etc are documented and understandable. * Take measures to ensure that a competent administrator would be expected to be able to adopt a predecessor's work in a reasonable amount of time. * Contingency planning and implementation to ensure continuity across site specific problems (including donated site withdrawal or outages). -* Keep the interested parties (Core Team, Security Team, FreeBSD Foundation, Port Management Team, etc), project members and community members appropriately informed. +* Keep the interested parties (Core Team, Security Team, FreeBSD Foundation, Ports Management Team, etc), project members and community members appropriately informed. * Give timely and authoritive answers to questions, or a direct referral to the appropriate party. * Aid other hat wearers and cluster administrators to get their job done. * Where practical and appropriate, use the Project's own product as a proving ground. * Make sure that it is easy for developers to know what hardware resources they have access to for project purposes. The lead cluster administrator answers to the FreeBSD Core Team. If a party is unhappy with a position that the hat wearer takes and is unable to change their mind, they may take the issue to the Core Team. The Core Team has the final say in the matter. If the lead cluster administrator is a member of the Core Team then a complaint may be made in confidence via the core secretary or another member if desired. Any of the following still require a sign-off from the Core Team: * New public facing services. * Planned withdrawal of public facing services. * New team members. Notable interaction with other hats: * The lead cluster administrator will consult with the Security Officer and the Security Team where appropriate but will be responsible for making decisions. However, the Security Officer may respond to security emergencies involving project infrastructure as necessary. -* The Port Management Team has a large resource footprint and arrangements will be made with them to effectively operate their resources within the constraints of the overall cluster operation. +* The Ports Management Team has a large resource footprint and arrangements will be made with them to effectively operate their resources within the constraints of the overall cluster operation. Earmarked resources: Some site resources are provided for specific purposes. Any such earmarking or use restrictions will be documented to make sure such resources are used as intended. diff --git a/website/content/en/press/press-rel-9.adoc b/website/content/en/press/press-rel-9.adoc index 1bc699753e..4568f639ac 100644 --- a/website/content/en/press/press-rel-9.adoc +++ b/website/content/en/press/press-rel-9.adoc @@ -1,33 +1,33 @@ --- title: "Hewlett-Packard donates blade cluster to FreeBSD" sidenav: about loadPress: false --- = Hewlett-Packard donates blade cluster to FreeBSD *The FreeBSD Foundation received a donation of a blade system from Hewlett-Packard for use as a third-party software build cluster. This 20-node HP BladeSystem cluster triples the speed of the build process for i386 packages.* -_"With this generous donation from HP, we are able to continuously produce up-to-date packages from more than 13000 ports of third-party software available in the FreeBSD Ports Collection, at about three times the rate of the previous hardware cluster,"_ said Kris Kennaway, member of the FreeBSD Port Management Team. +_"With this generous donation from HP, we are able to continuously produce up-to-date packages from more than 13000 ports of third-party software available in the FreeBSD Ports Collection, at about three times the rate of the previous hardware cluster,"_ said Kris Kennaway, member of the FreeBSD Ports Management Team. _"This directly benefits the users of FreeBSD through the rapid availability of new and updated software packages, and through the increased testing and QA of FreeBSD that the new hardware allows."_ _"We at HP recognize the important role of FreeBSD in the Internet's global network infrastructure, and we are happy that the HP BladeSystem cluster can contribute to the on-going success of the FreeBSD Foundation,"_ said Mark Potter, vice president of the Hewlett-Packard BladeSystem division. _"They're just standard i386 systems, architecturally, with a very nice ssh- and serial-based management server,"_ said Kennaway, who maintains the FreeBSD Ports cluster. Kennaway said FreeBSD has a few dozen other machines scattered around the globe for package builds. A big concentration of sparc machines hosted by Hiroki Sato in Japan include some large multiprocessor e4500's (10, 12 and 14 CPUs) that have been extremely valuable for SMP testing. Also, a couple of machines hosted by ISC, an amd64 hosted by Scott Long, three i386 machines at Yahoo! Korea, and sometimes Kennaway's own machines in Canada are used for the official package builds. The HP BladeSystem cluster is hosted at the Yahoo! datacenter in the San Francisco Bay area. In addition to Kennaway, Paul Saab and Peter Wemm from the FreeBSD project, and John Cagle from HP helped with blade system setup. == About The FreeBSD Project The FreeBSD Project provides an up-to-date and scalable modern operating system that offers high-performance, security, and advanced networking for personal workstations, Internet servers, routers, and firewalls. The FreeBSD packages collection includes popular software like Apache Web Server, Gnome, KDE, X.org X11 Window System, Python, Mozilla, and over 13,000 software suites. FreeBSD can be found on the Internet at http://www.FreeBSD.org/[http://www.FreeBSD.org]. == More Information: FreeBSD Ports webpage + http://www.freebsd.org/ports/[http://www.FreeBSD.org/ports/] + + FreeBSD Package building logs and errors webpage + http://pointyhat.freebsd.org/errorlogs/ diff --git a/website/content/en/status/report-2010-04-2010-06.html b/website/content/en/status/report-2010-04-2010-06.html index b33bdda59a..56ad0808b1 100644 --- a/website/content/en/status/report-2010-04-2010-06.html +++ b/website/content/en/status/report-2010-04-2010-06.html @@ -1,1588 +1,1588 @@ FreeBSD Quarterly Status Report
Skip site navigation (1) Skip section navigation (2)

Introduction

This report covers FreeBSD-related projects between April and June 2010. It is the second of the four reports planned for 2010, and contains 47 entries. During this period, a lot of work has gone into the development of new minor version of FreeBSD, 8.1-RELEASE, which should be released within days.

Thanks to all the reporters for the excellent work! We hope you enjoy reading.

Please note that the deadline for submissions covering the period between July and September 2010 is October 15th, 2010.

Google Summer of Code

Projects

FreeBSD Team Reports

Network Infrastructure

Kernel

Documentation

Userland Programs

Architectures

Ports

Miscellaneous


    Google Summer of Code


    Binary Package Patch Infrastructure — pkg_patch

    Links
    Wiki page URL: http://wiki.FreeBSD.org/IvanVoras/pkg_patch

    Contact: Ivan Voras <ivoras@FreeBSD.org>

    The pkg_patch project is about creating a binary package patch infrastructure which would allow users to patch their live system's packages in an easy and efficient way. It is a C program written to interface with libpkg (for things which are common to all pkg utilities) meant to be included in the base system when it is done. It comes with built-in mass patch creation and application commands. It is funded by Google Summer of Code 2010.

    -

    Open tasks:

    1. Finish the project.
    2. Get some testing for it.
    3. Convince the Port Management Team it is actually a Good +

      Open tasks:

      1. Finish the project.
      2. Get some testing for it.
      3. Convince the Ports Management Team it is actually a Good Thing to have even as an experimental feature.
      4. Agree upon the policy on which package patches will be created (i.e. from which point in time to which point in time), assuming the "stable" package tree idea has still not gotten traction.

      Collective Resource Limits (aka. Jobs)

      Links
      Project page on the wiki URL: http://wiki.FreeBSD.org/G%C3%A1borSoC2010
      Sources in Perforce URL: http://p4db.FreeBSD.org/depotTreeBrowser.cgi?FSPC=//depot/projects/soc2010/gabor_jobs/irix_jobs

      Contact: Gábor Kövesdán <gabor@FreeBSD.org>

      The SGI IRIX operating system has a concept, called job, which is used to group processes together and then apply resource limits on them. The purpose of this project is to implement this facility on FreeBSD.

      I spent most of the time familiarizing myself with how things are done inside the kernel, how syscalls work, etc. So far, I have the basic understanding needed and I added the most important syscalls to group processes together into jobs and manipulate collective resource limits on them.

      There is a bug, which I am tracking down at the moment, after this I can start to implement actual resource limit enforcement. For some of the limit types, it will be relatively easy but some others will take more effort and studies.

      Open tasks:

      1. Fix the showstopper bug, which prevent me working on actual limit enforcement.
      2. Implement limit enforcements for all of the limits supported by IRIX.
      3. Add support for userland facilities and make utilities jobs-aware, like showing jobs in ps(1), etc.

      ExtFS Status Report

      Links
      URL: http://wiki.FreeBSD.org/SOC2010ZhengLiu

      Contact: Zheng Liu <gnehzuil@gmail.com>

      This project has two goals: pre-allocation algorithm and ext4 read-only mode.

      The aim of pre-allocation algorithm is to implement a reservation window mechanism. Now this mechanism has been introduced. The performance comparison can be found on the wiki.

      The aim of ext4 read-only mode is to make it possible to read ext4 file system in read-only mode when the hard disk is formatted with default features. Currently it only supports a few features, such as extents, huge_file. Others features will be added, such as dir_index, uninit_bg, dir_nlink, flex_bg and extra_isize. My work resides in extfs and ext4fs branch of Perforce.

      File System Changes Notification

      Contact: Ilya Putsikau <iputsikau@gmail.com>

      The aim of the project is to implement an inotify-compatible file system change notification mechanism for FreeBSD and later, and add inotify support to linuxulator. The result, fsnotify is already functional but not yet compatible with inotify in some details.

      Open tasks:

      1. Add access permissions checks.
      2. Port inotify test cases.
      3. Fix compatibility issues.
      4. Add linuxulator support.

      Google Summer of Code 2010

      Links
      Summer of Code 2010 Projects URL: http://wiki.FreeBSD.org/SummerOfCode2010Projects

      Contact: Brooks Davis <brooks@FreeBSD.org>
      Contact: Tim Kientzle <kientzle@FreeBSD.org>
      Contact: Robert Watson <rwatson@FreeBSD.org>

      We are once again participating in the Google Summer of Code. This is our 6th year of participation and we hope to once again see great results from our 18 students. Coding officially began May 24th, and we are in the middle of the mid-term evaluation period. You can see and comment on weekly status reports on the mailing list or on the wiki.

      Making Ports Work with Clang

      Links
      URL: http://wiki.FreeBSD.org/SOC2010AndriusMorkunas
      URL: http://wiki.FreeBSD.org/PortsAndClang
      GSoC2010 patches URL: http://rainbow-runner.nl/~andrius/soc/
      All patches for ports URL: http://rainbow-runner.nl/clang/patches/

      Contact: Andrius Morkunas <hinokind@gmail.com>

      First part of the project is mostly complete. I added support for new PORTS_CC variable which should be used in make.conf instead of CC to change ports compiler. This allows user to change ports compiler easily, while still respecting USE_GCC.

      Some patches were written to get ports to work with Clang, and a lot of old patches written prior to the Google Summer of Code project were updated. There are still a lot of broken ports, and some that cannot be built because of Clang/LLVM bugs, but at this point, Clang can build most ports.

      Open tasks:

      1. Fix broken ports that do not work with Clang.
      2. Test patched ports with Clang, report Clang bugs.

      Namecache Improvements — dircache

      Links
      URL: http://wiki.FreeBSD.org/SOC2010GlebKurtsov

      Contact: Gleb Kurtsou <gk@FreeBSD.org>

      I have been reimplementing VFS namecache to make it granularly locked and supporting reliable full-path lookup without calling underlying file system routines. I have successfully implemented directory cache that works in idealized environment with tmpfs. I am currently working on adding support for entries without associated vnodes and for "weak" entries and incomplete cached path.

      Package Management Library — libpkg

      Links
      Wiki page URL: http://wiki.FreeBSD.org/SOC2010DavidForsythe
      Main project page URL: http://code.google.com/p/libpkg

      Contact: David Forsythe <dforsyth@FreeBSD.org>

      The libpkg library will allow for fairly fine grained control over package management.

      Presently libpkg has complete read functionality. Info and delete tools that have most of the current package tool features have already been implemented, and once they are completed they can be considered replacements for their counterparts.

      Once the write and logging aspects of the library are more mature, add and create tools can be created quickly. A new set of more maintainable package tools that leverage libpkg will hopefully be available soon after.

      Packet-Capturing Stack — ringmap

      Links
      Project-Page on Google Code URL: http://code.google.com/p/ringmap/
      Slides URL: http://ringmap.googlecode.com/files/ringmap_slides.pdf

      Contact: Alexander Fiveg <afiveg@FreeBSD.org>

      The ringmap stack is a complete FreeBSD packet-capturing mplementation specialized for very high-speed networks. Similar to the "zero-copy BPF" implementation, the idea of ringmap is to eliminate packet copy operations by using shared memory buffers. However, unlike the "zero-copy BPF" model, ringmap eliminates ALL packet copies during capturing: the network adapter's DMA buffer is mapped directly into user-space. The ringmap stack also adapts libpcap accordingly to provide userspace applications with access to the captured packets without any additional overhead.

      In the context of Google Summer of Code 2010:

      • The ringmap software was ported to 9-CURRENT.
      • Ringmap was redesigned to make it easier to port to other adapters and to integrate it with other network drivers.
      • Also ringmap was extended to be multi-threaded.

      Open tasks:

      1. Porting ringmap to 10GbE (integrating with ixgbe driver).
      2. Porting the entire ringmap code from 9-CURRENT to -STABLE.
      3. Evaluation tests.
      4. Documentation.

      Projects


      Clang Replacing GCC in the Base System

      Links
      URL: http://wiki.FreeBSD.org/BuildingFreeBSDWithClang

      Contact: Ed Schouten <ed@FreeBSD.org>
      Contact: Roman Divacky <rdivacky@FreeBSD.org>
      Contact: Brooks Davis <brooks@FreeBSD.org>
      Contact: Pawel Worach <pawel.worach@gmail.com>

      In the past quarter we imported Clang into FreeBSD and it is being built by default on i386/amd64/powerpc. We have not yet committed the necessary changes to let world compile with Clang.

      Some bugs and warnings were fixed in HEAD as a result of the Clang import and people are exploring more and more areas (DTrace, etc). There are some bug fixes in Clang/LLVM as well that stem from the import (unknown pragmas warnings, etc).

      Roman Divacky and Matthew Fleming are working on ELF writer in LLVM. This is meant as a replacement for assembler (currently we use an outdated GNU as(1)). This work is progressing nice, currently it is able to produce working variants of hello world in C and C++, and some other small programs from "configure run".

      Open tasks:

      1. Import of newer Clang/LLVM into HEAD.
      2. Help with ARM/MIPS/SPARC64.
      3. Start pushing src patches into HEAD.
      4. More testing of Clang on third-party applications (ports).
      5. More work on the ELF writer.

      DAHDI/FreeBSD Project

      Links
      URL: http://www.asterisk.org/dahdi/
      Project Status URL: https://spreadsheets.google.com/pub?key=0Arw6eRL10yIwdGhLdGJWUHF4b3ExQzBsd3BGd2tublE&hl=en&single=true&gid=0&output=html

      Contact: Max Khon <fjoe@samodelkin.net>

      The purpose of DAHDI/FreeBSD project is to make it possible to use FreeBSD as a base system for software PBX solutions.

      DAHDI (Digium/Asterisk Hardware Device Interface) is an open-source device driver framework and a set of hardware drivers for E1/T1, ISDN digital, and FXO/FXS analog cards [1]. Asterisk is one of the most popular open-source software PBX solutions [2].

      The project includes porting DAHDI framework and hardware drivers for E1/T1, FXO/FXS analog, and ISDN digital cards to FreeBSD. This also includes TDMoE support, software and HW echo cancellation (Octasic, VPMADT032), and hardware transcoding support (TC400B). The work is ongoing in the official DAHDI SVN repository with the close collaboration with DAHDI folks at Digium.

      The project is nearing completion. The DAHDI framework and hardware drivers telephony cards have been ported and tested. There are a number of success stories from early adopters who have been using E1/T1 and FXO/FXS cards on FreeBSD for several months.

      Distributed Audit

      Links
      Perforce repository URL: http://p4web.freebsd.org/@md=d&cd=//&c=wHa@//depot/projects/soc2010/disaudit/?ac=83
      Project Wiki URL: http://wiki.FreeBSD.org/SOC2010SergioLigregni

      Contact: Sergio Ligregni <ligregni@FreeBSD.org>

      90% of the functionality is working, the daemons sync two systems in a master-slave paradigm.

      Open tasks:

      1. Standardize the code to meet FreeBSD requirements.
      2. Implement SSL in network communication.
      3. Perform security improvements and bug fixing, strlxxx() functions, memcpy() instead of strcpy() when using non-char variables.
      4. Integrate with the current Audit subsystem.

      General-Purpose DMA Framework

      Links
      Project description on FreeBSD wiki URL: http://wiki.FreeBSD.org/SOC2010JakubKlama
      Project branch on Perforce URL: http://p4web.FreeBSD.org/@md=d&cd=//&c=eCv@//depot/projects/soc2010/jceel_dma/?ac=83

      Contact: Jakub Klama <jceel@FreeBSD.org>

      This project purpose is adding support for general purpose DMA engines found in most embedded devices. GPDMA framework provides a unified KOBJ interface to DMA engine drivers and unified programming interface to use direct memory transfers in kernel and userspace applications.

      This project is a part of Google Summer of Code 2010 and it is a work in progress. Current status can be observed on the wiki page.

      Open tasks:

      1. Add support for more DMA engines.
      2. Complete, clean up, and merge with HEAD.

      GEOM-Based Pseudo-RAID Implementation — geom_pseudoraid

      Links
      Code snapshot URL: http://acm.poly.edu/~spawk/geom_pseudoraid-20100715.tbz

      Contact: Boris Kochergin <spawk@acm.poly.edu>

      The old ata(4) driver is believed to be going away sometime in the future, to be replaced with ATA_CAM [1]. However, ATA pseudo-RAID support in FreeBSD, ataraid(4), is implemented as part of said ata(4) driver, which means that it, too, will be going away. It was decided that pseudo-RAID support is desirable and that it should be reimplemented in GEOM [2] [3], which this project aims to do.

      Currently, RAID-1 arrays can be used on VIA Tech V-RAID and Adaptec HostRAID controllers in a limited capacity. There is no support for writing metadata yet, so disks are not marked degraded, there is no rebuild support, etc. These features are planned, along with support for more hardware and RAID-0 and SPAN arrays.

      A major setback for the current code is that it uses the device(9) family of functions to identify ATA pseudo-RAID controllers and constructs arrays based on that information. Unfortunately, ATA_CAM does not appear to add its devices to the device tree, so that tactic cannot be used with ATA_CAM. While this is fine for development of the actual RAID parts of the code, the project will be somewhat useless in the absence of the old ata(4) driver. There has been talk of exporting PCI information to GEOM [4] [5], but the work does not appear to have been completed yet.

      Open tasks:

      1. Obtain documentation for or reverse-engineer metadata formats for which there is no write support in the ataraid(4) driver (for example, Adaptec HostRAID).
      2. Add CAM support for exporting PCI information to GEOM.

      GPIO Framework

      Links
      URL: http://wiki.FreeBSD.org/GPIO

      Contact: Luiz Otavio O Souza <loos.br@gmail.com>
      Contact: Oleksandr Tymoshenko <gonzo@FreeBSD.org>

      Implementation of General Purpose Input/Output interface for FreeBSD. Current GPIO bus implementation allows user to control pins from userland and it could be expanded to support various type of peripheral devices. So far there are two drivers:

      • gpioled provides simple led(4) functionality.
      • gpioiic implements I2C over GPIO.

      Framework is used in Alexandr Rybalko's port of FreeBSD to D-Link DIR-320 and in Luis Otavio O Souza's work of bringing FreeBSD to RouterBoard.

      New System Installer — pc-sysinstall

      Links
      Initial commit message URL: http://lists.FreeBSD.org/pipermail/svn-src-all/2010-June/025660.html
      BSDCan slides URL: http://www.BSDCan.org/2010/schedule/attachments/142_pc-sysinstall-kris-moore-2010.pdf

      Contact: Kris Moore <kris@pcbsd.org>
      Contact: M. Warner Losh <imp@FreeBSD.org>

      The new system installation backend, pc-sysinstall, was merged into HEAD recently and work is already underway to make it more functional and useful as a complete replacement to standard "sysinstall". It is written 100% in shell, not requiring any additional tools from what is standard to FreeBSD. The backend already supports a number of exciting features such as:

      • ZFS (Including support for raidz/mirror/multiple device pool setups).
      • Disk encryption via GELI(8).
      • Auto labeling of file systems with glabel(8).
      • Big disk support using GPT/EFI.
      • Full Installation Logging, which is saved to disk for post-install inspection.

      In addition to the features above, pc-sysinstall is unique, in that every install ends up being a scripted install. Front-ends, be it GUI- or text-based, simply generate the appropriate system configuration file, and pc-sysinstall does the grunt work of the actual installation. This is important for a couple of reasons. First, it makes the task of front-end development much easier by not needing to worry about a backend-driven program flow. Second it means that any front-end can be used to generate the installation configuration file, which can then be copied or modified to perform automated installs.

      While pc-sysinstall is still relatively new, it is already in use as the default backend for PC-BSD 8.0 and 8.1, and has been getting a very good reception and any bugs found are fixed quickly. A text-based front-end is already in the works which will allow installation media to be created without X11 support.

      OpenAFS Port

      Links
      OpenAFS home page URL: http://openafs.org
      FreeBSD port for the OpenAFS 1.5.75 release URL: http://web.mit.edu/freebsd/openafs/openafs.shar

      Contact: Benjamin Kaduk <kaduk@mit.edu>
      Contact: Derrick Brashear <shadow@gmail.com>

      AFS is a distributed network filesystem that originated from the Andrew Project at Carnegie-Mellon University; the OpenAFS client implementation has not been particularly useful on FreeBSD since the 4.X releases. Recent work on the OpenAFS codebase has updated it to be consistent with current versions of FreeBSD, and the client, though still considered experimental, is now relatively stable for light (single-threaded) use on 9-CURRENT. The auxiliary utilities for managing and examining the filesystem are functional, and reading and writing files works sufficiently well to copy /usr/src into and out of AFS. Compiling and running executables in AFS is unsuccessful, though, as mmap() is not always reliable.

      There are several known outstanding issues that are being worked on, but detailed bug reports are welcome at port-freebsd@openafs.org.

      Open tasks:

      1. Fix the {get,put}pages vnode operations for more reliable mmap() operation.
      2. Update VFS locking to allow the use of disk-based client caches as well as memory-based caches.
      3. Track down races and deadlocks that appear under load.
      4. Integrate with the bsd.kmod.mk kernel-module build infrastructure.

      Resource Containers

      Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

      As of now, FreeBSD only offers very rudimentary resource controls — resource limits for many resources (e.g. SysV IPC) are missing, and there is no way to set resource limits for jails. As a result, users who want to run many different workloads on a single physical machine often have to replace jails with several FreeBSD instances running in virtual machines.

      The goal of this project is to implement resource containers and a simple per-jail resource limits mechanism. Resource containers are also a prerequisite for other resource management mechanisms, such as Hierarchical Resource Limits, for "Collective Limits on Set of Processes (aka. Jobs)" Google Summer of Code 2010 project, for implementing mechanism similar to Linux cgroups, and might be also used to e.g. provide precise resource usage accounting for administrative or billing purposes.

      This project is being sponsored by The FreeBSD Foundation.

      V4L Support in Linux Emulator

      Links
      URL: http://opal.com/freebsd/sys/compat/linux/

      Contact: J.R. Oldroyd <fbsd@opal.com>

      Some bug fixes were applied, and the code was also tested and made to work with the cuse4bsd webcam driver, which supports a great many camera chipsets.

      The code is still only in 9-CURRENT. We were going to MFC it to 8.x but ran into the code freeze for 8.1, so missed that. However, the code does work on 8-STABLE. We will try to get it MFC'd for 8.2.


      FreeBSD Team Reports


      FreeBSD Bugbusting Team

      Links
      FreeBSD Support page URL: http://www.FreeBSD.org/support.html#gnats
      Resources and documentation available for Bugbusting URL: http://wiki.FreeBSD.org/BugBusting
      Information on Bugathons URL: http://wiki.FreeBSD.org/Bugathons
      Links to all of the auto-generated PR reports URL: http://people.FreeBSD.org/~linimon/studies/prs/
      PRs recommended for committer evaluation by the bugbusting team URL: http://people.FreeBSD.org/~linimon/studies/prs/recommended_prs.html
      PRs considered easy by the bugbusting team URL: http://people.FreeBSD.org/~linimon/studies/prs/easy_prs.html
      Summary Chart of FreeBSD PRs URL: http://people.FreeBSD.org/~linimon/studies/prs/prs_for_all_groups.html

      Contact: Gavin Atkinson <gavin@FreeBSD.org>
      Contact: Mark Linimon <linimon@FreeBSD.org>
      Contact: Remko Lodder <remko@FreeBSD.org>
      Contact: Volker Werth <vwe@FreeBSD.org>

      After a long hiatus, we aim to hold a bugathon on the weekend of the 6th - 9th August. Everybody is welcome to help resolve or progress PRs from the database. We appreciate the help of committers and non-committers alike, please join us on IRC in #freebsd-bugbusters on EFnet if you are free at any time over that weekend and can help. Please see the "Bugathon" URL for more information.

      Mark Linimon and Gavin Atkinson held a session on the State of Bugbusting at BSDCan, which was well attended and led to some interesting discussions. Time was also found to sit down with several committers to discuss long-standing PRs.

      The bugbusting team continue work on trying to make the GNATS PR database more accessible and easier for committers to find and resolve PRs.

      As a result, PRs continue to be classified as they arrive, by adding 'tags' to the subject lines corresponding to the kernel subsystem involved, or man page references for userland PRs. Reports are generated from these nightly, grouping related PRs in one place, sorted by tag or man page. Mark Linimon continues work on producing a new report, Summary Chart of PRs with Tags, which sorts tagged PRs into logical groups such as file system, network drivers, libraries, and so forth. The slice labels are clickable and may further subdivide the groups. The chart is updated once a day. You can consider it as a prototype for browsing "subcategories" of kernel PRs.

      The "recommended list" has been split up into "non-trivial PRs which need committer evaluation" and the "easy list" of trivial PRs, to try to focus some attention on the latter. Various new reports exist, including "PRs containing code for new device drivers", "PRs which are from FreeBSD vendors or OEMs", and "PRs referencing other BSDs".

      It is now possible for interested parties to be emailed a weekly, customized, report similar in style to the above. If you are interested in setting one up, contact linimon@FreeBSD.org.

      Our clearance rate of PRs, especially in kern and bin, seems to be improving. The number of non-ports PRs has stayed almost constant since the last status report.

      As always, anybody interested in helping out with the PR queue is welcome to join us in #freebsd-bugbusters on EFnet. We are always looking for additional help, whether your interests lie in triaging incoming PRs, generating patches to resolve existing problems, or simply helping with the database housekeeping (identifying duplicate PRs, ones that have already been resolved, etc). This is a great way of getting more involved with FreeBSD!

      Open tasks:

      1. Plan and manage the bugathon in August, and get as many people as possible interested in participating.
      2. Try to find ways to get more committers helping us with closing PRs that the team has already analyzed.

      FreeBSD Core Team Election

      Contact: Core Team <core@FreeBSD.org>

      The 2010 FreeBSD core team election was recently completed. The FreeBSD core team acts as the project's "board of directors" and is responsible for approving new src committers, resolving disputes between developers, appointing sub-committees for specific purposes (security officer, release engineering, port managers, webmaster, et cetera), and making any other administrative or policy decisions as needed. The core team has been elected by FreeBSD developers every 2 years since 2000, and this marks our 6th democratically elected core team.

      The new core team would like to thank outgoing members Kris Kennaway, Giorgos Keramidas, George V. Neville-Neil, Murray Stokely, and Peter Wemm for their service over the past two (and in some cases, many more) years.

      The core team would also especially like to thank Dag-Erling Smøgrav for running the election.

      The newly elected core team members are:

      • John Baldwin
      • Konstantin Belousov
      • Warner Losh
      • Pav Lucistnik
      • Colin Percival

      The returning core team members are:

      • Wilko Bulte
      • Brooks Davis
      • Hiroki Sato
      • Robert Watson

      Release Engineering Team

      Links
      URL: http://www.FreeBSD.org/releng/

      Contact: Release Engineering Team <re@FreeBSD.org>

      The Release Engineering Team has been working on the FreeBSD 8.1-RELEASE. At the time of this writing the final builds have been completed and uploaded to the master FTP site. The release announcement should be made within the next couple of days.

      The FreeBSD Foundation Status Report

      Links
      URL: http://www.FreeBSDFoundation.org

      Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

      We were proud to be a sponsor for BSDCan in May. We also committed to sponsoring MeetBSD 2010 Poland and California. We provided 12 travel grants for BSDCan.

      The Foundation and Core Team held a summit on BSD-licensed toolchains at BSDCan 2010.

      We officially kicked off five new projects that we are funding. They are BSNMP Improvements by Shteryana Shopova, Userland DTrace by Rui Paulo, FreeBSD jail-based virtualization by Bjoern Zeeb, DAHDI FreeBSD driver port by Max Khon, and Resource Containers project by Edward Tomasz Napierała.

      We continued our work on infrastructure projects to beef up hardware for package building, network testing, etc. This includes purchasing equipment as well as managing equipment donations.

      We are half way through the year and we have raised around $48,000 towards our goal of $350,000. Find out how to make a donation at http://www.FreeBSDFoundation.org/donate/.

      Our semi-annual newsletter will be published soon. Check out our website to find out more!


      Network Infrastructure


      Enhancing the FreeBSD TCP Implementation

      Links
      URL: http://caia.swin.edu.au/freebsd/etcp09/
      URL: http://caia.swin.edu.au/urp/newtcp/
      URL: http://www.FreeBSDfoundation.org/projects.shtml
      URL: http://people.FreeBSD.org/~lstewart/patches/tcp_ffcaia2008/

      Contact: Lawrence Stewart <lstewart@FreeBSD.org>

      SIFTR was recently imported into HEAD and will be backported to 8-STABLE in time to be included in 8.2-RELEASE.

      TCP reassembly queue autotuning will be ready for public testing within the next week and will be committed soon after. It too will be backported to 8-STABLE after an appropriate burn in period.

      Open tasks:

      1. Try SIFTR out and let me know if you run into any problems.
      2. Solicit external testing for and commit the reassembly queue autotuning patch.

      libnetstat(3)

      Links
      Wiki Page URL: http://wiki.FreeBSD.org/LibNetstat
      Patches URL: http://people.FreeBSD.org/~pgj/libnetstat/
      Perforce Depot (SoC 2009) URL: http://p4web.freebsd.org/@md=d&cd=//depot/projects/soc2009/&c=mGl@//depot/projects/soc2009/pgj_libstat/?ac=83

      Contact: Gábor Páli <pgj@FreeBSD.org>
      Contact: Aman Jassal <aman@FreeBSD.org>

      This project is about creating a wrapper library to support monitoring and management of networking with avoiding direct use of the FreeBSD kvm(3) and sysctl(3) interfaces. This approach would allow the kernel implementation to change and monitoring applications to be extended without breaking applications and requiring them to be recompiled. We decided to merge the sources from the last year's Summer of Code project back to the FreeBSD src/ repository piece by piece, and we have defined several phases of integration.

      • Standardize the in-kernel networking statistics structures.
      • Build a sysctl(3) interface, and add export routines.
      • Add a library, libnetstat(3) to work with the exported information, and to provide further functions in order to support extracting information via kvm(3). This library implements abstractions over the gathered data.
      • Adapt sources of the existing applications, i.e. netstat(1) and bsnmpd(1) to use the abstractions offered by the library, resulting in a cleaner and simpler code.
      • Add new applications on the top of the library, e.g. nettop(1).

      The first phase has been already posted for review. Note that we are looking for a sponsor with an src commit bit and enough time to represent the effort towards the Project.

      Open tasks:

      1. Review the sources.
      2. Pick a task from the list, and send patches.
      3. Comment the patches, help them to improve.

      Kernel


      Interrupt Threads

      Contact: John Baldwin <jhb@FreeBSD.org>

      For a while I have wanted to rework interrupt threads to address a few issues. The new design uses per-CPU queues of interrupt handlers. Interrupt threads are allocated by a CPU from a pool and bound to that CPU while draining that CPU's queue of handlers. Non-filter handlers can also reschedule themselves at the back of the current CPU's queue while executing. Filters with handlers are now always enabled and should provide a full replacement for the various uses of filters with "fast" taskqueues. A new class of "manual" handlers are also available which are not automatically scheduled, but are only explicitly scheduled from a filter. Thus, a filter can potentially schedule multiple handlers.

      The code has been tested on amd64, but it needs wider review and testing. I hope to start soliciting review and feedback soon with the goal of getting the code into 9.0.

      Jail-Based Virtualization

      Links
      FreeBSD Foundation Announcement URL: http://www.FreeBSDFoundation.org/project%20announcements.shtml#Bjoern
      Perforce Workspace URL: http://p4web.FreeBSD.org/@md=d&cd=//&c=Z8Q@//depot/user/bz/vimage/src/?ac=83

      Contact: Bjoern A. Zeeb <bz@FreeBSD.org>

      The project started with some cleanup on the network stack after all the import work and adjustments for virtualization to minimize changes to earlier branches. These made it into the tree already and to 8-STABLE, and it will be included in the upcoming 8.1 release.

      The first major task was to generalize the virtualization framework, so that virtualization of further subsystems would be easier and could be achieved with less duplication.

      In addition some documentation on the virtual network stack programming was written to help developers virtualizing their code. The interactive kernel debugger support was improved and libjail along with jls and netstat can work on core dumps now and query individual jails and attached virtual network stacks.

      The second major task was network stack teardown, a concept introduced with the network stack virtualization. The primary goal was to prototype a shutdown of the (virtual) network stacks from top to bottom, which means letting interfaces go last rather than first. Work in this area is still in progress and will have to continue to allow long term stability and a leak and panic free shutdown.

      The work on this project had been sponsored by the FreeBSD Foundation and CK Software GmbH. Special thanks also to John Baldwin and Philip Paeps for helping with review and suggestions.

      Open tasks:

      1. Merge stabilised change sets.
      2. Work further down the network stack freeing all resources for a stable, safe teardown.

      Kernel Event Timers Infrastructure

      Contact: Alexander Motin <mav@FreeBSD.org>

      Modern x86 systems include four different types of event timers: i8254, RTC, LAPIC, and HPET. First three are already supported by FreeBSD. Depending on hardware and loader tunables, periodic interrupts from them are used to trigger all time-based events in kernel. That code has a long history, that made it tangled and at the same time limited and hard-coded.

      New kernel event timers infrastructure was started to allow different event timer hardware to be operated in uniform way and to allow more features to be supported. Work consists of three main parts: writing machine-independent timer driver API and management code, updating existing drivers and improving HPET driver to support event timers.

      The new driver API provides unified support for both per-CPU (independent for every CPU core) and global timers in periodic and one-shot modes. Management code at this moment uses only periodic mode, while one-shot mode use is planned by later tickless kernel work.

      Different kinds of timers have different capabilities and could be present in hardware in different combinations. In every situation the infrastructure automatically chooses two best event timers to supply system with hardclock(), statclock(), and profclock() events. If some timer is not functioning — it will be replaced. If there is no second timer — it will be emulated. The administrator may affect that choice using loader tunables during boot and sysctl variables in run-time (kern.eventtimer.*, and so on).

      Most of the code was recently committed to HEAD. Now it is used by i386 and amd64 architectures.

      Open tasks:

      1. Troubleshoot possible hardware and software issues.
      2. Port other architectures to the new infrastructure.
      3. Implement tickless kernel, utilizing new features, such as per-CPU and one-shot timers.

      ZFS

      Links
      FreeBSD ZFS Wiki URL: http://wiki.FreeBSD.org/ZFS
      Latest FreeBSD ZFS development tree URL: http://perforce.FreeBSD.org/depotTreeBrowser.cgi?FSPC=//depot/user/pjd/zfs

      Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
      Contact: Martin Matuska <mm@FreeBSD.org>
      Contact: Xin Li <delphij@FreeBSD.org>

      The ZFS file system has been updated to version 15 on HEAD and it will be MFC'ed to 8-STABLE around September 13th, 2010. Work is in progress on porting the recent ZFS version 26 with deduplication functionality.

      Open tasks:

      1. Fix bugs, unresolved issues and to-dos in Perforce.

      Documentation


      The FreeBSD German Documentation Project

      Links
      Website of the FreeBSD German Documentation Project URL: http://doc.bsdgroup.de
      Mailing lists for the coordination of our work and the place where you can report bugs back to us URL: http://www.FreeBSD.de/mailinglists.html

      Contact: Johann Kois <jkois@FreeBSD.org>
      Contact: Benedict Reuschling <bcr@FreeBSD.org>

      A number of updates to the documentation were made since the last status report. We are especially grateful for the contributions from external people who sent the translations. People like Fabian Ruch, who updated the porters-handbook to the latest version (which had been on his to-do list for quite some time), and Benjamin Lukas, who did a great job with the from-scratch translation of the MAC chapter of the German handbook. We thank them both for their contributions and hope they will continue their efforts to enhance the German documentation.

      Frank Börner was released from Benedicts mentorship and is now a full committer to the German Documentation Project. We are always looking for fresh blood that is willing to be mentored by us as a first step in becoming committers for the documentation project themselves.

      Johann is keeping up the German website with the latest version. But we could use more translators for sections that are not fully translated yet.

      Open tasks:

      1. Read the translations and report bugs that you have found (even small ones).
      2. Translate new parts of the documentation and the website.

      The FreeBSD Hungarian Documentation Project

      Links
      Hungarian FreeBSD web pages URL: http://www.FreeBSD.org/hu/
      Hungarian FreeBSD documentation URL: http://www.FreeBSD.org/doc/hu/
      The FreeBSD Hungarian Documentation Project's Wiki Page URL: http://wiki.FreeBSD.org/HungarianDocumentationProject
      Perforce Deport for the FreeBSD Hungarian Documentation Project URL: http://p4web.FreeBSD.org/@md=d&cd=//depot/projects/docproj_hu/&c=aXw@//depot/projects/docproj_hu/?ac=83

      Contact: Gábor Kövesdán <gabor@FreeBSD.org>
      Contact: Gábor Páli <pgj@FreeBSD.org>

      Thanks to Katalin Konkoly, the first few chapters of the FreeBSD Handbook translation have been reviewed, therefore many typos and mistranslations were spotted and fixed. Apart from this, we are still keeping the existing documentation and web page translations up to date, currently without plans on further work. If you are interested in helping us, or you have any comments, or requests regarding the translations, do not hesitate to contact the project via the email addresses mentioned in the entry.

      Open tasks:

      1. Review translations and send feedback.
      2. Translate release notes.
      3. Add more article translations.

      The FreeBSD Japanese Documentation Project

      Links
      Japanese FreeBSD Web Pages URL: http://www.FreeBSD.org/ja/
      The FreeBSD Japanese Documentation Project's Web Page URL: http://www.jp.FreeBSD.org/doc-jp/

      Contact: Hiroki Sato <hrs@FreeBSD.org>
      Contact: Ryusuke Suzuki <ryusuke@FreeBSD.org>

      This project focuses on updating the www/ja and doc/ja_JP.eucJP/ trees. Since last year www/ja tree has been mostly synchronized with the English counterpart and doc/ja_JP.eucJP has also been updated steadily. We are now working on FreeBSD Handbook and Porter's Handbook.

      Open tasks:

      1. More Japanese translation of FreeBSD Handbook and contents of www.FreeBSD.org.
      2. Pre-/post-commit review of the translation.

      The FreeBSD Spanish Documentation Project

      Links
      Primer for translators URL: http://www.FreeBSD.org/doc/es/articles/fdp-es/

      Contact: Gábor Kövesdán <gabor@FreeBSD.org>
      Contact: Vicente Carrasco Vayá <carvay@FreeBSD.org>

      We need manpower. Existing documentation set has not been updated for quite some time because of lack of volunteers. Current members are busy with other projects and real life at the moment and we have not received anything from outside contributors. It is a shame because there are lots of users in Spain and Latin-America, as well. Besides, the world's first Free Software Street has been recently inaugurated in Spain. This obviously means that there is interest in free software but unfortunately, this translation project is not going very well nowadays.

      Open tasks:

      1. Review and update existing translations.

      Userland Programs


      BSD-Licensed grep in Base System

      Links
      Sources in Perforce URL: http://p4db.FreeBSD.org/depotTreeBrowser.cgi?FSPC=//depot/projects/soc2008/gabor_textproc/grep

      Contact: Gábor Kövesdán <gabor@FreeBSD.org>

      A portbuild test showed that grep is basically ready to enter HEAD, but there were a few failures that seem to be related. These have to be investigated and fixed before committing grep to 9-CURRENT.

      Open tasks:

      1. Investigate and fix some minor issues.

      BSD-Licensed iconv in Base System

      Links
      The latest patch for the base system URL: http://kovesdan.org/patches/iconv-20100708.diff

      Contact: Gábor Kövesdán <gabor@FreeBSD.org>

      The work has been completed and the GNU compatibility levels seems to be quite high. One exception is the fallback support. It is difficult to implement that facility in this implementation because the design is somewhat different. Probably, it will not be a big problem because that functionality is not even documented in the GNU version so few applications might use it.

      Open tasks:

      1. Run a portbuild test and solve possible problems that show up.

      FreeBSD Services Control — fsc

      Links
      URL: http://people.FreeBSD.org/~trhodes/fsc/

      Contact: Tom Rhodes <trhodes@FreeBSD.org>

      FreeBSD Services Control is a mix of binaries which integrate into the rc.d system and provide for service (daemon) monitoring. It knows about signals, pidfiles, and uses very few resources.

      The fsc daemon (fscd) runs in the background once the system has started. Services are then added to this daemon via the fscadm control utility, and from there they will be monitored. When they die, depending on the reason, they will be restarted. Certain signals may be ignored (list not decided) and fscd will remove that service from monitoring. Every action is logged to the system logging daemon. Additionally, the fscadm utility may be used to inquire about what services are monitored, their pidfile location, and current process ID.

      FSC provides several advantages over the third-party daemontools package. For example, fscd uses push notifications instead of polling; fscd is an internal, FreeBSD-maintained software package accessible to all developers, where daemontools would have to be a port and require us to maintain patches; fscd could be easily integrated with the current rc.d infrastructure.

      Partially based on the ideas of daemontools and Solaris Service Service Mangement Facility (SMF), this could be an extremely useful tool for FreeBSD systems.

      Open tasks:

      1. Testing. Get feedback on how it works in various environments.
      2. Code review.
      3. Other ideas on the rc.d integration.
      4. Update the manual pages.

      Architectures


      Flattened Device Tree for Embedded FreeBSD

      Links
      Project wiki pages URL: http://wiki.FreeBSD.org/FlattenedDeviceTree

      Contact: Rafal Jaworowski <raj@semihalf.com>

      The purpose of this project was to provide FreeBSD with support for the Flattened Device Tree (FDT) technology. A mechanism for describing computer hardware resources, which cannot be probed or self enumerated, in a uniform and portable way. The primary consumers of this technology are embedded FreeBSD platforms (ARM, MIPS, PowerPC), where a lot of designs are based on similar chips, but have different assignment of pins, memory layout, addresses ranges, interrupts routing and other resources.

      Current state highlights:

      • All code and documentation developed during the course of this project was merged with HEAD, which covers FDT support for the following platforms and systems:
      • Marvell ARM
        • DB-88F5182
        • DB-88F5281
        • DB-88F6281
        • DB-78100
        • SheevaPlug
      • Freescale PowerPC
        • MPC8555CDS
        • MPC8572DS
      • The FDT infrastructure (bus drivers, helper libraries, and routines shared across architectures and platforms) allows for easier porting to new platforms or variations. The initially supported systems offer a working example of how to migrate towards FDT approach.

      Work on this project was sponsored by the FreeBSD Foundation.

      Open tasks:

      1. Improve how-to and guidelines for new adopters (how to convert to FDT and so on).
      2. Migrate more existing embedded FreeBSD platforms (ARM, MIPS) to FDT approach.

      FreeBSD on the Sony Playstation 3

      Links
      Playstation 3 SVN repository URL: http://svn.FreeBSD.org/viewvc/base/user/nwhitehorn/ps3/

      Contact: Nathan Whitehorn <nwhitehorn@FreeBSD.org>

      Work has begun to port FreeBSD/powerpc64 to the IBM Cell-based Sony Playstation 3, using the OtherOS feature present on some models of the console. As of July 14, the FreeBSD boot loader is ported, and it is possible to netboot a kernel, which has support for the framebuffer, MMU, and device discovery. Once work on drivers for the network interface and interrupt controller is complete, it will be possible to boot the console multi-user.

      FreeBSD/avr32

      Links
      URL: http://wiki.FreeBSD.org/FreeBSD/avr32

      Contact: Oleksandr Tymoshenko <gonzo@FreeBSD.org>

      The FreeBSD/avr32 project was started by Arnar Mar Sing, and actively developed by him and Ulf Lilleengen. It successfully reached single-user stage but since then has not progressed much. At the moment I am trying to get it back into shape. So far some problems with toolchain on i386 host have been fixed, buildkernel succeeds, buildworld succeeds with some exceptions. Next step would be fixing pmap and bringing port back to single-user stage.

      FreeBSD/powerpc64

      Links
      Install CDs for powerpc64 URL: http://people.FreeBSD.org/~nwhitehorn/FreeBSD-9.0-20100715-SNAP-powerpc64/

      Contact: Nathan Whitehorn <nwhitehorn@FreeBSD.org>

      On July 13, FreeBSD/powerpc64 was integrated into HEAD. This provides support for fully 64-bit operation on 64-bit PowerPC machines conforming to the Book-S specification, including the PowerPC 970, Cell, and POWER4-7. Hardware support is currently limited to Apple machines, although this should expand in the near future.

      Currently supported hardware:

      • Apple Xserve G5
      • Apple Power Macintosh G5
      • Apple iMac G5

      FreeBSD/sparc64

      Contact: Marius Strobl <marius@FreeBSD.org>

      Since the last status report some issues with cas(4) have been fixed, allowing it to work with Sun GigaSwift Ethernet 1.0 MMF cards (Cassini Kuheen, part no. 501-5524) as well as the on-board interfaces of Sun Fire B100s server blades (for the Sun Fire B1600 platform).

      Support for Fujitsu (Siemens) PRIMEPOWER 250 based on SPARC64 V CPUs has been added. PRIMEPOWER 450, 650, and 850 likely also work but have not been tested. This also means that the building blocks for support of machines based on SPARC64 VI and VII CPUs like the Fujitsu/Sun SPARC Enterprise Mx000 series are now in place, but they need testing as well.

      The problems with Schizo version 7 bridges (actually the firmware of these machines) triggering panics during boot finally should be solved.

      The work on getting Sun Fire V1280 supported has been stalled due to access to such machines no longer being available.

      The above mentioned improvements are/will be available in FreeBSD 8.1-RELEASE and 7.4-RELEASE.

      Open tasks:

      1. Access to machines based on SPARC64 VI and VII CPUs, like the Fujitsu/Sun SPARC Enterprise Mx000 series would be appreciated.
      2. Someone adding support for 64-bit SPARC V9 to Clang/LLVM, and getting it on par with GCC would be appreciated.

      Ports


      Chromium Web Browser

      Links
      Main chromium site URL: http://chromium.hybridsource.org
      PR for chromium port URL: http://www.FreeBSD.org/cgi/query-pr.cgi?pr=146302

      Contact: Ruben <chromium@hybridsource.org>

      Chromium is a Webkit-based web browser that is largely BSD-licensed. It works very well on FreeBSD and supports new features like HTML 5 video. This effort uses a new hybrid-source model, where the FreeBSD patches are largely kept closed for a limited time. I submitted Chromium to ports a couple of months ago and recently updated the submission to the stable 5.0.375 branch. The port is ready to be committed pending final legal approval by the FreeBSD Foundation. Further work remains to port Chromium to FreeBSD completely, such as porting the task manager fully and making sure extensions work properly.

      FreeBSD Haskell

      Links
      Wiki Page of the Project URL: http://wiki.FreeBSD.org/Haskell
      FreeBSD Haskell Ports URL: http://www.FreeBSD.org/ports/haskell.html
      The freebsd-haskell Mailing List URL: http://www.haskell.org/mailman/listinfo/freebsd-haskell

      Contact: Gábor Páli <pgj@FreeBSD.org>
      Contact: Giuseppe Pilichi <jacula@FreeBSD.org>
      Contact: Ashish Shukla <ashish@FreeBSD.org>

      Our efforts on porting the generalized, general-purpose purely functional programming language, Haskell has rallied, since two new committers, Giuseppe Pilichi and Ashish Shukla joined recently, forming the FreeBSD Haskell Team. Over the last months, FreeBSD/i386 and FreeBSD/amd64 have become Tier-1 platforms, featuring officially supported vanilla binary distributions for the Glasgow Haskell Compiler starting from version 6.12.1. We introduced a unified ports infrastructure for Haskell Cabal ports, which also makes possible the direct translation of Cabal package descriptions to FreeBSD ports. The number of Haskell package ports increases steadily.

      Open tasks:

      1. Improve support for Haskell Cabal packages and their translation.
      2. Create a port for Haskell Platform.
      3. Add more Haskell package ports.
      4. Test and send feedback.

      Ports Collection

      Links
      URL: http://www.FreeBSD.org/ports/
      URL: http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing-ports/
      URL: http://portsmon.FreeBSD.org/index.html
      URL: http://www.FreeBSD.org/portmgr/index.html
      URL: http://blogs.FreeBSDish.org/portmgr/
      URL: http://www.twitter.com/freebsd_portmgr/
      URL: http://www.facebook.com/group.php?gid=135441496471197
      URL: http://tinderbox.marcuscom.com/

      Contact: Thomas Abthorpe <portmgr-secretary@FreeBSD.org>
      Contact: Port Management Team <portmgr@FreeBSD.org>

      A significant part of quarter two was spent coordinating efforts for inclusion of Xorg 7.5, KDE 4, GNOME 2, plus preparation of ports for the 8.1 release process. Due to the success of enforcing Feature Safe ports commits during 7.3-RELEASE, it was continued for the recent src/ freeze.

      The port count is approaching 22,000 ports. The open PR count currently floats at about 1200 entries.

      Since the last report, we added four new committers, and had two old committers rejoin us.

      The Ports Management Team is very grateful to the FreeBSD Foundation for sponsoring two new head nodes for the ports building cluster, pointyhat. Each of the new head nodes has a larger capacity, both with regard to performance but also in amount of space available for the staging areas, allowing for faster, and thus more, build cycles. Additionally, having two head nodes will allow us to dedicate one of them for building production-ready binary packages, adding predicability for our users to when what types of packages are available for installation, and dedicate the other for regression testing of large port updates, ports infrastructure improvements, the cluster scheduling code, and FreeBSD itself. Over the last few weeks, Mark Linimon has been working hard to get the first of the two new nodes online and has already completed its first package build. This has involved a substantial rework of our custom codebase.

      The Ports Management team have been running -exp runs on an ongoing basis, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates. Of note, -exp runs were done for:

      • ale: Update of math/gmp.
      • delphij: Changes to Mk/bsd.ldap.mk.
      • gahr: Inclusion of USE_GL=glew.
      • pgollucci: Changes to Mk/bsd.*apache.mk plus updates to devel/apr and www/apache*.
      • Testing of x11/xorg, x11/gnome2, x11/kde4, and lang/mono
      • A test run make fetch run.
      • A test run for devel/gettext.
      • mm: Inclusion of USE_XZ.
      • ale: Request to switch default mysql from 5.0-EOL to 5.1-GA.

      alepulver's Licensing Framework Summer of Code project has made - it into the tree and the Port Management Team is currently + it into the tree and the Ports Management Team is currently assessing the fallout and it will come up with guidelines and documentation in due time.

      Open tasks:

      1. Looking for help fixing ports broken on 9-CURRENT.
      2. Looking for help with Tier-2 architectures.
      3. Most ports PRs are assigned, we now need to focus on testing, committing, and closing.

      Miscellaneous


      BSD-Day@2010

      Links
      URL: http://wiki.freebsd.org/BSDDay_2010

      Contact: Gábor Páli <pgj@FreeBSD.org>

      The purpose of this one-day event is to gather Central European developers of today's open-source BSD systems to popularize their work and their organization, and to provide an interface for real-life communication. There are no formalities, no papers, and no registration or participation fee. However the invited developers are encouraged to give a talk on their favorite BSD-related topic or join the live forum, then have a beer with the other folks around. The goal is to motivate potential future developers and users, especially undergraduate university students to work with BSD systems.

      This year's BSD-Day will be held in Budapest, Hungary at Eötvös Loránd University, Faculty of Informatics on November 20, 2010.

      Open tasks:

      1. Apply as a developer, we are still looking for BSD people in the area.

      BSDCan

      Links
      BSDCan 2010 URL: http://www.BSDCan.org/2010/

      Contact: Dan Langille <dvl@FreeBSD.org>

      BSDCan 2010 was our 7th conference. As has become the custom, a FreeBSD developer summit was held in the two days before the conference. Record numbers attended the Dev Summit which carried over into the conference proper. It was great to see representatives from so many more companies. I saw many great ideas take root and the start of cooperation on several projects.

      The talks during the Dev Summit are beginning to attract a wider audience, and we have been talking about opening this up to the general audience by creating a fourth track at BSDCan 2011.

      As impossible as it sounds, each year has seen an increase in the quality of talks and the number of proposals submitted.

      Open tasks:

      1. I need people to help with various pre-conference tasks: website updates, booking travel, etc.

      meetBSD 2010 — The BSD Conference

      Links
      URL: http://www.meetbsd.org
      URL: http://picasaweb.google.com/meetbsd/MeetBSD2010Day1#
      URL: http://picasaweb.google.com/meetbsd/MeetBSD2010Day2#
      URL: http://picasaweb.google.com/meetbsd/MeetBSD2010SocialEvent#

      Contact: meetBSD Information <info@meetbsd.org>

      meetBSD 2010 took place on July 2 - 3 in Krakow, Poland at the Faculty of Mathematics and Computer Science building of the Jagiellonian University.

      The gathering was a much successful event which brought together developers, contributors, and users of the BSD systems from around the world. We had many interesting presentations, of various character and appeal for the diversified audience.

      Attendees had a chance for taking the BSD Certification exam during the conference, as well as the advantage of face to face side conversations and discussions, which continued long during the social event on Friday night!

      The conference presentation slides are already available for download. Video recordings edition is being finalized, and their publication is expected shortly.

      We hope you enjoyed the event and had great time in Krakow. See you again soon!

      News Home | Status Home

    diff --git a/website/content/en/status/report-2012-04-2012-06.html b/website/content/en/status/report-2012-04-2012-06.html index a71dd542d2..f4b45bb520 100644 --- a/website/content/en/status/report-2012-04-2012-06.html +++ b/website/content/en/status/report-2012-04-2012-06.html @@ -1,835 +1,835 @@ FreeBSD Quarterly Status Report
    Skip site navigation (1) Skip section navigation (2)

    Introduction

    This report covers FreeBSD-related projects between April and June 2012. This quarter was highlighted by having a new Core Team elected, which took office on July 11th to start its work with a relatively high number of new members. Note that this is the second of the four reports planned for 2012.

    Thanks to all the reporters for the excellent work! This report contains 17 entries and we hope you enjoy reading it.

    Please note that the deadline for submissions covering the period - between July and December 2012 is February 17th, 2013.

    Projects

      Userland Programs

      FreeBSD Team Reports

      Kernel

      Network Infrastructure

      Documentation

      Architectures

      • FreeBSD/arm on ARM Fast Models Simulator for Cortex-A15 MPCore + between July and December 2012 is February 17th, 2013.

        Projects

          Userland Programs

          FreeBSD Team Reports

          Kernel

          Network Infrastructure

          Documentation

          Architectures

          Ports

          Miscellaneous


            Projects



            Userland Programs


            FreeBSD Services Control (fsc)

            Contact: Tom Rhodes <trhodes@FreeBSD.org>

            FSC has been moved into the ports system (see sysutils/fsc) and continues to improve outside of the ports tree. Some interesting work is being done in the area of services control, system boot, and a simplification of the process. Stay tuned for more information in status reports that follow.

            Open tasks:

            1. Test, test, test. Feedback is really important to this project.

            Replacing the Regular Expression Code

            Links
            TRE home page URL: http://laurikari.net/tre/

            Contact: Gábor Kövesdán <gabor@FreeBSD.org>

            It has been decided to implement the optimizations and extensions as a more isolated layer and not directly in TRE itself. Since the last report there has been some progress in this direction and the code has been significantly refactored. It does not work yet in this new form but it is close to a working state. Apart from this, the multiple pattern matching needs some debugging and some minor features are missing.

            Open tasks:

            1. Finish multiple pattern heuristic regex matching.
            2. Implement GNU-specific regex extensions.
            3. Test performance, standard-compliance and correct behavior.

            FreeBSD Team Reports


            FreeBSD Documentation Project

            Links
            URL: http://wiki.freebsd.org/GoogleCodeIn/2011Status
            URL: http://wiki.freebsd.org/201208DevSummit
            URL: http://wiki.freebsd.org/DocIdeaList

            Contact: <freebsd-doc@FreeBSD.org>

            We continue to make progress in committing the work produced as part of Google Code-In 2011; an overview of the status is at http://wiki.freebsd.org/GoogleCodeIn/2011Status. Doc committers and GCIN mentors are encouraged to go through the list and help shepherd outstanding tasks into the tree.

            We are planning a full day of Documentation Summit on the day preceding the August 2012 DevSummit in Cambridge, UK. This follows a successful DocSummit day held at BSDCan in May 2012. Further details are available at: http://wiki.freebsd.org/201208DevSummit.

            A doc sprint took place over IRC (#bsddocs on EFnet) in early July, setting out plans for reviving the marketing team and a strong desire for a new, more organized website.

            A lot of progress and momentum has built up with creating and updating documentation and website content over the last few months. Also read the doceng report for the recent infrastructure improvements.

            Anyone wishing to help with this effort is welcome to join us and say hello either on the freebsd-doc mailing list, or #bsddocs on EFnet IRC.

            Open tasks:

            1. Review the website content and remove outdated parts or update when applicable.
            2. Go through the doc idea list on the wiki and start working them out.

            The FreeBSD Core Team

            Links
            Announcement URL: http://docs.FreeBSD.org/cgi/mid.cgi?1342030291.6001.80.camel

            Contact: Core Team <core@FreeBSD.org>

            The FreeBSD Project is pleased to announce the completion of the 2012 Core Team election. The FreeBSD Core Team acts as the project's "Board of Directors" and is responsible for approving new src committers, resolving disputes between developers, appointing sub-committees for specific purposes (security officer, release engineering, port managers, webmaster, et cetera), and making any other administrative or policy decisions as needed. The Core Team has been elected by FreeBSD developers every 2 years since 2000.

            Peter Wemm rejoins the Core Team after a two-year hiatus, with new members Thomas Abthorpe, Gavin Atkinson, David Chisnall, Attilio Rao and Martin Wilke joining incumbents John Baldwin, Konstantin Belousov and Hiroki Sato.

            The complete newly elected core team is:

            • Thomas Abthorpe <tabthorpe@FreeBSD.org>
            • Gavin Atkinson <gavin@FreeBSD.org>
            • John Baldwin <jhb@FreeBSD.org>
            • Konstantin Belousov <kib@FreeBSD.org>
            • David Chisnall <theraven@FreeBSD.org>
            • Attilio Rao <attilio@FreeBSD.org>
            • Hiroki Sato <hrs@FreeBSD.org>
            • Peter Wemm <peter@FreeBSD.org>
            • Martin Wilke <miwi@FreeBSD.org>

            The new Core Team would like to thank outgoing members Wilko Bulte, Brooks Davis, Warner Losh, Pav Lucistnik, Colin Percival and Robert Watson for their service over the past two (and in some cases, many more) years.

            The Core Team would also especially like to thank Dag-Erling Smørgrav for running the election.

            -

            The FreeBSD Port Management Team

            Links
            +

            The FreeBSD Ports Management Team

            Links
            URL: http://www.FreeBSD.org/ports/
            URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
            URL: http://portsmon.freebsd.org/index.html
            URL: http://www.freebsd.org/portmgr/index.html
            URL: http://blogs.freebsdish.org/portmgr/
            URL: http://www.twitter.com/freebsd_portmgr/
            URL: http://www.facebook.com/portmgr

            Contact: Thomas Abthorpe <portmgr-secretary@FreeBSD.org>
            Contact: Port Management Team <portmgr@FreeBSD.org>

            The ports tree slowly approaches 24,000 ports. The PR count still is close to 1200.

            In Q2 we added 7 new committers and took in one commit bit for safe keeping.

            The Ports Management team have been running -exp runs on an ongoing basis, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates. Of note, -exp runs were done for:

            • automake update
            • cmake update
            • xorg update
            • png update
            • Fix make reinstall
            • Implement USE_QT4 in bsd.ports.mk
            • KDE4 update
            • XFCE4 update
            • bison update
            • perl5.14 as default
            • ruby1.9 as default
            • ruby1.8 update
            • bsdsort regression test

            A lot of focus during this period was put into getting the ports tree into a ready state for FreeBSD 9.1.

            A significant step forward was the implementation of OptionsNG.

            A record number of Port Managers attended BSDCan 2012, with five being present to partake in the week of events, culminating in a portmgr PR closing session that dealt with 18 PRs in one day. You can see a group photo at . While you are there, please click on the "Like" icon.

            Beat Gaetzi has been doing ongoing tests with the ports tree to ensure a smooth transition from CVS to Subversion. The tree was successfully migrated the weekend of June 14, 2012.

            Open tasks:

            1. Looking for help getting ports to build with clang.
            2. Looking for help fixing ports broken on CURRENT. (List needs updating, too.)
            3. Looking for help with Tier-2 architectures.
            4. ports broken by src changes.
            5. ports failing on pointyhat.
            6. ports failing on pointyhat-west.
            7. ports that are marked as BROKEN.
            8. When did that port break?
            9. Most ports PRs are assigned, we now need to focus on testing, committing and closing.

            Kernel


            FreeBSD/at91 Improvements

            Links
            FreeBSD on ATMEL AT91 Wiki URL: http://wiki.freebsd.org/FreeBSDAtmel

            Contact: Warner Losh <imp@FreeBSD.org>

            FreeBSD's Atmel support has languished for some time. A number of improvements were urgently needed as demand for newer SoCs has materialized. New SoC support is not hard, but it does wind up copying a lot of code. I have started down the path to make it easier to do. I had planned on making it table driven. But then I discovered with dts files that Atmel was producing.

            So, I plan on moving to using Atmel's .dsti files, or variations on them. They have .dsti files for all the AT91SAM9 parts. This should allow us to support new SoCs and boards faster.

            However, there are some challenges with this approach. Pin multiplexing seems undefined in Atmel's dts file. Only a few of the devices are well-defined at the present time. And the encoding seems to be immature.

            So we have a target-rich port that is quite ripe for refactoring.

            Open tasks:

            1. Update the base system libfdt to a version that supports include.
            2. Write a .dtsi for Atmel AT91RM9200.
            3. Write .dti files for all supported boards.
            4. Help sort out the pin multiplexing issue.
            5. Refactor existing board files to make new ones easier in the interim.
            6. Knock yourself out and implement board support for new CPUs.

            Network Infrastructure


            Multipath TCP (MPTCP) for FreeBSD

            Links
            URL: http://caia.swin.edu.au/newtcp/mptcp/
            TCP Extensions for Multipath Operation with Multiple Addresses (draft) URL: http://tools.ietf.org/html/draft-ietf-mptcp-multiaddressed-09
            "MultiPath TCP — Linux Kernel implementation" home page URL: http://mptcp.info.ucl.ac.be/

            Contact: Nigel Williams <njwilliams@swin.edu.au>
            Contact: Lawrence Stewart <lastewart@swin.edu.au>
            Contact: Grenville Armitage <garmitage@swin.edu.au>

            Work is underway to create an IETF draft-compatible Multipath TCP implementation for the FreeBSD kernel.

            A key goal of the project is to create a research platform to investigate a range of multipath related transport issues including congestion control, retransmission strategy and packet scheduling policy. We also aim to provide full interoperability with the Linux kernel implementation being developed at Université catholique de Louvain.

            We expect to release code and results at the project's home page as it progresses.

            SMP-Friendly pf(4)

            Links
            Project SVN branch URL: http://svnweb.freebsd.org/base/projects/pf/head/
            Alpha announcement email thread URL: http://lists.freebsd.org/pipermail/freebsd-pf/2012-June/006643.html

            Contact: Gleb Smirnoff <glebius@FreeBSD.org>

            The project is aimed at moving the pf(4) packet filter out of single mutex, as well as in general improving of its FreeBSD port.

            The project is near its finish, the code is planned to go into head after more testing and benchmarking. If you are interested in details, please see the corresponding email thread on freebsd-pf (see links).

            Open tasks:

            1. Rewrite the pf(4) ioctl() interface so that it does not utilize in-kernel structures. That would make ABI more stable and ease future development.

            Documentation


            The FreeBSD Japanese Documentation Project

            Links
            Japanese FreeBSD Web Page URL: http://www.FreeBSD.org/ja/
            The FreeBSD Japanese Documentation Project Web Page URL: http://www.jp.FreeBSD.org/doc-jp/

            Contact: Hiroki Sato <hrs@FreeBSD.org>
            Contact: Ryusuke Suzuki <ryusuke@FreeBSD.org>

            Our translation work has slightly moved on to handbook from the www/ja (CVS) or htdocs (SVN) subtree, since almost translated web page contents were updated to the latest English counterparts.

            During this period, we translated the 8.3-RELEASE announcement and published it in a timely manner. Newsflash and some other updates in the English version were also translated as soon as possible.

            For FreeBSD Handbook, translation work of the "cutting-edge" and "printing" sections have been completed. Some updates in the "linuxemu" and "serialcomms" section were done. At this moment, "bsdinstall", "cutting-edge", "desktop", "install", "introduction", "kernelconfig", "mirrors", "multimedia", "pgpkeys", "ports", "printing", and "x11" chapters are synchronized with the English versions.

            Open tasks:

            1. Further translation work of outdated documents in ja_JP.eucJP subtree.

            Architectures


            FreeBSD/arm on ARM Fast Models Simulator for Cortex-A15 MPCore Processor

            Links
            Cortex-A15 product page URL: http://www.arm.com/products/processors/cortex-a/cortex-a15.php
            Fast Models product page URL: http://www.arm.com/products/tools/models/fast-models.php

            Contact: Zbigniew Bodek <zbb@semihalf.com>
            Contact: Rafal Jaworowski <raj@semihalf.com>
            Contact: Tomasz Nowicki <tn@semihalf.com>

            ARM Fast Models is platform which helps software developers debug systems in parallel with SoC design, speeding up and improving system development. This work is bringing up FreeBSD on ARM Fast Models system based on ARM Cortex-A15 and peripheral components. It works in single user mode, using a compiled-in kernel RAM disk minimal root file system.

            Current FreeBSD support includes:

            • L1, L2 cache, Branch Predictor
            • Dual-core (SMP) support setup in WB cache mode
            • Cortex-A15 integrated Generic Timer
            • Drivers for ARM peripheral components:
              • PL011 UART controller
              • PL390 GIC - Generic Interrupt Controller
              • SP804 Dual Timer

            Next steps:

            • Quad-core (SMP) support
            • Multi-user mode

            Ports


            BSD-licensed Sort Utility (GNU sort(1) Replacement)

            Links
            FreeBSD port of BSD sort(1) URL: http://www.freebsd.org/cgi/cvsweb.cgi/ports/textproc/bsdsort/
            IEEE Std 1003.1-2008 sort(1) specification URL: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html

            Contact: Oleg Moskalenko <oleg.moskalenko@citrix.com>
            Contact: Gábor Kövesdán <gabor@FreeBSD.org>

            BSD sort(1) has been made the default sort utility in 10-CURRENT. It is compatible with the latest GNU sort(1), version 8.15, except that the multi-threaded mode is not enabled by default.

            Open tasks:

            1. When the track record of the BSD sort(1) allows, remove GNU sort(1) from -CURRENT.
            2. Improve reliability of the multi-threaded sort and investigate the possibility of making it the default compilation mode.
            3. Investigate possibility of factoring out the sort functionality into a standalone library so that other utilities can also make use of it.

            FreeBSD Haskell Ports

            Links
            FreeBSD Haskell wiki page URL: http://wiki.freebsd.org/Haskell
            FreeBSD Haskell ports repository URL: https://github.com/freebsd-haskell/freebsd-haskell/

            Contact: Gábor PÁLI <pgj@FreeBSD.org>
            Contact: Ashish SHUKLA <ashish@FreeBSD.org>

            We are proud to announce that the FreeBSD Haskell Team has updated the Haskell Platform to 2012.2.0.0, GHC to 7.4.1 as well as updated existing ports to their latest stable versions. We also added a number of new Haskell ports, and their count in FreeBSD Ports tree is now 336.

            Open tasks:

            1. Test GHC to work with clang/LLVM.
            2. Add an option to the lang/ghc port to be able to build it with already installed GHC instead of requiring a separate GHC bootstrap tarball.
            3. Commit pending Haskell ports to the FreeBSD Ports tree.
            4. Add more ports to the Ports Collection.

            KDE/FreeBSD

            Links
            KDE/FreeBSD home page URL: http://FreeBSD.kde.org
            area51 URL: http://FreeBSD.kde.org/area51.php

            Contact: KDE FreeBSD <kde@FreeBSD.org>

            The team has made many releases and upstreamed many fixes and patches. The latest round of releases include:

            • KDE SC: 4.8.3, 4.8.4 (in ports) and 4.8.95 (in area51)
            • Qt: 4.8.1, 4.8.2
            • PyQt: 4.9.1; SIP: 4.13.2; QScintilla 2.6.1
            • KDevelop: 2.3.1; KDevPlatform: 1.3.1
            • Calligra: 2.4.2, 2.4.3
            • Amarok: 2.5.90 (in area51)
            • CMake: 2.8.8
            • Digikam (and KIPI-plugins): 2.6.0

            As a result — according to PortScout — kde@ has 393 ports, of which 91% are up-to-date.

            The team is always looking for more testers and porters so please contact us and visit our home page.

            Open tasks:

            1. Test KDE SC 4.8.95.
            2. Test KDE PIM 4.8.95.
            3. Update out-of-date ports, see PortScout for a list.

            Portbuilder

            Links
            Git Repository URL: https://github.com/DragonSA/portbuilder
            README URL: https://github.com/DragonSA/portbuilder/blob/0.1.5.2/README
            TODO URL: https://github.com/DragonSA/portbuilder/blob/0.1.5.2/TODO

            Contact: David Naylor <naylor.b.david@gmail.com>

            Since the last update there has been 2 feature releases and 4 bug-fix releases. A highlight of the changes made:

            • Support has been added for:
              • -j: controlling concurrency per stage
              • pkgng: next generation package manager
              • installing packages via repository
              • dynamic defaults (loaded from /etc/make.conf)
              • new options framework (aka OptionsNG)
            • Some of the fixes include:
              • correct assertions
              • correct build logic
              • retry when kevent receives EINTR
              • correctly detecting installed ports
              • many fixes in the build logic

            A benchmark was run timing portbuilder against a standard ports build of KDE (x11/kde4) in a clean chroot(8) environment. Portbuilder achieved a build time of 2:21:16 compared to ports build time of 4:47:21 for an decreased build time of 51% from using portbuilder.

            Redports

            Links
            URL: http://www.redports.org/

            Contact: Bernhard Froehlich <decke@FreeBSD.org>

            There was good progress in the last half a year and a lot of support from different parties to make redports a stable and fast service.

            A long known security concern within tinderbox was raised at the BSD-Day in Vienna which was addressed by beat. That improves security and isolation of the concurrent running jobs a lot and gives me peace of mind.

            We also recently got two beefy machines from the FreeBSD Foundation which increases computing power a lot. So no more backlogs and your jobs finish much quicker.

            But as usual now that we have enough power I was able to make another promise come true and integrated Ports QAT functionality into redports. Ports QAT was an automated services that did a buildtest after each commit to the official FreeBSD ports tree. If a build fails it sends out mails and logfiles to the committer. That finds bad commits quickly and allows the committer to fix it before the first user notices. The former service stopped about 2 years ago and we had no proper replacement for that task at hand. Now that this is fully integrated into redports it also gives us all the nice benefits of a common platform.

            Open tasks:

            1. Automatic build incoming patches from Ports PRs in redports and send results to GNATS database.
            2. People want an GCC testing environment on redports where all ports are build with lang/gcc47. To make that happen we need to patch the ports framework to handle that and correctly bootstrap with base GCC. This also gives us the possibility to build all our binary packages with a modern gcc and is easy to use for regular users. Contributors?

            Xorg on FreeBSD

            Links
            URL: http://wiki.freebsd.org/Xorg
            URL: http://trillian.chruetertee.ch/ports/browser/trunk

            Contact: Martin Wilke <miwi@FreeBSD.org>
            Contact: Koop Mast <kwm@FreeBSD.org>
            Contact: Niclas Zeising <zeising@FreeBSD.org>
            Contact: Eitan Adler <eadler@FreeBSD.org>

            During the beginning of this period, an update to the xorg distribution for FreeBSD was made, dubbed xorg 7.5.2. This update included a new flag, WITH_NEW_XORG, to get a more recent xorg distribution for those with modern hardware. To get KMS support for recent Intel graphics chipsets WITH_KMS must also be set. This requires a recent FreeBSD 10-CURRENT or FreeBSD 9-STABLE.

            Open tasks:

            1. Switch to use FreeGLUT instead of libGLUT, since the latter is old and has there is no upstream support or releases any more. Work on this is mostly done.
            2. Update the xorg distribution to what is in the development repository. The xorg project recently did a new release, and the development repository contains this release. It needs more testing before it can be merged, and a CFT was sent out in the beginning of June. Work on this is ongoing.
            3. Decide how to handle the new and old xorg distributions. In recent xorg, a lot of legacy driver support has been dropped, therefore we need to maintain two xorg distributions to not loose a lot of hardware drivers. Currently, this is done by setting the flag WITH_NEW_XORG in /etc/make.conf, but a more practical solution is needed. This is especially important since the flag is not very user friendly, and since there currently will be no official packages for the new distribution.

            Miscellaneous


            BSD-Day 2012

            Links
            BSD-Day 2012 web site URL: http://bsdday.eu/2012
            Video recordings of the talks at YouTube URL: http://www.youtube.com/playlist?list=PL13D5471D8ECF08C9
            Event photo album URL: https://picasaweb.google.com/116452848880746560170/BSDDay2012?authkey=Gv1sRgCN3twLrxuaeongE

            Contact: Gábor Páli <pgj@FreeBSD.org>

            For this year, we moved the time of the event earlier by six months, so it was held on May 5, 2012 and it was co-located with the Austrian Linuxweeks (Linuxwochen Österreich) in Vienna. We had many sponsors, like the freshly joined FreeBSD Foundation, iXsystems, FreeBSDMall, BSD Magazine, allBSD.de Projekt, that enabled us to continue our previously launched series of multi-project BSD developer summits all around Central Europe.

            To kick off, there was a "stammtisch" (local beer meetup) organized in the downtown of Vienna, at Kolar on the Friday evening before the event — as usual. Then it was followed by the event on Saturday that brought many interesting topics from the world of FreeBSD, OpenBSD and NetBSD: running NetBSD as an embedded system for managing VOIP applications, introduction to the Capsicum security framework, relayd(8), the load balancer and proxy solution for OpenBSD, status update of the developments around the FreeBSD ports tree, using DVCSs in clouds, firewalling with pfSense, and mfsBSD. Please consult the links in the report for the details.

            News Home | Status Home
            diff --git a/website/content/en/status/report-2012-07-2012-09.html b/website/content/en/status/report-2012-07-2012-09.html index 9e763df51d..a60acc6639 100644 --- a/website/content/en/status/report-2012-07-2012-09.html +++ b/website/content/en/status/report-2012-07-2012-09.html @@ -1,695 +1,695 @@ FreeBSD Quarterly Status Report
            Skip site navigation (1) Skip section navigation (2)

            Introduction

            This report covers FreeBSD-related projects between July and September 2012. This is the third of the four reports planned for 2012.

            Highlights from this quarter include successful participation in Google Summer of Code, major work in areas of the source and ports trees, and a Developer Summit attended by over 30 developers.

            Thanks to all the reporters for the excellent work! This report contains 12 entries and we hope you enjoy reading it.

            Projects

            FreeBSD Team Reports

            Kernel

            Documentation

            Ports

            Miscellaneous

            FreeBSD in Google Summer of Code


              Projects


              FreeBSD on Altera FPGAs

              Links
              CTSRD Project URL: http://www.cl.cam.ac.uk/research/security/ctsrd/
              CHERI URL: http://www.cl.cam.ac.uk/research/security/ctsrd/cheri.html

              Contact: Brooks Davis <brooks@FreeBSD.org>
              Contact: Robert Watson <rwatson@FreeBSD.org>
              Contact: Bjoern Zeeb <bz@FreeBSD.org>

              In the course of developing the CHERI processor as part of the CTSRD project SRI International's Computer Science Laboratory and the University of Cambridge Computer Laboratory have developed support for a number of general purpose IP cores for Altera FPGAs including the Altera Triple Speed Ethernet (ATSE) MAC core, the Altera University Program SD Card core, and the Altera JTAG UART. We have also added support for general access to memory mapped devices on the Avalon bus via the avgen bus. We have implemented both nexus and flattened device tree (FDT) attachments for these devices.

              In addition to these softcore we have developed support for the Terasic multi-touch LCD and are working to provide support for the Terasic HDMI Transmitter Daughter Card. Both of these work with common development and/or reference boards for Altera FPGAs. They do require additional IP cores which we plan to release to the open source community in the near future.

              With exception of the ATSE and HDMI drivers we have merged all of these changes to FreeBSD-CURRENT. We anticipate that these drivers will be useful for users who with to run FreeBSD on either hard or soft core CPUs on Altera FPGAs.

              This work has been sponsored by DARPA, AFRL, and Google.

              Native iSCSI Target

              Links

              Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

              During the July-September time period, the Native iSCSI Target project was officially started under sponsorship from the FreeBSD Foundation. Before the end of September I've written ctld(8), the userspace part of the target, responsible for handling configuration, accepting incoming connections, performing authentication and iSCSI parameter negotiation, and handing off connections to the kernel. For the time being, I've reused some parts of protocol-handling code from the istgt project; since ctld(8) only handles the Login phase, the code can be rewritten in a much simpler and shorter way in the future.

              Parallel rc.d execution

              Links
              URL: https://github.com/buganini/rcexecr
              URL: https://github.com/kil/rcorder

              Contact: Kuan-Chung Chiu <buganini@gmail.com>
              Contact: Kilian <kklimek@uos.de>

              There are two implementations to make rc.d execution parallel. Compared to Kil's rcorder, rcexecr brings more concurrence and provides more flexibility than older "early_late_divider" mechanism but require more invasive /etc patch. Both implementations have switch to toggle parallel execution. Further modification/integration needs more discussion.

              Open tasks:

              1. Refine /etc/rc.d/* to eliminate unnecessary waiting.

              FreeBSD Team Reports


              FreeBSD Bugbusting Team

              Links
              URL: http://www.FreeBSD.org/support.html#gnats
              URL: https://wiki.freebsd.org/BugBusting

              Contact: Eitan Adler <eadler@FreeBSD.org>
              Contact: Gavin Atkinson <gavin@FreeBSD.org>
              Contact: Oleksandr Tymoshenko <gonzo@FreeBSD.org>

              In August, Eitan Adler (eadler@) and Oleksandr Tymoshenko (gonzo@) joined the Bugmeister team. At the same time, Remko Lodder and Volker Werth stepped down. We extend our thanks to Volker and Remko for their work in the past, and welcome Oleksandr and Eitan. Eitan and Oleksandr have been working hard on migrating from GNATS, and have made significant progress on evaluating new software, and creating scripts to export data from GNATS.

              The bugbusting team continue work on trying to make the contents of the GNATS PR database cleaner, more accessible and easier for committers to find and resolve PRs, by tagging PRs to indicate the areas involved, and by ensuring that there is sufficient info within each PR to resolve each issue.

              As always, anybody interested in helping out with the PR queue is welcome to join us in #freebsd-bugbusters on EFnet. We are always looking for additional help, whether your interests lie in triaging incoming PRs, generating patches to resolve existing problems, or simply helping with the database housekeeping (identifying duplicate PRs, ones that have already been resolved, etc). This is a great way of getting more involved with FreeBSD!

              Open tasks:

              1. Further research into tools suitable to replace GNATS.
              2. Get more users involved with triaging PRs as they come in.
              3. Assist committers with closing PRs.

              FreeBSD Foundation

              Links
              Semi-annual newsletter URL: http://www.freebsdfoundation.org/press/2012Jul-newsletter.shtml

              Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

              The Foundation hosted and sponsored the Cambridge FreeBSD developer summit in August 2012.

              We were represented at the following conferences: OSCON July 2012, Texas LinuxFest, and Ohio LinuxFest.

              We negotiated/supervised Foundation funded projects: Distributed Security Audit Logging, Capsicum Component Framework, Native iSCSI Target Scoping, and Growing UFS Filesystems Online.

              We negotiated, supervised, and funded hardware needs for FreeBSD co-location centers.

              We welcomed Kirk McKusick to our board of directors. He took over the responsibility of managing our investments.

              We visited companies to discuss their FreeBSD use and to help facilitate collaboration with the Project.

              We managed FreeBSD vendor community mailing list and meetings.

              We created a high quality FreeBSD 9 brochure to help promote FreeBSD.

              Published our semi-annual newsletter that highlighted Foundation funded projects, travel grants for developers, conferences sponsored and other ways the Foundation supported the FreeBSD Project.

              We hired a technical writer to help with FreeBSD marketing/promotional material.

              We began work on redesigning our website.

              The FreeBSD Core Team

              Contact: Core Team <core@FreeBSD.org>

              Along with the change in the Core Team membership, several related roles changed hands. Gabor Pali assumed the role of core secretary from Gavin Atkinson, and David Chisnall replaced Robert Watson as liaison to the FreeBSD Foundation. The Core Team felt there was no longer a need for a formal security team liaison, so that role was retired.

              In the third quarter, the Core Team granted access for 2 new committers and took 2 commit bits into safekeeping.

              -

              The Core Team worked with the Port Management Team and Cluster +

              The Core Team worked with the Ports Management Team and Cluster Administrators to set a date to stop providing CVS exports for the ports repository, which is February 28, 2013. In the meantime, the CVS export for 9.1-RELEASE was restored.


              Kernel


              FreeBSD on ARMv6/ARMv7

              Links

              Contact: freebsd-arm mailing list <freebsd-arm@FreeBSD.org>

              Support for ARMv6 and ARMv7 architecture has been merged from project branch to HEAD. This code covers the following parts:

              • General ARMv6/ARMv7 kernel bits (pmap, cache, assembler routines, etc...)
              • ARM Generic Interrupt Controller driver
              • Improved thread-local storage for cpus >=ARMv6
              • Driver for SMSC LAN95XX and LAN8710A ethernet controllers
              • Marvell MV78x60 support (multiuser, ARMADA XP kernel config)
              • TI OMAP4 and AM335x support (multiuser, no GPU or graphics support, kernel configs for Pandaboard and Beaglebone)
              • LPC32x0 support (multiuser, frame buffer works with SSD1289 LCD controller. Embedded Artists EA3250 kernel config)

              This work was a result of a joint effort by many people, including but not limited to: Grzegorz Bernacki (gber@), Aleksander Dutkowski, Ben R. Gray (bgray@), Olivier Houchard (cognet@), Rafal Jaworowski (raj@) and Semihalf team, Tim Kientzle (kientzle@), Jakub Wojciech Klama (jceel@), Ian Lepore (ian@), Warner Losh (imp@), Damjan Marion (dmarion@), Lukasz Plachno, Stanislav Sedov (stas@), Mark Tinguely and Andrew Turner (andrew@). Thanks to all, who contributed by submitting code, testing and giving valuable advice.

              Open tasks:

              1. More hardware bring-ups and more drivers
              2. Finish SMP support
              3. VFP/NEON support

              Documentation


              The FreeBSD Japanese Documentation Project

              Links
              Japanese FreeBSD Web Page URL: http://www.FreeBSD.org/ja/
              The FreeBSD Japanese Documentation Project Web Page URL: http://www.jp.FreeBSD.org/doc-jp/

              Contact: Hiroki Sato <hrs@FreeBSD.org>
              Contact: Ryusuke Suzuki <ryusuke@FreeBSD.org>

              Web page (htdocs): Newsflash and some other updates in the English version were translated to keep them up-to-date. Especially "security incident on FreeBSD infrastructure" was translated and published in a timely manner.

              FreeBSD Handbook: Big update in the "advanced-networking". With this update, merging translation results from the handbook in the local repository of Japanese documentation project into the main repository was completed. This chapter is still outdated and needs more work. The other sections have also constantly been updated. Especially, new subsection "Using pkgng for Binary Package Management" was added to "ports" section and "Using subversion" subsection was added to "mirrors" section.

              Article: Some progress was made in "Writing FreeBSD Problem Reports" and "Writing FreeBSD Problem Reports" articles.

              Open tasks:

              1. Further translation work of outdated documents in the ja_JP.eucJP subtree.

              Ports


              KDE/FreeBSD

              Links
              KDE/FreeBSD home page URL: http://FreeBSD.kde.org
              area51 URL: http://FreeBSD.kde.org/area51.php

              Contact: KDE FreeBSD <kde@FreeBSD.org>

              The KDE/FreeBSD team have continued to improve the experience of KDE software and Qt under FreeBSD. The latest round of improvements include:

              • Fixes for building Qt with libc++ and C++11
              • Fixes for Solid-related crashes
              • Fix battery detection in battery monitor plasmoid

              The team has also made many releases and upstreamed many fixes and patches. The latest round of releases include:

              • KDE SC: 4.9.1 (area51) and 4.8.4 (ports)
              • Qt: 4.8.3 (area51)
              • PyQt: 4.9.4 (area51); QScintilla 2.6.2 (area51); SIP: 4.13.3 (area51)
              • Calligra: 2.4.3, 2.5-RC2, 2.5.0. 2.5.1, 2.5.2 (area51) and 2.4.3, 2.5.0, 2.5.1 (ports)
              • Amarok: 2.6.0 (area51)
              • CMake: 2.8.9 (ports)
              • Digikam (and KIPI-plugins): 2.7.0, 2.8.0, 2.9.0 (area51) and 2.7.0, 2.9.0 (ports)
              • QtCreator: 2.6.0-beta (area51)
              • many smaller ports

              The team is always looking for more testers and porters so please contact us at kde@FreeBSD.org and visit our home page at http://FreeBSD.kde.org.

              Open tasks:

              1. Please see 2012 Q4 Status Report
              2. Updating out-of-date ports, see PortScout for a list

              Ports Collection

              Links
              URL: http://www.FreeBSD.org/ports/
              URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
              URL: http://portsmon.freebsd.org/index.html
              URL: http://www.freebsd.org/portmgr/index.html
              URL: http://blogs.freebsdish.org/portmgr/
              URL: http://www.twitter.com/freebsd_portmgr/
              URL: http://www.facebook.com/portmgr

              Contact: Thomas Abthorpe <portmgr-secretary@FreeBSD.org>
              Contact: Port Management Team <portmgr@FreeBSD.org>

              The ports tree approaches 24,000 ports, while the PR count still is above 1000.

              In Q3 we added 2 new committers and took in two commits bit for safe keeping.

              The Ports Management team had performed multiple -exp runs, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates.

              Beat Gaetzi took over the role of sending out fail mails, a role that Pav Lucistnik had previously held. Beat also undertook the task of converting the Ports tree from CVS to Subversion.

              Florent Thoumie stepped down from his role on portmgr, he was instrumental in maintaining the legacy pkg_* code.

              Open tasks:

              1. Most ports PRs are assigned, we now need to focus on testing, committing and closing.

              Miscellaneous


              FreeBSD Developer Summit, Cambridge, UK

              Links
              Developer Summit Home Page URL: https://wiki.freebsd.org/201208DevSummit

              Contact: Robert Watson <rwatson@FreeBSD.org>

              In the end of August, there was an "off-season" Developer Summit held in Cambridge, UK at the University of Cambridge Computer Laboratory. This was a three-day event, with a documentation summit scheduled for the day before. The three days of the main event were split into three sessions, with two tracks in each. Some of them even involved ARM developers from the neighborhoods which proven to be productive, and led to further engagement between the FreeBSD community and ARM.

              The schedule was finalized on the first day, spawning a plethora of topics to discuss, followed by splitting into groups. A short summary from each of the groups was presented in the final session and then published at the event's home page on the FreeBSD wiki. This summit contributed greatly to arriving to a tentative plan for throwing the switch to make clang the default compiler on HEAD. This was further discussed on the mailing list, and has now happened, bringing us one big step closer to a GPL-free FreeBSD 10. As part of the program, an afternoon of short talks from researchers in the Cambridge Computer Laboratory involved either operating systems work in general or FreeBSD in particular. Robert Watson showed off a tablet running FreeBSD on a MIPS-compatible soft-core processor running on an Altera FPGA.

              In association with the event, a dinner was hosted by St. John's college and co-sponsored by Google and the FreeBSD Foundation. The day after the conference, a trip was organized to Bletchley Park, which was celebrating Turing's centenary in 2012.


              FreeBSD in Google Summer of Code


              Google Summer of Code 2012

              Links
              FreeBSD Summer of Code page URL: http://www.freebsd.org/projects/summerofcode.html
              Summer of Code 2012 projects URL: https://wiki.freebsd.org/SummerOfCode2012

              Contact: FreeBSD Summer of Code Administrators <soc-admins@FreeBSD.org>

              Over the Summer of 2012, FreeBSD were once again granted a place to participate in the Google Summer of Code program. We received a total of 32 project proposals, and were ultimately given 15 slots for university students to work on open source projects mentored by existing FreeBSD developers.

              We were able to accept a wide spread of proposals, covering both the base system and the ports infrastructure. We had students working on file systems, file integrity checking, and parallelization in the ports collection. Students worked on kernel infrastructure, including one project to support CPU resource limits on users, processes and jails, and one student improving the BSD callout(9) and timer facilities. Two students worked on the ARM platform, widely used in embedded systems and smart phones; one student worked on a significant cleanup and improvements to the Flattened Device Tree implementation code, while the other ported FreeBSD to the OMAP3-based BeagleBoard-xM device. One student worked on improving IPv6 support in userland tools, whilst another worked on BIOS emulation for the BHyVE BSD-licensed hypervisor, new in FreeBSD 10. Other students worked on EFI boot support, userland lock profiling and an automated kernel crash reporting system.

              Overall, a significant proportion of the code produced has or will be integrated into FreeBSD in one form or another. All of the work is available in our Summer Of Code Subversion repository, and some of the work has already been merged back into the main repositories.

              FreeBSD is once again grateful to Google for being selected to participate in Summer of Code 2012.

              News Home | Status Home

              diff --git a/website/content/en/status/report-2013-07-2013-09.html b/website/content/en/status/report-2013-07-2013-09.html index b939432207..3efedb55d2 100644 --- a/website/content/en/status/report-2013-07-2013-09.html +++ b/website/content/en/status/report-2013-07-2013-09.html @@ -1,1350 +1,1350 @@ FreeBSD Quarterly Status Report
              Skip site navigation (1) Skip section navigation (2)

              Introduction

              This report covers FreeBSD-related projects between July and September 2013. This is the third of four reports planned for 2013.

              We have had another very active three months in the FreeBSD world, including two Developer Summits (BSDCam and EuroBSDcon) that will be covered in separate status reports. FreeBSD continues to push hard on security, with improvements to both the performance and reliability of the random number generation, and more compartmentalisation in programs in the base system. For developers, there is work on a new modern debugger. There is also a significant amount of of modernization in the support for Objective-C and Ada via ports, making FreeBSD a first-rate platform for developing in either language, in addition to the existing C++11 and C11 support already present in the base system. Server users will be pleased to see improvements in the iSCSI stack and scalability allowing over a million I/O operations per second on commodity hardware, while desktop users will see improvements in X support for new GPUs and for possible X replacements.

              Thanks to all the reporters for the excellent work! This report contains 30 entries and we hope you enjoy reading it.

              The deadline for submissions covering between October and - December 2013 is January 14th, 2014.

              FreeBSD Team Reports

              Projects

              Kernel

              Architectures

              Userland Programs

              Ports

              Documentation

              Google Summer of Code

              Miscellaneous


                FreeBSD Team Reports


                FreeBSD Core Team

                + December 2013 is January 14th, 2014.

                FreeBSD Team Reports

                Projects

                Kernel

                Architectures

                Userland Programs

                Ports

                Documentation

                Google Summer of Code

                Miscellaneous


                  FreeBSD Team Reports


                  FreeBSD Core Team

                  Contact: FreeBSD Core Team <core@FreeBSD.org>

                  In the third quarter of 2013, the Core Team focused on officially launching pkg.freebsd.org, the Project's official pkg(8) repository, in cooperation with the - Port Management Team, the Security Team, and the Cluster + Ports Management Team, the Security Team, and the Cluster Administration Team. At the same time, there are plans to gradually deprecate the use of the old pkg_add(1), allowing pkg(8) to be the default binary package management solution for FreeBSD, arriving with 10.0-RELEASE. Thomas Abthorpe has been appointed to the role of liaison between the Core Team and the Ports Management Team, in order to make the collaboration more effective.

                  David Chisnall has joined the group that publishes the Quarterly Status reports and compiled a special status report on the results of the BSDCan 2013 Developer Summit. David also took the lead role on the organization of an off-season developer summit in Cambridge, UK, which was finally held at the end of August. For the items discussed in Cambridge, preparation of a detailed report is still in progress.

                  There were src commit bits issued for 5 new developers and most of the src commits being idle more than 12 months have been taken into safekeeping as result of a major cleanup to the repository access file in July, performed by Gavin Atkinson.

                  -

                  FreeBSD Port Management Team

                  Links
                  +

                  FreeBSD Ports Management Team

                  Links
                  URL: http://www.FreeBSD.org/ports/
                  URL: http://www.freebsd.org/doc/en/articles/contributing-ports/
                  URL: http://portsmon.freebsd.org/index.html
                  URL: http://www.freebsd.org/portmgr/index.html
                  URL: http://blogs.freebsdish.org/portmgr/
                  URL: http://www.twitter.com/freebsd_portmgr/
                  URL: http://www.facebook.com/portmgr
                  URL: http://lists.freebsd.org/mailman/listinfo/freebsd-pkg-fallout

                  - Contact: FreeBSD Port Management Team <portmgr@FreeBSD.org> + Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

                  The ports tree contains approximately 24,400 ports, while the PR count exceeds 1,900. In the third quarter, we added four new committers and took in six commit bits for safekeeping.

                  A significant amount of effort has gone into tweaking and manipulating the infrastructure to modernize and update it, in preperation for pkg(8) replacing the old pkg_add(1) infrastructure, as well as preparing for FreeBSD 10.0 with Clang as default compiler, libc++ as the default C++ standard library, and iconv(1) integrated into libc.

                  Automated procedures for quality assurance have been implemented, notably pkg-fallout. All porters are encouraged to subscribe to the associated mailing list (see links), and do their part to fix ports for pkg(8) and Clang readiness.

                  Many iterations of tests were run to ensure that as many packages as possible would be available for the 9.2 release.

                  Open tasks:

                  1. Most ports PRs are assigned, we now need to focus on testing, committing, and closing.

                  FreeBSD Postmaster Team

                  Links
                  URL: http://lists.freebsd.org/mailman/listinfo/freebsd-fortran
                  URL: http://lists.freebsd.org/mailman/listinfo/freebsd-pkg-fallout
                  URL: http://lists.freebsd.org/mailman/listinfo/freebsd-users-jp

                  Contact: FreeBSD Postmaster Team <postmaster@FreeBSD.org>

                  In the third quarter of 2013, the FreeBSD Postmaster Team has implemented the following items that may be interest of the general public:

                  • Created the freebsd-fortran list, requested by Anton Shterenlikht.
                  • Created the freebsd-pkg-fallout list, requested by Baptiste Daroussin.
                  • Created the freebsd-users-jp list, requested by Hiroki Sato
                  • Retired the freebsd-mozilla list, requested by Florian Smeets.
                  • Worked with the FreeBSD Cluster Administrators to enable TLS support on incoming and outgoing mail servers.
                  • Started discussions and exploration of current and possible future mail and spam filtering.
                  • Started the process for retiring the aic7xxx mailing list. Completion of this is scheduled for 12 October 2013.

                  FreeBSD Release Engineering Team

                  Links
                  FreeBSD 9.2-RELEASE schedule URL: http://www.FreeBSD.org/releases/9.2R/schedule.html
                  FreeBSD 10.0-RELEASE schedule URL: http://www.FreeBSD.org/releases/10.0R/schedule.html
                  FreeBSD Virtual Machine Images URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/
                  FreeBSD Development Snapshots URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/ISO-IMAGES/

                  Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

                  The FreeBSD Release Engineering Team has completed the 9.2-RELEASE process. The release cycle changed with a last-minute addition of 9.2-RC4. The 9.2-RELEASE was announced September 30, four weeks behind the original schedule.

                  The FreeBSD 10.0-RELEASE cycle has started, and testing is strongly encouraged. For testing purposes, both installation images and virtual machine images exist on the FreeBSD Project FTP servers.

                  Open tasks:

                  1. Test 10.0-CURRENT and report problems.

                  Projects


                  Static Code Analysis

                  Links
                  Coverity Scan URL: http://scan.coverity.com/
                  Clang Static Analyzer Scan for FreeBSD URL: http://scan.freebsd.your.org/
                  Clang Static Analyzer Home Page URL: http://clang-analyzer.llvm.org/

                  Contact: Ulrich Spoerlein <uqs@FreeBSD.org>

                  With our own (old and unstable) instance of Coverity Prevent gone, we have now fully transitioned to the Scan project run by Coverity (see links), which Open Source projects can use to learn about possible defects in their source code.

                  We also continue to run our code base through the Static Analyzer that is shipped with Clang/LLVM. It cannot track the state of the code over time, but has the benefit that everyone can use it without any special setup. See the home page at the links section for more information on the Clang Static Analyzer project in general, and head over to the FreeBSD Clang Static Analyzer Scan page (see links) to see those possible defects (no signup required).

                  We are looking for a co-admin for both of these projects to increase the bus-factor and the chance of survival for these services. Fame and fortune await!

                  Open tasks:

                  1. Maybe turn on email reports for new defects to the internal list of FreeBSD developers.
                  2. Find co-admin.
                  3. Fix the defects reported by Coverity and Clang.

                  Kernel


                  AES-NI Improvements for GELI

                  Links
                  URL: http://svnweb.freebsd.org/changeset/base/255187

                  Contact: John-Mark Gurney <jmg@FreeBSD.org>

                  An enhancement to the AES-NI implementation for OpenCrypto, the kernel's cryptography framework, has been committed that significantly improves AES-XTS and AES-CBC decryption performance. This gives geli(8) around a three times performance boost on gnop(8) using AES-XTS compared to the old code.

                  These improvements are available to users of the OpenCrypto framework and crypto(4).

                  Atomic "close-on-exec"

                  Links
                  URL: https://wiki.freebsd.org/AtomicCloseOnExec

                  Contact: Jilles Tjoelker <jilles@FreeBSD.org>

                  If threads or signal handlers call fork() and exec(), file descriptors may be passed undesirably to child processes, which may lead to hangs (if a pipe is not closed), exceeding the file descriptor limit, and security problems (if the child process has lower privilege). One solution is various new APIs that set the "close-on-exec" flag atomically with allocating a file descriptor. Some existing software will use the new features if present or will even refuse to compile without them.

                  With mkostemp(), dup3(), and a change to modes of fopen() and freopen(), everything proposed in Austin Group issue #411 has now been implemented. For all POSIX-specified functions that allocate file descriptors, it is possible to request that the new descriptor be set close-on-exec atomically.

                  Additionally, many file descriptors used internally by libc and libutil now have the close-on-exec bit set.

                  Continuation of the Newcons Project

                  Links
                  Newcons project branch URL: http://svn.freebsd.org/base/user/ed/newcons/

                  Contact: Aleksandr Rybalko <ray@FreeBSD.org>

                  The Newcons project is aimed to replace the old syscons(4)-based virtual terminals. The main objectives are: support Unicode characters, and move away from the dependency on fixed VGA and VESA graphics modes and built-in BIOS services.

                  This project was originally started by Ed Schouten, and it already featured the following features (among many others) in 2013:

                  • Unicode fonts with Latin, Cyrillic and some more simple character sets.
                  • Unicode output support.
                  • Graphics mode support.
                  • Text mode support.
                  • sysmouse(4) support, without copy/paste.

                  And these have been extended by the following items recently:

                  • History, that is, the ability to scroll through the terminal history. The old, separate history buffer has been removed.
                  • The history is implemented by a circular buffer which has no risk of overflow, and scrolling appears "unlimited".
                  • VT_PROCESS mode, a way to hold the terminal and prevent terminal switching. For example, X.Org uses this feature to prevent the user from switching to a non-X terminal.
                  • drm2/fb_helper, the KMS driver. This binds Newcons to framebuffers created the DRM-enabled video drivers in the kernel (such as i915kms and radeonkms).
                  • Dynamic attachment of VT drivers, vt_allocate() to allow attaching console video drivers at a later point where framebuffer owner can manage the initialization. This is for KMS and devices without early graphics support.

                  Supported startup modes for KMS:

                  • Start without VT graphics drivers, then load KMS.
                  • Start with VGA, then load KMS.
                  • Preload KMS, then the KMS driver will be attached to the output.
                  • Preload KMS, start with VGA, then KMS driver will replace the VGA output.

                  This project is being sponsored by The FreeBSD Foundation. Many thanks to Ed Schouten, who started the Newcons project and did most of the work.

                  Open tasks:

                  1. Implement a Generic Framebuffer interface, a simple interface to offer direct access to the framebuffer from the userland (via /dev/fb*) and automatic management of virtual terminals by Newcons.
                  2. Mouse support, copy/paste using sysmouse(4).
                  3. Improve locking.
                  4. Bug fixes.
                  5. Integrate into FreeBSD head.
                  6. Integrate into FreeBSD 10.0.
                  7. Implement mapping non-ASCII characters to Unicode on keyboard input.
                  8. Adapt existing screen savers.
                  9. Last but not least, testing is welcome!

                  GEOM Direct Dispatch and Fine-Grained CAM Locking

                  Links
                  Project SVN branch URL: http://svnweb.freebsd.org/base/projects/camlock/
                  Project patches URL: http://people.freebsd.org/~mav/camlock_patches/

                  Contact: Alexander Motin <mav@FreeBSD.org>

                  Last year's high-performance storage vendor summit reported a performance bottleneck in the FreeBSD block storage subsystem, limiting peak performance to around 300-500K IOPS. While that is still more than enough for average systems, detailed investigation has shown a number of places that require radical improvement. The unmapped I/O support implemented early this year has already improved I/O performance by about 30% and moved more focus toward GEOM and CAM subsystems scalability. Fixing these issues was the goal of this project.

                  The existing GEOM design assumed most I/O handling was to be done by only two kernel threads (g_up() and g_down()). That simplified locking in some cases, but limited potential SMP scalability and created additional scheduler overhead. This project introduces the concept of direct I/O dispatch into GEOM for cases where it is known to be safe and efficient. That implies marking some GEOM consumers and providers with one or two new flags, declaring situations when a direct function call can be used instead of normal request queuing. That permits avoiding any context switches inside GEOM for the most widely used topologies, simultaneously processing multiple I/Os from multiple calling threads.

                  Having GEOM pass through multiple concurrent calls down to the underlying layers exposed major lock congestion in CAM. In the existing CAM design, all devices connected to the same ATA/SCSI controller share a single lock, which can be quite busy due to multiple controller hardware accesses and/or code logic. Experiments have shown that applying only the above GEOM direct dispatch changes burns up to 60% of system CPU time or even more in attempts to obtain these locks by multiple callers, killing any benefits of GEOM direct dispatch.

                  To overcome this scaling limitation, a new fine-grained CAM locking design was implemented. It implies splitting the big per-SIM locks into several smaller ones: per-LUN locks, per-bus locks, queue locks, etc. After these changes, the remaining per-SIM lock protects only the controller driver internals, reducing lock congestion down to an acceptable level and keeping compatibility with existing drivers.

                  Together, the GEOM and CAM changes double the peak I/O rate, reaching up to 1,000,000 IOPS on contemporary hardware.

                  The changes were tested by a number of people and will be committed into FreeBSD head and merged to stable/10 after the end of the FreeBSD 10.0 release cycle.

                  The project is sponsored by iXsystems, Inc.

                  Open tasks:

                  1. More reviews, more stability and performance tests.

                  Native iSCSI Stack

                  Links
                  URL: https://wiki.freebsd.org/Native%20iSCSI%20target

                  Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                  Due to the quickly approaching time of 10.0-RELEASE, the priorities for the native iSCSI stack shifted somewhat, from performance optimizations to making sure the new stack is reliable, feature-complete, and is able to interoperate correctly with various implementations. Plenty of time was invested into testing and debugging, mostly on the initiator side, to make sure it works correctly with other targets, such as Solaris COMSTAR, and behaves properly in edge conditions like connection problems. Nevertheless, some fundamental optimizations, such as Immediate Data support, were implemented. The documentation has improved, and there will be a new section added to the FreeBSD Handbook describing the use of the new stack.

                  The new stack was committed to head and will ship as part of 10.0-RELEASE. There is ongoing work on fixing issues reported by early adopters.

                  This project is being sponsored by The FreeBSD Foundation.

                  Open tasks:

                  1. Fix newly reported issues.
                  2. Improve performance.

                  Reworking random(4)

                  Contact: Mark Murray <markm@freebsd.org>
                  Contact: Arthur Mesh <arthurmesh@gmail.com>
                  Contact: Dag-Erling Smørgrav <des@freebsd.org>

                  Random numbers require a lot more thought and preparation than would naively appear to be the case. For simulations, number sequences that are repeatable but sufficiently disordered are often needed to achieve required experimental duplication ability, and many programmers are familiar with these. For cryptography, it is essential that an attacker not be able to predict or guess the output sequence, thus giving a source of security-critical secret material for uses such as passwords or "key material".

                  FreeBSD's random number generator, available as the pseudo-file /dev/random produces unpredictable numbers intended for cryptographic use, and is thus a Cryptograpically-Secured Pseudo-Random Number Generator, or CSPRNG. The security is given by careful design of the output generator (based on a block cipher) and input entropy accumulation queues. The latter uses hashes to accumulate stochastic information harvested from various places in the kernel to provide highly unpredictable input to the generator. The algorithm for doing this, Yarrow, by Schneier et al, may be found by web search.

                  FreeBSD's CSPRNG also allowed for certain stochastic sources, deemed to be "high-quality", to directly supply the random(4) device without going through Yarrow. With recent revelations over possible government surveillance and involvement in the selection of these "high-quality" sources, it is felt that they can no longer be trusted, and must therefore also be processed though Yarrow.

                  The matter was discussed at various levels of formality at the Cambridge Developer Summit in August, and at EuroBSDcon 2013 in September.

                  This work is now done, and the random(4) CSPRNG is now brought to a more paranoid, modern standard of distrust with regard to its entropy sources. Infrastructure work was also done to facilitate certain entropy-source choices for the convenience of the system administrators.

                  Future work is now going ahead with the implementation of the Fortuna algorithm by Ferguson and Schneier as an upgrade or alternative to Yarrow. Initially a choice will be presented, and decisions on the future of the CSPRNG processing algorithms in use will be made in the future as needs arise.

                  Open tasks:

                  1. Implement FIPS 800-90b support.
                  2. A full, in-depth review of entropy.

                  SDIO Driver

                  Links
                  SDIO Project Page URL: https://wiki.freebsd.org/SDIO
                  Source Code URL: https://github.com/kibab/freebsd/tree/kibab-dplug

                  Contact: Ilya Bakulin <ilya@bakulin.de>

                  SDIO is an interface designed as an extension of the existing SD card standard, to allow connecting different peripherals to the host with the standard SD controller. Peripherals currently sold at the general market include WLAN/BT modules, cameras, fingerprint readers, and barcode scanners. The driver is implemented as an extension to the existing MMC bus, adding a lot of new SDIO-specific bus methods. A prototype of the driver for the Marvell SDIO WLAN/BT (Avastar 88W8787) module is also being developed, using the existing Linux driver as the reference.

                  SDIO card detection and initialization already work, most needed bus methods are implemented and tested. There is an ongoing work to design a good locking model for the stack. The WiFi driver is able to load firmware onto the card and initialize it.

                  Open tasks:

                  1. SDIO stack: Design a locking model, define how the interrupts should be processed (on SDIO controller level, MMC stack level and by child drivers).
                  2. Marvell SDIO WiFi: connect to the FreeBSD network stack, write the code to implement required functions (such as sending and receiving data, network scanning, and so on).
                  3. Implement detach path. It cannot be tested on the DreamPlug used for development, because the DreamPlug does not have an external SDIO-capable slot.

                  VirtIO Network Multiqueue

                  Links
                  URL: http://svnweb.freebsd.org/changeset/base/255112

                  Contact: Bryan Venteicher <bryanv@freebsd.org>

                  The VirtIO network driver, vtnet(4), is used by FreeBSD systems running on hypervisors including bhyve(4) and Linux's KVM. It recently gained support for multiple queues, along with a significant cleanup and support for a few additional features.

                  VMware VMXNET3 Driver

                  Links
                  URL: http://lists.freebsd.org/pipermail/freebsd-current/2013-August/043494.html
                  URL: http://svnweb.freebsd.org/base/head/sys/dev/vmware/vmxnet3/

                  Contact: Bryan Venteicher <bryanv@freebsd.org>

                  A port of the OpenBSD vmx(4) ethernet driver for VMware virtual machines has been committed. The driver can be used in place of the VMware Tools vmxnet3 driver, which currently does not support 10.0-RELEASE (or anything past 9.0-RELEASE).

                  Open tasks:

                  1. Performance improvements, multiqueue support.
                  2. Merge to stable/9.

                  Architectures


                  FreeBSD on Cubieboard2

                  Links
                  URL: http://svnweb.freebsd.org/changeset/base/254056

                  Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>

                  Initial support of Allwinner A20 SoC is committed to head. The A20 SoC on Cubieboard2 is pin-to-pin compatible with the A10 in Cubieboard1 and FreeBSD supports the following peripherals:

                  • USB EHCI
                  • GPIO

                  Open tasks:

                  1. Get the EMAC Ethernet driver working. Need more help from network driver experts.
                  2. Add more drivers.

                  FreeBSD/EC2

                  Links
                  FreeBSD/EC2 Status Page URL: http://www.daemonology.net/freebsd-on-ec2/
                  AWS Marketplace Listing URL: https://aws.amazon.com/marketplace/pp/B00AA25MLK/

                  Contact: Colin Percival <cperciva@freebsd.org>

                  FreeBSD images are available for use in EC2 for 8.3-RELEASE, 8.4-RELEASE, 9.0-RELEASE, 9.1-RELEASE, and 9.2-RELEASE. In 9.2-RELEASE, FreeBSD runs in EC2 using an unpatched source tree, but it needs the XENHVM kernel configuration.

                  Starting from FreeBSD 10.0-ALPHA3, the GENERIC kernel configuration now contains all the XENHVM bits needed to allow FreeBSD to run in EC2 natively. Consequently, FreeBSD 10.0 will be the first release for which FreeBSD/EC2 is purely "bits off the ISO". This also means that starting with 10.0 it will be possible to use freebsd-update(8) for all base system updates — in earlier releases it was necessary to recompile the XENHVM kernel manually.

                  Due to FreeBSD's use of HVM virtualization, running on "old" EC2 instance types (m1, m2, c1, t1) requires that FreeBSD pretends to be Windows, which unfortunately results in paying the higher "windows" EC2 instance prices. On "new" EC2 instances (cc1, cc2, cg1, cr1, hi1, hs1, and m3) FreeBSD can run as a "unix" image at the lower rate.

                  Open tasks:

                  1. Test FreeBSD 10.0-ALPHAs/BETAs/RCs as they become available. Plenty of new Xen code has been committed recently and there are probably bugs to find before the release.
                  2. Keep nagging Amazon to provide more instance types which FreeBSD can run on without paying a "Windows tax".
                  3. Provide some mechanism for instance configuration via EC2 user-data. This might involve using cloud-init, or it might be a new system.

                  FreeBSD/pseries

                  Links
                  URL: http://svnweb.freebsd.org/changeset/base/255643

                  Contact: Andreas Tobler <andreast@freebsd.org>
                  Contact: Nathan Whitehorn <nwhitehorn@freebsd.org>

                  Starting with FreeBSD 10.0-ALPHA4, the projects/pseries branch has been merged into FreeBSD head. This allows FreeBSD/powerpc64 to run in an IBM POWER logical partition and on certain classes of older IBM-type PowerPC hardware.

                  Open tasks:

                  1. Test, possibly on real hardware. Most testing and development was conducted with the emulated LPAR target in QEMU. Please send any testing reports to the freebsd-ppc mailing list.

                  FreeBSD/sparc64

                  Contact: Marius Strobl <marius@FreeBSD.org>

                  There are several things going on with the FreeBSD/sparc64 port.

                  After having fixed all remaining problems and starting with 9.2-RELEASE, releases for this architecture are cross-built on the FreeBSD Project cluster. As one might already have noticed, this means that from now on, sparc64 install sets and images including those for ALPHA, BETA, and RC builds, are available alongside those for the other platforms supported by FreeBSD. Since August 2013, automatically cross-built monthly FreeBSD/sparc64 snapshots are distributed via the official project mirrors. Hopefully, this can soon be extended further with freebsd-update(8) support for sparc64.

                  The X.Org ports have been fixed to work on sparc64 when built with the WITH_NEW_XORG knob. However, it still needs to be evaluated whether the recently committed update to Mesa 9.1.6 has introduced any breakage.

                  Superpages for ARMv7

                  Links
                  URL: http://static.usenix.org/events/osdi02/tech/full_papers/navarro/navarro.pdf
                  URL: http://wiki.freebsd.org/ARMSuperpages
                  URL: http://blogs.arm.com/software-enablement/1079-transparent-superpages-for-freebsd-on-arm
                  URL: https://wiki.freebsd.org/201309DevSummit?action=AttachFile&do=view&target=semihalf-superpages_armv7.pdf
                  URL: http://svnweb.freebsd.org/changeset/base/254918

                  Contact: Zbigniew Bodek <zbb@semihalf.com>
                  Contact: Grzegorz Bernacki <gjb@semihalf.com>
                  Contact: Rafał Jaworowski <raj@semihalf.com>

                  The ARM architecture is becoming more and more prevalent, with increasing usage beyond the mobile and embedded space. Among the more interesting industry trends emerging in the recent months, there has been the concept of "ARM server". Top-tier companies like Dell and HP have already started to develop such systems.

                  Key to the success of FreeBSD in these new areas is dealing with the sophisticated features of the platform, for example adding support for superpages.

                  The objective of this project is to enable FreeBSD/arm to utilize superpages, allowing efficient use of TLB translations (by enlarging TLB coverage), leading to improved performance in many applications and scalability. This is intended to work on ARMv7-based processors, however compatibility with ARMv6 will be preserved.

                  The following steps have been made since the last status report:

                  • The pmap(9) module has been adjusted to fully utilize superpages.
                  • Found and fixed minor bugs in superpage management.
                  • Implemented the pmap_advise() routine.
                  • Performed extensive testing and benchmarking:
                    • Giga Updates Per Second (GUPS) benchmark: 34% lower memory access latency and 34% higher updates ratio.
                    • LMbench: 38% lower memory latency.
                    • Self-hosted buildworld: 20% shorter, using GCC.
                  • Final integration into FreeBSD head.

                  This project is jointly sponsored by The FreeBSD Foundation and Semihalf.

                  Open tasks:

                  1. Adjust pmap to resolve the demotion issue caused by the continuous active queue scanning in VM.
                  2. Support for 64KB page size.
                  3. Move pv_flags to page table entry descriptors.

                  Userland Programs


                  Capsicum

                  Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>

                  Capsicum is the FreeBSD sandboxing subsystem, which presents programmers with a capability module allowing fine-grained delegation of rights to less-privileged processes. Casper is a friendly daemon that provides services to sandboxed processes, allowing policy-based access to privileged services such as DNS resolution.

                  The work on Capsicum and related projects (such as Casper, libnv, etc.) is progressing nicely. An overhaul of the cap_rights_t was committed to FreeBSD head and will be included in 10.0. This allows us to have more capability rights on file descriptors than the previous limit of 64 rights, which was almost reached. This change is not backward compatible, so it was very important to get it into 10.0.

                  libnv, used for communication between Casper services and consumers, but which will hopefully be used more widely, is finalized and comes with a nice set of regression tests.

                  The number of applications sandboxed using the Capsicum framework is increasing. We have around 10 of them already in base and more that are not yet committed.

                  This project is being sponsored by the FreeBSD Foundation.

                  Open tasks:

                  1. Finish documentation of Casper and its services.
                  2. Implement regression tests for Casper services.
                  3. Finish documentation for libnv.
                  4. Start making libc more sandbox-friendly, that is, modifying functions such as strerror(3), strsignal(3), localtime(3), login_get*(), getservent(3), getprotent(3), and getrpcent(3) which currently open files on first use, which might be too late if we are already in a capability-mode sandbox.
                  5. Rethink the system.filesystem Casper service to allow for easy compartmentalization of various command-line tools that operate on multiple files.

                  LLDB Debugger Port

                  Links
                  URL: https://wiki.freebsd.org/lldb

                  Contact: Ed Maste <emaste@freebsd.org>

                  LLDB is the debugger project in the LLVM family. It supports the Mac OS X, Linux, and FreeBSD platforms.

                  A number of improvements have been made to the port since the previous status update. Unit test failures have been triaged and have defects entered in LLDB's bug tracker. In combination with the lldb buildbot this allows for the quick identification of new failures introduced by other ongoing development. Core file support has also been added.

                  An LLDB snapshot has been imported into the FreeBSD base system and is available as of SVN revision 255722. It is not yet built by default but may be enabled by adding WITH_LLDB= to src.conf(5).

                  This project is sponsored by DARPA/AFRL in collaboration with SRI International and the University of Cambridge.

                  Open tasks:

                  1. Support live debugging of multithreaded processes.
                  2. Fix amd64 watchpoints.
                  3. Add support for remote debugging (gdbserver, debugserver).
                  4. Add support for kernel debugging.
                  5. Verify i386 and arm architectures.
                  6. Implement MIPS target support.
                  7. Verify cross-debugging.
                  8. Investigate and fix test suite failures.

                  Ports


                  FreeBSD Ada Ports

                  Links
                  URL: http://www.dragonlace.net

                  Contact: John Marino <marino@FreeBSD.org>

                  A few years ago, Ada-based ports almost completely disappeared from the Ports Collection. This was not surprising, as FSF GNAT, the only open-source Ada compiler, ceased to build correctly on any BSD flavor. Previously-built bootstrap compilers would not run on modern FreeBSD, and certainly not on amd64. The first step, see the link for details, was to patch GCC in order to fix GNAT not only on FreeBSD, but DragonFly, NetBSD, and OpenBSD as well. New bootstraps for both i386 and amd64 platforms were produced during this effort. Ada compilers on FreeBSD now pass 100% of the ACATS and GCC testsuites.

                  With the introduction of the first new Ada compiler port, the GCC 4.6-based lang/gnat-aux, the GNAT Programming Studio (a multilanguage integrated development environment), XML/Ada, and GtkAda were among the first Ada ports resurrected.

                  With the latest compiler, lang/gcc-aux based on GCC 4.7, a cohesive Ada framework was created with the new USES= framework. Currently around 20 ports are part of this framework including Florist, ASIS, GPRbuild, QtAda, AdaControl, AdaBrowse, PolyOrb, and AWS (Ada Web Server).

                  The GNAT AUX compiler is also still in use to serve as a basis for the GNATDroid ports which are FreeBSD-to-Android Ada+C cross-compilers. However, these will soon be integrated into the Ada Framework.

                  At this point, it looks like FreeBSD (shared with DragonFly via DPorts) has taken the crown from Debian as the recognized best Ada development platform. The FreeBSD versions of the software are more recent and the Ports Collection has ports not available on Debian, such as LibSparkCrypto, the Matreshka library, and the Ahven unit tester.

                  Future work potentially includes converting GCC AUX to GCC 4.8 to acquire better Ada 2012 support, importing Spark 2014 into ports when it arrives and to continue to add new Ada ports to the framework.

                  FreeBSD Python Ports

                  Links
                  The FreeBSD Python Team Page URL: https://wiki.FreeBSD.org/Python
                  IRC channel URL: irc://freebsd-python@irc.freenode.net

                  Contact: FreeBSD Python Team <python@FreeBSD.org>

                  We are currently working on cleaning up the lang/python* ports to improve their compatibility with the original upstream build behaviour and to reduce the need for FreeBSD-specific build patches. A first step was made in September by reducing the flags injected into the different Python interpreter versions.

                  The first tasks have been completed to support the installation of packages for different Python ports. A new metaport structure has replaced the original Python port behaviour, and will be enhanced over the next months to enable improved installation support of packages for different Python versions at the same time.

                  The Python ports framework was enhanced with automated packaging list creation and replacement macros, which improve the compatibility with multiple Python versions and reduce the packaging list sizes.

                  PyPy was heavily enhanced over the last couple of months. Major updates to the port solved integration issues and a new pypy-devel port for snapshots and previews was added. Since the PyPy 3 release, there is a new pypy3-devel port available to provide not only compatibility for Python 2.x specific scripts, but also for those using the 3.x language specification.

                  IronPython found its way into the FreeBSD ports tree, providing an implementation of the Python language based on .NET and Mono.

                  Open tasks:

                  1. Develop a high-level and lightweight Python Ports Policy.
                  2. Chase the unification of Distribute (devel/py-distribute) and Setuptools (devel/py-setuptools*).
                  3. Add support for granular dependencies (for example >=1.0 or < 2.0).
                  4. Look at what adding pip (Python Package Index) support looks like.
                  5. More tasks can be found on the Team's wiki page (see links).

                  GNOME/FreeBSD

                  Links
                  URL: http://www.FreeBSD.org/gnome/

                  Contact: FreeBSD GNOME Team <gnome@FreeBSD.org>

                  Glib 2.36 and Gtk 3.8 were imported into the ports tree. The GNOME Team is currently working on improving the quality of GNOME 3.6. The version of multimedia/cheese shipped with GNOME 3 is now able to use devd(8) to find the camera through multimedia/webcamd. Several build improvements have been made to the www/webkit-gtk3 port, however it still is rather fragile.

                  MATE, a desktop environment forked from the now-unmaintained codebase of GNOME 2, is about ready to go in.

                  GNOME 2 will be removed at some point in the near future. How or when this will happen is not yet clear.

                  Open tasks:

                  1. Test the update. Contact the maintainers if it is suspected that a port does not work with the newer version of devel/glib20.
                  2. Update the FreeBSD GNOME website with recent changes in the ports tree, add new items in preparation for GNOME 3 and Mate, etc.
                  3. Continue working on GNOME 3.6, stability and missing features.
                  4. Import MATE into the ports tree.

                  GNUstep on FreeBSD

                  Contact: David Chisnall <theraven@FreeBSD.org>

                  GNUstep is the open source implementation of the Objective-C APIs based on the OpenStep specification that Apple brands as Cocoa. The similarities between the FreeBSD and OS X libc make FreeBSD an attractive target platform for porting OS X applications, with the addition of GNUstep.

                  The GNUstep ports in FreeBSD have now been updated to the latest releases and now build with the GNUstep Objective-C runtime and Clang 3.3, with the non-fragile ABI by default. This means that all of the modern features of Objective-C are supported, including Automatic Reference Counting (ARC) and recent syntax improvements.

                  The devel/gnustep meta-port will install all of the core GNUstep libraries, ready for development. The x11/gnustep-app meta-port will install all of the GNUstep-based applications and libraries currently in the ports tree. Many of these are old and not well-tested with later GNUstep release, so consider them experimental at present. We are currently working on updating them, including moving from some abandoned upstream locations to the GNUstep Applications Project (GAP), which has taken over maintenance of a number of older GNUstep programs.

                  X.Org on FreeBSD

                  Links
                  X11 Team roadmap (WIP) URL: https://wiki.freebsd.org/Graphics
                  Ports-related status URL: https://wiki.freebsd.org/Xorg
                  Ports-related development repository URL: http://trillian.chruetertee.ch/ports/browser/trunk
                  AMD GPU status URL: https://wiki.freebsd.org/AMD_GPU
                  DRM generic code update branch on GitHub URL: https://github.com/dumbbell/freebsd/tree/kms-drm-update-38

                  Contact: FreeBSDX11 Team <x11@FreeBSD.org>

                  Mesa 9.1 (libGL and dri) was updated in ports. This includes experimental ports for libEGL and libgles2: they are dependencies of the experimental ports for Wayland and Weston.

                  The radeonkms driver was committed to FreeBSD head in the end of August and will be part of 10.0-RELEASE. It received several fixes since the initial commit and now seems quite stable. However, one missing major feature is support for suspend/resume: the GPU almost always locks up during resume on the test computer.

                  Thanks to the update of Mesa and the update of x11-drivers/xf86-video-ati to 7.2.0 in the ports tree, every pieces are in place to allow users to use recent AMD video cards (up to HD7000, maybe some HD8000).

                  The driver will now only receive bug fixes and focus will move on the update of the DRM generic code and the i915 driver.

                  The generic DRM code, shared by the i915kms and radeonkms video drivers is quite old now. Work has started to update and sync it with that of Linux 3.8. This code is available on GitHub.

                  The expected benefits are:

                  • Fixes in the framebuffer code, which would help the future deployment of Newcons.
                  • Preliminary support for minor devices (that is, control versus render nodes).
                  • Support for setmaster and dropmaster, which allows to run multiple X sessions.

                  François Tigeot from DragonFly is also working on updates to their DRM code, and the X11 team is planning to share the effort.

                  An experimental devd(8) backend was added to the x11-servers/xorg-server port. This allows X.Org to use devd(8) to detect and configure input devices (for example, keyboards and mices) dynamically.

                  Our current wiki articles are used to describe projects and report status. However, they lack some consistency and links between them. We started to think about reorganizing them to:

                  • Improve the coordination between the ports and the kernel efforts.
                  • Make the information more accessible.

                  Nothing is visible yet on the wiki.

                  Open tasks:

                  1. Keep tracking Mesa 9.2 or later and xorg-server 1.14. Both are currently blocked, but it is good to keep track of what upstream is doing.
                  2. Test and report successes and failures for AMD GPUs.
                  3. Wayland builds now. Work is being done on Weston to see if there are any run-time issues. Weston is the reference compositor for Wayland.
                  4. Improve the devd(8) backend for x11-servers/xorg-server, so the HAL option can be removed completely.

                  Documentation


                  FreeBSD Documentation Project Primer Edit

                  Links
                  URL: http://www.freebsd.org/doc/en/books/fdp-primer/book.html

                  Contact: Warren Block <wblock@FreeBSD.org>

                  The FreeBSD Documentation Project Primer had not changed at the same rate as the documents themselves. Some sections were outdated and others were verbose and confusing, while information on new changes to the documentation were not described at all. In July, Warren gave the entire FDP Primer a fairly intense edit for simplicity and clarity. Chapters and sections were moved into a more logical order, and information was updated to be a better guide to the current state. Markup examples were added and revised. Style guidelines were also extended and updated. The Primer is now far more consistent and usable. As always, there is still room for improvement, and additions or corrections are encouraged.

                  Open tasks:

                  1. An introductory chapter on writing manual pages with mdoc(7) would be an excellent addition.

                  The entities Documentation Branch

                  Links
                  URL: http://svnweb.freebsd.org/changeset/doc/42226

                  Contact: René Ladan <rene@FreeBSD.org>

                  The entities project branch has been successfully merged into the main documentation branch per revision 42226 of the doc repository (see link). The purpose of this branch was to remove the duplicated definitions of authors in both authors.ent and developers.ent. The latter file has been removed after migrating its contents to the former file. While most changes are not visible to end users, the Committer's Guide was changed to accomodate for changes related to adding a new committer. Translators were also informed of the update. The largest hurdle mentioned in the last report, processing the <email> element, was solved with the help of Gábor Kövesdán.


                  Google Summer of Code


                  Download Manager Service for the Ports Collection

                  Links
                  Project wiki page URL: https://wiki.freebsd.org/SummerOfCode2013/IntellegentDownloadManager
                  More information on DMS URL: https://wiki.freebsd.org/IdeasPage/IDMS

                  Contact: Ambarisha Bhatlapenumarthi <ambarisha@freebsd.org>
                  Contact: Xin Li <delphij@freebsd.org>

                  This is a Google Summer of Code 2013 project that aims to replace the fetch(1)-based method for getting distribution files, such as source tarballs, for the third-party applications (ports) with an intelligent Download Manager Service (see links for more information).

                  All the modules highlighted in the project wiki have been completed (see links). Specifically:

                  • A service that receives and serves download requests. It samples download speeds from different mirrors and uses this information to pick the best mirror on the next request. It can migrate jobs between mirrors if it realizes that a complete download from a different mirror would be faster than proceeding with the mirror it is currently using.
                  • A status dump feature has also been added to the client (dmget) which dumps the information about active downloads, speeds from mirrors, etc.

                  Open tasks:

                  1. The implementation (especially job migration and dumping status) has not been tested thoroughly. Test the code, write more unit and regression tests.

                  Miscellaneous


                  The FreeBSD Foundation

                  Links
                  URL: http://www.FreeBSDFoundation.org/

                  Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                  The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of our funding is used to support FreeBSD development projects, conferences and developer summits, purchase equipment to grow and improve the FreeBSD infrastructure, and provide legal support for the Project.

                  We listened to our donors who asked us to have more fundraising efforts throughout the year. This quarter we had the second of three fundraising campaigns planned for 2013. We started the quarter having raised $365,291. By the end of the quarter, we raised $410,000 for the year. These early donations have made a significant impact on our fundraising efforts this year.

                  Some of the highlights from this past quarter include:

                  • Projects completed last quarter:
                    • ARM Superpages
                    • Documentation project infrastructure enhancements
                  • Projects in progress:
                    • Native iSCSI kernel stack
                    • Newcons console driver
                  • Projects that started last quarter:
                    • Capsicum Integration
                    • Network Stack Layer 2 Modernization
                  • Platinum Sponsor for EuroBSDCon, had six Foundation representatives attend the conference and the Developer Summit, sponsored 7 developers to attend the conference, and sponsored the Developer Summit.
                  • Sponsored the Cambridge Developer Summit, and sponsored 2 developers to attend this event.
                  • Attended Indianapolis LinuxFest July 27, FOSSCON in Philadelphia August 10, Ohio LinuxFest in Columbus September 14, and LinuxCon in New Orleans September 16-17, to promote FreeBSD.
                  • Met with the FreeBSD Core Team to discuss their goals and to discuss areas that we can help.
                  • Met with the Documentation Team to talk about helping them update their website as well as what other areas we can help them with.
                  • Recognized Dag-Erling Smørgrav at EuroBSDCon for his contributions to FreeBSD.
                  • Became a sponsor of vBSDCon, a new conference in Washington, DC.
                  • Hired Glen Barber as a full-time employee to do system administration work and to help with release engineering.
                  • Hired Cinthy Tanko as a part-time administrative assistant to help with day-to-day Foundation activities.
                  • Purchased hardware to be placed in our NYI colo to support the building and distribution of new style packages in advance of FreeBSD 10.
                  • Provided teleconferencing services to the Core Team to support their monthly conferences.
                  News Home | Status Home
                  diff --git a/website/content/en/status/report-2013-10-2013-12.html b/website/content/en/status/report-2013-10-2013-12.html index ab8b67c09f..cd89a0e953 100644 --- a/website/content/en/status/report-2013-10-2013-12.html +++ b/website/content/en/status/report-2013-10-2013-12.html @@ -1,1619 +1,1619 @@ FreeBSD Quarterly Status Report
                  Skip site navigation (1) Skip section navigation (2)

                  Introduction

                  This report covers FreeBSD-related projects between October and December 2013. This is the last of four reports planned for 2013.

                  The last quarter of 2013 was very active for the FreeBSD community, much like the preceding quarters. Many advances were made in getting FreeBSD to run on ARM-based System-on-Chip boards like Cubieboard, Rockchip, Snapdragon, S4, Freescale i.MX6, and Vybrid VF6xx. FreeBSD is also becoming a better platform for Xen and the Amazon Elastic Compute Cloud. There are plans for FreeBSD to become a fully supported compute host for OpenStack. The I/O stack has again received some performance boosts on multi-processor systems through work touching the CAM and GEOM subsystems, and through better adaptation of UMA caches to system memory constraints for ZFS. The FreeBSD Foundation did an excellent job in this quarter, and many of their sponsored projects like VT-d and UEFI support, iSCSI stack, Capsicum, and auditdistd are about to complete. At the same time, new projects like Automounter and Intel GPU updates have just been launched. The Newcons project has been merged into -CURRENT, which will make it possible to finally move to the latest version of X.Org in the Ports Collection. Efforts are also under way to improve testing with Jenkins and Kyua. It is an exciting time for users and developers of FreeBSD!

                  Thanks to all the reporters for the excellent work! This report contains 37 entries and we hope you enjoy reading it.

                  The deadline for submissions covering between January and - March 2014 is April 7th, 2014.

                  FreeBSD Team Reports

                  Projects

                  Kernel

                  Architectures

                  Userland Programs

                  Ports

                  Miscellaneous


                    FreeBSD Team Reports


                    FreeBSD Cluster Administration Team

                    + March 2014 is April 7th, 2014.

                    FreeBSD Team Reports

                    Projects

                    Kernel

                    Architectures

                    Userland Programs

                    Ports

                    Miscellaneous


                      FreeBSD Team Reports


                      FreeBSD Cluster Administration Team

                      Contact: FreeBSD Cluster Administration Team <admins@>

                      The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the project relies on for its distributed work and communications to be synchronised. In the last quarter of 2013, they continued general maintenance of the FreeBSD cluster across all sites.

                      In addition to general upkeep tasks, additional cluster-related items were addressed. Some of these items include:

                      • Added several machines for the Kyua testing framework.
                      • Replaced failed hardware hosting various web services.
                      • Coordinated with the FreeBSD Security Officer and Ports Management Teams to implement signed binary packages.
                      • Added the redports.org machines to the list of machines managed by the Cluster Administration Team.
                      • Began discussion with contacts at Yandex regarding the addition of a mirror site for binary packages and Subversion repositories.

                      FreeBSD Core Team

                      Contact: FreeBSD Core Team <core@FreeBSD.org>

                      The FreeBSD Core Team constitutes the project's Board of Directors, responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape.

                      In the fourth quarter of 2013, the Core Team finally reached its previous goal of launching the official repositories for pkg(8)-based binary packages. The Core Team also unified the commit bit expiration policies for all Project repositories, allowing committers to idle for 18 months before their commit bits are automatically taken into safekeeping. This was then followed by an extension to suspension of cluster accounts for the committers who lost all of their commit bits. This helps to improve the security of the Project server cluster by temporarily disabling inactive accounts. In addition to the above efforts, Thomas Abthorpe resurrected the Grim Reaper service which helps to enforce the aforementioned policy.

                      With the work of John Baldwin, Hiroki Sato, and others, many licenses in the base system source code have been revisited and cleaned up. Furthermore, the Core Team is hoping that the situation can be improved by introducing periodic automated checks of the license agreements, and by providing developers guidelines on questions of licensing. John Baldwin and David Chisnall have been guiding the work of the FreeBSD Graphics Team on moving to the newer version of X.Org and related software in the Ports Collection, in coordination with the switch to Newcons on FreeBSD 10.x.

                      It was a busy quarter for the src repository as well. The Core Team was happy to welcome Jordan K. Hubbard (jkh) back, who has recently returned to the FreeBSD business, and joined iXsystems as project manager and release engineer of FreeNAS. In addition to this, there were three commit bits offered for new developers, two committers were upgraded, one commit bit was taken for safekeeping, and one src bit was reactivated.

                      -

                      FreeBSD Port Management Team

                      Links
                      +

                      FreeBSD Ports Management Team

                      Links
                      URL: http://www.FreeBSD.org/ports/
                      URL: http://www.freebsd.org/doc/en/articles/contributing-ports/
                      URL: http://portsmon.freebsd.org/index.html
                      URL: http://www.freebsd.org/portmgr/index.html
                      URL: http://blogs.freebsdish.org/portmgr/
                      URL: http://www.twitter.com/freebsd_portmgr/
                      URL: http://www.facebook.com/portmgr
                      URL: http://plus.google.com/communities/108335846196454338383

                      - Contact: FreeBSD Port Management Team <portmgr@FreeBSD.org> + Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

                      The FreeBSD Ports collection is a package management system for the FreeBSD operating system, providing an easy and consistent way of installing software packages. The FreeBSD Ports Collection now contains approximately 24,500 ports, while the PR count exceeds 1,900.

                      -

                      The FreeBSD Port Management Team ensures that the FreeBSD ports +

                      The FreeBSD Ports Management Team ensures that the FreeBSD ports developer community provides a Ports Collection that is functional, stable, up-to-date, and full-featured. Its secondary responsibility is to coordinate among the committers and developers who work on it. As part of these efforts, we added three new committers, took in three commit bits for safe keeping, and reinstated one commit bit in the fourth quarter of 2013.

                      Ongoing effort went into testing larger changes, as many as eight a week, including sweeping changes to the tree, moderization of the infrastructure, and basic quality assurance (QA) runs. Many iterations of tests against 10.0-RELEASE were run to ensure that the maximum number of packages would be available for the release.

                      We now have pkg(8) packages for the releases 8.3, 8.4, 9.1, 9.2, 10.0 and -CURRENT on pkg.FreeBSD.org. During this same time, further enhancements were put into pkg(8), including secure package signing.

                      -

                      Commencing November 1, the Port Management Team undertook a +

                      Commencing November 1, the Ports Management Team undertook a portmgr-lurkers pilot project in which ports committers - could volunteer to assist the Port Management Team for a + could volunteer to assist the Ports Management Team for a four-month duration. The first two candiates are Mathieu Arnold (mat) and Antoine Brodin (antoine).

                      Ongoing maintenance goes into redports.org, including QAT runs, ports and security updates.

                      Open tasks:

                      1. As previously noted, many PRs continue to languish; we would like to see some committers dedicate themselves to closing as many as possible!

                      FreeBSD Postmaster Team

                      Links
                      URL: http://lists.freebsd.org/mailman/listinfo/svn-src-stable-10
                      URL: http://lists.freebsd.org/mailman/listinfo/ctm-src-10
                      URL: http://lists.freebsd.org/mailman/listinfo/ctm-src-10-fast
                      OpenPGP Keys section in the Committer's Guide URL: http://www.freebsd.org/doc/en/articles/committers-guide/pgpkeys.html

                      Contact: FreeBSD Postmaster Team <postmaster@FreeBSD.org>

                      In the fourth quarter of 2013, the FreeBSD Postmaster Team has implemented the following items that may be interest of the general public:

                      • Retired the freebsd-aic7xxx mailing list.
                      • Created a graphics-team alias, requested by Niclas Zeising.
                        • -
                      • Worked with the FreeBSD Port Management Team to set up +
                      • Worked with the FreeBSD Ports Management Team to set up portmgr-lurkers so port managers can move addresses between those two aliases at their discretion.
                      • Created the lists associated with the new stable/10 branch: svn-src-stable-10, ctm-src-10, and ctm-src-10-fast.
                      • Redirected the vbox alias to the emulation list, requested by Bernhard Fröhlich.
                      • Continued a discussion on current and possible future mail and spam filtering.
                      • Disbanded lua and transferred it to Baptiste Daroussin, requested by Matthias Andree and Baptiste Daroussin.
                      • Modified the list moderators/administrators for ports-secteam, requested by Dag-Erling Smørgrav.
                      • Assisted Warren Block with an update to the "OpenPGP Keys for FreeBSD" section of the Committer's Guide.

                      FreeBSD Release Engineering Team

                      Links
                      FreeBSD 10.0-RELEASE schedule URL: http://www.FreeBSD.org/releases/10.0R/schedule.html
                      FreeBSD Virtual Machine Images URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/
                      FreeBSD Development Snapshots URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/ISO-IMAGES/

                      Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

                      The FreeBSD Release Engineering Team is finishing the 10.0-RELEASE cycle. The release cycle changed with two last-minute release candidate builds, each addressing fixes critical to include in the final release.

                      The FreeBSD 10.0-RELEASE cycle is expected to be completed by mid-January, approximately eight weeks behind the original schedule.


                      Projects


                      CBSD

                      Links
                      URL: http://www.bsdstore.ru
                      URL: https://github.com/olevole/cbsd

                      Contact: Oleg Ginzburg <olevole@olevole.ru>

                      CBSD is another FreeBSD jail management solution, aimed at combining various features, such as racct(8), vnet, zfs(8), carp(4), and hastd(8), into a single tool. This provides a more comprehensive way to build application servers using pre-installed jails with a typical set of software, and requires minimal effort to configure.

                      Open tasks:

                      1. Proper English translation of the website and the documentation.

                      Jenkins Continuous Integration for FreeBSD

                      Links
                      Vendor Summit presentation URL: http://www.ixsystems.com/whats-new/jenkins-bhyve-and-webdriver-continuous-integration-testing-on-freenas/

                      Contact: Craig Rodrigues <rodrigc@FreeBSD.org>

                      At the November 2013 FreeBSD Vendor Summit, some work was presented that Craig Rodrigues has been doing with Continuous Integration and Testing at iXsystems. Craig's presentation described how iXsystems is using modern best practices for building and testing the FreeNAS code. Jenkins is a framework for doing continuous builds and integration that is used by hundreds of companies. BHyve (BSD Hypvervisor) is the new virtual machine system which will be part of FreeBSD 10. Webdriver is a Python toolkit for testing web applications. By combining these technologies, iXsystems is developing a modern and sophisticated workflow for testing and improving the quality of FreeNAS.

                      Ed Maste from The FreeBSD Foundation was interested in this work, and based on this interest, it is now being ported to FreeBSD. Currently, a machine in the FreeBSD cluster has been allocated for this purpose, where a bhyve(4)-based virtual machine was set up and Jenkins was installed. The remainder is still in progress.

                      Open tasks:

                      1. Finish setting up Jenkins.
                      2. Add more builds to Jenkins.
                      3. Integrate testing with Jenkins.

                      Kernel


                      GEOM Direct Dispatch and Fine-Grained CAM Locking

                      Links
                      Slides from EuroBSDCon 2013, also describing this project URL: http://people.freebsd.org/~mav/disk.pdf
                      CAM improvements in the stable/10 branch URL: http://svnweb.freebsd.org/changeset/base/260387
                      GEOM improvements in the stable/10 branch URL: http://svnweb.freebsd.org/changeset/base/260385

                      Contact: Alexander Motin <mav@FreeBSD.org>

                      The CAM and GEOM multi-processor scalability improvement project has completed. The corresponding code has been committed to FreeBSD head and recently merged to the stable/10 branch; it shall appear in 10.1-RELEASE.

                      As part of this project, cam(4) (the ATA/SCSI subsystem) has received more fine-grained locking for better utilization of multi-core systems. In addition, the locking in geom(4) (the block storage subsystem) has also been polished, and a new direct dispatch functionality was implemented to spread the load between multiple threads and processors, and reduce the number of context switches.

                      Thanks to these cam(4) and geom(4) changes, the peak I/O rate has doubled on contemporary hardware, reaching up to 1,000,000 IOPS!

                      This project was sponsored by iXsystems, Inc.

                      Open tasks:

                      1. Some CAM controller drivers (SIMs) could also be optimized to get more benefits from this project, utilizing the new locking models and direct command completions from multiple interrupt threads.

                      Intel 802.11n NIC (iwn(4)) Work

                      Links

                      Contact: Adrian Chadd <adrian@freebsd.org>

                      There has been a large amount of work on iwn(4) over the last six months:

                      • New hardware support: 2xxx, 6xxx, 1xx series hardware.
                      • Many bugs were fixed, including scanning, association, EAPOL related fixes.
                      • iwn(4) now natively works with 802.11n rates from the net80211 rate control code, rather than mapping non-11n rates to 11n rates.

                      Open tasks:

                      1. There are still some scan hangs, due to how net80211 scans a single channel at a time. This needs to be resolved.
                      2. The transmit, receive, scan and calibration code needs to be refactored out of if_iwn.c and into separate source files.
                      3. There still seem to be some issues surrounding 2 GHz versus 5 GHz association attempts leading to firmware assertions, especially on the Intel 4965 NIC.

                      Intel GPU Driver Update

                      Contact: Konstantin Belousov <kib@FreeBSD.org>

                      This project will update the Intel graphics chipset driver, i915kms, to a recent snapshot of the Linux upstream code. The update will provide at least 1.5 years of bugfixes from the Intel team, and introduce support for the newest hardware — in particular Haswell and ValleyView. The IvyBridge code will also be updated. The addition of several features which are required to update X.Org and Mesa is also planned.

                      This project was sponsored by The FreeBSD Foundation.

                      Native iSCSI Stack

                      Links
                      URL: https://wiki.freebsd.org/Native%20iSCSI%20target

                      Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                      iSCSI is a popular block storage protocol. Under this project, a new, fast, and reliable kernel-based iSCSI initiator (client) and target (server) have been implemented.

                      During October to December, the work focused on performance and scalability. The target and the initiator now spread the load over multiple kernel threads, and the locking is optimized to reduce contention. This makes better use of multiple processor cores.

                      Work to finish iSER support is ongoing. All those optimizations will be gradually merged to head in February, and are expected to merged back to stable/10 and finally arrive in 10.1-RELEASE.

                      This project was sponsored by The FreeBSD Foundation.

                      New Automounter

                      Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                      Research and prototyping has begun on a new project to implement autofs(4) — an automounter filesystem — and its userland counterpart, automountd(8). The idea is to provide a very similar user experience to the automounters available on Linux, MacOS X, and Solaris, including using the same map format. The automounter will also integrate with directory services, such as LDAP.

                      This project was sponsored by The FreeBSD Foundation.

                      UEFI Boot

                      Links
                      UEFI wiki page URL: https://wiki.freebsd.org/UEFI
                      UEFI project branch URL: http://svnweb.freebsd.org/base/projects/uefi/

                      Contact: Ed Maste <emaste@FreeBSD.org>

                      The Unified Extensible Firmware Interface (UEFI) provides boot- and run-time services for x86 computers, and is a replacement for the legacy BIOS. This project will adapt the FreeBSD loader and kernel boot process for compatibility with UEFI firmware, found on contemporary servers, desktops, and laptops.

                      In 2013, The FreeBSD Foundation sponsored Benno Rice for a short project to improve the UEFI bootloader. This resulted in a working proof-of-concept in the UEFI project branch, but it was not ready to be merged to FreeBSD head.

                      Ed Maste has taken that original work and, with review feedback from Konstantin Belousov, been preparing it for integration into FreeBSD head. Some changes have been merged to head already. The rest will be merged as they are refined.

                      Intel provided a motherboard and CPU for the project, which proved invaluable for addressing bugs that did not appear while testing with the QEMU emulator.

                      This project was sponsored by The FreeBSD Foundation.

                      Open tasks:

                      1. Resolve a 32- versus 64-bit libstand(3) build issue.
                      2. Merge kernel parsing of EFI memory map metadata.
                      3. Integrate the EFI framebuffer with vt(9) (also known as Newcons).
                      4. Connect efiloader to the build.
                      5. Document manual installation for dual-boot configurations.
                      6. Integrate UEFI configuration with the FreeBSD installer.
                      7. Support secure boot.

                      UMA/ZFS and RPC/NFS Performance Improvements

                      Links
                      Discussion of the ZFS/UMA changes URL: http://docs.freebsd.org/cgi/mid.cgi?52894C92.60905

                      Contact: Alexander Motin <mav@FreeBSD.org>

                      The performance of ZFS and NFS was suboptimal in FreeBSD, so we have recently investigated some possible improvement paths. The uma(9) memory allocator caching code was improved to adapt better to system memory constraints. Combined with other virtual memory subsystem improvements done in the previous years, it should be safe to actively use uma(9) caches now. Their use in ZFS for ZIO/ARC may be enabled via the vfs.zfs.zio.use_uma loader(8) tunable, which is now the default for amd64, where it is recommended. Use of uma(9) caches for LZ4 compression buffers is unconditionally enabled on all architectures as it is has no serious drawbacks. On systems with many CPUs, these changes doubled the performance in the benchmarks.

                      Several areas of the NFS server stack (RPC, FHA, DRC) got a number of fixes and performance optimizations that significantly improve performance and reduce the CPU usage in a number of tests. Together with the ZFS memory allocator changes mentioned above, it was possible to reach 200K NFS block read IOPS and 55K SPEC NFS IOPS.

                      The code was committed to head. The uma(9) ZFS commits have been already merged to stable/10, and the remainder will be done soon as well.

                      This project was sponsored by iXsystems, Inc.

                      Open tasks:

                      1. The SPEC NFS test hits lock congestion on several global locks in the file system layer when a quite intensive READDIRPLUS NFS request is received. Fixing this problem could improve performance on large systems even further.

                      Updated vt(9) System Console

                      Links
                      Project wiki page URL: https://wiki.freebsd.org/Newcons

                      Contact: Aleksandr Rybalko <ray@FreeBSD.org>
                      Contact: Ed Maste <emaste@FreeBSD.org>
                      Contact: Ed Schouten <ed@FreeBSD.org>

                      Colloquially known as Newcons, vt(9) is a modern replacement for the existing, quite old, virtual terminal emulator called syscons(4). Initially motivated by the lack of Unicode support in syscons(4), the project was later expanded to cover the new requirement of supporting Kernel Mode Switching (KMS).

                      The project is now approaching completion and is ready for wider testing, as the related code was already merged to FreeBSD head. Hence, vt(9) can be tested easily by replacing the following two lines in the kernel config file:

                      device sc
 device vga

                      with the following ones:

                      device vt
 device vt_vga

                      Major highlights:

                      • Unicode support.
                      • Double-width character support for CJK characters.
                      • xterm(1)-like terminal emulation.
                      • Support for Kernel Mode Setting (KMS) drivers (i915kms, radeonkms).
                      • Support for different fonts per terminal window.
                      • Simplified drivers.

                      Brief status of supported architectures and hardware:

                      • amd64 (VGA/i915kms/radeonkms) — works.
                      • ARM framebuffer — works.
                      • i386 (VGA/i915kms/radeonkms) — works.
                      • IA64 — untested.
                      • MIPS — untested.
                      • PPC and PPC64 — works, but without X.Org yet.
                      • SPARC — works on certain hardware (e.g., Ultra 5).
                      • vesa(4) — in progress.
                      • i386/amd64 nVidia driver — need testing.
                      • Xbox framebuffer driver — need testing.

                      Known Issues:

                      • Switching to vty0 from X.Org on Fatal events will not work.
                      • Certain hardware (e.g., Lenovo X220) get a black screen when i915kms is preloaded.
                      • Scrolling can be slow;
                      • Screen borders are not cleared when changing fonts.
                      • vt(9) locks up with the gallant12x22 font in VirtualBox.

                      This project was sponsored by The FreeBSD Foundation.

                      Open tasks:

                      1. Create sub-directories for vt(9) under /usr/share/ to store key maps and fonts.
                      2. Implement remaining features supported by vidcontrol(1).
                      3. Write the vt(9) manual page.
                      4. Support keyboard handled directly by device kbd (without kbdmux(4)).
                      5. CJK fonts (in progress).

                      Architectures


                      FreeBSD Host Support for OpenStack and OpenContrail

                      Links
                      URL: http://www.openstack.org
                      URL: http://www.opencontrail.org
                      URL: https://github.com/Semihalf/openstack-devstack
                      URL: https://github.com/Semihalf/openstack-nova
                      URL: https://github.com/Semihalf/contrail-vrouter
                      URL: https://blueprints.launchpad.net/nova/+spec/freebsd-compute-node

                      Contact: Grzegorz Bernacki <gjb@semihalf.com>
                      Contact: Michał Dubiel <md@semihalf.com>
                      Contact: Rafał Jaworowski <raj@semihalf.com>

                      OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources in a data center. OpenContrail is a network virtualization (SDN) solution comprising a network controller, a virtual router, and an analytics engine, which can be integrated with cloud orchestration systems like OpenStack or CloudStack.

                      The goal of this work is to enable FreeBSD as a fully supported compute host for OpenStack, using OpenContrail virtualized networking. The main areas of development are the following:

                      • OpenStack compute driver (nova-compute) for the FreeBSD bhyve(4) hypervisor.
                      • OpenContrail vRouter (forwarding-plane kernel module) port to FreeBSD.
                      • Integration and performance optimizations.

                      The current state of development features a working demo of OpenStack with compute node components running on a FreeBSD host:

                      • The native bhyve(4) hypervisor is driven by a nova-compute component for spawning guest instances and a nova-network component for providing simple networking between those guests.
                      • The nova-network approach (based on local host bridging) is becoming an obsolete technology in OpenStack and was used here only for demonstration and proof-of-concept purposes, without exploring all the possible features.
                      • The main objective is to move to OpenContrail-based networking, therefore becoming compliant with the modern OpenStack networking API ("neutron").

                      This project was sponsored by Juniper Networks, Inc.

                      Open tasks:

                      1. Decide how to integrate bhyve(4) with nova-compute, either natively or via the libvirt management layer.

                      FreeBSD on Cubieboard{1,2}

                      Links
                      EMAC driver code URL: https://github.com/tsgan/allwinner_a10/blob/master/if_emac.c

                      Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>

                      Cubieboard is a single-board computer based on the AllWinner A10 SoC, popular on cheap tablets, phones and media PCs. The second version enhances the board mainly by replacing the AllWinner A10 SoC with an AllWinner A20 which contains 2 ARM Cortex-A7 MPCore CPUs and 2 Mali-400 GPUs (Mali-400MP2). In the last few months, work has continued on their FreeBSD port, and some work was done on the EMAC 10/100 Ethernet driver (see link). The driver is now in a good shape, however the RX side is very slow and there is need to have an external DMA driver that can be used in this case.

                      FreeBSD on Freescale i.MX6 processors

                      Links
                      Announcement of Wanboard support URL: http://lists.freebsd.org/pipermail/freebsd-arm/2013-November/006877.html

                      Contact: Ian Lepore <ian@freebsd.org>

                      The i.MX range is a family of Freescale Semiconductor proprietary microprocessors for multimedia applications based on the ARM architecture and focused on low power consumption. The i.MX6x series is based on the ARM Cortex A9 solo, dual, or quad cores. Initial support for them has been committed to head, and merged to stable/10. All members of the i.MX6 family (Solo, Dual, and Quad core) are supported, but SMP support on the multi-core SoCs has not yet been enabled.

                      Initial driver support includes:

                      • USB (EHCI)
                      • Ethernet (Gigabit)
                      • SD Card
                      • UART

                      The initial hardware bringup was done on Wandboard hardware, see the announcement on freebsd-arm in the links section for more information.

                      Open tasks:

                      1. Write drivers for additional on-chip hardware, including I2C, SPI, AHCI, audio, and video.
                      2. Add support to FreeBSD-crochet script to generate Wandboard images

                      FreeBSD on Freescale Vybrid VF6xx

                      Links
                      URL: http://svnweb.freebsd.org/changeset/base/258057

                      Contact: Ruslan Bukin <br@freebsd.org>

                      Basic support for the Freescale Vybrid Family VF6xx heterogeneous ARM Cortex-A5/M4 System-on-Chip (SoC) was added to FreeBSD head. The Vybrid VF6xx family is an implementation of the new modern Cortex-A5-based low-power ARM SoC boards. Vybrid devices are ideal for applications including simple HMI in appliances and industrial machines, secure control of infrastructure and manufacturing equipment, energy conversion applications such as motor drives and power inverters, ruggedized wired and wireless connectivity, and control of mobile battery-operated systems such as robots and industrial vehicles.

                      Supported device drivers:

                      • NAND Flash Controller (NFC)
                      • USB Enhanced Host Controller Interface (EHCI)
                      • General-Purpose Input/Output (GPIO)
                      • Universal Asynchronous Receiver/Transmitter (UART)

                      Also supported:

                      • Generic Interrupt Controller (GIC)
                      • MPCore timer
                      • ffec Ethernet driver

                      Open tasks:

                      1. Add support for a number of different VF5xx- and VF6xx-based development boards.
                      2. Expand device driver support, including framebuffer and other devices.

                      FreeBSD on Newer ARM Boards

                      Links
                      FreeBSD on Radxa Rock URL: https://wiki.freebsd.org/FreeBSD/arm/Radxa%20Rock
                      URL: http://svnweb.freebsd.org/changeset/base/256949
                      Some preliminary sources for Snapdragon board IFC6410 URL: https://github.com/tsgan/qualcomm

                      Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>

                      Rockchip is a series of SoC (System on Chip) integrated circuits that are mainly for embedded systems applications in mobile entertainment devices such as smartphones, tablets, e-books, set-top boxes, media players, personal video, and MP3 players. Due to their evolution from the MP3/MP4 player market, most Rockchip ICs feature advanced media decoding logic but lack integrated cellular radio basebands. Initial support for the Rockchip RK3188 (Quad core Cortex A9) SoC is committed to head. Now FreeBSD runs on Radxa Rock and it supports the following peripherals:

                      • Existing DWC OTG driver in host mode
                      • GPIO

                      Some work was also done on initial support for the Qualcomm Snapdragon S4 SoC, featuring the Krait CPU, which is considered a "platform" for use in smartphones, tablets, and smartbook devices. Krait has many similarities with the ARM Cortex-A15 CPU and is also based on the ARMv7 instruction set. A minimal console driver was written, and FreeBSD's early boot messages can be now seen on the serial console. The timer driver works too, and the boot now stops at the mountroot prompt.

                      FreeBSD/EC2

                      Links
                      FreeBSD/EC2 status page URL: http://www.daemonology.net/freebsd-on-ec2/
                      Configinit URL: http://www.daemonology.net/blog/2013-12-09-FreeBSD-EC2-configinit.html

                      Contact: Colin Percival <cperciva@freebsd.org>

                      An Amazon Machine Image (AMI) is a special type of virtual appliance that is used to create a virtual machine within the Amazon Elastic Compute Cloud ("EC2"). It serves as the basic unit of deployment for services delivered using EC2. Such AMIs are available for 8.3-RELEASE and later FreeBSD releases, and every ALPHA, BETA, and RC of FreeBSD 10.0. Starting from FreeBSD 10.0-BETA1, FreeBSD/EC2 images are running "fully supported" FreeBSD binaries, and starting from FreeBSD 10.0-RC1, FreeBSD/EC2 images include a "configinit" system for autoconfiguration using EC2 user-data.

                      Due to limitations of old (m1, m2, c1, t1) instance types, "Windows"-labelled images are required for those instance types; however all of the recent instances types — m3 (general purpose), c3 (high-CPU), and i2 (high-I/O) — support FreeBSD at the "unix" pricing rates.

                      The maintainer of this platform considers it to be ready for production use.

                      Open tasks:

                      1. Hand over the task of building FreeBSD AMIs to the Release Engineering Team.
                      2. Get Amazon to add "FreeBSD" to the list of platforms supported by EC2, so that it can stop showing up as "Other Linux".

                      FreeBSD/Xen

                      Links
                      FreeBSD PVH wiki page URL: http://wiki.xen.org/wiki/FreeBSD_PVH

                      Contact: Roger Pau Monné <royger@FreeBSD.org>
                      Contact: Justin T. Gibbs <gibbs@FreeBSD.org>

                      Xen is a native (bare-metal) hypervisor providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently. Xen 4.4 will bring a virtualization mode called PVH — PV (paravirtualization) in an HVM (fully-virtual) container. This is essentially a paravirtualized guest using paravirtualized drivers for boot and I/O. Otherwise it uses hardware virtualization extensions, without the need for emulation.

                      After merging the changes to improve Xen PVHVM support, work has shifted on getting PVH DomU support on FreeBSD. Patches have been posted, and after a couple of rounds of review, the series looks almost ready for merging into head. Also, very initial patches for FreeBSD PVH Dom0 support has been posted. So far the posted series only focuses on getting FreeBSD booting as a Dom0 and being able to interact with the hardware.

                      This project was sponsored by Citrix Systems R&D, and Spectra Logic Corporation.

                      Open tasks:

                      1. Finish reviewing and commit the PVH DomU support.
                      2. Work on PVH Dom0 support.

                      Intel IOMMU (VT-d, DMAR) Support

                      Links
                      URL: http://svnweb.freebsd.org/changeset/base/257251
                      URL: http://svnweb.freebsd.org/changeset/base/259512

                      Contact: Konstantin Belousov <kib@FreeBSD.org>

                      An Input/Output Memory Management Unit (IOMMU) is a Memory Management Unit (MMU) that connects a Direct Memory Access-capable (DMA-capable) I/O bus to main memory; therefore, I/O virtualization is performed by the chipset. An example IOMMU is the graphics address remapping table (GART) used by AGP and PCI Express graphics cards. Intel has published a specification for IOMMU technology as Virtualization Technology for Directed I/O, abbreviated VT-d.

                      A VT-d driver was committed to head and stable/10, so busdma(9) is now able to utilize VT-d. The feature is disabled by default, but it may be enabled via the hw.dmar.enable loader(8) tunable — see the links for more information. The immediate plans include increasing the support for this kind of hardware by testing and providing workarounds for specific issues, and by adding features of the next generation of Intel IOMMU. Hopefully, the existing and new consumers of VT-d will start to use the driver soon.

                      This project was sponsored by The FreeBSD Foundation.


                      Userland Programs


                      auditdistd(8)

                      Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>

                      The auditdistd(8) daemon is responsible for distributing audit trail files over TCP/IP networks securely and reliably. Currently, the daemon uses Transport Layer Security (TLS) for communication, but only server-side certificates are verified, based on the certificate's fingerprint. The ongoing work will make it possible to use client-side certificates and will support more complete public-key infastructure, which includes validation of the entire certificate chain, including revocation checking against Certification Revocation Lists at every level. From now on, auditdistd(8) will support TLSv1.2 and PFS modes only. In addition, it will be possible to send audit trail files to multiple receivers.

                      The work will be completed at the beginning of February 2014.

                      This project was sponsored by The FreeBSD Foundation.

                      Base GCC Updates

                      Contact: Pedro Giffuni <pfg@FreeBSD.org>

                      The GCC compiler in the FreeBSD base system is on its way to deprecation and is only used by some Tier-2 platforms at this time. While Clang is much better in many aspects, we still cannot use all the new features that it brings in the base system until we can drop GCC completely. As a stop-gap solution, several bug fixes and features from Apple GCC and other sources have been ported to our version of GCC 4.2.1 to make it more compatible with Clang. FreeBSD's GCC has added more warnings and some enhancements like -Wmost and -Wnewline-eof. An implementation for Apple's blocks extension is now available, too, and it will be very useful to enhance FreeBSD's support for Apple's Grand Central Dispatch (GCD).

                      Open tasks:

                      1. A merge from head to stable/9 is being considered but it disables nested functions by default, so the impact on the Ports Collection needs to be evaluated.
                      2. No further development of GCC 4.2 in the base system is planned.

                      BSDInstall ZFSBoot

                      Links
                      URL: http://lists.freebsd.org/mailman/listinfo/freebsd-sysinstall
                      Original Root-on-ZFS instuctions on the FreeBSD Wiki URL: https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE

                      Contact: Allan Jude <freebsd@allanjude.com>
                      Contact: Devin Teske <dteske@FreeBSD.org>
                      Contact: Warren Block <wblock@FreeBSD.org>

                      BSDInstall has been the default installation program since FreeBSD 9.0-RELEASE. However, it could not utilize one of the best features of FreeBSD, ZFS.

                      The ZFSBoot project started at EuroBSDCon 2013 and reached stable status in December, just in time for FreeBSD 10.0-RELEASE. Currently, ZFSBoot implements root-on-ZFS with 4k partition alignment, optional forced 4k sectors, optional geli(8) full disk encryption, and support for boot environments.

                      As part of ZFSBoot, BSDInstall itself also received a number of updates, including enhanced debugging, more scriptability, a new keymap selection menu, and a number of other small changes to streamline the installation process. The new keymap menu allows the user to test the selected keymap before continuing, to ensure it is the desired keymap. Minor changes were made to the network configuration dialogues to make the identification of wireless interfaces easier.

                      A number of additional features are also planned. The user should be able to create additional datasets and adjust the properties on all datasets in an interactive menu. There should also be integration with BSDConfig to allow users to install packages and the various other functionality that was previously provided by sysinstall.

                      Open tasks:

                      1. Interactive dataset editor.
                      2. Dataset property editor.
                      3. Consider using shell geom(4) parser.
                      4. BSDConfig integration.
                      5. UFS as a file system option, to allow users to create encrypted UFS installs.
                      6. Optionally make the boot pool UFS or reside on USB device(s).
                      7. Further streamline the installation process.

                      Capsicum and Casper

                      Links
                      URL: http://freebsdfoundation.blogspot.com/2013/12/freebsd-foundation-announces-capsicum.html

                      Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>

                      Capsicum is a lightweight OS capability and sandbox framework implementing a hybrid capability system model. The Casper daemon enables sandboxed application to use functionality normally unavailable in capability-mode sandboxes.

                      The Casper daemon, libcasper, libcapsicum(3), libnv(3) and Casper services (system.dns, system.grp, system.pwd, system.random and system.sysctl) have been committed to FreeBSD head. The tcpdump(8) utility in head now uses the system.dns service to do DNS lookups. The kdump(1) utility in head now uses the system.pwd and system.grp services to convert user and group identifiers to user and group names.

                      There is ongoing work to sandbox more applications. If you are interested in helping to make FreeBSD more secure and would like to learn about Capsicum and Casper, do not hesitate to contact Pawel — he can provide candidate programs that could use sandboxing.

                      This project was sponsored by The FreeBSD Foundation.

                      Centralized Panic Reporting

                      Links
                      Usage instructions URL: http://www.daemonology.net/blog/2013-11-06-automated-freebsd-panic-reporting.html

                      Contact: Colin Percival <cperciva@freebsd.org>

                      With the sysutils/panicmail port, a mechanism is now in place for automated submission of kernel panic reports to a central location. It is hoped that this will prove useful, as similar systems have for other operating systems, in identifying common panics so that developers can be alerted and they can be fixed faster.

                      In the first two months that this mechanism has been in place, 28 kernel panics have been reported. This is nowhere near enough to be useful, so readers are strongly encouraged to install the sysutils/panicmail port and follow the instructions to enable it.

                      Open tasks:

                      1. Get more systems set up to automatically submit panic reports!

                      FreeBSD Test Suite

                      Links
                      Project page URL: http://wiki.FreeBSD.org/TestSuite
                      Continuous testing infrastructure URL: http://kyua1.nyi.FreeBSD.org/
                      Mailing list announcement URL: http://lists.freebsd.org/pipermail/freebsd-testing/2013-December/000109.html
                      Blog post URL: http://julipedia.meroh.net/2013/12/introducing-freebsd-test-suite.html

                      Contact: Julio Merino <jmmv@FreeBSD.org>

                      The FreeBSD Test Suite project aims to equip FreeBSD with a comprehensive test suite that is easy to run out of the box and during the development of the system. The test suite is installed into /usr/tests/ and the kyua(1) command-line tool (devel/kyua in the Ports Collection) is used to run them.

                      The benefits of having a test suite that is easy to use and continuously run are obvious: regressions can be caught sooner rather than later and the Release Engineering Team can better assess the quality of the tree before deciding to cut a release. Additionally, because we choose to install the tests, we allow any end user to perform sanity checks on new installations of the system on their particular hardware configuration — a very attractive thing to do when deploying production servers.

                      During the last few months, we have added the necessary pieces to the build system to support building and installing test programs of various kinds. To demonstrate the functionality of these, some test programs were added and others were migrated from the old testing tree in tools/regression/ to the new layout for tests.

                      The current test suite should be seen as a proof of concept at this point: it is only composed of a small set of test programs and the goal is to get the infrastructure in place before mass-migrating existing test code and/or importing external tests.

                      As part of this work, two new releases of Kyua were published. Of special interest is the addition of a TAP-compliant backend so that existing tests from tools/regression/ can be plugged into the test suite with minimum effort.

                      As of December 31st, the basic continuous testing infrastructure is up and running, see the links section for the home page. For further information, please see the related announcement and blog post on the subject (also in the links section).

                      Open tasks:

                      1. We have three machines for the test cluster. At the moment, only one of them is in use to continuously test amd64 on both head and stable/10. We need to figure out the right level of parallelization to put other machines to use — but a first easy cut may be to just test different architectures (with the help of QEMU).
                      2. Related to the above, the Kyua reporting engine needs significant tuning to make the reports nice and clean. Ideally, Kyua should be able to coalesce results from different runs into a single location and generate cohesive reports out of them. Fixing this is a high priority.
                      3. A tutorial on writing tests for FreeBSD has been proposed for AsiaBSDCon 2014. The outcome of the proposal is still unknown, but stay tuned!
                      4. Port, port, and port more tests to the new test suite. A test suite is worthless if it does not validate stuff. Stay tuned for a request for help once we have put all basic pieces in place and have streamlined the migration process.

                      The LLDB Debugger

                      Links
                      URL: https://wiki.freebsd.org/lldb

                      Contact: Ed Maste <emaste@FreeBSD.org>

                      LLDB is the debugger in the LLVM family of projects. It supports Mac OS X, Linux, and FreeBSD, with ongoing work to support Windows.

                      In the last quarter of 2013, LLDB gained support for live (ptrace(2)-based) debugging of multithreaded processes on FreeBSD. Initial FreeBSD MIPS target support has also been committed, along with a number of endianness fixes in the general LLDB infrastructure.

                      The LLDB snapshot in the FreeBSD tree was updated to r196322. Currently disabled by default, it will be enabled for amd64 after the import of Clang 3.4. In the interim, it may be enabled by adding WITH_LLDB= to src.conf(5).

                      This project was sponsored by DARPA/AFRL, SRI International, and University of Cambridge.

                      Open tasks:

                      1. Update the in-tree snapshot to build after the Clang 3.4 import.
                      2. Fix amd64 watchpoints.
                      3. Test and fix the i386 port.
                      4. Implement FreeBSD ARM support.
                      5. Add support for kernel debugging (live local and remote debugging, and core files).
                      6. Fix the remaining test suite failures.
                      7. Enable by default on the amd64 architecture.

                      Ports


                      FreeBSD Python Ports

                      Links
                      The FreeBSD Python Team page URL: https://wiki.FreeBSD.org/Python
                      IRC channel URL: irc://freebsd-python@irc.freenode.net

                      Contact: FreeBSD Python Team <python@FreeBSD.org>

                      Python is a widely used general-purpose, high-level programming language. For many operating systems, Python is a standard component; it ships with FreeBSD as well. A lot of progress has been made around the FreeBSD Python ports in the last quarter.

                      The devel/py-distribute port has been replaced by the refreshed devel/py-setuptools port, which comes with a lot of features that simplify the methods of installing Python packages. The change also led us to install everything through Setuptools now, which resembles PyIP a bit and allows us to perform some major cleanup on the distutils installation behaviour.

                      The implicit lang/python build and run-time dependency was removed from the ports infrastructure. Every port now depends on a specific Python version or on the lang/python metaport. This prevents compatibility issues for ports that depend on Python 2.x OR Python 3.x exclusively, but use the python command, which might point to a version of incompatible user choice.

                      The lang/python27 port was updated to version 2.7.6, and the lang/python33 port was updated to version 3.3.3, and the lang/pypy port was updated to version 2.2.1.

                      We are currently working on the necessary infrastructure quirks to support different Python versions for the same port. Most of the work has been done and needs to be tested before it can be integrated.

                      Open tasks:

                      1. Develop a high-level and lightweight Python Ports Policy.
                      2. Add support for granular dependencies (for example >=1.0 or <2.0).
                      3. Look at what adding pip support looks like.
                      4. Convert all USE_PYDISTUTILS=easy_install entries to yes and remove the use of easy_install from the ports infrastructure.
                      5. More tasks can be found on the team's wiki page (see links).

                      GNOME/FreeBSD

                      Links
                      URL: http://www.FreeBSD.org/gnome/
                      Import of MATE URL: http://svnweb.freebsd.org/changeset/ports/334661

                      Contact: FreeBSD GNOME Team <gnome@FreeBSD.org>

                      GNOME is a desktop environment and graphical user interface that runs on top of a computer operating system. GNOME is part of the GNU Project and can be used with various Unix-like operating systems, including FreeBSD.

                      In this quarter, MATE 1.6 was finally imported into the Ports Collection, thanks to the efforts of Jeremy Messenger. MATE is a desktop environment forked from the now-unmaintained code base of GNOME 2, therefore it is basically a replacement for GNOME 2. Users wanting to keep GNOME 2 as their desktop are advised to switch to MATE since GNOME 2 will be replaced by GNOME 3 in the near future. This switch will be announced in advance, so people will have time to move to MATE if they have not already. The complete MATE-based desktop environment can be installed via the x11/mate port, or, for a minimal install, x11/mate-base.

                      Our home page is quite out of date. An update for it for GNOME 3.6 is underway. Part of this update is rewriting and updating the old GNOME porting guide as a chapter of the Porter's Handbook.

                      Another major task required for getting a bleeding-edge GNOME to build on FreeBSD mostly out-of-the box is moving to JHbuild with some custom rules. This is done to find and fix compile issues on other BSDs more quickly.

                      Open tasks:

                      1. GNOME 2 ports still need to be sorted out to evaluate which GNOME 2 components will be gone or be replaced with their newer GNOME 3 versions. This task is currently halted until we can get the documentation into a shape good enough to gather the issues and document the migration, including how to avoid the migration if the upgrade is not preferred. (This does not mean we do not want to know about issues with upgrading, though).
                      2. Help the X11 Team with Cairo 1.12, since the next version of GNOME 3 (3.12) will need an up-to-date version of Pango and GTK 3.

                      KDE/FreeBSD

                      Links
                      KDE/FreeBSD home page URL: http://FreeBSD.kde.org
                      area51 URL: http://FreeBSD.kde.org/area51.php
                      Out-of-date ports URL: http://portscout.freebsd.org/kde@freebsd.org.html

                      Contact: KDE FreeBSD Team <kde@FreeBSD.org>

                      KDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Solaris, Microsoft Windows, and OS X systems. The KDE/FreeBSD Team have continued to improve the experience of KDE software and Qt under FreeBSD.

                      During the last quarter, the team has kept most of the KDE and Qt ports up-to-date, working on the following releases:

                      • KDE SC (area51): 4.11.2, 4.11.3, 4.11.4
                      • Qt: 4.8.5 and 5.2 (area51)
                      • PyQt: 4.10.3; SIP: 4.15.2; QScintilla2: 2.8
                      • Qt Creator 2.8.0
                      • KDevelop: 4.5.2
                      • Calligra: 2.7.5
                      • CMake: 2.8.12, 2.8.12.1

                      As a result, according to PortScout, our team has 464 ports (down from 473), of which 88.15% are up-to-date (down from 98.73%). iXsystems Inc. continues to provide a machine for the team to build packages and to test updates. iXsystems Inc. has been providing the KDE/FreeBSD Team with support for quite a long time and we are very grateful for that.

                      As usual, the team is always looking for more testers and porters, so please contact us or visit our home page (see links). It would be especially useful to have more helping hands on tasks such as getting rid of the dependency on the defunct HAL project and providing integration with KDE's Bluedevil Bluetooth interface.

                      Open tasks:

                      1. Update out-of-date ports, see links for a list.
                      2. Worke on KDE 4.12 and Qt 5.
                      3. Make sure the whole KDE stack (including Qt) builds and works correctly with Clang and libc++.
                      4. Remove the dependency on HAL.

                      Wine/FreeBSD

                      Links
                      Wine wiki page URL: http://wiki.FreeBSD.org/Wine
                      Wine on amd64 wiki page URL: http://wiki.FreeBSD.org/i386-Wine
                      Wine homepage URL: http://www.winehq.org/

                      Contact: Gerald Pfeiffer <gerald@FreeBSD.org>
                      Contact: David Naylor <dbn@FreeBSD.org>

                      Wine is a free and open source software application that aims to allow applications designed for Microsoft Windows to run on Unix-like operating systems, such as FreeBSD. The Wine/FreeBSD Team have continued to improve the experience of Wine under FreeBSD.

                      During the fourth quarter of 2013, the team has kept Wine updated by porting:

                      • Stable releases: 1.6 and 1.6.1
                      • Development releases: 1.7.4 through 1.7.8

                      The ports have included packages built for amd64 (available through the Ports Collection).

                      The Wine ports have been kept up-to-date with the changes in the Ports Collection, including some improvements:

                      • Building with Clang by default (via USES=compiler:c11).
                      • Conditional X11 support (on by default; allowing for headless instances of Wine).
                      • Staging support and other ports best practices.

                      Support in improving the experience of Wine on FreeBSD is needed. Key areas including fixing regressions, adding copy protection scheme support, and fixing regressions when using Wine under FreeBSD/amd64.

                      Open tasks:

                      1. Open Tasks and Known Problems (see links for the wiki page).
                      2. FreeBSD/amd64 integration (see links for the i386-Wine wiki page).
                      3. Porting WoW64 and Wine64.

                      X.Org on FreeBSD

                      Links
                      X11 roadmap and supported hardware matrix URL: https://wiki.freebsd.org/Graphics
                      Ports-related development repository URL: http://trillian.chruetertee.ch/ports/browser/trunk
                      CFT for Cairo 1.12 and 8.x survey URL: http://lists.freebsd.org/pipermail/freebsd-x11/2014-January/014003.html

                      Contact: FreeBSD X11 Team <x11@FreeBSD.org>

                      The newer graphics stack (WITH_NEW_XORG) is now built by default on head and is provided as binary packages from the official FreeBSD pkg(8) repository for 11-CURRENT. The major updates are:

                      • X.Org server 1.12.
                      • Mesa 9.1.
                      • Recent Intel and Radeon X.Org drivers, using exclusively the KMS kernel drivers available in FreeBSD 9.x (Intel) and FreeBSD 10.x (Radeon).

                      This change makes X.Org on FreeBSD head work out-of-the-box on workstations and laptops based on recent Intel and Radeon GPUs. FreeBSD 10.x will follow in a few weeks or months.

                      Some software has started to require Cairo 1.12, for example GTK+ 3.10 and Pango. Unfortunately, this version of Cairo triggers a bug in the old Intel driver (2.7.1, installed when WITH_NEW_XORG is not set), which causes display artifacts. A "Call For Testers" mail was posted on the freebsd-x11 mailing-list (see the links above) to gather information about the behavior on other configurations (new Intel driver and non-Intel drivers). As of this writing, the reports received talk about improvements or, at least, no change noticed.

                      To better manage changes such as the WITH_NEW_XORG and the Cairo 1.12 changes mentioned above, we asked on the freebsd-x11 mailing-list if people are using FreeBSD 8.x on their desktop computers and why they do not upgrade to FreeBSD 9.x or 10.x. So far, we received very few answers to this.

                      The Radeon KMS driver in FreeBSD 10.x is now considered stable, especially now that integrated GPUs are properly initialized. One of the next steps will be to merge this to stable/9.

                      A "Graphics" wiki article (see links) was created to centralize and coordinate the work being done on both the ports and the kernel. It contains the following important information:

                      • A roadmap of the team.
                      • A matrix of supported hardware.
                      • Instructions on upgrading to KMS.
                      • Project status and results.

                      This starting page then points to project- and topic-specific articles where more detailed information is available.

                      Open tasks:

                      1. Report why FreeBSD 8.x is still used on your desktop and why moving to FreeBSD 9.x or 10.x is not an option.
                      2. Report about the Cairo 1.12 update on your system.
                      3. See the "Graphics" wiki page for up-to-date information.

                      Xfce/FreeBSD

                      Links
                      The FreeBSD Xfce Team's wiki page URL: https://wiki.freebsd.org/Xfce
                      Core URL: https://people.freebsd.org/~olivierd/xfce-core-unstable.html
                      Parole URL: https://people.freebsd.org/~olivierd/parole-unstable.html

                      Contact: FreeBSD Xfce Team <xfce@FreeBSD.org>

                      Xfce is a free software desktop environment for Unix and Unix-like platforms, such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use. The FreeBSD Xfce Team has kept most of the Xfce ports up-to-date, while fixing many issues along the way in this quarter.

                      Currently, the following components with the following versions are available:

                      • Applications:
                        • Orage (4.10.0)
                        • Midori (0.5.6)
                        • xfce4-terminal (0.6.3)
                        • xfce4-parole (0.5.3, 0.5.4)
                      • Panel plugins:
                        • xfce4-whiskermenu-plugin (1.2.0, 1.2.1, 1.2.2, 1.3.0)
                        • xfce4-mailwatch-plugin (1.2.0)
                        • xfce4-wmdock-plugin (0.6.0)

                      We helped Midori's upstream switch from Waf (Python script) to CMake. Xfce now also supports Gtk2, Gtk3, and the new WebKitGtk API, available from the 2.x branch, not present in our ports tree at the moment, though. Most of the ports now use stage directories, with only some plugins left to convert.

                      We also removed obsolete ports:

                      • x11-themes/lila-xfwm4 (Xfwm4 theme)
                      • multimedia/xfce4-media (multimedia player)
                      • net-im/xfce4-messenger-plugin

                      Besides, we followed the development of the Xfce core components and Parole closely. See the links for documentation on how to upgrade those libraries.

                      Open tasks:

                      1. Fix Midori's build on DragonFly, through DPorts.
                      2. Fix build of the Granite framework (it is an extension to Gtk and Midori uses it) on FreeBSD 10 and head. Those are mostly LLVM failures.
                      3. Add support for Berkeley DB 5 and higher to Orage.

                      Miscellaneous


                      The FreeBSD Foundation

                      Links
                      URL: http://www.FreeBSDFoundation.org/
                      Semi-annual newsletter URL: http://www.freebsdfoundation.org/press/2013Dec-newsletter
                      FreeBSD Journal URL: http://freebsdjournal.com/

                      Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                      The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of the funding is used to support FreeBSD development projects, conferences and developer summits, purchase equipment to grow and improve the FreeBSD infrastructure, and provide legal support for the Project.

                      We held our year-end fundraising campaign. We are still processing donations and will post the final numbers by mid-January. We are extremely grateful to all the individuals and organizations that supported us and the Project by making a donation in 2013. We have already started our fundraising efforts for 2014.

                      Some of the highlights from this past quarter include:

                      • We sponsored or are sponsoring the following projects:
                        • Projects completed last quarter: Capsicum, Casper daemon, and Intel I/O Memory Management Unit driver.
                        • Projects in progress: Native in-kernel iSCSI stack, network stack layer 2 modernization, UEFI boot, updated vt(9) system console.
                        • Projects started last quarter: Automounter, Intel graphics driver update.
                      • Continued work on the FreeBSD Journal, our new online FreeBSD magazine, which debuts on January 27th (see links).
                      • Sponsored, organized, and ran the Bay Area Developer Summit.
                      • Sponsored and attended the first ever vBSDCon, which had an impressive attendance.
                      • Sponsored and attended the OpenZFS developer summit.
                      • Represented the foundation at the following conferences: All Things Open in Raleigh, NC and LISA in Washington, DC.
                      • Sponsored the FreeBSD 20th Birthday Party, held in San Francisco.
                      • Attended the ICANN meeting in Buenos Aires in November and gave a short presentation on the change from BIND to unbound in FreeBSD 10.0 during the ccNSO Tech Day.
                      • Met with a few companies to discuss their FreeBSD use, what they would like to see supported in FreeBSD, and assist with collaboration between them and the Project.
                      • Purchased an 80-core server to reside at Sentex for the Project to use for stability, scalability, and performance improvements. It is a big step forwards for the Foundation in providing this kind of hardware to the Project's developers. It will let us test our scaling to 80 simultaneous cores and 1 TB of RAM. It will also be used to do performance analysis on large workloads, such as large databases etc.
                      • Acquired a second rack to use at Sentex.
                      • We received a commitment from VMware, Inc. for BSD-licensed drivers. They also committed to a yearly silver level donation.
                      • Signed up as a Google Compute trusted tester for the Project.
                      • Funded a project to produce a white paper titled Managed Services Using FreeBSD at NYI.
                      • Finally, we published our semi-annual newsletter (see links) highlighting what we did to support the FreeBSD Project and Community in 2013.
                      News Home | Status Home
                      diff --git a/website/content/en/status/report-2014-01-2014-03.html b/website/content/en/status/report-2014-01-2014-03.html index 730c5a3833..037ff8e9a4 100644 --- a/website/content/en/status/report-2014-01-2014-03.html +++ b/website/content/en/status/report-2014-01-2014-03.html @@ -1,2035 +1,2035 @@ FreeBSD Quarterly Status Report
                      Skip site navigation (1) Skip section navigation (2)

                      Introduction

                      This report covers FreeBSD-related projects between January and March 2014. This is the first of four reports planned for 2014.

                      The first quarter of 2014 was, again, a hectic and productive time for FreeBSD. The Ports team released their landmark first quarterly stable branch. FreeBSD continues to grow on the ARM architecture, now running on an ARM-based ChromeBook. SMP is now possible on multi-core ARM systems. bhyve, the native FreeBSD hypervisor, continues to improve. An integral test suite is taking shape, and the Jenkins Continuous Integration system has been implemented. FreeBSD patches to GCC are being forward-ported, and LLDB, the Clang/LLVM debugger is being ported. Desktop use has also seen improvements, with work on Gnome, KDE, Xfce, KMS video drivers, X.org, and vt, the new console driver which supports KMS and Unicode. Linux and Wine binary compatibility layers have been improved. UEFI booting support has been merged to head. The FreeBSD Foundation continues to assist in moving FreeBSD forward, sponsoring conferences and meetings and numerous development projects. And these are only some of the things that happened! Read on for even more.

                      Thanks to all the reporters for the excellent work! This report contains 41 entries and we hope you enjoy reading it.

                      The deadline for submissions covering between April and - June 2014 is July 7th, 2014.

                      FreeBSD Team Reports

                      Projects

                      Kernel

                      Architectures

                      Userland Programs

                      Ports

                      Documentation

                      Miscellaneous


                        FreeBSD Team Reports


                        FreeBSD Cluster Administration Team

                        + June 2014 is July 7th, 2014.

                        FreeBSD Team Reports

                        Projects

                        Kernel

                        Architectures

                        Userland Programs

                        Ports

                        Documentation

                        Miscellaneous


                          FreeBSD Team Reports


                          FreeBSD Cluster Administration Team

                          Contact: FreeBSD Cluster Administration Team <admins@>

                          The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:

                          • Assimilated master email configurations into a single source control repository.
                          • Moved the FreeBSD web server CGI services to a new location (sponsored).
                          • Further enhanced upon our internal monitoring utilities.
                          • Added a Russian pkg(8) mirror, hosted by Yandex.
                          • Moved the FreeBSD Foundation web services to a new server (sponsored).

                          This project was sponsored by The FreeBSD Foundation.

                          FreeBSD Core Team

                          Contact: FreeBSD Core Team <core@FreeBSD.org>

                          The FreeBSD Core Team constitutes the project's Board of Directors, responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape.

                          The first quarter of 2014 was very active for the Core Team. John Baldwin and David Chisnall kept coordinating the work required for providing a newer version of X.Org for 9.x and 10.x systems. Now that vt(4), a successor to syscons(4) that offers a KMS-enabled console, has been merged to both stable/9 and stable/10, an alternative pkg(8) repository is in preparation for wider testing of vt(4) and the new X.Org version. In addition to that, John Baldwin published the policy on licenses for new files and files with non-standard licenses. Thanks to the efforts of Gavin Atkinson, FreeBSD has again made it into the Google Summer of Code program, for the tenth time. David Chisnall reported that both libc++ and libstdc++ can now be built, as all of the standards-compliant implementations of the required numerical functions have been added.

                          The Core Team conducted an annual review among the Project teams and hats, where team members had to declare whether they wished to continue their service. As a result, Florian Smeets replaced David Wolfskill in the lead role of the Postmaster Team, and Glen Barber assumed the head Release Engineer position from Ken Smith. The Core Team congratulates Florian and Glen, and thanks David and Ken for their long-standing work.

                          The Core Team approved chartering the Ports Security Team, which is established to maintain security updates for the ported applications. In coordination with the Port Management Team, pkg_tools was eventually deprecated and tagged with an End-of-Life date, in order to clear the way for - pkg(8). The Port Management Team also requested a + pkg(8). The Ports Management Team also requested a way to make it possible to track userland ABI and KBI changes reliably for the Ports Collection. Ideally this can be achieved by increasing the value of __FreeBSD_version on each fix, therefore the corresponding discussion concluded in freezing the ABI note tag for releases in order to keep the size of binary patches for freebsd-update(8) low. A related Errata Notice is about to be published soon.

                          Only a single commit bit was taken for safekeeping. We did not have new committers to the src/ repository in this quarter.

                          FreeBSD Documentation Engineering Team

                          Links
                          Announcement of Warren Block's addition URL: http://lists.freebsd.org/pipermail/freebsd-doc/2014-February/023265.html

                          Contact: FreeBSD Documentation Engineering Team <doceng@FreeBSD.org>

                          The FreeBSD Documentation Engineering Team is responsible for defining and following up on the documentation goals for the committers in the Documentation project. The team is pleased to announce a new member — Warren Block. In early March, the FreeBSD Documentation Engineering Team members assumed responsibility for the FreeBSD Webmaster Team.

                          -

                          FreeBSD Port Management Team

                          Links
                          +

                          FreeBSD Ports Management Team

                          Links
                          URL: http://www.FreeBSD.org/ports/
                          URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
                          URL: http://portsmon.freebsd.org/index.html
                          URL: http://www.freebsd.org/portmgr/index.html
                          URL: http://blogs.freebsdish.org/portmgr/
                          URL: http://www.twitter.com/freebsd_portmgr/
                          URL: http://www.facebook.com/portmgr
                          URL: http://plus.google.com/communities/108335846196454338383

                          Contact: Thomas Abthorpe <portmgr-secretary@FreeBSD.org>
                          Contact: Frederic Culot <culot@FreeBSD.org>
                          - Contact: FreeBSD Port Management Team <portmgr@FreeBSD.org> + Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

                          -

                          The role of the FreeBSD Port Management Team is to ensure that the +

                          The role of the FreeBSD Ports Management Team is to ensure that the FreeBSD Ports Developer community provides a ports collection that is functional, stable, up-to-date and full-featured. It is also to coordinate among the committers and developers who work on it.

                          The ports tree slowly approaches the 25,000 ports threshold, while the PR count exceeds 1,800. In the first quarter, we added four new committers, took in three commit bits for safe keeping, and reinstated one commit bit.

                          In January, the longest serving port manager, Joe Marcus Clarke, stepped down from his active duties on the team. At a similar time Ion-Mihai Tetcu also stepped down from his duties. Fortunately, as a result of the first portmgr-lurkers intake, we were able to replace them with Mathieu Arnold and Antoine Brodin.

                          Commencing March 1, the second intake of portmgr-lurkers started active duty on portmgr for a four month duration. The next two candidates are Alexey Dokuchaev and Frederic Culot.

                          This quarter also saw the release of the first quarterly branch, namely 2014Q1. This branch is intended to provide a stable and high-quality ports tree, with patches related to security fixes as well as packaging and runtime fixes being backported from head.

                          Ongoing maintenance goes into redports.org, including QAT runs and ports and security updates.

                          Open tasks:

                          1. As previously noted, many PRs continue to languish. We would like to see committers dedicate themselves to closing as many as possible.

                          FreeBSD Postmaster Team

                          Contact: FreeBSD Postmaster Team <postmaster@FreeBSD.org>

                          The FreeBSD Postmaster Team is responsible for mail being correctly delivered to the committers' email addresses, ensuring that the mailing lists work, and should take measures against possible disruptions of project mail services, such as having troll-, spam- and virus-filters.

                          In the first quarter of 2014, the team has implemented these items that may be interest of the general public:

                          • Continued a discussion on current and possible future mail and spam filtering.
                          • Discovered more of what needs to be done for a new year (with respect to email archives), did what we could, and recorded the steps for next time.
                          • Added Kubilay Kocak to donations, requested by Pietro Cerutti.
                          • Added Warren Block to doceng.
                          • Made sure portmgr receives bounces for pkg-fallout messages.
                          • Created a jenkins-admin mail alias.
                          • Enabled Mailman password reminder emails again.
                          • Discovered that all Mailman cron jobs were disabled in November during upgrades. Enabled those again. This caused problems like digests not being sent.

                          FreeBSD Release Engineering Team

                          Links
                          FreeBSD 9.3-RELEASE schedule URL: http://www.FreeBSD.org/releases/9.3R/schedule.html
                          FreeBSD 9.3-RELEASE todo list URL: http://www.FreeBSD.org/releases/9.3R/todo.html
                          FreeBSD development snapshots URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/ISO-IMAGES/

                          Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

                          The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

                          In early January, the team became aware of several last-minute showstopper issues in FreeBSD 10.0, which led to an extension in the final release builds. FreeBSD 10.0-RELEASE was announced on January 20, two months behind the original schedule.

                          The schedule for the FreeBSD 9.3-RELEASE cycle has been written and posted to the website, and the release cycle will begin early May.

                          There is ongoing work to integrate support for embedded architectures as part of the release build process. At this time, support exists for a number of ARM kernels, in particular the Raspberry Pi, BeagleBone, and WandBoard.

                          This project was sponsored by The FreeBSD Foundation.


                          Projects


                          Jenkins Continuous Integration for FreeBSD

                          Links
                          Jenkins CI server in FreeBSD cluster URL: https://jenkins.FreeBSD.org
                          Jenkins on FreeBSD project status URL: https://wiki.freebsd.org/Jenkins#Jenkins_for_FreeBSD_status
                          Video and slides of March 13, 2014 presentation at Bay Area FreeBSD User Group (BAFUG) URL: https://wiki.freebsd.org/Jenkins#Presentations_and_Working_Groups
                          Jenkins, libvirt, and bhyve URL: http://empt1e.blogspot.ru/2014/03/using-jenkins-libvirt-slave-plugin-with.html
                          Jenkins Continuous Integration URL: http://jenkins-ci.org
                          Ansible URL: http://www.ansible.com

                          Contact: Craig Rodrigues <rodrigc@FreeBSD.org>
                          Contact: Jenkins Administrators <jenkins-admin@FreeBSD.org>
                          Contact: FreeBSD Testing <freebsd-testing@FreeBSD.org>

                          Jenkins is a framework used by many companies and open source projects for Continuous Integration (CI). CI allows developers to commit code to a Source Code Management (SCM) system such as Subversion, and then have automated programs check out, build, and test the code. Jenkins is implemented in the Java language.

                          Ed Maste reviewed some CI work that Craig Rodrigues had done for the FreeNAS project with Jenkins, and encouraged him to set up something similar for the FreeBSD Project. With the help of the FreeBSD Cluster Administration Team, he set up a FreeBSD machine running two bhyve virtual machines, jenkins-9.FreeBSD.org and jenkins-10.FreeBSD.org. He set up software builds of head and several stable branches on these machines. The status of these builds is visible on a web interface accessible at jenkins.FreeBSD.org. When any of the builds fail, emails are sent to freebsd-current or freebsd-stable. Emails are also sent directly to the list of people who recently committed code to Subversion since the last successful build.

                          As part of the Jenkins setup, Craig Rodrigues encountered problems with running Java on FreeBSD 9.2 and FreeBSD 10.0. Both problems stemmed from changes to the FreeBSD Virtual Memory (VM) subsystem. On FreeBSD 9.2-RELEASE, running Jenkins under Java would cause the kernel to panic. This was a known problem, and fixed in 9.2.-RELEASE-p3. On FreeBSD 10.0-RELEASE, Java processes would randomly crash. Disabling the vm.pmap.pcid_enabled sysctl(3) variable seemed to fix the problem. In kern/187238, Henrik Gulbrandsen submitted fixes to the FreeBSD VM to address this problem. Konstantin Belousov committed the fixes to head, where they are being tested now.

                          During the setup of the bhyve VMs which run Jenkins processes, Craig Rodrigues wrote scripts to start bhyve VMs from the rc.d bootup scripts, which were then published at GitHub.

                          On February 19, 2014, Craig Rodrigues notified the FreeBSD developers that Jenkins was running in the FreeBSD cluster, and that they could look at the web interface to see the status of builds.

                          On March 13, 2014, Craig Rodrigues gave a presentation of the Jenkins work at the Bay Area FreeBSD User Group (BAFUG) in Mountain View, California, USA. Video of the presentation was recorded and put online by iXsystems.

                          Craig Rodrigues assembled a team of volunteers, jenkins-admin, to help maintain jenkins.FreeBSD.org and expand the use of Jenkins CI used in the FreeBSD cluster. jenkins-admin consists of the following people working in the following areas:

                          • R. Tyler Croy is both a FreeBSD developer and a Jenkins developer. He will be working on fixing bugs in Jenkins specific to FreeBSD. He is first looking at fixing the libpam4j library which is used by Jenkins to interface with the PAM system for user authentication. The released version of libpam4j does not currently work on FreeBSD.
                          • Li-Wen Hsu maintains the devel/jenkins port. He set up a Jenkins build which runs the scan-build static analyzer which is part of LLVM.
                          • Steven Kreuzer has experience administering Jenkins systems. He set up several builds on jenkins.FreeBSD.org, including a Jenkins build of the FreeBSD documentation. He is looking into using Ansible for automatic provisioning of VMs running Jenkins in the FreeBSD cluster.
                          • Craig Rodrigues will be running a Continuous Testing working group at the FreeBSD Devsummit in Ottawa on May 15, 2014. He will also give a Jenkins presentation on May 17, 2014. He is interested in working with Julio Merino to integrate Jenkins and Kyua. They have exchanged some emails about this on the freebsd-testing list.
                          • Steve Wills maintains the devel/jenkins-lts port. He has implemented several builds at jenkins.FreeBSD.org which detect commits to the FreeBSD ports repository, and then build the ports tree using Poudrière.

                          At the end of March, Roman Bogorodskiy reported to jenkins-admin that he has successfully run the Jenkins libvirt plugin with his libvirt modifications to integrate with bhyve. He provided a link to a blog posting where he described his experience.

                          This project was sponsored by iXsystems, Inc.

                          Open tasks:

                          1. Obtain certificates for LDAP and web servers, to replace self-signed certificates.
                          2. Set up more Jenkins builds of the FreeBSD base repository on different branches and with different configurations.
                          3. Set up more Jenkins builds of the FreeBSD ports repository on different FreeBSD versions.
                          4. Integrate with Kyua, so that Jenkins can run Kyua tests and report the results directly in the native Jenkins web UI where test results are reported.
                          5. Write scripts which can take a Jenkins build of FreeBSD, and boot the results in a bhyve VM or on real hardware.
                          6. Fix libpam4j on FreeBSD.
                          7. Continuous Testing working group at Devsummit on May 15, 2014
                          8. Jenkins presentation at BSDCan on May 17, 2014

                          ZFSguru

                          Links
                          Home page URL: http://zfsguru.com/

                          Contact: Jason Edwards <sub.mesa@gmail.com>

                          ZFSguru is a multifunctional server appliance with a strong emphasis on storage. It wants to deliver all the great BSD and ZFS technology to a wider audience, while at the same time pleasing more advanced users as well with unique features and customization.

                          A vanilla ZFSguru installation comes with only Samba and a web-interface setup, but can be extended easily by installing addons called services to add functionality as desired. This prevents users from running programs they do not need and do not want. Advanced users can still use ZFSguru as they would a normal FreeBSD installation with a 100% ZFS setup (Root-on-ZFS). ZFSguru does not strip away anything, and uses a GENERIC-like kernel with only some additional settings added like InfiniBand networking, Device Polling and AltQ. This means you can use a ZFSguru installation as you would use a FreeBSD installation.

                          In the first month of 2014, ZFSguru has released beta9 version of the web interface. This release brings vastly improved support for Samba and NFS configuration. In particular, it adds a convenient drag-and-drop interface for Samba permissions. This allows novice users to configure access to shares in various configurations. It allows both control and usability, with no manual being necessary in order to operate it. This is the ZFSguru style.

                          New system versions have been released, based on FreeBSD 9.2, 10.0, and head. The experimental head version has vt(4) and X.org 7.12.4 and the Intel/Radeon KMS graphics drivers. That is, the latest and greatest of FreeBSD graphics development. The ZFSguru project plans to release stable/10 builds in the near future which also have the MFCed patches for vt(4), the KMS-enabled system console with Unicode support. Please see the vt(4) entry for more information.

                          Support for ZFS version 5000 is now universal across 9.2, 10.0 and head builds. LZ4 compression is the key feature for ZFS version 5000. Otherwise users are advised to keep their pool versions as is, to be as compatible as you can with as many ZFS platforms as possible. Only upgrade the pool as you desire its functionality, forfeiting the compatibility with older storage platforms.

                          Open tasks:

                          1. ZFSguru beta10 will increase the compatibility of newly added Samba functionality with non-Gecko browsers. It will also fix some minor bugs as well as speed up some pages by having a redesigned remote database system called GuruDB.
                          2. ZFSguru beta11 will add the one major feature still missing in ZFSguru: the Migration Manager. This allows users to maintain a file with all the configuration of their ZFSguru installation. It can be used like a firmware — restoring the machine to the exact state and configuration of the snapshot configuration. It allows users to maintain a backup of their ZFSguru configuration and allows upgrading to a newer ZFSguru system version without any hassle.
                          3. Automated system builds should bring more system image releases.
                          4. New website with new forum and new login system.
                          5. Developer website with GitLab setup, allowing bug reports, code contributions, wiki, and wall messages. Note that GitLab has also been provided as a ZFSguru service, for those interested in trying GitLab.

                          Kernel


                          ASLR and PIE

                          Links
                          Blog post with latest status update URL: http://0xfeedface.org/blog/lattera/2014-04-03/awesome-freebsd-aslr-progress
                          Shawn's ASLR branch URL: https://github.com/lattera/freebsd/tree/soldierx/lattera/aslr
                          Olivér's ASLR branch URL: https://github.com/opntr/opBSD/tree/op/stable/10-aslr

                          Contact: Shawn Webb <lattera@gmail.com>
                          Contact: Olivér Pintér <oliver.pntr@gmail.com>

                          Address space layout randomization (ASLR) is a computer security technique involved in protection from buffer overflow attacks. In order to prevent an attacker from reliably jumping to a particular exploited function in memory, ASLR involves randomly arranging the positions of key data areas of a program, including the base of the executable and the positions of the stack, heap, and libraries, in a process' address space.

                          We have added ASLR support to all architectures. As the primary developers behind this feature have the most access to amd64, the focus of development is on the amd64 architecture. Other architectures, such as ARM, have known bugs with our current ASLR implementation and we are working hard to fix them. We added support for Position-Independent Executables (PIEs) in a number of applications in base.

                          Open tasks:

                          1. Shawn has access to a Raspberry Pi (RPI). PIE is 90% broken. Debug and fix major issues on the RPI. The existing NX stack protections are not obeyed on RPI. Properly implemented ASLR requires a NX stack.
                          2. Shawn will be receiving a sparc64 box on April 6, 2014. He will test ASLR on sparc64, identifying and fixing any bugs that pop up.
                          3. Olivér has identified one or more bugs with the Linuxulator. He will be looking into that and fixing those.
                          4. Shawn will be cleaning up code and adding support for PIE to more applications in base. He will also add PIE support to the ports framework for general consumption.
                          5. Shawn will be giving a presentation regarding ASLR at BSDCan 2014.

                          Intel GPU Driver Update

                          Contact: Konstantin Belousov <kib@FreeBSD.org>

                          The project to update the Intel graphics chipset driver (i915kms) to a recent snapshot of the Linux upstream code continues. Progress was delayed by external circumstances, but it is hoped to reach a useful milestone in the near future.

                          This project was sponsored by The FreeBSD Foundation.

                          Native iSCSI Stack

                          Links
                          URL: https://wiki.freebsd.org/Native%20iSCSI%20target

                          Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                          The new FreeBSD in-kernel iSCSI stack was functionally complete in FreeBSD 10.0-RELEASE, but ongoing enhancements and bug fixes are being committed to FreeBSD head, with a plan to merge them back to stable/10 in time for FreeBSD 10.1-RELEASE.

                          Many issues have been resolved, including very slow operation with data digests enabled, bugs in persistent reservations which impacted Hyper-V Failover Cluster, and a negotiation problem affecting Dell Equallogic users.

                          There have also been numerous enhancements, such as support for redirections, which are necessary for some High Availability setups, and the ability to modify session parameters in the iscsictl utility. Previously it was necessary to remove the session and add it again.

                          This project was sponsored by The FreeBSD Foundation.

                          New Automounter

                          Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                          The automount project is nearing the functional prototype stage, and a call for testing is expected in the next month. The userspace portion consists of the automountd(8) daemon, which is designed to be fully compatible with its counterparts in OS X, Solaris, and Linux, and which is nearly complete. Work on the kernel component continues.

                          This project was sponsored by The FreeBSD Foundation.

                          PCI SR-IOV Infrastructure

                          Links
                          Work in progress on GitHub URL: https://github.com/rysto32/freebsd/tree/iov_ixgbe

                          Contact: Ryan Stone <rstone@FreeBSD.org>

                          PCI Single Root I/O Virtualization (SR-IOV) is an optional part of the PCIe standard that provides hardware acceleration for the virtualization of PCIe devices. When SR-IOV is in use, a function in a PCI device (known as a Physical Function, or PF) will present multiple Virtual PCI Functions (VF) on the PCI bus. These VFs are fully independent PCI devices that have access to the resources of the PF. For example, on a network interface card, VFs could transmit and receive packets independent of the PF.

                          The most obvious use case for SR-IOV is virtualization. A hypervisor like bhyve could instantiate a VF for every VM and use PCI passthrough to assign the VFs to the VMs. This would allow multiple VMs to share access to the PCI device without having to do any expensive communication with the hypervisor, greatly increasing performance of performing I/O from a VM.

                          There are two parts to this project. The first is implementing an API in the PCI subsystem for creating VFs and configuring standard PCI features like BARs. The second part is updating individual drivers for PCI devices that support SR-IOV to configure their VFs. For example, a network interface driver will typically have to assign a MAC address to a VF and configure the interface to route packets destined for that MAC address to the VF.

                          At this point only SR-IOV support for the ixgbe(4) driver is planned. The PCI subsystem API is designed to be generic and should support SR-IOV on any device, but fairly extensive driver work is necessary to support SR-IOV, which is currently not planned due to lack of time and hardware.

                          At present, ixgbe(4) is able to create VFs and the ixgbevf driver is able to pass traffic. There is still a fair amount of work to support VLAN tags, multicast addresses, and other features on the VFs. Also, the VF configuration needs to be better integrated with the PF initialization path to ensure that resets of the PF do not interrupt operation of the VFs.

                          This project was sponsored by Sandvine, Inc.

                          SDIO Driver

                          Links
                          SDIO project page on FreeBSD wiki URL: https://wiki.freebsd.org/SDIO
                          Source code URL: https://github.com/kibab/freebsd/tree/mmccam

                          Contact: Ilya Bakulin <ilya@bakulin.de>

                          SDIO is an interface designed as an extension of the existing SD card standard, allowing connection of different peripherals to the host with the standard SD controller. Peripherals currently sold on the general market include WLAN/BT modules, cameras, fingerprint readers, and barcode scanners. The FreeBSD driver is implemented as an extension to the existing MMC bus, adding a lot of new SDIO-specific bus methods. A prototype of the driver for the Marvell SDIO WLAN/BT (Avastar 88W8787) module is also being developed, using the existing Linux driver as a reference.

                          SDIO card detection and initialization already work; most needed bus methods are implemented and tested.

                          The WiFi driver is able to load firmware onto the card and initialize it. Migration of the MMC stack to the new locking model is necessary in order to work with SDIO cards effectively. The FreeBSD CAM implementation is believed to be a good choice. There is ongoing work to implement an MMC transport for CAM.

                          Open tasks:

                          1. SDIO stack: finish CAM migration. The XPT layer is almost ready. What is missing is a SIM module, for which a modified version of the SDHCI controller driver will be used, and a peripheral module, where porting the mmcsd(4) driver is required.
                          2. Marvell SDIO WiFi: connect it to the FreeBSD network stack, write the code to implement required functions, such as sending and receiving data, network scanning and so on.

                          UEFI Boot

                          Links
                          Project page on the wiki URL: https://wiki.freebsd.org/UEFI

                          Contact: Ed Maste <emaste@FreeBSD.org>

                          The Unified Extensible Firmware Interface (UEFI) provides boot- and run-time services for x86 computers, and is a replacement for the legacy BIOS. This project will adapt the FreeBSD loader and kernel boot process for compatibility with UEFI firmware, found on contemporary servers, desktops, and laptops.

                          Starting with Rui Paulo's i386 EFI loader, Benno Rice developed a working proof-of-concept amd64 loader in 2013 under sponsorship from the FreeBSD Foundation. After refinement, that work has now been merged from the projects/uefi Subversion branch into FreeBSD head. The project includes the infrastructure to build a UEFI-enabled loader, and the kernel-side changes to parse metadata provided by the loader.

                          A number of integration tasks remain, with a plan to have UEFI installation and boot support merged to stable/10 in time for FreeBSD 10.1-RELEASE.

                          This project was sponsored by The FreeBSD Foundation.

                          Open tasks:

                          1. Document manual installation, including dual-boot configurations.
                          2. Implement chain-loading from UFS/ZFS file systems.
                          3. Integrate UEFI configuration with the FreeBSD installer.
                          4. Support secure boot.

                          Updated vt(4) System Console

                          Links
                          Project wiki page URL: https://wiki.freebsd.org/Newcons

                          Contact: Aleksandr Rybalko <ray@FreeBSD.org>
                          Contact: Ed Maste <emaste@FreeBSD.org>
                          Contact: Ed Schouten <ed@FreeBSD.org>

                          vt(4) is a modern replacement for the existing, quite old, virtual terminal emulator called syscons(4). Initially motivated by the lack of Unicode support and infrastructural issues in syscons(4), the project was later expanded to cover the new requirement to support Kernel Mode Setting (KMS).

                          The project is now in head, stable/10 and stable/9 branches. Hence, vt(4) can be tested by using the VT kernel configuration (i386 and amd64) or by replacing two lines in the GENERIC kernel configuration file:

                          device sc
 device vga

                          with the following ones:

                          device vt
 device vt_vga

                          Or, to use for UEFI testing, add the following lines instead:

                          device vt
 device vt_efifb

                          Major highlights:

                          • Unicode support.
                          • Double-width character support for CJK characters.
                          • xterm(1)-like terminal emulation.
                          • Support for Kernel Mode Setting (KMS) drivers (i915kms, radeonkms).
                          • Support for different fonts per terminal window.
                          • Simplified drivers.

                          Brief status of supported architectures and hardware:

                          • amd64 (VGA/i915kms/radeonkms) — works.
                          • ARM framebuffer — works.
                          • i386 (VGA/i915kms/radeonkms) — works.
                          • IA64 — untested.
                          • MIPS — untested.
                          • PPC and PPC64 — work, but without X.Org yet.
                          • SPARC — works on certain hardware (e.g., Ultra 5).
                          • vesa(4) — in progress.
                          • i386/amd64 nVidia driver — not supported. VGA should be used (VESA planned).
                          • Xbox framebuffer driver — will be deleted as unused.

                          This project was sponsored by The FreeBSD Foundation.

                          Open tasks:

                          1. Create sub-directories for vt(4) under /usr/share/ to store key maps and fonts.
                          2. Implement the remaining features supported by vidcontrol(1).
                          3. Write the vt(4) manual page. (This is in progress.)
                          4. Support direct handling of keyboard by the kbd device (without kbdmux(4)).
                          5. CJK fonts. (This is in progress).
                          6. Address performance issues on some architectures.
                          7. Switch to vt(4) by default.

                          Architectures


                          bhyve

                          Links
                          bhyve FAQ and Talks URL: http://www.bhyve.org
                          Talk: bhyve Past, Present, Future URL: http://www.youtube.com/watch?v=lTOiSyu0-MA

                          Contact: Peter Grehan <grehan@FreeBSD.org>
                          Contact: Neel Natu <neel@FreeBSD.org>
                          Contact: John Baldwin <jhb@FreeBSD.org>
                          Contact: Tycho Nightingale <tychon@FreeBSD.org>
                          Contact: Allan Jude <freebsd@allanjude.com>

                          bhyve is a Type-1 hypervisor that runs on the FreeBSD platform. It currently only runs FreeBSD (9.x or later) and Linux guests; current development efforts aim at widening support for other x86 64-bit operating systems. After a great deal of work by all involved, bhyve was shipped as part of FreeBSD 10.0-RELEASE. Increased interest in bhyve and the first usable versions have provided great feedback and many bug reports.

                          A number of important improvements have been made to bhyve this quarter:

                          • Optionally ignore accesses to unimplemented MSRs
                          • Support soft power-off via the ACPI S5 state for bhyve guests
                          • Graceful shutdown via ACPI on SIGTERM
                          • Fix an issue with virtio-blk devices on Linux guests with more than 4GB of RAM
                          • Increase the block-layer backend maximum requests to match AHCI command queue depth
                          • Add SMBIOS support
                          • Improve support for nmdm, opening the tty non-blocking
                          • Add HPET device emulation
                          • Implement the Virtual Interrupt Delivery and Posted Interrupt Processing VT-x features on newer Intel CPUs
                          • Add support for booting FreeBSD/i386 guests
                          • Add virtualized XSAVE support for features like AVX
                          • Add Support for booting from ZFS with bhyveload

                          Open tasks:

                          1. Improve documentation.
                          2. Write Handbook chapter for bhyve.
                          3. Merge fixes and features back to stable/10.
                          4. Support for booting with UEFI instead of userspace loaders.
                          5. CSM BIOS boot support for FreeBSD (which has no UEFI support currently).
                          6. Add support for virtio-scsi.
                          7. Improve virtio-net, add offload features, support multiple queues.
                          8. Implement Intel 82580 and e1000 NIC emulation.
                          9. Netmap support.
                          10. Flexible networking backend: wanproxy, vhost-net.
                          11. Improve resource accounting.
                          12. Move to a single process model, instead of bhyveload and bhyve.
                          13. Support running bhyve as non-root.
                          14. Add filters for popular VM file formats (VMDK, VHD, QCOW2).
                          15. Implement an abstraction layer for video (no X11 or SDL in base system).
                          16. Support for VNC as a video output.
                          17. Implement USB and Sound.
                          18. Suspend/resume support.
                          19. Live Migration.
                          20. Nested VT-x support (bhyve in bhyve).
                          21. Support for other architectures (ARM, MIPS, PPC).

                          FreeBSD Host Support for OpenStack and OpenContrail

                          Links
                          URL: http://www.openstack.org
                          URL: http://www.opencontrail.org
                          URL: https://github.com/Semihalf/openstack-devstack
                          URL: https://github.com/Semihalf/openstack-nova
                          URL: https://github.com/Semihalf/contrail-vrouter
                          URL: https://blueprints.launchpad.net/nova/+spec/freebsd-compute-node

                          Contact: Grzegorz Bernacki <gjb@semihalf.com>
                          Contact: Michał Dubiel <md@semihalf.com>
                          Contact: Dominik Ermel <der@semihalf.com>
                          Contact: Rafał Jaworowski <raj@semihalf.com>

                          OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources in a data center. OpenContrail is a network virtualization (SDN) solution comprising a network controller, virtual router, and analytics engine, which can be integrated with cloud orchestration systems like OpenStack or CloudStack.

                          The goal of this work is to make FreeBSD a fully supported compute host for OpenStack using OpenContrail virtualized networking. The main areas of development are:

                          • Libvirt hypervisor driver for bhyve.
                          • Support for bhyve (via the libvirt compute driver) and the FreeBSD platform in overall in nova-compute.
                          • Port OpenContrail vRouter (forwarding plane kernel module) to FreeBSD.
                          • Port OpenContrail Agent (network controller node) to FreeBSD.
                          • Integration, performance optimization.

                          The current state of development allows for a working demo of OpenStack with compute node component running on a FreeBSD host:

                          • The native bhyve hypervisor is driven by a nova-compute component for spawning guest instances using libvirt and a nova-network component for providing simple networking using bridges between guest VMs.
                          • QEMU might also be used instead of bhyve this way.
                          • The main goal on the networking side is to use the OpenContrail solution, compliant with the modern OpenStack networking API ("neutron").

                          Also, an initial port of the OpenContrail vRouter kernel module has been completed. It successfully handles all networking on the host.

                          This project was sponsored by Juniper Networks.

                          FreeBSD on Chromebook

                          Links
                          Manual URL: https://wiki.freebsd.org/FreeBSD/arm/Chromebook

                          Contact: Ruslan Bukin <br@FreeBSD.org>

                          One model of Chromebook is an ARMv7 Cortex-A15 personal computer powered by a Samsung Exynos 5 Dual System-on-Chip. As of the current status of this project, such laptops can be booted with FreeBSD from USB flash — it works stably (including SMP) and it can build third-party applications. The display and keyboard work.

                          Thanks to Peter Grehan for providing hardware.

                          Open tasks:

                          1. Implement keyboard polling mode.
                          2. Add support for the upcoming second generation of Chromebook.
                          3. Write SD, SATA drivers.

                          FreeBSD/arm64

                          Links
                          Project branch in the Subversion repository URL: http://svnweb.freebsd.org/base/projects/arm64/
                          GitHub repository URL: https://github.com/zxombie/aarch64-freebsd-sandbox

                          Contact: Andrew Turner <andrew@FreeBSD.org>

                          Arm64 is the name of the in-progress port of FreeBSD to the ARMv8 CPU when it is in AArch64 mode. Until recently, all ARM CPU designs were 32-bit only. With the introduction of the ARMv8 architecture, ARM has added a new 64-bit mode. This new mode has been named AArch64.

                          Progress has been good on getting FreeBSD to build and run on the ARM Foundation model. FreeBSD is able to be built for this architecture, however, it requires a number of external tools including objdump(1) and ld(1). These tools are provided by an external copy of binutils until replacements can be written.

                          FreeBSD will run the early boot on the Foundation model. The loader has been ported to the AArch64 UEFI environment and can load and run the kernel. The kernel is able to create the initial page tables to be able to run from virtual memory. It can then execute C code to parse the memory map provided by the loader. This is as far as the kernel currently boots.

                          This work is now happening in the FreeBSD Subversion repository in a project branch, see the links.

                          Open tasks:

                          1. Implement an initial pmap(9) layer.
                          2. Write the missing machine-dependent code.
                          3. Test on real hardware.

                          FreeBSD/armv6hf

                          Contact: Andrew Turner <andrew@FreeBSD.org>

                          FreeBSD has been updated to allow it to use the VFP variant of the ARM EABI ABI. The VFP unit is the ARM hardware to perform floating-point operations. This changes the ABI to improve the performance of code that uses floating-point operations. By default, FreeBSD already uses the ARM EABI on all releases from 10.0.

                          This is important for FreeBSD/arm users running code with floating-point operations on ARMv6 or ARMv7 SoC systems. It removes the need for the slow software floating-point support in libc. This is mostly compatible with the existing ABI, with the exception of how floating-point values are passed into functions. Because no floating-point values are passed to the kernel, the armv6 and armv6hf kernels will work with either userland.

                          As part of this change, some support functions have been updated to use the VFP unit when available. The existing armv6 target architecture will be kept for cases where the SoC lacks a VFP unit, or existing binaries that are incompatible with the new ABI.

                          Open tasks:

                          1. Testing.
                          2. More testing.

                          SMP on Multi-Core ARM Systems

                          Links
                          Announcement URL: http://lists.freebsd.org/pipermail/freebsd-arm/2014-April/007886.html

                          Contact: Ian Lepore <ian@FreeBSD.org>
                          Contact: Olivier Houchard <cognet@ci0.org>
                          Contact: Wojciech Macek <wma@semihalf.com>

                          FreeBSD now supports Symmetrical MultiProcessing (SMP) on a variety of ARM multi-core systems. The effort to bring SMP to ARM has been underway for quite some time, but a major push by the FreeBSD ARM developer community over the past two months has resulted in robust production-ready SMP support.

                          An ever-growing number of ARM-based development boards and small low-power computer systems are available with multi-core processors. FreeBSD is now able to make good use of all that computing power, making such systems more attractive to both end users and vendors looking to create products based on similar designs.

                          As of r264138 in FreeBSD head, SMP is now enabled by default in the configuration files for all currently-supported systems that have multi-core processors. This includes systems based on the following processor families:

                          • Allwinner A20
                          • Freescale i.MX6
                          • Marvell Armada XP
                          • Samsung Exynos 5
                          • Texas Instruments OMAP4

                          We plan to merge this work to stable/10 in time for 10.1-RELEASE.

                          This project was sponsored by Microsemi, Inc., and Semihalf sp.j.


                          Userland Programs


                          auditdistd(8)

                          Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>

                          The auditdistd(8) daemon is responsible for distributing audit trail files over TCP/IP networks securely and reliably.

                          The daemon now supports client-side certificates, which can be used to automatically configure the receiver side — the directory name for received trail files is determined based on the commonName field in client's certificate. There is no need any more to add every sender to the receiver's configuration file.

                          The sender's functionality was extended to allow sending audit trail files to multiple receivers.

                          Complete Public Key Infrastructure (PKI) support is now implemented, including full certificate chain verification, Certificate Revocation Lists (CRL) verification at every level and support for multiple Certificate Authority (CA) certificates.

                          This project was sponsored by The FreeBSD Foundation.

                          External Toolchain Improvements

                          Links
                          Brooks Davis' XCC work URL: https://wiki.freebsd.org/ExternalToolchain

                          Contact: Warner Losh <imp@FreeBSD.org>

                          Building on the work that Brooks Davis did to enable external Clang toolchains, this project hopes to generalize that to GCC, as well as support different versions of these compilers simultaneously for the FreeBSD base system and the kernel. We also hope get to the point that a port can be cross-compiled entirely from scratch with no initial binary artifacts.

                          Open tasks:

                          1. Setup Subversion project repository.
                          2. Fix issues with differences of interpretation of the -B argument between GCC and Clang.
                          3. Support building the entire tree based only on xdev-built compilers.
                          4. Support building the entire tree based only on ports-built GCC compilers.
                          5. Support full bootstrapping of FreeBSD to new platforms.

                          Forward Port FreeBSD GCC

                          Contact: Warner Losh <imp@FreeBSD.org>

                          Not all of the FreeBSD changes to GCC have been reflected upstream. A large amount of the platform support as well as a couple of minor improvements like the kernel formatting checker need to be forward ported (and if possible, moved upstream into GCC).

                          We will be targeting the FreeBSD ports tree lang/gcc* ports for these efforts to (optionally) include them in these builds. Some variation from normal builds may be required due to bootstrapping issues when combined with the external toolchain enhancements project.

                          FreeBSD Test Suite

                          Links
                          Project page URL: http://wiki.FreeBSD.org/TestSuite
                          Testing cluster URL: http://kyua1.nyi.FreeBSD.org/
                          Roadmap for the project URL: http://julipedia.meroh.net/2014/01/freebsd-test-suite-goals-and-planning.html
                          AsiaBSDCon 2014 tutorial materials URL: https://drive.google.com/a/meroh.net/#folders/0B08g-X1kPkSYNmlEdTB5RjlFbkk

                          Contact: Julio Merino <jmmv@FreeBSD.org>

                          The FreeBSD Test Suite project aims to equip FreeBSD with a comprehensive collection of tests that are easy to run out of the box and during the development of the system. The test suite is installed into /usr/tests/ and the kyua(1) command-line tool (devel/kyua in the Ports Collection) is used to run them. See the project page for more details.

                          Since the last status report, we have been hard at work polishing the framework in many different areas. The highlights are:

                          • A roadmap for the project has been prepared and published, see links.
                          • Many tests have been added to the test suite thanks to the work of various developers and, in particular, a good bunch of old tests from src/tools/regression/ have been incorporated into the new test suite. As of this writing, there are 509 test cases continuously running.
                          • The testing infrastructure in the stable/10 branch has been synced to head. It should now be possible to seamlessly MFC changes to the stable branch along with their tests, if any.
                          • The testing cluster, which only issued amd64 builds, has been extended to perform i386 builds as well. Additionally, a canary machine has been put in place so that changes to the cluster configuration can be properly validated before deployment.
                          • A tutorial on Kyua and the FreeBSD Test Suite was given at AsiaBSDCon 2014. The tutorial materials are available for public consumption, please consult the links.
                          • Both Kyua's and ATF's upstream sites have been moved to GitHub, mostly due to the discontinuation of file downloads in Google Code.

                          Open tasks:

                          1. Enable the build of the test suite by default.
                          2. Add alerting for failed or missing test runs from the testing cluster.
                          3. Add bhyve support to the testing cluster for faster turnaround times.
                          4. Simplify and improve Kyua HTML reports. In particular, reports will be coalesced into single HTML files for easier management and will include more useful details for debugging. Such details are the revision at which the system was built, the date and duration of the whole run, or the list of installed packages, to mention a few examples.
                          5. Add JUnit XML output to Kyua for better integration with Jenkins. This work is actively ongoing and should be ready for prime time at BSDCan 2014.

                          LLDB Debugger Port

                          Links
                          URL: https://wiki.FreeBSD.org/lldb

                          Contact: Ed Maste <emaste@FreeBSD.org>

                          LLDB is the debugger project associated with Clang/LLVM. It supports the Mac OS X, Linux, and FreeBSD platforms, with ongoing work on Windows. It builds on existing components in the larger LLVM project, for example using Clang's expression parser and LLVM's disassembler.

                          The majority of work since the last status update has been on bugfixes and implementation of the remaining functionality missing on FreeBSD. Most of these improvements are now in the LLDB snapshot in the base system, which has been updated to upstream Subversion revision r202189. Some highlights of the new update include:

                          • Improvements to the remote GDB protocol client.
                          • Bug fixes for big-endian targets.
                          • Initial support for libdispatch (GCD) queues in the debuggee.
                          • Add "step-avoid-libraries" setting.
                          • IO subsystem improvements (including initial work on a curses GUI).
                          • Support hardware watchpoints.
                          • Improved unwinding through hand-written assembly functions.
                          • Handle DW_TAG_unspecified_parameters for variadic functions.
                          • Fix Ctrl+C interrupting a running inferior process.
                          • Various bug fixes for memory leaks, LLDB segfaults, the C++ demangler, ELF core files, DWARF debug info, and others.

                          LLDB is currently not yet built by default and may be enabled by adding WITH_LLDB= to src.conf(5). A port will be made available for those who wish to track ongoing development more closely.

                          This project was sponsored by DARP/AFRL, SRI International, and University of Cambridge.

                          Open tasks:

                          1. Add support for remote debugging (gdbserver-compatible debugserver).
                          2. Add support for local and core file kernel debugging.
                          3. Implement, fix or test support on all non-amd64 architectures.
                          4. Verify cross-debugging.
                          5. Investigate and fix test suite failures.
                          6. Package LLDB as a port.
                          7. Enable by default in the base system for working architectures.

                          Ports


                          Chromium

                          Links
                          Chromium website URL: http://www.chromium.org/Home
                          Development repository on GitHub URL: https://github.com/gliaskos/freebsd-chromium
                          Chromium on FreeBSD wiki URL: https://wiki.freebsd.org/Chromium

                          Contact: Chromium on FreeBSD Team <freebsd-chromium@FreeBSD.org>

                          Chromium is the open source web browser project from which Google Chrome draws its source code. The browsers share the majority of code and features, though there are some minor differences in features and they have different licensing. Over the last four years, the Chromium team has been busy with porting Chromium to FreeBSD. This involves patching the browser so that it runs on FreeBSD, tracking and documenting security updates, and merging patches back upstream.

                          While there are already several browsers available for FreeBSD, advantages of Chromium are:

                          • Quick response from upstream to security issues, resulting in approximately bi-weekly updates.
                          • A testbed for security features of FreeBSD, like Capsicum. While support for this capability and sandbox framework is currently not included in the browser, a proof-of-concept implementation for an early version of Chromium was realized within a single weekend.

                          George Liaskos and René Ladan are currently busy with submitting the remaining patches specific to FreeBSD back upstream. Apart from making future updates easier, it sometimes also improves the overall code quality.

                          Jonathan Anderson recently updated the Capsicum patches for Chromium and is talking to upstream about them.

                          Open tasks:

                          1. Advocate FreeBSD. While patches are getting accepted by both humans and bots, it is not an official platform so attitude varies from developer to developer. While René Ladan thinks it is a bit early, it might be fruitful to investigate what is required to make FreeBSD (and possibly OpenBSD) an official platform in terms of both hardware and procedures.
                          2. If you feel comfortable with large source trees, you can try to build the Git version of Chromium on FreeBSD. If you are also comfortable with signing Google's Contributor License Agreement, you can join in testing and submitting patches upstream.

                          FreeBSD Ada Ports

                          Links
                          URL: http://www.dragonlace.net
                          SPARK 2014 URL: http://www.spark-2014.org/about/

                          Contact: John Marino <marino@FreeBSD.org>

                          Ada is a structured, statically typed, imperative, wide-spectrum, and object-oriented high-level computer programming language, extended from Pascal and other languages, originally targeted at embedded and real-time systems. The number of Ada ports in the collection has grown significantly since the last report six months ago. There are almost 50 Ada-related ports now, with new ones getting added all the time.

                          The previous plan was to move from the GCC 4.7-based GNAT compiler to a GCC 4.8-based one, but finally GCC 4.8 was skipped and now a GCC 4.9-based GNAT is the standard Ada compiler, which fully supports the new ISO standard, Ada 2012. Moving to a newer compiler allowed several important ports like PolyOrb and GPRBuild to be upgraded to the latest available versions. In fact, almost every Ada port is currently at its most recent upstream version.

                          For non-Windows-based Ada development, FreeBSD and DragonFly are now undisputed as the go-to platforms. The other candidates are Debian and Fedora, but there are few Ada softwares on those platforms that are not also in the FreeBSD ports tree, and the FreeBSD versions are much newer. The Ports Collection also features software not found anywhere else such as the USAFA's Ironsides DNS server, libsparkcrypto, matreshka, GNATDroid (Android cross-compiler) and several developer libraries.

                          A desired addition to the Ada ports will be SPARK 2014 (see links), which should cement FreeBSD as an option for professional, safety-critical application development. This package should have its first release by early summer.

                          GCC in the Ports Collection

                          Links
                          Upstream GCC URL: http://gcc.gnu.org

                          Contact: Gerald Pfeifer <gerald@FreeBSD.org>

                          While the age old version of the GNU Compiler Collection (GCC) in the base system is on its way out with FreeBSD 10 and later, there are many users who want—and some platforms which need—to use GCC.

                          For that purpose there are various versions of GCC in the ports tree, including lang/gcc46, lang/gcc47, lang/gcc48 and lang/gcc49 which track upstream snapshots of the respective release branches, and more importantly lang/gcc which serves as the canonical version of GCC and is the default when a port requests USE_GCC=yes as well as for some cases of USES=compiler.

                          With a lot of help from Christoph Moench-Tegeder who fixed many ports and made a fair number respect CXXFLAGS, LDFLAGS and friends, we managed to update the canonical version from GCC 4.6.4 to GCC 4.7.3. Many of Christoph's fixes also benefit Clang and other modern compilers.

                          For users of lang/gcc, this upgrade proved very smooth, and we generally recommend using this port over version specific ones.

                          After ten years of service lang/gcc34 retired, as did lang/gcc44 after half that timespan.

                          On a related note, with the help of John Marino, the license of the GCC ports now properly reflects the combination of GPLv3 for the compiler itself and GPLv3 with GCC Runtime Library Exception for the runtime. The latter is the key in making it possible to use GCC for building and distributing non-free software.

                          Open tasks:

                          1. Move lang/gcc from GCC 4.7 to GCC 4.8.

                          GNOME/FreeBSD

                          Links
                          GNOME FreeBSD page URL: http://www.freebsd.org/gnome
                          JHbuild info and results URL: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD
                          MATE staging repository (might break) URL: https://github.com/jlmess77/mate-ports
                          GNOME staging repository (might break) URL: http://marcuscom.com/downloads/marcusmerge

                          Contact: FreeBSD GNOME Team <gnome@FreeBSD.org>

                          GNOME is a desktop environment and graphical user interface that runs on top of a computer operating system. GNOME is part of the GNU Project and can be used with various Unix-like operating systems, including FreeBSD.

                          Preparations for merging GNOME 3 are moving forward. The work on the documentation is falling behind a bit, but we got some solid feedback on the rough work to keep this moving forward as well. In the meantime, deprecation of ports that need the old GNOME 2 desktop ports has begun. These ports will break when the GNOME desktop components are updated to the GNOME 3 version.

                          Thanks to a combined effort by Ryan Lortie (GNOME developer), Ting-Wei Lan (upstream contributor), and Koop Mast, we now have a FreeBSD-powered JHbuild tinderbox. JHbuild is a build system that allows building GNOME upstream code. Twice a day, it will attempt to build Gnome components from a specific branch, usually the git master branch, to catch compile issues. A positive side effect is that it lets upstream know GNOME still lives on non-Linux systems. It also exposes the GNOME code base to the Clang compiler and libc++. Since the start of this project over a hundred issues have been fixed.

                          Gustau Perez has stepped up and put together a port set in the ports-experimental tree of our development repository with GNOME 3.12. It was decided to polish GNOME 3.12. It will be merged when the preparation work has (mostly) finished, and we are happy with the stability of GNOME 3.12.

                          Gustau Perez also ported Cinnamon 2.0 to FreeBSD. It will appear in the Ports Collection after GNOME 3 has been merged.

                          MATE 1.8 was released at the beginning of April, Eric Turgeon of GhostBSD had volunteered to do that update for FreeBSD. Note that this update is still based on GTK+, version 2. The GTK+ 3-based MATE is on the roadmap for 1.10.

                          Open tasks:

                          1. Finish the work needed to be done before GNOME 3 can be merged at all. Documentation work, port deprecation, and so on.
                          2. Finish porting of MATE 1.8.
                          3. Update Cairo to 1.12 in coordination with the Graphics Team.

                          KDE/FreeBSD

                          Links
                          KDE/FreeBSD Home Page URL: http://FreeBSD.kde.org
                          area51 URL: http://FreeBSD.kde.org/area51.php
                          PortScout Status URL: http://portscout.freebsd.org/kde@freebsd.org.html

                          Contact: KDE/FreeBSD Team <kde@FreeBSD.org>

                          KDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Solaris, Microsoft Windows, and OS X systems. The KDE/FreeBSD Team have continued to improve the experience of KDE software and Qt under FreeBSD.

                          During this quarter, the team has kept most of the KDE and Qt ports up-to-date, working on the following releases:

                          • KDE SC: 4.12.2, 4.12.3, and 4.12.4; Workspace: 4.11.6, 4.11.7, and 4.11.8
                          • Qt: 5.2.1
                          • KDevelop: 4.6.0
                          • Digikam (and KIPI-plugins): 3.5.0

                          As a result — according to PortScout — kde@ has 526 ports (up from 464), of which 98.86% are up-to-date (up from 88.15%). iXsystems continues to provide a machine for the team to build packages and to test updates. They have been providing the KDE/FreeBSD team with support for quite a long time and we are very grateful for that.

                          A major change has been the deprecation of the KDE3 ports and the move of the KDE4_PREFIX to LOCALBASE. Also, work on Qt5 continues to maturity. Raphael Kubo da Costa has been working with upstream to ensure Baloo (Nepomuk successor in KDE SC 4.13) compiles and runs on non-Linux systems. His work not only benefits FreeBSD but other BSDs and OS X.

                          As usual, the team is always looking for more testers and porters, so please contact us and visit our home page (see links). It would be especially useful to have more helping hands on tasks such as getting rid of the dependency on the defunct HAL project and providing integration with KDE's Bluedevil Bluetooth interface.

                          This project was sponsored by iXsystems, Inc.

                          Open tasks:

                          1. Update out-of-date ports, see PortScout for a list.
                          2. Work on Qt 5.
                          3. Make sure the whole KDE stack (including Qt) builds and works correctly with Clang and libc++.
                          4. Remove the dependency on HAL.

                          libvirt/bhyve Support

                          Links
                          bhyve Driver URL: http://libvirt.org/drvbhyve.html
                          libvirt Home Page URL: http://libvirt.org/
                          Developer Blog URL: http://empt1e.blogspot.ru/search/label/libvirt

                          Contact: Roman Bogorodskiy <novel@FreeBSD.org>

                          Libvirt is a virtualization library providing a common API for various hypervisors (Qemu/KVM, Xen, LXC, and others), and also a popular library used by a number of projects. Libvirt 1.2.2, released on March, 2014, was the first release to include bhyve support. Enabling bhyve support allows consumers to use bhyve in libvirt-ready applications without major efforts.

                          Currently, libvirt supports almost all essential features of bhyve, such as Virtual Machine lifecycle (start, stop), bridged networking, and virtio/SATA driver support. The work continues to implement more API calls and to cover more of features offered by bhyve.

                          Open tasks:

                          1. FreeBSD port of netcf is needed for adding interface driver support to libvirt.

                          OpenAFS on FreeBSD

                          Links
                          OpenAFS homepage URL: http://openafs.org

                          Contact: Benjamin Kaduk <bjk@FreeBSD.org>

                          AFS is a distributed network filesystem that originated from the Andrew Project at Carnegie-Mellon University. OpenAFS is an open-source implementation of the AFS protocol derived from IBM AFS, which was released under the IBM Public License. OpenAFS on FreeBSD (the net/openafs port) is suitable for light use, but is not yet production ready.

                          We got a chance to pick up this porting project after some hiatus. Recent work focused on investigating the bugs preventing the use of a disk cache for caching file data. An internal "lookupname" abstraction was intended to return an unlocked, referenced vnode, but instead returned a locked, referenced vnode, leading to various failure modes depending on the number of kernel debugging options enabled.

                          Open tasks:

                          1. Track down an issue involving incorrect reference counts on the AFS root vnode that cause warnings on shutdown.
                          2. Audit the locking in all the vnode operations code — it is expected that there remain some incorrectly locked areas, though none that present visible issues under light load.

                          The Graphics Stack on FreeBSD

                          Links
                          Graphics stack roadmap and supported hardware matrix URL: https://wiki.freebsd.org/Graphics
                          WITH_NEW_XORG status URL: https://wiki.freebsd.org/Graphics/WITH_NEW_XORG
                          Ports-related development repository URL: http://trillian.chruetertee.ch/ports/browser/trunk

                          Contact: FreeBSD Graphics Team <graphics-team@FreeBSD.org>

                          On the kernel side, the Radeon KMS driver was merged in stable/9 and will be available in FreeBSD 9.3-RELEASE. Now both the 9.x and 10.x branches share the same support for Intel and AMD GPUs.

                          The next big tasks are the updates of the DRM generic code and the i915 driver. Both are making good progress and the DRM update should hopefully be ready for wider testing during April. An update of the Radeon driver is on the to-do list, but nothing is scheduled yet.

                          On the ports tree and packages side, the update to Cairo 1.12 mentioned in the last quarterly report is ready to be committed, as people who tested it either reported improvements or no regressions. As a reminder, the switch from Cairo 1.10 to 1.12 causes display artifacts with xf86-video-intel 2.7.1, but fixes similar problems with other hardware/driver combinations. Furthermore, Cairo 1.12 is required by Pango 1.36.0, GTK+ 3.10 and Firefox 27.0. A Heads up mail will be posted to the freebsd-x11 mailing-list when this update goes live.

                          In the graphics stack's ports development tree, new Mesa ports are being worked on. Those ports are required to support GLAMOR (the GL-based 2D acceleration library used by Radeon HD 7000+ cards for instance) and OpenCL (using the GPU to perform non-graphical calculations). We were able to execute some "Hello World" OpenCL programs and play with OpenCL in darktable, but there are some compatibility issues between Clover (Mesa's libOpenCL implementation) and Clang/libc++.

                          We are preparing an alternate pkg(8) repository with packages built with WITH_NEW_XORG. The goal is to ease the usage of the KMS drivers and move forward with the graphics stack updates. The main pkg(8) repository will still use the default setting (WITH_NEW_XORG set on head, but not on the stable branches).

                          This will pave the way to the deprecation ofWITH_NEW_XORG and the removal of the older stack. The current plan is to do this after 10.0-RELEASE End-of-Life, scheduled on January 31st, 2015. By that time, the only supported releases will be 8.4-RELEASE, 9.3-RELEASE and 10.1-RELEASE. FreeBSD 9.3 and 10.1 will be fully equipped to work with the newer stack. Unfortunately, FreeBSD 8.x misses the required kernel DRM infrastructure: supporting X.Org here cripples progress on the graphics stack and, once WITH_NEW_XORG is gone, we will not support 8.x as a desktop any more. Therefore, please upgrade to 9.3 or 10.1 when they are available.

                          Open tasks:

                          1. See the Graphics and WITH_NEW_XORG wiki pages for up-to-date information.

                          Using CentOS 6.5 as Linux Base

                          Links
                          Work in Progress URL: http://github.com/xmj/linux-ports
                          ports/187786 URL: http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/187786

                          Contact: Johannes Meixner <xmj@chaot.net>

                          The Linux emulation layer relies on a Linux base distribution along with Linux ports of relevant non-base software. Fedora 10 was imported in 2006, and it shows — current Linux software like Skype 4, Sublime Text 2, or even modern games fail to run with the provided libraries.

                          CentOS 6.5 was released in December 2013 and will be supported until 2017, making it an ideal basis for an update to the ports infrastructure. Built upon the work of Carlos Jacobo Puga Medina, all ports using Linux have been updated to work with either Fedora 10 or CentOS 6.5.

                          The goal of this project is to make CentOS 6.5 the default Linux distribution, so that FreeBSD users can enjoy running modern Linux binaries without having to resort to virtualization à la VirtualBox, or even dual-booting.

                          This project was sponsored by Goldener Grund OÜ.

                          Open tasks:

                          1. Clean up Mk/bsd.linux-*.mk and fix errors detected in ports/187786.
                          2. Revert making c6 the default (in the git repository).
                          3. Testing.
                          4. Review patches and import into the ports tree (any help appreciated).
                          5. Make c6 the default (after sufficient testing) within the ports tree.

                          Wine/FreeBSD

                          Links
                          Wine Wiki Page URL: http://wiki.FreeBSD.org/Wine
                          Wine on amd64 Wiki Page URL: http://wiki.FreeBSD.org/i386-Wine
                          Wine Home Page URL: http://www.winehq.org/

                          Contact: Gerald Pfeifer <gerald@FreeBSD.org>
                          Contact: David Naylor <dbn@FreeBSD.org>

                          Wine is a free and open source software application that aims to allow applications designed for Microsoft Windows to run on Unix-like operating systems, such as FreeBSD. The Wine project has been in maintenance mode this quarter and has updated the ports for the following versions:

                          • Stable releases: 1.6.2
                          • Development releases: 1.7.9 through 1.7.15

                          The ports have packages built for amd64, available through the ports emulators/i386-wine and emulators/i386-wine-devel.

                          Open tasks:

                          1. See the Open Tasks and Known Problems sections on the Wine wiki page.
                          2. FreeBSD/amd64 integration, consult the i386-Wine wiki page for the details.
                          3. Port WoW64 (supporting Windows 32-bit and 64-bit from the same port) and Wine64.

                          Xfce/FreeBSD

                          Links
                          URL: https://wiki.freebsd.org/Xfce
                          Development repository URL: https://svn.redports.org/olivierd/xfce4/
                          URL: https://people.freebsd.org/~olivierd/xfce-core-unstable.html
                          ports/183690 URL: http://www.freebsd.org/cgi/query-pr.cgi?pr=183690

                          Contact: FreeBSD Xfce Team <xfce@FreeBSD.org>

                          Xfce is a free software desktop environment for Unix and Unix-like platforms, such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use. The Xfce team continues to keep each piece of the Xfce Desktop up to date.

                          The latest commits concerned:

                          • Applications:
                            • Midori (0.5.7)
                            • xfburn (0.5.0)
                            • xfce4-parole (0.5.4)
                            • xfce4-taskmanager (1.0.1)
                            • xfce4-tumbler (0.1.30)
                          • Panel plugins:
                            • xfce4-clipman-plugin (1.2.5)
                            • xfce4-equake-plugin (1.3.4)
                            • xfce4-wavelan-plugin (0.5.11)
                            • xfce4-whiskermenu-plugin (1.3.2)

                          We also follow development of core components (available in your repository). See the links for documentation on how to upgrade those libraries.

                          • garcon (0.3.0)
                          • libxfce4menu (4.11.1)
                          • libxfce4util (4.11.0)
                          • xfce4-appfinder (4.11.0)
                          • xfce4-desktop (4.11.4)
                          • xfce4-dev-tools (4.11.0)
                          • xfce4-panel (4.11.0)
                          • xfce4-parole (0.6.0)
                          • xfce4-settings (4.11.2)
                          • xfce4-session (4.11.0)
                          • xfce4-wm (4.11.1)
                          • xfce4-xkb-plugin (0.7.0)

                          Open tasks:

                          1. Add support of DragonFly for xfce4-taskmanger.
                          2. Finish replacing Tango icon theme with GNOME, in order to close ports/183690 (see links, Midori remains to be fixed).

                          Documentation


                          ZFS Chapter of the Handbook

                          Links
                          Preview ZFS Handbook URL: http://www.allanjude.com/zfs_handbook/zfs.html
                          Slides from AsiaBSDCon 2014 URL: http://www.allanjude.com/talks/AsiaBSDCon_2014_-_WIP_-_ZFS_Handbook.pdf

                          Contact: Allan Jude <freebsd@allanjude.com>
                          Contact: Benedict Reuschling <bcr@FreeBSD.org>
                          Contact: Warren Block <wblock@FreeBSD.org>

                          ZFS is one of the premier features of FreeBSD. The current documentation in the Handbook and elsewhere online is severely lacking. Much of the original documentation from Sun and Oracle has disappeared, moved, or is about the proprietary version of ZFS.

                          New users have many questions about ZFS and yet there exists a great deal more bad advice about ZFS than proper documentation. The current ZFS chapter of the FreeBSD Handbook starts off with the required steps to configure an i386 machine to run ZFS. This is more likely to scare off a new user than to educate them about how to properly use ZFS.

                          At BSDCan 2013, the process of writing an entirely new chapter of the Handbook on ZFS was started. Currently this chapter consists of approximately 16,000 words covering all subcommands of the zpool(8) and zfs(8) utilities, delegation, tuning and a section devoted to definitions and explanations of the terms and features of ZFS.

                          The remaining section is the FAQ, to help users address the most common problems they might run into with ZFS. It would be useful to hear experiences, questions, misconceptions, gotchas, stumbling blocks, and suggestions for the FAQ section from other users. Also, it would be good to have a use cases section that highlights some of the cases where ZFS provides advantages over traditional file systems.

                          Please send suggestions to the freebsd-doc mailing list.

                          This project was sponsored by ScaleEngine, Inc.

                          Open tasks:

                          1. Technical review by Matt Ahrens (co-creator of ZFS).
                          2. Improve delegation section.
                          3. Improve tuning section, add new sysctls added in head.
                          4. Add section on jails and the jailed property.
                          5. Add FAQ section.
                          6. Add Use Cases section.
                          7. General editing and review.

                          Miscellaneous


                          FreeBSD Participating in Summer of Code 2014

                          Links
                          URL: http://gsoc.FreeBSD.org
                          URL: https://wiki.freebsd.org/SummerOfCode2014

                          Contact: Gavin Atkinson <gavin@FreeBSD.org>
                          Contact: Glen Barber <gjb@FreeBSD.org>
                          Contact: Wojciech Koszek <wkoszek@FreeBSD.org>

                          FreeBSD is pleased to have been accepted as a participating organization in Google's Summer of Code 2014. This will be the tenth time we have participated in the program, having been selected to participate every year since its introduction.

                          This year, the administrators made a special attempt to spread the word about Summer of Code around universities, including making contact with around 350 mainly Polish, British, African and American universities to advertise the Summer of Code program, with a particular focus on FreeBSD's participation. We made contact with both technical departments and student societies. Posters were produced in several languages, and FreeBSD committers and users were encouraged to distribute these posters around their local universities.

                          FreeBSD received a total of 39 proposals from students, and were subsequently granted 15 slots from Google. We are now facing the unpleasant challenge of trying to decide which of the 39 proposals to select, taking into account the quality, desirability and feasibility of each proposal, as well as ensuring we will be able to provide an excellent mentoring experience to each selected student. All mentors have volunteered to mentor, and we pair students with mentors primarily based on the prospective mentor's areas of expertise, interest in the project, also taking into account the desire to pair students up with mentors in similar time zones in order to improve the student experience. The final list of accepted students is expected to be announced on the 21st April.

                          The FreeBSD Foundation

                          Links
                          URL: http://www.FreeBSDFoundation.org/
                          FreeBSD Journal URL: http://freebsdjournal.com/

                          Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                          The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of the funding is used to support FreeBSD development projects, conferences and developer summits, purchase equipment to grow and improve the FreeBSD infrastructure, and provide legal support for the Project.

                          We published the first issue of the FreeBSD Journal, our new on-line FreeBSD magazine. The positive feedback from both the FreeBSD and outside communities has been incredible. This quarter we began work on articles and promotion for the second issue. We also started working on a dynamic version of the magazine that can be read in many web browsers including those that run on FreeBSD.

                          This year we are earmarking more funding towards FreeBSD advocacy and education. You will see more literature, white papers, articles, and so on to help promote FreeBSD.

                          The Foundation held a board meeting in Berkeley, California, in January. We discussed longer term strategy and planning for the year. We put together our 2014 budget with a plan of raising at least $1,000,000 and spending $900,000.

                          Two Foundation funded projects were completed. The first, co-sponsored by Google, integrated the Casper daemon into FreeBSD. The second was auditdistd(8) improvements for the FreeBSD cluster.

                          Work continued on these Foundation-sponsored projects: Intel graphics driver update by Konstantin Belousov, UEFI boot support for amd64 by Ed Maste, autofs automounter and in-kernel iSCSI stack enhancements and bug fixes by Edward Tomasz Napierała, and updated vt(4) system console by Aleksandr Rybalko. A more detailed project update for each of the above projects can be found within this quarterly status report.

                          We were a Gold Sponsor for NYCBSDCon 2014 in New York, February 8, which was attended by several board members. We were represented at SCALE in Los Angeles, February 22-23, and ICANN in Singapore, March 22-25.

                          We were a sponsor for AsiaBSDCon in Tokyo, March 15-16. Board member Hiroki Sato was the conference organizer. Board members Kirk McKusick and George V. Neville-Neil taught tutorials and Kirk gave a keynote. Board member Dru Lavigne manned the foundation table and spoke at one of the sessions.

                          We became a Gold+ sponsor for BSDCan 2014, May 16-17 and have started reaching out to vendors to attend the developer summit that runs in the two days before BSDCan.

                          Board members George, Kirk, and Robert Watson pushed to finish the final draft of the next edition of their book The Design and Implementation of the FreeBSD Operating System.

                          ITWire editor Sam Varghese published an interview with Kirk and Foundation technical manager Ed Maste about the status of secure boot on FreeBSD.

                          The FreeBSD Logo is now officially a registered trademark to represent the FreeBSD operating system. We are working to expand the registration beyond just the FreeBSD operating system, but currently still have to use the TM symbol when using it on apparel and other non-operating-system items. We continued reviewing requests and granting permission to use FreeBSD trademarks.

                          After finishing the 10.0-RELEASE, Foundation system administrator and release engineer Glen Barber began work on adding support for FreeBSD/arm image builds as part of the release build process. As a result of this work, FreeBSD/arm images are produced as part of the weekly development snapshot builds, and are available from any of the FreeBSD FTP mirrors. Supported kernel configurations currently include BEAGLEBONE, RPI-B, PANDABOARD, WANDBOARD-QUAD, and ZEDBOARD.

                          George visited six large FreeBSD users in the Bay Area in February. These meetings are conducted to help facilitate collaboration between FreeBSD customers and the FreeBSD Project. It is an opportunity to exchange information on what the customers are doing and what is being worked on in the Project. It is also an opportunity to try to connect customers with the appropriate FreeBSD developers who may be working on areas of FreeBSD that interest these customers.

                          News Home | Status Home
                          diff --git a/website/content/en/status/report-2014-04-2014-06.html b/website/content/en/status/report-2014-04-2014-06.html index 1a099e3fc5..010eac2e46 100644 --- a/website/content/en/status/report-2014-04-2014-06.html +++ b/website/content/en/status/report-2014-04-2014-06.html @@ -1,1355 +1,1355 @@ FreeBSD Quarterly Status Report
                          Skip site navigation (1) Skip section navigation (2)

                          Introduction

                          This report covers FreeBSD-related projects between April and June 2014. This is the second of four reports planned for 2014.

                          The second quarter of 2014 was a very busy and productive time for the FreeBSD Project. A new FreeBSD Core Team was elected, the FreeBSD Ports Management Team branched the second quarterly stable branch, the FreeBSD Release Engineering Team was in the process of finalizing the FreeBSD 9.3-RELEASE cycle, and many exciting new features have been added to FreeBSD.

                          Thanks to all the reporters for the excellent work! This report contains 24 entries and we hope you enjoy reading it.

                          The deadline for submissions covering the period from July to - September 2014 is October 7th, 2014.

                          FreeBSD Team Reports

                          Projects

                          Kernel

                          • PostgreSQL Performance Improvements
                          • Running FreeBSD as an Application on Top of the Fiasco.OC + September 2014 is October 7th, 2014.

                            FreeBSD Team Reports

                            Projects

                            Kernel

                            Architectures

                            Ports

                            Documentation

                            Miscellaneous


                              FreeBSD Team Reports


                              FreeBSD Core Team

                              Contact: FreeBSD Core Team <core@FreeBSD.org>

                              The FreeBSD Core Team constitutes the project's "Board of Directors", responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape.

                              Topics for core this quarter have included some far-reaching policy reviews and some significant changes to the project development methodology.

                              In May, a new release policy was published and presented at the BSDCan developer conference by John Baldwin. The idea is that each major release branch (for example, 10.X) is guaranteed to be supported for at least five years, but individual point releases on each branch, like 10.0-RELEASE, will be issued at regular intervals and only the latest point release will be supported.

                              Another significant change did not receive approval. When the change to the Bylaws reforming the core team election process was put to the vote of all FreeBSD developers, it failed to reach a quorum.

                              June saw the culmination of a long running project to replace the project's bug tracking system. As of June 3, the FreeBSD project has switched to Bugzilla as its bug tracking system. All of the history of GNATS PRs has been preserved, so there is no need to re-open old tickets. Work is still going on to replicate some of the integration tweaks that had been applied to GNATS, but all necessary functionality has been implemented and the project is already seeing the benefits of the new capabilities brought by Bugzilla.

                              An election to select core members for the next two year term of office took place during this period. We would like to thank retiring members of core for their years of service. The new core team provides continuity with previous core teams: about half are incumbents from the previous team, and several former core team members have returned after a hiatus. Core now includes two members of the FreeBSD Foundation board and one other Foundation staff member, aiding greater coordination at the top level of the project. At the same time the core-secretary role was passed on to a new volunteer.

                              Other activities included providing consultation on licensing terms for software within the FreeBSD source tree, and oversight of changes to the membership of postmaster and clusteradm.

                              Three new src commit bits were issued during this quarter, and one was taken into safekeeping.

                              -

                              FreeBSD Port Management Team

                              Links
                              +

                              FreeBSD Ports Management Team

                              Links
                              URL: http://www.FreeBSD.org/ports/
                              URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
                              URL: http://portsmon.freebsd.org/index.html
                              URL: http://www.freebsd.org/portmgr/index.html
                              URL: http://blogs.freebsdish.org/portmgr/
                              URL: http://www.twitter.com/freebsd_portmgr/
                              URL: http://www.facebook.com/portmgr
                              URL: http://plus.google.com/communities/108335846196454338383

                              Contact: Frederic Culot <portmgr-secretary@FreeBSD.org>
                              - Contact: FreeBSD Port Management Team <portmgr@FreeBSD.org> + Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

                              The ports tree slowly approaches the 25,000 ports threshold, while the PR count is slightly below 1800.

                              In Q2 we added three new committers, took in one commit bit for safekeeping, and reinstated one commit bit.

                              In May, Thomas Abthorpe was replaced by Frederic Culot as portmgr secretary, and Steve Wills became a member of the portmgr team.

                              Commencing July 1, the third intake of portmgr-lurkers started active duty on portmgr for a four month duration. The next two candidates are William Grzybowski and Nicola Vitale.

                              This quarter also saw the release of the second quarterly branch, namely 2014Q2. This branch was not only built for 10 (as 2014Q1) but for 9 as well (both i386 and amd64).

                              Open tasks:

                              1. As previously noted, many PRs continue to languish, we would like to see committers dedicate themselves to closing as many as possible.

                              FreeBSD Release Engineering Team

                              Links
                              FreeBSD 9.3-RELEASE schedule URL: http://www.freebsd.org/releases/9.3R/schedule.html
                              FreeBSD development snapshots URL: http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/

                              Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

                              The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, and announcing code freezes and maintaining the respective branches, among other things.

                              In early May, the FreeBSD 9.3-RELEASE cycle entered the code slush phase. The FreeBSD 9.3-RELEASE cycle is nearing the final phases, and 9.3-RC3 builds will be starting soon. 9.3-RC3 is planned to be the final release candidate for this release cycle, and at the time of this writing, 9.3-RELEASE should be available on schedule.

                              Work is ongoing to integrate support for embedded architectures into the release build process. At this time, support exists for a number of ARM kernels, in particular the Raspberry Pi, BeagleBone, and WandBoard.

                              Additionally, work is in progress to produce virtual machine images as part of the release cycle, supporting various cloud services such as Microsoft Azure, Amazon EC2, and Google Compute Engine.

                              This project was sponsored by The FreeBSD Foundation.


                              Projects


                              Chelsio iSCSI Offload Support

                              Links

                              Contact: Sreenivasa Honnur <shonnur@chelsio.com>

                              Building on the new in-kernel iSCSI target and initiator stack released in FreeBSD 10.0, Chelsio Communications has begun developing an offload interface to take advantage of the hardware offload capabilities of Chelsio T4 and T5 10 and 40 gigabit Ethernet adapters.

                              The code currently implements a working prototype of offload for the initiator side, and target side offload should begin shortly. The code will be released under the BSD license and is expected to be completed later in the year and be committed to FreeBSD-HEAD, and will likely ship in a FreeBSD release in early 2015.

                              Open tasks:

                              1. Complete testing and debugging of the initiator offload.
                              2. Start development of target offload.
                              3. Create hardware-independent offload APIs, based on experiences with target and initiator proof-of-concept implementations.

                              CUSE4BSD

                              Links
                              Commit URL: http://svnweb.freebsd.org/changeset/base/266581

                              Contact: Hans Petter Selasky <hselasky@FreeBSD.org>

                              The so-called CUSE4BSD has been imported into the base system of FreeBSD-11. CUSE is short for character device in userspace. The CUSE library is a wrapper for the devfs(8) kernel functionality which is exposed through /dev/cuse. In order to function, the CUSE kernel code must either be enabled in the kernel configuration file or loaded separately as a module. Follow the commit message link to get more information.

                              FreeBSD and Summer of Code 2014

                              Links
                              URL: http://gsoc.FreeBSD.org
                              URL: https://wiki.freebsd.org/SummerOfCode2014

                              Contact: Gavin Atkinson <gavin@FreeBSD.org>
                              Contact: Glen Barber <gjb@FreeBSD.org>
                              Contact: Wojciech Koszek <wkoszek@FreeBSD.org>

                              FreeBSD received 39 project proposals this year, many of which were of a very high standard. After a difficult selection process narrowing these down into the slots we had been allocated, a total of 16 projects were selected to participate in Google Summer of Code 2014 with FreeBSD.

                              The projects selected span a wide range of areas within FreeBSD, covering both the base system and ports infrastructure, userland and kernel. We have students working on firewall optimisation, ports packaging tools, embedded systems, debugging infrastructure, improved Unicode support, enhancements to the loader and to the installer, and several other areas of work. We are just over halfway through the allocated time this year, and are very much looking forward to integrating code produced by these projects into FreeBSD.

                              This is the tenth time FreeBSD has taken part in Google's Summer of Code, and we are grateful to Google to have accepted us as a participating organisation.

                              New Automounter

                              Links

                              Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                              Deficiencies in the current automounter, amd(8), are a recurring problem reported by many FreeBSD users. A new automounter is being developed to address these concerns.

                              The automounter is a cleanroom implementation of functionality available in most other Unix systems, using proper kernel support implemented via an autofs filesystem. The automounter supports a standard map format, and will integrate with the Lightweight Directory Access Protocol (LDAP) service.

                              The project is at the early testing stage. A patch will be released as part of a broader call for testing after additional review on some critical components (in particular, the autofs filesystem). After fixing reported problems, the code will be committed to FreeBSD 11-CURRENT. It is expected to ship in the FreeBSD 10.2 release.

                              This project was sponsored by The FreeBSD Foundation.

                              Open tasks:

                              1. Fix bad interaction with fts(3).
                              2. Debug a problem with Kerberos NFS mounts.

                              pkg(8)

                              Links
                              The main pkg(8) git repository. URL: https://github.com/freebsd/pkg
                              The preferred place to raise bug reports concerning pkg(8). URL: https://github.com/freebsd/pkg/issues

                              Contact: Baptiste Daroussin <bapt@FreeBSD.org>
                              Contact: Bryan Drewery <bdrewery@FreeBSD.org>
                              Contact: Matthew Seaman <matthew@FreeBSD.org>
                              Contact: Vsevolod Stakhov <vsevolod@FreeBSD.org>
                              Contact: The pkg mailing list <freebsd-pkg@FreeBSD.org>

                              pkg(8) is the new package management tool for FreeBSD. It is now the only supported package management tool for FreeBSD releases from 10.0-RELEASE, including the upcoming 9.3-RELEASE. pkg(8) is available on all currently supported releases. Support for the legacy pkg_tools is due to be discontinued at the beginning of September 2014.

                              The release of pkg(8) 1.3 is imminent. This includes major improvements in the dependency solver. Now we can:

                              • Switch versions of, for example, Perl or PHP and resolve all the conflicts with packages that depend on them automatically. No more need to manually switch package origins.
                              • Deal more gracefully with complex upgrade or install scenarios.
                              • Sandbox operations dealing with freshly downloaded data until it can be verified as trustworthy by checking the package signature.
                              • Deal with provides-and-requires style of dependencies, so for example we can say "this package needs to use a web server" and allow that dependency to be fulfilled by apache or nginx or any other alternative that provides web-server functionality.

                              Beyond the next release, we have work in progress on allowing ranges of versions in dependency rules and handling a selection of "foreign" package repositories, such as CPAN or CTAN or PyPi.

                              There are plans to use pkg(8) to package up the base system. Along with other benefits, this will allow writing a universal installer: download one installer image and from there install any available version of FreeBSD, including snapshots.

                              We are also intending to use pkg(8) within the ports tree at package-build time to handle fulfilling build dependencies. This opens the possibility of installing build-dependencies by downloading binary packages, which means you can install a package with customized options with the minimum amount of time spent compiling anything else.

                              Open tasks:

                              1. We are sorely lacking a comprehensive testing setup. Integrating automated regression testing into the development cycle is becoming an imperative.
                              2. We need testers who can run development versions of pkg in as many distinct types of use-cases as possible, and report feedback from their experiences to the freebsd-pkg@freebsd.org mailing list or our issues list on github.

                              QEMU bsd-user-Enabled Ports Building

                              Links
                              Overview of technology URL: https://wiki.freebsd.org/QemuUserModeHowTo
                              Status of ports building URL: http://dirty.ysv.freebsd.org/
                              Master repository for collaboration URL: https://github.com/seanbruno/qemu-bsd-user

                              Contact: Stacey Son <sson@FreeBSD.org>
                              Contact: Juergen Lock <nox@FreeBSD.org>
                              Contact: Sean Bruno <sbruno@FreeBSD.org>

                              The ports-mgmt/poudriere-devel port is capable of building ports via an emulator. Configuration of the miscellaneous binary image activator is required prior to a poudriere-devel run.

                              ARMV6, MIPS32 and MIPS64 packages can be produced via full emulation. There are several packages that block a full run of builds. They can be viewed on the "Status of ports building" link.

                              To build packages via emulation, on current or latest stable/10:

                              Clone the github repository, and switch to the bsd-user branch. Then run:

                              ./configure --static \
                              --target-list="arm-bsd-user i386-bsd-user \
                              mips-bsd-user mips64-bsd-user mips64el-bsd-user \
                              mipsel-bsd-user ppc-bsd-user ppc64-bsd-user sparc-bsd-user \
                              sparc64-bsd-user x86_64-bsd-user"

                              gmake; gmake install

                              Then set up the binmiscctl tools to do some evil hackery to redirect execution of armv6 binaries to qemu:

                              binmiscctl add armv6 --interpreter \ "/usr/local/bin/qemu-arm" --magic \ "\x7f\x45\x4c\x46\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02 \
                              \x00\x28\x00" --mask "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff \
                              \xff\xff\xff\xff\xfe\xff\xff\xff" --size 20 --set-enabled

                              Install poudriere-devel from ports. It knows how to set up things.

                              Create a poudriere jail to do all the magic:

                              poudriere jail -c -j 11armv632 -m svn -a armv6 \
                              -v head

                              Now run poudriere against that jail to build all the ports:

                              poudriere bulk -j 11armv632 -a

                              Nullfs mount the ports tree into the jail:

                              mkdir /usr/local/poudriere/jails/11armv632/usr/ports
                              mount -t nullfs /usr/ports /usr/local/poudriere/jails/11armv632/usr/ports

                              To chroot into the jail:

                              mount -t devfs devfs /usr/local/poudriere/jails/11armv632/dev
                              chroot /usr/local/poudriere/jails/11armv632/

                              Open tasks:

                              1. PPC on AMD64 emulation. This is a work in progress as there appear to be some serious issues running the bsd-user binary on big-endian hardware. Justin Hibbits is working on this.
                              2. SPARC64 on AMD64 emulation is non-functional and instantly segfaults. We are looking for someone to poke at the bits here.
                              3. External Toolchain, XDEV support. There is partial support for using an AMD64 toolchain that can output binaries for other architecture (e.g., using an AMD64 toolchain to build MIPS64 packages). We are currently tracking a linking issue with ports-mgmt/pkg. Thanks to Warner Losh, Baptiste Daroussin, Dimitry Andric for poking at bits in here to make the XDEV target useful.
                              4. Signal handling. The MIPS/ARMV6 target stills display a failure that manifests itself when building devel/p5-Sys-SigAction.
                              5. Massive documentation update needed. These modifications actually allow chrooting into a MIPS or ARMv6 environment and using native toolchains and libraries to prototype software for a target platform.

                              RPC/NFS and CTL/iSCSI Performance Optimizations

                              Contact: Alexander Motin <mav@FreeBSD.org>

                              The FreeBSD RPC stack, used as a base for its NFS server, received multiple optimizations to improve performance and SMP scalability. Algorithmic optimizations reduced processing overhead, while improved locking allowed it to scale up to at least 40 processor cores without significant lock congestion. Combined with some other kernel optimizations, the peak NFS request rate increased by many times, reaching up to 600K requests per second on modern hardware.

                              The CAM Target Layer (CTL), used as the base for the new kernel iSCSI server, also received a series of locking optimizations which allowed its peak request rate to increase from ~200K to ~600K IOPS with the potential of reaching a rate of 1M requests per second. That rate is sufficient to completely saturate 2x10Gbit Ethernet links with 4KB requests. For comparison, the port of net/istgt (user-level iSCSI server) on the same hardware with an equivalent configuration showed only 100K IOPS.

                              There is also ongoing work on improving CTL functionality. It was already made to support three of four VMware VAAI storage acceleration primitives (net/istgt supports 2), while the goal is to reach full VAAI support during next months.

                              With all these improvements, and earlier improvements in CAM, GEOM, ZFS, and a number of other kernel areas coming soon, FreeBSD 10.1 may become the fastest storage release ever. ;)

                              These projects are sponsored by iXsystems, Inc.

                              ZFSguru

                              Links
                              URL: http://zfsguru.com
                              URL: http://zfsguru.com/news/stateoftheproject/2014

                              Contact: Jason Edwards <sub.mesa@gmail.com>

                              ZFSguru is a multifunctional server appliance with a strong emphasis on storage. ZFSguru began as simple web-interface frontend to ZFS, but has since grown into a FreeBSD derivative with its own infrastructure. The scope of the project has also grown with the inclusion of add-on packages that add functionality beyond the traditional NAS functionality found in similar product like FreeNAS and NAS4Free. ZFSguru aims to be a true multifunctional server appliance that is extremely easy to setup and can unite both novice and more experienced users in a single user interface. The modular nature of the project combats the danger of bloat, whilst still allowing extended functionality to be easily deployed.

                              Where development in the first quarter of this year brought drag-and-drop permissions for Samba and NFS, development in the second quarter focused on strengthening the infrastructure of the project. A new library and toolkit solution dubbed 'Mesa' is in the works, providing a cleaner foundation to the project. A new master server providing secure remote services is being setup, to be located in a high-speed datacenter. But most importantly, a new system build infrastructure has shown great progress and will soon be able to provide automated system builds to our users. This not only improves the frequency of system releases but also frees much developer time to be spent on different areas of the project.

                              Furthermore, a new website and forum is being worked on, replacing the old-fashioned website that offers only limited functionality. The new website will be linked to the server database, providing real-time updates about the project.

                              In addition, a new platform for collaborative development is in the works. A service addon has been created for the GitLab project, which is a drop-in replacement of the popular GitHub website. The choice was made to host our own solution and not rely on GitHub itself. In retrospect this appears to be a good decision. The recent development where GitHub removed projects after DCMA takedowns being sent is incompatible with the philosophy of free-flow-of-information, which the ZFSguru project is a strong proponent of. By hosting our own solution, we have avoided any dependency on third party projects.

                              It is expected that after the infrastructure of the project has been revamped, work on the web-interface itself can continue. New functionality such as GuruDB and Service Bulletins provide a tighter connection between the server infrastructure and the web-interface. The Migration Manager is one of the last remaining features still missing in the web-interface. This functionality provides an easy way to upgrade the current system by performing a new clean installation, but migrate all relevant configuration to the new installation. It also allows to backup all system configuration in a single file to be stored on a different machine should things go awry.

                              A longer version of this status report giving a wider perspective on the project can be found at the stateoftheproject link.


                              Kernel


                              PostgreSQL Performance Improvements

                              Links
                              URL: https://www.kib.kiev.ua/kib/pgsql_perf_v2.0.pdf

                              Contact: Konstantin Belousov <kib@FreeBSD.org>

                              Analysis of the performance of the latest 9.3 version of PostgreSQL on FreeBSD-CURRENT has been performed. The issues which prevented good scalability on a 40-core machine were determined, and changes prototyped which solve the bottlenecks.

                              The URL above provides a paper which contains a detailed explanation of the issues and solutions, together with a graph demonstrating the effects on scalability.

                              This project was sponsored by The FreeBSD Foundation.

                              Running FreeBSD as an Application on Top of the Fiasco.OC Microkernel

                              Links
                              L4 microkernel family URL: http://en.wikipedia.org/wiki/L4_microkernel_family
                              A brief description of the project on the FreeBSD wiki (short talk during FreeBSD DevSummit in Cambridge) URL: https://wiki.freebsd.org/201407DevSummit/BSDUserspace

                              Contact: Ilya Bakulin <ilya@bakulin.de>

                              Fiasco.OC belongs to the L4 microkernel family. A microkernel provides a bare minimum of services to the applications running on top of it, unlike traditional kernels that incorporate complex code like IP stacks and device drivers. This allows a dramatic decrease in the amount of code running in the privileged mode of the CPU, achieving higher security while still providing an acceptable level of performance.

                              Running an operating system kernel on top of the microkernel allows leveraging any software that was developed for that operating system. The OS kernel runs in user-mode side-by-side with other microkernel applications such as real-time components. Multiple OSes, each with their userland applications, can even be run in parallel, thus allowing construction of products where processing of corporate data is strictly separated from the processing of private data.

                              The project aims to create a port of FreeBSD to the Fiasco.OC microkernel, a high performance L4 microkernel developed by TU Dresden. Existing ports of OpenBSD and Linux are used as a reference. This will allow the use of unique FreeBSD features like ZFS in L4-based projects.

                              Open tasks:

                              1. Finish opensourcing the port of L4OpenBSD/amd64 made by genua mbh. This is a work in progress.
                              2. Publish the sources of the L4FreeBSD port that is largely based on the L4OpenBSD code.
                              3. Improve the port, the first task being adopting the pmap(9) module to work with L4 microkernel memory allocation services.

                              SDIO Driver

                              Links
                              SDIO project page on FreeBSD Wiki URL: https://wiki.freebsd.org/SDIO
                              Source code URL: https://github.com/kibab/freebsd/tree/mmccam

                              Contact: Ilya Bakulin <ilya@bakulin.de>

                              SDIO is an interface designed as an extension of the existing SD card standard, which allows the connecting of different peripherals to a host with a standard SD controller. Peripherals currently sold on the general market include WLAN/BT modules, cameras, fingerprint readers, and barcode scanners. Additionally, SDIO is used to connect some peripherals in products like Chromebooks and Wandboards. A prototype of the driver for the Marvell SDIO WLAN/BT (Avastar 88W8787) module is also being developed, using the existing Linux driver as the reference.

                              SDIO card detection and initialization already work. Most necessary bus methods are implemented and tested.

                              The WiFi driver is able to load firmware onto the card and initialize it. A rewrite of the MMC stack as a transport layer for the CAM framework is in progress. This will allow utilization of the well-tested CAM locking model and debug features.

                              Open tasks:

                              1. SDIO stack: finish CAM migration. The initialization of the MMC/SD card is implemented in the XPT layer, but cannot be tested with real hardware because of the lack of any device drivers that implement peripheral drivers and SIMs for CAM MMC. The plan is to use a modified version of the BeagleBone Black SDHCI controller driver for the SIM and a modified version of mmcsd(4) as a peripheral driver.
                              2. Marvell SDIO WiFi: connect to the FreeBSD network stack, write the code to implement required functions (such as sending/receiving data, network scanning and so on).

                              TMPFS Stability

                              Contact: Konstantin Belousov <kib@FreeBSD.org>
                              Contact: Peter Holm <pho@FreeBSD.org>

                              Extensive testing of tmpfs(5) using the stress2 kernel test suite was done. The issues found were debugged and fixed.

                              Most of the problems are related to bugs in the interaction of the vnode and node lifetime, culminating in e.g., unmount races and dotdot lookup bugs.

                              This project was sponsored by The FreeBSD Foundation.

                              UEFI Boot

                              Links
                              FreeBSD UEFI wiki page URL: https://wiki.freebsd.org/UEFI
                              FreeBSD snapshots URL: http://www.freebsd.org/snapshots/

                              Contact: Ed Maste <emaste@FreeBSD.org>
                              Contact: Nathan Whitehorn <nwhitehorn@FreeBSD.org>

                              The Unified Extensible Firmware Interface (UEFI) provides boot- and run-time services for x86 and other computers. For the x86 architecture it replaces the legacy BIOS. This project will adapt the FreeBSD loader and kernel boot process for compatibility with UEFI firmware, found on contemporary servers, desktops, and laptops.

                              Ed and Nathan completed a number of integration tasks over the past three months. Nathan added a first-stage loader, boot1.efi, to support chain-loading the rest of the system from a UFS filesystem. This allows the UEFI boot process to proceed in a similar fashion as with BIOS boot. Nathan also added UEFI support to the FreeBSD installer and release image creation script.

                              The EFI framebuffer requires the vt(4) system console — a framebuffer driver is not implemented for the legacy syscons(4) console. Ed added automatic vt(4) selection to the UEFI boot path.

                              Snapshots are now built as dual-mode images, and should boot via both BIOS and UEFI. Our plan is to merge the UEFI and vt(4) work to stable/10 to appear in FreeBSD 10.1-RELEASE.

                              This project was sponsored by The FreeBSD Foundation.

                              Open tasks:

                              1. Document manual installation, including dual-boot configurations.
                              2. Implement boot1.efi for ZFS file systems.
                              3. Add support for UEFI variables stored in non-volatile memory (NVRAM).
                              4. Debug boot failures with certain UEFI firmware implementations.
                              5. Support secure boot.

                              Updated vt(4) System Console

                              Links
                              Project wiki page URL: https://wiki.freebsd.org/Newcons

                              Contact: Aleksandr Rybalko <ray@FreeBSD.org>
                              Contact: Ed Maste <emaste@FreeBSD.org>
                              Contact: Ed Schouten <ed@FreeBSD.org>
                              Contact: Warren Block <wblock@FreeBSD.org>

                              The vt(4) (aka Newcons) project provides a replacement for the legacy syscons system console. It brings a number of improvements, including better integration with graphics modes and broader character set support.

                              Since the last report, vt(4) gained the ability to make early driver selection. vt(4) selects the best successfully-probed driver before most other kernel subsystems are initialized. Also, to facilitate migration from syscons(4) to vt(4), multiple virtual terminal subsystems in the kernel are now supported. It is controlled by a small module with just one kernel environment variable. Users can select the virtual terminal system to use by setting kern.vty=sc or kern.vty=vt.

                              The GENERIC kernel configuration for the amd64 and i386 platforms now includes both syscons(4) and vt(4) by default. This configuration is also planned to be in FreeBSD 10.1-RELEASE.

                              The project finally received a man page, so now vt(4) is not only the project name, but also a link to its documentation. Great thanks to Warren Block for that.

                              Major highlights:

                              • Unicode support.
                              • Double-width character support for CJK characters.
                              • xterm(1)-like terminal emulation.
                              • Support for Kernel Mode Setting (KMS) drivers (i915kms, radeonkms).
                              • Support for different fonts per terminal window.
                              • Simplified drivers.

                              Brief status of supported architectures and hardware:

                              • amd64 (VGA/i915kms/radeonkms) — works.
                              • ARM framebuffer — works.
                              • i386 (VGA/i915kms/radeonkms) — works.
                              • IA64 — untested.
                              • MIPS — untested.
                              • PPC and PPC64 — work, but without X.Org yet.
                              • SPARC — works on certain hardware (e.g., Ultra 5).
                              • vesa(4) — in progress.
                              • i386/amd64 nVidia driver — not supported. VGA should be used (VESA planned).
                              • Xbox framebuffer driver — will be deleted as unused.

                              This project was sponsored by The FreeBSD Foundation.

                              Open tasks:

                              1. Implement the remaining features supported by vidcontrol(1).
                              2. Write manual pages for vt(4) drivers and kernel interfaces.
                              3. Support direct handling of keyboard by the kbd device (without kbdmux(4)).
                              4. CJK fonts. (This is in progress).
                              5. Address performance issues on some architectures.
                              6. Switch to vt(4) by default.
                              7. Convert keyboard maps for use with vt(4).
                              8. Implement compatibility mode to be able to use single-byte charsets/key-codes in vt(4).

                              Architectures


                              FreeBSD/arm64

                              Links
                              URL: http://svnweb.freebsd.org/base/projects/arm64/

                              Contact: Andrew Turner <andrew@FreeBSD.org>

                              Arm64 is the name of the in-progress port of FreeBSD to the ARMv8 CPU when it is in AArch64 mode. Until recently, all ARM CPU designs were 32-bit only. With the introduction of the ARMv8 architecture, ARM has added a new 64-bit mode. This new mode has been named AArch64.

                              Booting FreeBSD on the ARM Foundation Model has made a lot of progress since the last status report. An initial pmap implementation has been written. With this, FreeBSD is able to enter the Machine Independent boot code. The required autoconf functions have been added allowing FreeBSD to start scheduling tasks. Finally the cpu_switch and copystr functions were added. With these two, FreeBSD will boot to the mountroot prompt.

                              Work has started on supporting exceptions, including interrupts. This will allow more developers to start working on device drivers.

                              Open tasks:

                              1. Finish exception and interrupt handling
                              2. Read the Device Tree or ACPI tables from UEFI
                              3. Test on real hardware

                              Ports


                              FreeBSD Python Ports

                              Links
                              The FreeBSD Python Team Page URL: https://wiki.FreeBSD.org/Python
                              IRC channel URL: irc://freebsd-python@irc.freenode.net

                              Contact: FreeBSD Python Team <python@FreeBSD.org>

                              We are pleased to announce the availability of conflict-free Python package support across different Python versions based on the USES=uniquefiles feature recently introduced to the Ports framework. A Python package can be marked as buildable and installable in parallel for different Python versions at the same time on the same host. The package building tools, however, do not support this feature yet and the Python team will work closely with portmgr and the pkg developers to enable support on a global ports and packages scale.

                              In May and June a huge clean-up operation took place to remove the last bits and pieces targeting easy_install. In the beginning of July we committed the final changes to remove easy_install support completely from the ports framework. This greatly simplifies the infrastructure and allows us to modernize and maintain it with less effort.

                              We added Python 3.4, removed Python 3.1 after its end of life, updated the setuptools ports to version 5.1 and PyPy's development version to 2.3.1. The latest Python 2.7.8 and an updated setuptools will hit the tree shortly.

                              Our upstreaming effort continues to produce good outcomes for simplifying maintenance and reducing complexity.

                              Looking forward, one of the top priorities is to comply with the USES framework in the foreseeable future and to roll out a consistent maintainer policy for integrating new Python-related ports into the tree.

                              Open tasks:

                              1. Migrate bsd.python.mk to the Uses framework.
                              2. Develop a high-level and lightweight Python Ports Policy.
                              3. Add support for granular dependencies (for example >=1.0,<2.0).
                              4. See what adding pip (Python Package Index) support will require.
                              5. Add default QA targets and functions for Python ports (TEST_DEPENDS, regression-test, etc.)
                              6. More tasks can be found on the team's wiki page (see links).
                              7. To get involved, come and say "hi" on IRC and let us know what you are interested in!

                              KDE/FreeBSD

                              Links
                              KDE/FreeBSD home page URL: http://FreeBSD.kde.org
                              area51 URL: http://FreeBSD.kde.org/area51.php

                              Contact: KDE/FreeBSD Team <kde@FreeBSD.org>

                              The KDE/FreeBSD team has continued to improve the experience of KDE software and Qt under FreeBSD.

                              During this quarter, the team has kept most of the KDE and Qt ports up-to-date, working on the following releases:

                              • KDE SC: 4.12.5; Workspace: 4.11.9

                              As a result — according to PortScout — kde@ has 526 ports (up from 526), of which 84.63% are up-to-date (down from 98.86%). iXsystems Inc. continues to provide a machine for the team to build packages and to test updates. iXsystems Inc. has been providing the KDE/FreeBSD team with support for quite a long time and we are very grateful for that.

                              As usual, the team is always looking for more testers and porters so please contact us at kde@FreeBSD.org and visit our home page at http://FreeBSD.kde.org. It would be especially useful to have more helping hands on tasks such as getting rid of the dependency on the defunct HAL project and providing integration with KDE's Bluedevil Bluetooth interface.

                              Open tasks:

                              1. Updating out-of-date ports, see PortScout for a list
                              2. Removing the dependency on HAL

                              The Graphics Stack on FreeBSD

                              Links
                              Graphics stack roadmap and supported hardware matrix URL: https://wiki.freebsd.org/Graphics
                              WITH_NEW_XORG repository announce URL: http://lists.freebsd.org/pipermail/freebsd-announce/2014-July/001570.html
                              Ports-related development repository URL: http://trillian.chruetertee.ch/ports/browser/trunk

                              Contact: FreeBSD Graphics team <x11@FreeBSD.org>

                              We were generally short on time this quarter. We made less progress than expected on all fronts.

                              The alternate pkg(8) repository, built with WITH_NEW_XORG, is now available. This alleviates the need for users to rebuild their ports with WITH_NEW_XORG. See the announcement, linked above for further information.

                              Thanks to a contribution from Jan Kokemüller, Radeon 32bit ioctls are now working on 64bit hosts. This was tested successfully with Wine and StarCraft II on FreeBSD 9.x and 11. This required modifications to emulators/i386-wine-devel so that it works with WITH_NEW_XORG, and the creation of a new port, libtxc_dxtn, to support the texture compression used by StarCraft II. We have not yet had the time to polish everything, so this still requires manual steps.

                              The DRM generic code update is ready, but it breaks the current i915 driver. Therefore, the i915 driver must be updated before anything is committed.

                              Compared to the previous status report, OpenCL test programs are running fine now, thanks to upgrades and fixes to libc++ and Clang. The relevant ports are still not ready to hit the ports tree, unfortunately.

                              Open tasks:

                              1. See the "Graphics" wiki page for up-to-date information.

                              Documentation


                              Quarterly Status Reports

                              Contact: Quarterly Status Report Team <monthly@FreeBSD.org>

                              These quarterly status reports help the FreeBSD community stay up-to-date with the happenings in and around the project. Updates from FreeBSD teams, new features being developed in- or out-of-tree, products derived from FreeBSD, and FreeBSD events are all welcome additions to the status reports.

                              The Monthly team has been busy since the last report, with longtime organizer Gábor Páli having stepped down from the team — thank you Gábor for all your hard work! This has left something of a void in the preparation of this report, for which the call for items was issued quite late. To help fill the void, Warren Block and Benjamin Kaduk have been added to the monthly@ team, joining Glen Barber, Gavin Atkinson, Ed Maste, and the rest of the team in preparing this report. Special thanks to Glen for doing most of the work while simultaneously getting 9.3-RELEASE out the door!

                              The next cycle is sooner than you think! The deadline for submitting entries for the Q3 report is October 7th, 2014.

                              This project was sponsored by The FreeBSD Foundation.

                              Open tasks:

                              1. Submit reports for Q42014 to monthly@FreeBSD.org!

                              Miscellaneous


                              FreeBSD Host Support for OpenStack and OpenContrail

                              Links
                              URL: http://www.openstack.org
                              URL: http://www.opencontrail.org
                              URL: https://github.com/Semihalf/openstack-devstack
                              URL: https://github.com/Semihalf/openstack-nova
                              URL: https://github.com/Semihalf/contrail-vrouter
                              URL: https://blueprints.launchpad.net/nova/+spec/freebsd-compute-node

                              Contact: Grzegorz Bernacki <gjb@semihalf.com>
                              Contact: Michal Dubiel <md@semihalf.com>
                              Contact: Dominik Ermel <der@semihalf.com>
                              Contact: Rafal Jaworowski <raj@semihalf.com>

                              OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources in a datacenter.

                              OpenContrail is a network virtualization (SDN) solution comprising network controller, virtual router, and analytics engine, which can be integrated with cloud orchestration systems like OpenStack or CloudStack.

                              The goal of this work is to enable FreeBSD as a fully supported compute host for OpenStack using OpenContrail virtualized networking. The main areas of development are:

                              • Libvirt hypervisor driver for bhyve.
                              • Support for bhyve (via libvirt compute driver) and the overall FreeBSD platform in nova-compute.
                              • OpenContrail vRouter (forwarding plane kernel module) port to FreeBSD.
                              • OpenContrail Agent (network controller node) port to FreeBSD.
                              • Integration and performance optimizations.

                              Since the last report the following items have been completed, which allow for a working demo of an OpenStack compute node on a FreeBSD host using OpenContrail for network virtualization:

                              • Port of the OpenContrail vRouter kernel module for FreeBSD (MPLS over GRE mode only)
                              • Port of the OpenContrail Agent for FreeBSD
                              • FreeBSD version of a Devstack installation/configuration script with support for the OpenContrail solution (Compute node components only)

                              A demo was presented at the DevSummit during BSDCan2014 in Ottawa. Also, a meetup regarding the subject was organized in Krakow, Poland.

                              Work on this project is sponsored by Juniper Networks.

                              The FreeBSD Foundation

                              Links
                              URL: http://www.FreeBSDFoundation.org/
                              FreeBSD Journal URL: http://freebsdjournal.com/

                              Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                              The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of the funding is used to support FreeBSD development projects, conferences and developer summits, purchase equipment to grow and improve the FreeBSD infrastructure, and provide legal support for the Project.

                              We published our third issue of the FreeBSD Journal. We have over 2700 subscriptions so far. We continued working on the digital edition, which will allow subscribers to read the magazine in different web browsers, including those than run on FreeBSD. This will be available for the July/August issue of the Journal.

                              We hired Anne Dickison, on a freelance basis, as our new marketing director, to help us promote the Foundation and Project.

                              The annual board meeting was held in Ottawa, Canada, in May. Directors and officers were elected, and we did some long-term planning. We worked on our vision, core values, project road mapping, and our near-term goals. We also met with the core team to discuss roles and responsibilities, project roadmapping, and what we can do to help the Project more.

                              We were a Gold+ sponsor for BSDCan, May 16-17 and provided 7 travel grants for developers to attend the conference. We also were the sponsor for both the developer and vendor summits.

                              Justin Gibbs gave a FreeBSD presentation at a FreeBSD user's internal technology summit. Company visits like this help users understand the Project structure better and gives us a chance to communicate what FreeBSD people are working on as well as learn what different companies are doing with FreeBSD, as well as what they'd like to see supported. We can then help facilitate collaboration between the companies and FreeBSD developers.

                              We were represented at Great Wide Open, April 2-3 (greatwideopen.org), Texas LinuxFest, June 13-14 (texaslinuxfest.org), and SouthEast LinuxFest, June 20-22 (southeastlinuxfest.org).

                              Hardware was purchased to support an upgrade at Sentex. A new high-capacity 1Gbps switch was deployed to allow for more systems to be added to the test lab. The main file server and development box was upgraded to allow more users in the lab simultaneously.

                              We purchased hardware, including package builders, and a larger server to allow NYI to be a full replica of all Project systems, comparable to what is in place at Yahoo Inc. and ISC.

                              We worked with our lawyer to create an NDA between the Foundation and individuals for third party NDAs. This allows developers who need access to proprietary documents, to go through the Foundation, via an NDA for access.

                              FreeBSD Foundation Systems Administrator and Release Engineer, Glen Barber, continued work on producing regularly-updated FreeBSD/arm snapshots for embedded devices, such as the Raspberry Pi, ZedBoard, and BeagleBone.

                              In addition to producing weekly development snapshots from the head/ and stable/ branches, with feedback and help from Ed Maste, Glen finished work to produce release images that will, by default, provide debugging files for userland and kernel available on the FreeBSD Project FTP mirrors. Note that the debugging files will not be included on the bootonly.iso, disc1.iso, or dvd1.iso images due to the size of the resulting images.

                              Foundation staff member Konstantin Belousov completed an investigation into poor performance of PostgreSQL on FreeBSD. This uncovered scalability problems in the FreeBSD kernel, and changes to address these issues are in progress.

                              Some previously completed Foundation-sponsored projects received enhancements or additional work. The ARM superpages project was completed last year, but is now enabled by default in FreeBSD-CURRENT. Many stability fixes and enhancements have been committed to the in-kernel iSCSI stack. The iSCSI project was released in FreeBSD 10.0. Many stability fixes and enhancements have been committed and will be included in FreeBSD 10.1.

                              Work continues on the Foundation-sponsored autofs automount daemon, UEFI boot support, the updated vt(4) system video console, virtual machine images, and the Intel graphics driver update. Foundation-sponsored work resulted in 226 commits to FreeBSD over the April to June period.

                              News Home | Status Home
                              diff --git a/website/content/en/status/report-2014-10-2014-12.html b/website/content/en/status/report-2014-10-2014-12.html index 021b2e7cab..8c347aa0e2 100644 --- a/website/content/en/status/report-2014-10-2014-12.html +++ b/website/content/en/status/report-2014-10-2014-12.html @@ -1,2131 +1,2131 @@ FreeBSD Quarterly Status Report
                              Skip site navigation (1) Skip section navigation (2)

                              Introduction

                              This report covers FreeBSD-related projects between October and December 2014. This is the last of four reports planned for 2014.

                              The fourth quarter of 2014 included a number of significant improvements to the FreeBSD system. In particular, compatibility with other systems was enhanced. This included significant improvements to the Linux compatibility layer, used to run Linux binaries on FreeBSD, and the port of WINE, used to run Windows applications. Hypervisor support improved, with FreeBSD gaining the ability to run as domain 0 on Xen's new high-performance PVH mode, bhyve gaining AMD support, and new tools for creating FreeBSD VM images arriving.

                              This quarter was also an active time for the toolchain, with numerous improvements to the compiler, debugger, and other components, including initial support for C++14, which should be complete by FreeBSD 10.2.

                              Thanks to all the reporters for the excellent work!

                              The deadline for submissions covering the period from January to March 2015 is April 7th, 2015.

                              FreeBSD Team Reports

                              Projects

                              Kernel

                              Architectures

                              Userland Programs

                              Ports

                              Documentation

                              Miscellaneous


                                FreeBSD Team Reports


                                FreeBSD Release Engineering Team

                                Links
                                FreeBSD development snapshots URL: http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/

                                Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

                                The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

                                The FreeBSD 10.1-RELEASE cycle completed on November 14th, marking the second official release point from the stable/10 branch, just short of three weeks later than the original schedule anticipated.

                                Work to produce virtual machine images for platforms not currently supported has continued, with focus aimed primarily at Amazon EC2, Google Compute Engine, and Openstack.

                                With huge thanks to Ian Lepore and Warner Losh, new ports exist for FreeBSD/arm where u-boot is required. Work has been in progress since late December to migrate the existing FreeBSD/arm release build tools to utilize the new ports.

                                This project was sponsored by The FreeBSD Foundation.

                                Ports Collection

                                Links
                                URL: http://www.FreeBSD.org/ports/
                                URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
                                URL: http://portsmon.freebsd.org/index.html
                                URL: http://portscout.freebsd.org/
                                URL: http://www.freebsd.org/portmgr/index.html
                                URL: http://blogs.freebsdish.org/portmgr/
                                URL: http://www.twitter.com/freebsd_portmgr/
                                URL: http://www.facebook.com/portmgr
                                URL: http://plus.google.com/communities/108335846196454338383

                                Contact: Frederic Culot <portmgr-secretary@FreeBSD.org>
                                - Contact: Port Management Team <portmgr@FreeBSD.org> + Contact: Ports Management Team <portmgr@FreeBSD.org>

                                As of the end of Q4 the ports tree holds more than 24,000 ports, and the PR count is just over 1,400. As during the previous quarter the tree saw a sustained activity with almost 6,000 commits and more than 1,600 ports PRs closed!

                                In Q4, five new developers were granted a ports commit bit (gordon@, jmg@, jmmv@, bofh@, truckman@) and six were taken in for safekeeping (sylvio@, pclin@, flz@, jsa@, anders@, motoyuki@).

                                On the management side, miwi@ decided to step down from his portmgr duties in November. No other changes were made to the team during Q4.

                                This quarter also saw the release of the fourth quarterly branch, namely 2014Q4.

                                On the QA side, 39 exp-runs were performed to validate sensitive updates or cleanups.

                                Open tasks:

                                1. A tremendous work was done on the PR front in Q4 and we would be very pleased to see committers dedicate themselves to closing as many as possible in 2015 as well!

                                2. 2014 is the year that saw the highest number of commits in all of our ports tree's history! As for the PR front and to keep our beloved tree in good shape, we would love to see the same commitment from our developers next year!

                                The FreeBSD Core Team

                                Contact: FreeBSD Core Team <core@FreeBSD.org>

                                The FreeBSD Core Team constitutes the project's "Board of Directors", responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape.

                                During the fourth quarter of 2014, the FreeBSD Core team saw the culmination of a long-running project to rebuild the FreeBSD Forums. The chosen solution was to license XenForo; core would like to thank the FreeBSD Foundation for paying the licensing costs of this software.

                                Much discussion ensued concerning the "New Support Model" following Core's meeting at EuroBSDCon in September. It was recognised that trying to change the model immediately before 10.1-RELEASE was far too late, and the change will be targeted at 11.0-RELEASE.

                                In order to ensure that 10.1-RELEASE shipped with support for up-to-date X Window Systems and KDE4, core approved the switch to 'new Xorg' as the default in time for building the packages for that release.

                                Git was officially promoted from beta to an officially supported version control system. Git is available as a read-only resource for downstream consumers and contains an exported copy from SVN, the primary and only read-write repository. The FreeBSD git repositories (exported from the master SVN version control) will shortly be available at https://git.freebsd.org/, and core has been active in ensuring that there is a sufficient body of Git administrators available with access to appropriate documentation in order to maintain a good git service.

                                Core mediated in disputes between a number of committers over some updates to system sources, and fielded complaints about code quality of some other work in critical areas. While such disagreements will occasionally occur, core is promoting the routine use of the Phabricator service in order to review work before committal. Catching problems early is in the project's best interests, and discussion of changes in an open review context should minimize confrontational demands for immediate back-out of changes.

                                Core is working on a charter for a proposed new QA team, to encompass members of the Release Engineering and Security teams, as well as committers with interests in standards compliance. It is envisioned that the QA team will take responsibility for merging code from HEAD into the STABLE branches, run integration testing against those updates and handle merging patches and bug-fixes submitted to the FreeBSD project from third parties.

                                During this quarter, core issued two new commit bits, and also took two commit bits into safe-keeping.


                                Projects


                                bhyve

                                Links
                                bhyve FAQ and talks URL: http://www.bhyve.org

                                Contact: Peter Grehan <grehan@FreeBSD.org>
                                Contact: Neel Natu <neel@FreeBSD.org>
                                Contact: John Baldwin <jhb@FreeBSD.org>
                                Contact: Tycho Nightingale <tychon@FreeBSD.org>
                                Contact: Allan Jude <freebsd@allanjude.com>

                                bhyve is a hypervisor that runs on the FreeBSD/amd64 platform. At present, it runs FreeBSD (8.x or later), Linux i386/x64, OpenBSD i386/amd64, and NetBSD/amd64 guests. Current development is focused on enabling additional guest operating systems and implementing features found in other hypervisors.

                                Support for AMD processors was committed to -CURRENT in October 2014. This has also been merged to 10-STABLE and will be included in the 10.2 release.

                                A bhyve status update presentation was given at the FreeBSD Vendor Summit in Nov 2014. The slides are available at http://people.freebsd.org/~neel/bhyve/bhyve_update_vendor_summit_2014.pdf.

                                A number of improvements have been made to bhyve this quarter:

                                • OpenBSD/i386 guests are now able to boot with multiple vcpus.
                                • NetBSD/amd64 guests are now fully supported.
                                • Improvements to the AHCI emulation to be more resilient under heavy load.
                                • Various improvements to PIC emulation to be able to boot legacy guests.
                                • A fully featured RTC device emulation that allows date/time changes by the guest and supports periodic and alarm interrupts.
                                • Consolidate all timer emulations in vmm.ko. This enables the use of a single clocksource for all timer emulations.
                                • Allow tracing of every exception incurred by a guest. This is useful when debugging guest double and triple faults.
                                • Emulate platform-specific MSRs accessed by recent Linux guests.
                                • Various bug fixes to grub-bhyve to boot OpenBSD/i386 and Centos 4.x guests.
                                • grub-bhyve is now able to connect to an nmdm(4) console using the --cons-dev option.

                                Open tasks:

                                1. Improve documentation.

                                2. bhyveucl is a script for starting bhyve instances based on a libUCL config file. More information at https://github.com/allanjude/bhyveucl.

                                3. CSM BIOS boot support for non UEFI-aware guests.

                                4. Add support for virtio-scsi.

                                5. Improve virtio-net, add offload features, support multiple queues.

                                6. Implement Intel 82580 and e1000 NIC emulation.

                                7. Netmap support.

                                8. Flexible networking backend: wanproxy, vhost-net.

                                9. Move to a single process model, instead of bhyveload + bhyve.

                                10. Support running bhyve as non-root.

                                11. Add filters for popular VM file formats (VMDK, VHD, QCOW2).

                                12. Implement an abstraction layer for video (no X11 or SDL in base system).

                                13. Support for VNC as a video output.

                                14. Suspend/resume support.

                                15. Live Migration.

                                16. Nested VT-x support (bhyve in bhyve).

                                17. Support for other architectures (ARM, MIPS, PPC).

                                Clang, llvm, and lldb Updated to 3.5.0

                                Links
                                LLVM 3.5.0 Release Notes URL: http://llvm.org/releases/3.5.0/docs/ReleaseNotes.html
                                Clang 3.5.0 Release Notes URL: http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html

                                Contact: Dimitry Andric <dim@FreeBSD.org>
                                Contact: Ed Maste <emaste@FreeBSD.org>
                                Contact: Roman Divacky <rdivacky@FreeBSD.org>

                                Just before the end of the year, we updated clang, llvm, and lldb in the base system to 3.5.0 release. These all contain numerous improvements. Please see the linked release notes for more detailed information.

                                This is the first release that requires C++11 support to build. At this point, FreeBSD 10.0 and later provide that support, at least on x86.

                                In the near future, more components from llvm.org will be updated in base, with libc++ and libcompiler-rt most likely being the first to be updated.

                                Thanks to Ed Maste, Roman Divacky, Andrew Turner, Justin Hibbits, and Antoine Brodin for their invaluable help with this import.

                                Open tasks:

                                1. While most ports that were impacted by this update have already been fixed, there are still a few that do not work with the clang 3.5.0 update.

                                  In most cases, this is due to relatively simple issues, such as new warnings, or slightly stricter error checking (primarily for C++ programs). Fixing those issues should not take too much work.

                                2. There are still some open issues with the ARM, PowerPC, and Sparc64 architectures, and any help in this area is very much appreciated.

                                External Toolchain

                                Links
                                URL: https://wiki.freebsd.org/ExternalToolchain

                                Contact: Baptiste Daroussin <bapt@FreeBSD.org>
                                Contact: Warner Losh <imp@FreeBSD.org>
                                Contact: Brooks Davis <brooks@FreeBSD.org>

                                The main goal of the external toolchain project is to be able to build world and kernel with a non-default toolchain. It can be helpful to:

                                • Prepare a migration to a newer version of toolchain components.
                                • Port FreeBSD to a new architecture
                                • Upgrade from a FreeBSD that ships with GCC 4.2 to a version that ships with clang 3.5+ (which needs a more modern toolchain than GCC 4.2 to bootstrap).

                                The initial external toolchain work only supported clang. It has been extended to support recent GCC (4.9.1 has been tested) and recent binutils (2.24 and 2.25).

                                A large number of fixes have been committed to HEAD to support incompatible behaviour changes between ld(1) from binutils 2.17.50 (the version in base) and binutils 2.24+.

                                A large number of warnings have been deactivated when building the kernel to make sure it is possible to build the kernel with recent GCC (first 4.6 and then 4.9.1)

                                The build system has been changed to build libc++ as the C++ standard library implementation when a recent enough GCC (4.6+) is used to build world.

                                To simplify using an external toolchain, the following pre-seeded configurations have been added to the ports tree:

                                • amd64-xtoolchain-gcc
                                • powerpc64-xtoolchain-gcc
                                • sparc64-xtoolchain-gcc

                                Those packages will depend on special versions of GCC (minimalistic cross-built ready GCC) and on binutils. To use them, run: make CROSS_TOOLCHAIN=powerpc64-gcc TARGET=powerpc TARGET_ARCH=powerpc64

                                As a result of this effort, it is possible to successfully build and run a kernel and world built with GCC 4.9.1 and binutils 2.24 on sparc64, amd64 (with minor tweaks), powerpc and powerpc64.

                                Open tasks:

                                1. Patch GCC 4.9 to support FreeBSD mips, arm and aarch64 and submit the patches to upstream.

                                2. Adapt and upstream the aarch64 patches for binutils 2.25.

                                3. Add more pre-seeded configurations.

                                FreeBSD on Google Cloud

                                Links
                                The script used to create gcloud images URL: https://github.com/swills/FreeBSD-gcloud
                                More detail on how to create and use a gcloud image URL: https://plus.google.com/112202779615695172291/posts/eYajb8JKerY

                                Contact: Steve Wills <swills@FreeBSD.org>

                                Google Cloud is a cloud computing platform that allows users to run hosted services and servers in a cloud maintained by Google. The goal of this project is to provide an easy way to create and manage FreeBSD installations running on Google Cloud.

                                The good news:

                                FreeBSD 10.1 runs fine. You can create an image and start it up and login via standard ssh, via the gcloud command or via the web console (ssh in a web browser window). More details on how to do all this can be found in the links. Basically, you should be able to gcutil addimage freebsd-101-release-amd64-20150101032704 gs://swills-test-bucket/FreeBSD-10.1-RELEASE-amd64-20150101032704.tar.gz

                                Then spin up an image using gcloud compute instances create --zone us-central1-b --image freebsd-101-release-amd64-20150101032704 --boot-disk-size 20GB gtest1

                                These commands are part of the google-cloud-sdk port, which contains all the commands to interact with Google Cloud. There is also a google-daemon port which is used in running instances to create users and set them up and a google-startup-scripts port which handles running startup/shutdown scripts as specified in node metadata.

                                Additionally, the firstboot-growfs port has been brought back so that new instances will grow their root filesystem. (Thanks to Colin Percival for having created that port initially.)

                                There is also a firstboot-freebsd-update port which can be used to update a system on first boot but is currently disabled (see below). Similarly, the firstboot-pkgs port/scripts will install specified packages on first boot.

                                Overall, Google Cloud Compute is quite nice; instances spin up in about 60 seconds and it is very reasonably priced with automatic discounts for longer term usage. There is a $300 credit for first time users that also makes it free to try out. That credit covers quite a lot of time, and the instances are pretty fast, as well, even the ones without SSDs.

                                The bad news:

                                Google does not make sharing non-official images as easy as AWS, so you have to create your own using my public tar file. The tar file was created using the script in the links section. That script can be used to produce customized images, even though there are no official image (nor will there be any time soon).

                                There are some issues running FreeBSD on Google Cloud, listed in the tasks section.

                                Open tasks:

                                1. The 8 and 16 cpu instances seem to reboot randomly.

                                2. Repeated UFS panics that Google folks have reported, but I do not think those are particular to Google Cloud. The panic message is "ffs_valloc dup alloc".

                                3. Running freebsd-update causes the system to become unbootable, so updates do not work. (Reboots work fine otherwise.)

                                4. There is no gcimagebundle command in the Ports Collection so you cannot easily create an image from a running machine.

                                5. There are a few minor issue with the startup script that is supposed to regenerate ssh keys (for when you create an image from an existing system).

                                6. 10.1 works, but 10.0 does not boot; other versions remain untested.

                                7. The kern.vm_guest sysctl node does not detect that it is in a guest.

                                8. The vtnet driver needs wq disabled on 16 cpu boxes, but it is just disabled everywhere for now since that is easier.

                                9. There is work needed for the Google safe_format_and_mount command which formats and mounts newly attached disks, but this is just a nicety really.

                                10. I need to look into irq affinity for vtnet.

                                11. We need to support virtualized clocks; bryanv@ is working on this. In fact, all his ongoing work in the virtualization area would probably make things work better.

                                12. It would be nice if there was the ability to disable the spinner before the loader, which clutters up the console log. The ability to disable it is in HEAD; hopefully it will be MFCd to 10-STABLE before 10.2.

                                FreeBSD on the Acer C720 Chromebook

                                Links
                                URL: http://blog.grem.de/pages/c720.html

                                Contact: Michael Gmelin <freebsd@grem.de>

                                The Acer C720 Chromebook is a powerful but inexpensive laptop designed to run Google's Chrome OS. This project aims to bring FreeBSD to the C720, providing an easy way for people to experience FreeBSD on hardware which is widely available and inexpensive.

                                As of this update, most system features work, including the keyboard, WiFi, sound, VESA graphics, touchpad, and USB. The battery life is a reasonable 5 to 6 hours (compare to the published 8.5 hour lifetime for Chrome OS.

                                Open tasks:

                                1. Streamline patches and merge them into HEAD.

                                2. Make suspend/resume work (depends on Haswell support).

                                Git Integration

                                Links
                                URL: https://lists.freebsd.org/mailman/listinfo/freebsd-git
                                URL: https://www.kernel.org/pub/software/scm/git/docs/git-svn.html
                                URL: https://github.com/git/git/commit/83c9433e679635f8fbf8961081ea3581c93ca778
                                URL: https://wiki.freebsd.org/GitWorkflow
                                URL: https://github.com/freebsd/freebsd
                                URL: https://bugs.freebsd.org/bugzilla

                                Contact: Git discussion list <freebsd-git@FreeBSD.org>

                                Several FreeBSD developers have expressed interest in improving the tools and documentation to facilitate the use of the Git source code management (SCM) system when working with FreeBSD code. Some highlights of the work in this area include the following:

                                • At Alfred Perlstein's request, a new mailing list freebsd-git@FreeBSD.org was created for discussion of git use in the FreeBSD project.
                                • Alfred Perlstein submitted a patch to git. This patch allows a developer to work on a source code tree in git and use git-svn to push changes from this tree directly to a Subversion repository and set Subversion properties. Before this patch, git-svn did not properly set Subversion properties. This is important for FreeBSD developers because the FreeBSD Subversion repo will block commits which do not properly set certain Subversion properties. The git project accepted this change in changeset 83c9433.
                                • Alfred Perlstein updated the Git Workflow wiki document to include information for using git-svn to commit to the FreeBSD Subversion repository.
                                • Bartek Rutkowski wrote a script which integrates Github and FreeBSD Bugzilla. When a user files a Github pull request against the FreeBSD source code tree on Github, this script will open a new PR in FreeBSD Bugzilla. This will allow users to contribute code and patches via Github pull requests, and have the request tracked by FreeBSD developers in Bugzilla. Github pull requests cannot currently be directly merged into the FreeBSD source tree on Github, because the main source code repository is currently Subversion. The FreeBSD source code tree on Github is a read-only mirror of the FreeBSD Subversion repository. Craig Rodrigues coordinated with Bartek Rutkowski and bugmeister@FreeBSD.org to move forward on this, and provide Bartek Rutkowski with enough access to Bugzilla to open PR's via a script.

                                Open tasks:

                                1. The Github integration script is not deployed yet and is not active for all pull requests against the FreeBSD source tree on Github. Bartek Rutkowski and bugmeister@FreeBSD.org need to work out the final details for deploying this script into production. The script must be accessible via HTTP POST requests because it uses the Github REST API. Bartek Rutkowski and bugmeister@FreeBSD.org need to reach agreement on where this script lives, and do a security audit.

                                Jenkins Continuous Integration for FreeBSD

                                Links
                                Jenkins CI server in FreeBSD cluster URL: https://jenkins.freebsd.org
                                FreeBSD and Jenkins OS Testing URL: http://jenkins-ci.org/content/freebsd-project-use-jenkins-os-testing
                                Kyua and Jenkins Testing Framework for BSD URL: https://wiki.freebsd.org/201411DevAndVendorSummit?action=AttachFile&do=view&target=kyua_jenkins.pdf
                                PAM authentication problems in FreeBSD URL: https://issues.jenkins-ci.org/browse/JENKINS-21507
                                Issue to update Jenkins to JNA 4.1.0 for FreeBSD URL: https://issues.jenkins-ci.org/browse/JENKINS-24521
                                Kyua Quickstart Guide URL: https://github.com/rodrigc/kyua/wiki/Quickstart-Guide
                                Igor tool for proofreading documentation URL: http://www.freshports.org/textproc/igor/
                                FreeBSD_Doc-igor Jenkins build URL: https://jenkins.freebsd.org/job/FreeBSD_Doc-igor
                                FreeBSD_HEAD_sparc64 Jenkins build URL: https://jenkins.freebsd.org/job/FreeBSD_HEAD_sparc64/
                                Susan Stanziano from Xinuous ran kyua tests URL: https://lists.freebsd.org/pipermail/freebsd-testing/2014-November/000609.html
                                Andy Zhang from Microsoft ran kyua tests URL: https://lists.freebsd.org/pipermail/freebsd-testing/2014-December/000697.html
                                Ngie Cooper imported NetBSD tests URL: https://lists.freebsd.org/pipermail/svn-src-all/2014-October/092212.html
                                Steve Wills ran tests in Google Compute Engine URL: http://lists.freebsd.org/pipermail/freebsd-testing/2015-January/000713.html
                                Kyua submitted to Homebrew project URL: https://github.com/Homebrew/homebrew/pull/32346
                                Brian Gardner submits jtreg tests URL: https://github.com/freebsd/freebsd-ci/pull/3
                                Ahmed Kamal offers to help with Saltstack URL: https://lists.freebsd.org/pipermail/freebsd-testing/2015-January/000723.html
                                MIPS builds URL: https://lists.freebsd.org/pipermail/freebsd-testing/2015-January/000722.html

                                Contact: Craig Rodrigues <rodrigc@FreeBSD.org>
                                Contact: Jenkins Administrators <jenkins-admin@FreeBSD.org>
                                Contact: FreeBSD Testing <freebsd-testing@FreeBSD.org>

                                Since the last status report, many people have contributed help in various areas to help with Continuous Integration and Testing in FreeBSD. Some of the highlights include:

                                • The Jenkins project mentioned on their blog how FreeBSD is using Jenkins and kyua to run OS-level tests.
                                • Craig Rodrigues submitted patches to upgrade Jenkins to use JNA 4.1.0. The Jenkins project accepted these patches [JENKINS-24521] in the Jenkins 1.586 release. This fixed problems with PAM authentication support in Jenkins on FreeBSD [JENKINS-21507].
                                • Craig Rodrigues gave a presentation "Kyua and Jenkins Testing Framework" for BSD at the Developer and Vendor summit on November 3, 2014 in San Jose, California. In the presentation, Craig Rodrigues described how, for every commit to the FreeBSD source tree, nearly 3000 tests are run using kyua inside a bhyve virtual machine. The kyua test results are exported to JUnit XML format, which is then used by Jenkins to generate web-based test reports with graphs.
                                • Li-Wen Hsu set up a Jenkins build named FreeBSD_Doc-igor to run the Igor tool written by Warren Block. Igor proofreads FreeBSD documentation and reports various errors.
                                • Craig Rodrigues set up a Jenkins build named FreeBSD_HEAD_sparc64 to build the FreeBSD HEAD branch for the sparc64 architecture
                                • Ngie Cooper imported more tests from NetBSD. After this import, there are now over 3000 tests in the /usr/tests directory.
                                • Susan Stanziano from Xinuous ran kyua tests and provided feedback about test errors, running the tests in a bhyve VM.
                                • Andy Zhang from Microsoft ran kyua tests and provided feedback about test errors running in a Hyper-V 2012R2 VM.
                                • Steve Wills ran the FreeBSD tests in Google Compute Engine and provided the test results.
                                • Craig Rodrigues submitted a formula to create a package for kyua in the Homebrew packaging system on OS X. The Homebrew project accepted this. Now, kyua can easily be installed on OS X via a Homebrew package. Hopefully this will make it easier to share more test infrastructure and scripts with OS X.
                                • Craig Rodrigues submitted to the Debian project a kyua package. Approval for this is still pending. A package will make it much easier to install kyua on Linux distributions which use Debian packages such as Debian, Ubuntu, and Linux Mint. Hopefully this will make it easier to share more test infrastructure and scripts with Linux.
                                • Brian Gardner submitted scripts to run the Regression Test Harness for OpenJDK (jtreg). The test results are in JUnit XML format, which can be natively imported into Jenkins.
                                • Ahmed Kamal, an experienced devops expert and past contributor to the Ubuntu project, offered to help Craig Rodrigues with improving the automation and deployment of Jenkins nodes in the FreeBSD cluster using the Saltstack automation framework. Ahmed is interested in helping the FreeBSD project.
                                • Craig Rodrigues worked with Adrian Chadd to set up Jenkins builds of MIPS targets. The next step will be to get kyua tests running inside a QEMU MIPS VM.

                                Open tasks:

                                1. Set up more builds based on different architectures.

                                2. Improve the maintenance of nodes in the Jenkins cluster using devops frameworks such as Saltstack.

                                3. Get feedback for improving the Kyua Quickstart Guide.

                                4. People interested in helping out should join the freebsd-testing@FreeBSD.org list.

                                Migration to ELF Tool Chain Tools

                                Links
                                URL: http://elftoolchain.sourceforge.net

                                Contact: Ed Maste <emaste@freebsd.org>

                                The ELF Tool Chain project provides BSD-licensed implementations of compilation tools and libraries for building and analyzing ELF objects. It started as part of FreeBSD but has moved to a standalone project to encourage wider participation from others in the open-source developer community. FreeBSD's libelf and libdwarf are now imported from upstream sources in contrib/elftoolchain.

                                ELF Tool Chain provides a set of tools equivalent to the GNU Binutils suite. This project's goal is to import these tools into the FreeBSD base system so that we have a set of up-to-date and maintained tools that also provide support for new CPU architectures of interest, such as arm64.

                                The following tools have now been imported and are available by setting the src.conf knob WITH_ELFTOOLCHAIN_TOOLS=yes:

                                • addr2line
                                • nm
                                • size
                                • strings
                                • strip (elfcopy)

                                A ports exp-run uncovered some bugs in these tools. The bugs are being fixed in the FreeBSD source tree and are in the process of being committed to the upstream project.

                                ELF Tool Chain's readelf will be enabled as well once some missing functionality in ELF note parsing is added. ELF Tool Chain's elfcopy provides equivalent functionality to Binutils' objcopy, and accepts the same command-line arguments. For it to be a viable replacement for all uses of objcopy in the base system, it must gain support for writing portable exectuable (PE) format binaries, which are used by UEFI boot code.

                                The ELF Tool Chain project does not currently provide replacements for as, ld, and objdump. For FreeBSD these tools will likely be obtained from the LLVM project.

                                This project was sponsored by The FreeBSD Foundation.

                                Open tasks:

                                1. Import readelf.

                                2. Add missing functionality to readelf.

                                3. Add missing functionality to elfcopy and migrate the base system build.

                                4. Fix issues found by fuzzing inputs to the tools.

                                5. Switch the default to WITH_ELFTOOLCHAIN_TOOLS.

                                pkg(8)

                                Links
                                URL: https://github.com/freebsd/pkg

                                Contact: The pkg team <pkg@FreeBSD.org>

                                The package development team has released pkg(8) 1.4. This release fixes lots of bugs and adds some new features:

                                • Stricter checking of paths passed via the plist
                                • Change the ABI string to be closer to MACHINE_ARCH
                                • Add three-way merge functionality
                                • Add conservative upgrade support for multi repository configurations
                                • Multirepository priority

                                An important part of the development direction for the 1.4 release was stabilizing the existing features and improving the pkg(8) experience on small/embedded machines (reducing memory usage and speeding up operations).

                                pkg(8) is not only the FreeBSD Package Manager, but also the Package Manager for DragonflyBSD. Support has been added to build pkg(8) on OS X and Linux. This work will allow other Operating Systems the option of adopting pkg(8) to manage their packages and bring new developers into the project.

                                Open tasks:

                                1. Add more regression tests.

                                2. Package the FreeBSD base system.

                                3. Allow using mtree as a plist when creating a package.

                                4. Implement flexible dependencies.

                                5. Test the development branch.

                                6. More developers are needed, check the Issues on Github.


                                Kernel


                                FreeBSD Xen

                                Links
                                FreeBSD PVH DomU wiki page URL: http://wiki.xen.org/wiki/FreeBSD_PVH
                                FreeBSD PVH Dom0 wiki page URL: http://wiki.xen.org/wiki/FreeBSD_Dom0

                                Contact: Roger Pau Monné <royger@FreeBSD.org>
                                Contact: Justin T. Gibbs <gibbs@FreeBSD.org>

                                During this quarter almost all pending Xen changes have been committed, enabling FreeBSD to be used as Dom0 under the new PVH mode. The set of features supported by FreeBSD is still limited, but it should allow for basic usage of FreeBSD as Dom0. Support for booting Xen from the FreeBSD boot loader will be committed very soon to HEAD.

                                Apart from testing on a variety of hardware, work has now shifted to improve PVH support in Xen itself in order to have feature parity with a traditional PV Dom0 and to declare the PVH ABI as stable.

                                Regarding guest improvements (running FreeBSD as a DomU), there is also ongoing work to add unmapped IO support to Xen blkfront, which is blocked pending some modifications to the generic bounce buffer code.

                                This project was sponsored by Citrix Systems R&D, and Spectra Logic Corporation.

                                Open tasks:

                                1. Test on different hardware.

                                2. Improve the performance of the netback and blkback backends.

                                3. Work with upstream Xen to improve PVH and make it stable.

                                4. Improve generic bounce buffer code for unmapped bios in order to support blkfront alignment requirements.

                                Linux Emulation Layer, the Linuxulator

                                Links
                                URL: https://svnweb.freebsd.org/base/user/dchagin/lemul/
                                URL: https://reviews.freebsd.org/differential/query/i9Ua2XMYQtNX/

                                Contact: Dmitry Chagin <dchagin@FreeBSD.org>

                                The main goal of the Linux emulation layer project is the execution on FreeBSD of multithreaded Linux applications that require the glibc library version 2.20 or later to be available. Glibc 2.20 requires a Linux kernel (or emulation thereof) of version 2.6.32 or later. The main obstacle preventing this is that the current Linuxulator uses native FreeBSD processes for emulating Linux threads. This leads to several problems, including problems with process reparenting and dethreading, wait() and signal handling. It would be much better to reuse the FreeBSD kernel code for thread management than to create a completely new codebase for pseudothread management in the Linuxulator.

                                At present, the linux emulation layer project has implemented all of the necessary system calls for supporting glibc 2.20, and more, bringing the emulated Linux kernel version to 2.6.32:

                                • Using native threads for emulating Linux threads
                                • Implemented VDSO support, including DWARF for signal trampolines, which are needed for stack unwinding in pthread_cancel()
                                • Implemented the "vsyscall hack", used by some Linux-based distributions, including CentOS 6
                                • Implemented the epoll() system call emulation
                                • Added support for x86_64
                                • Many bugs were fixed

                                The project's code is located in the FreeBSD Project's Subversion repository at base/user/dchagin/lemul (a little bit old). To facilitate merging the improvements back to head, several patches have been placed on reviews.FreeBSD.org with the tag #lemul. Nearly half of the patches have already been approved by Ed Maste and Edward Tomasz Napierała.

                                Open tasks:

                                1. Review and merge the lemul branch to head within the next month or two.

                                2. Implement native and Linuxulator inotify() system calls.

                                3. Implement the ptrace() system call for the x86_64 Linuxulator.

                                4. Implement the signalfd() and timerfd() system calls for the Linuxulator.

                                5. Implement Priority Inheritance Futexes for the Linuxulator.

                                6. Extend xucred support, required for many Linux applications.

                                PCI SR-IOV Infrastructure

                                Links
                                URL: https://github.com/rysto32/freebsd/commits/iov_ixl

                                Contact: Ryan Stone <rstone@FreeBSD.org>

                                PCI Single Root I/O Virtualization (SR-IOV) is an optional part of the PCIe standard that provides hardware acceleration for the virtualization of PCIe devices. When SR-IOV is in use, a function in a PCI device (known as a Physical Function, or PF) will present multiple Virtual PCI Functions (VF) on the PCI bus. These VFs are fully independent PCI devices that have access to the resources of the PF. For example, on a network interface card, VFs could transmit and receive packets independently of the PF.

                                The most obvious use case for SR-IOV is virtualization. A hypervisor like bhyve could instantiate a VF for every VM and use PCI passthrough to assign the VFs to the VMs. This would allow multiple VMs to share access to the PCI device without having to do any expensive communication with the hypervisor, greatly increasing the performance of I/O within a VM.

                                Work on the core PCI infrastructure is complete and undergoing review. Currently it is planned to commit the PCI infrastructure to head by the end of January.

                                In addition to the PCI infrastructure, individual PCI drivers must be extended to implement SR-IOV. An SR-IOV implementation is in progress for the ixl(4) driver, which supports the Intel XL710 family of 40G and 10G NICs. Currently it is planned to have this in review by the end of January. An implementation for ixgbe(4) is also in progress, but there is no timeline for completion.

                                This project was sponsored by Sandvine Inc..

                                Process Management

                                Contact: Konstantin Belousov <kib@FreeBSD.org>
                                Contact: Peter Holm <pho@FreeBSD.org>

                                There were several improvements made to FreeBSD's process management last quarter.

                                The Reaper facility was added, allowing a process to reliably track the running and exiting state of the whole subtree of its processes. It is intended to improve tools like timeout(1) or poudriere, by making it impossible for a runaway grandchild to escape the controlling process. The feature was designed based on similar facilities in DragonFlyBSD and Linux, with some references to Solaris contracts. Committed to HEAD in r275800.

                                The FreeBSD suspension code does not ensure that the system, both software and hardware, is in a steady and consistent state. One aspect is usermode process activity, which is not yet stopped, continuing to making requests to the hardware. It is not realistic to expect drivers to be able to correctly handle the calls after SUSPEND_CHILD.

                                We developed a facility to stop usermode threads at safe points, where they are known to not own and to not wait for kernel resources, in particular, not waiting for device requests finishing. It is based on the existing single-threading code, but extending it to allow external thread to put some processes into stopped state. Also, a facility to sync filesystems before suspend was added, to ensure that consistent metadata and as much as possible of the cached user data are on stable storage, to minimize the damage that could be caused by a failed resume.

                                The code stressed some parts of the system and has led to discovery of a number of bugs in different areas, including process management, buffer cache, and syscall handlers. The bugs were fixed, and the fixes and features commmitted by a series culminating in r275745.

                                During the work described above, it was noted that process spinlock duties are significantly overloaded (the same is true for the process lock). The spinlock was split into per-feature locks in r275121. As a result, it was also possible to eliminate recursion on it in r275372.

                                This project was sponsored by The FreeBSD Foundation.

                                Secure Boot

                                Links
                                URL: https://wiki.freebsd.org/SecureBoot

                                Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                                UEFI Secure Boot is a mechanism that requires boot drivers and operating system loaders to be cryptographically signed by an authorized key. It will refuse to execute any software that is not correctly signed, and is intended to secure boot drivers and operating system loaders from malicious tampering or replacement.

                                This project will deliver the initial phase of secure boot support for FreeBSD and consists of:

                                • creating ports/packages of the gnu-efi toolchain, Matthew Garrett's shim loader, and sbsigntools
                                • extending the shim to provide an API for boot1.efi to load and verify binaries signed by keys known to the shim
                                • writing uefisign(8), a BSD-licensed utility to sign EFI binaries using Authenticode, as mandated by the UEFI specification.

                                This project was sponsored by The FreeBSD Foundation.

                                Open tasks:

                                1. Ensure that the signature format properly matches UEFI spec requirements.

                                2. Verify that correctly signed, incorrectly signed, and unsigned loader components are handled properly.

                                3. Investigate signed kernel ELF objects (including modules).

                                Timer Function Support for Linuxulator

                                Contact: Bjoern A. Zeeb <bz@FreeBSD.org>

                                Since 2006, initial support for Linux timer function compatibility support was present but untested. This update corrects the initial implementation and makes it available to the 32-bit Linuxulator on amd64, not just on i386.

                                Starting with FreeBSD 10.1, this enables users to run another FPGA high-level synthesis toolchain and emulation platform on a FreeBSD system.

                                This project was sponsored by DARPA, and AFRL.

                                Updating OpenCrypto

                                Links
                                r275732 changeset URL: https://svnweb.freebsd.org/changeset/base/r275732
                                FreeBSD Foundation announces IPsec Enhancement Project URL: http://freebsdfoundation.blogspot.com/2014/08/freebsd-foundation-announces-ipsec.html

                                Contact: John-Mark Gurney <jmg@FreeBSD.org>

                                The project adds support for AES-GCM and AES-CTR modes to the OpenCrypto framework. Both software and AES-NI accelerated versions are functional, working and committed. Ermal Luçi (eri@) is working on adding support for the additional modes to IPsec.

                                This project was sponsored by The FreeBSD Foundation, and Netgate.

                                Open tasks:

                                1. Commit the port that provides the NIST KAT vectors so that the tests committed can run.


                                Architectures


                                FreeBSD on POWER8

                                Links
                                URL: https://wiki.freebsd.org/POWER8
                                URL: http://www.tyan.com/campaign/openpower/

                                Contact: Nathan Whitehorn <nwhitehorn@freebsd.org>
                                Contact: Justin Hibbits <jhibbits@freebsd.org>
                                Contact: Adrian Chadd <adrian@freebsd.org>

                                IBM and the OpenPOWER Foundation are pushing for a wider software and hardware ecosystem for POWER8-based systems. Beginning January 3, we have been doing bringup work on a Tyan GN70-BP010 POWER8 server, a quad-core 3 GHz system with 32 hardware threads.

                                The main target for the port is the PowerKVM hypervisor provided on OpenPOWER hardware. This uses the same software interfaces as the PowerVM hypervisor already supported on earlier POWER hardware. The target is to have this operation mode fully supported by FreeBSD 10.2. FreeBSD currently runs under the hypervisor when using a mass storage driver other than the built-in virtualized SCSI; the issues with the SCSI driver should be solved shortly.

                                The longer-term goal is to also operate on the bare system. This requires interaction with the OPAL system firmware and the development of device drivers for the on-board PCI, console, and interrupt controller hardware. As of January 4, the FreeBSD kernel had printed initial messages to the console.

                                This project was sponsored by FreeBSD Foundation.

                                Open tasks:

                                1. Fix virtualized SCSI driver in PowerKVM.

                                2. Write OPAL drivers.

                                3. Integrate loader(8) with petitboot bootloader.

                                FreeBSD/arm64

                                Links
                                URL: https://wiki.freebsd.org/arm64
                                URL: https://github.com/FreeBSDFoundation/freebsd/tree/arm64-dev

                                Contact: Andrew Turner <andrew@FreeBSD.org>
                                Contact: Ed Maste <emaste@FreeBSD.org>
                                Contact: Zbigniew Bodek <zbb@semihalf.com>

                                There is growing interest in ARM's 64-bit architecture. Officially named AArch64, it is also known as ARMv8 and arm64. Andrew Turner started initial work on the FreeBSD/arm64 port at the end of 2012.

                                The FreeBSD Foundation is now collaborating with ARM, Cavium, the Semihalf team, and Andrew Turner to port FreeBSD to arm64, and significant progress was made on the port over the last quarter of 2014.

                                As of the end of the year, FreeBSD boots to single-user mode on arm64, executing both static and dynamic applications. Patches in review allow FreeBSD to boot to multi-user mode, and these are expected to be merged soon. This includes implementing many stub functions in userland and the kernel. With this, FreeBSD has booted to multi-user mode on both the ARM Foundation Model and the QEMU full system emulation.

                                Cavium has supplied a software simulator of their Thunder X hardware. Bringup of FreeBSD has started on this including writing new drivers for the ARM Generic Interrupt Controller v3 (GICv3) and a preliminary driver for the PCIe root complex. With these, FreeBSD is able to boot on this simulator in preparation for testing on hardware. Further work is progressing to add full PCIe bringup and to add support for the GICv3 Interrupt Translation Services (ITS) for MSI-X.

                                Further improvements have been made to the loader to allow it to take the Flattened Device Tree data from UEFI and pass it to the kernel. In the kernel, busdma, CPU identification, and improvements to interrupt handling have been made, along with preliminary KDB support.

                                Hardware for testing the port will be installed in the FreeBSD Test Cluster hosted by Sentex Communications. The first reference platform, Cavium's ThunderX, is expected to arrive in the cluster in mid-January.

                                This project was sponsored by The FreeBSD Foundation, ARM, and Cavium.

                                Open tasks:

                                1. Bring up and test kernel support on real hardware.

                                2. Implement the remaining userland libraries and binaries.

                                3. Produce installable images.


                                Userland Programs


                                libxo: Generate Text, XML, JSON, and HTML Output

                                Links
                                libxo: The Easy Way to Generate text, XML, JSON, and HTML output. URL: http://juniper.github.io/libxo/libxo-manual.html

                                Contact: Marcel Moolenaar <marcel@FreeBSD.org>

                                Many FreeBSD utilities provide insight into the operational state of a running FreeBSD system and as such are used regularly to monitor the system. These utilities provide their output in a human readable form and sometimes even optimized for the limited width of traditional terminals. Often times these utilities are used by other programs that want to present the output in different ways or as part of other user interfaces. For such use cases, it is infinitely better to work with machine-readable output instead of human-readable output.

                                Juniper Networks has created a library called libxo, which makes it easy for utilities to emit output in various formats. By default, text output is emitted, but with the introduction of the --libxo option this can be changed to XML, JSON, and HTML. The FreeBSD project has imported this library into the base system and is in the process of rewriting utilities to use libxo.

                                Related to this, FreeBSD now also has the xo utility that allows scripts to grow the same capabilities. Instead of using echo or printf in scripts, output can be done using the xo utility.

                                The df, w, and wc utilities have been converted to use libxo. The netstat utility is in the process of being converted and others are planned.

                                Open tasks:

                                1. FreeBSD contains a lot of utilities that could benefit from having the ability to emit various output formats, too many for a few people to convert in time for FreeBSD 11.0-RELEASE. If you or your company would like to see a particular utility converted, consider learning about libxo and trying to perform the conversion of said utility to help out.

                                mandoc(1) Support

                                Links
                                URL: http://mdocml.bsd.lv

                                Contact: Baptiste Daroussin <bapt@FreeBSD.org>
                                Contact: Ulrich Spoerlein <uqs@FreeBSD.org>
                                Contact: The Documentation Team <docs@FreeBSD.org>

                                mandoc(1) has been made the default manual page formatter on HEAD — man(1) will use mandoc(1) to format manual pages by default, then fall back to groff(1) if it fails.

                                This change also fixes an issue with the FreeBSD man(1) command not being able to properly deal with ".so" in gzipped manual pages.

                                The documentation team has spent a lot of time fixing issues reported by mandoc(1) in the FreeBSD manual pages. This greatly improves the quality of our manual pages.

                                Most manual pages with remaining issues are from contrib/, for which changes should be reported and fixed upstream.

                                The "manlint" target has also been switched to use mandoc -Tlint, which results in the target being more useful when working on manual pages.

                                Some groff(1) versus mandoc(1) formatting differences have been spotted and reported to mandoc's upstream developers.

                                Open tasks:

                                1. Switch makewhatis(1) to the version shipped with mandoc(1).

                                2. Figure out a way to detect mandoc(1)-unfriendly manpages in ports and create catpages with groff(1) for them.

                                3. Remove groff(1) from the base system.


                                Ports


                                GNOME on FreeBSD

                                Links
                                FreeBSD GNOME Team website URL: http://www.freebsd.org/gnome
                                FreeBSD Team development repo URL: https://github.com/freebsd/freebsd-ports-gnome
                                GNOME jhbuild buildbot wiki pages. URL: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD

                                Contact: FreeBSD GNOME Team <gnome@FreeBSD.org>

                                The FreeBSD GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for FreeBSD. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel.

                                This quarter was an exciting time for the GNOME Team. We imported GNOME 3.14.0 and CINNAMON 2.2.16 into the ports tree. At the same time, we removed the old GNOME 2.32 desktop. And two weeks later we updated GNOME to 3.14.2 and CINNAMON to 2.4.2, which was collected while the preparation for the initial GNOME 3.14.0 import was under way.

                                We moved our development repo to GitHub. The repo is structured as follows: the master branch is vanilla FreeBSD Ports, and we have theme branches for topics such as the porting of MATE 1.9 (the mate-1.10 branch) and GNOME 3.15 (the gnome-3.16 branch). The GNOME 3.14 branch (gnome-3.14) is not used or updated any more because the content has been committed to ports, but is kept around for the history.

                                Open tasks:

                                1. The GNOME website is stale. Work is starting on updating the development section. We could use some help here.

                                2. MATE 1.10 porting is under way; the latest 1.9 releases are available in the mate-1.10 branch.

                                3. GNOME 3.16 porting is under way, and is available in the gnome-3.16 branch.

                                KDE on FreeBSD

                                Links
                                URL: https://freebsd.kde.org/
                                URL: https://freebsd.kde.org/area51.php
                                URL: https://wiki.freebsd.org/KDE
                                URL: https://mail.kde.org/mailman/listinfo/kde-freebsd
                                URL: http://portscout.freebsd.org/kde@freebsd.org.html

                                Contact: KDE on FreeBSD team <kde@FreeBSD.org>

                                The KDE on FreeBSD team focuses on packaging and making sure that the experience of KDE and Qt on FreeBSD is as good as possible.

                                As mentioned last quarter, Alonso Schaich (alonso@) became a committer and since then has made good progress helping his mentors Raphael Kubo da Costa (rakuco@) and Max Brazhnikov (makc@) maintain all Qt and KDE-related ports.

                                This quarter, Qt 5.3 was finally committed to the ports tree. Extensive work was required, including cleaning up and/or changing a lot of the Qt5 ports infrastructure to make it both easier to maintain the Qt ports as well as finally make it possible to build newer versions when older ones are already installed on the system.

                                We have also updated KDE in our experimental area51 repository and committed several updates to other ports such as KDevelop and KDE Telepathy. Overall, we have worked on the following releases:

                                • CMake 3.1.0 (in area51, exp-run in progress for it to be committed to the ports tree)
                                • Calligra 2.8.6 (in area51)
                                • KDE 4.14.2 (committed to ports), 4.14.3 (in area51)
                                • KDE Telepathy 0.8.0 (committed to ports)
                                • KDevelop 4.7.0 (committed to ports)
                                • Qt 5.3.2 (committed to ports)

                                Tobias Berner has contributed patches to update QtCreator to 3.3.0 as well as KDE Frameworks 5 ports which are under review for inclusion in our experimental area51 repository.

                                Open tasks:

                                1. Update Qt5 to 5.4.0.

                                2. Try to contribute to the work on getting rid of HAL on FreeBSD, which seems to be gaining more traction recently.

                                3. Add KDE Frameworks 5 ports to our experimental repository.

                                Linux Emulation Ports

                                Links
                                contains additions for CentOS 6.5 64bit ports URL: https://github.com/allanjude/linux-ports
                                contains a work in progress of the Fedora 20 ports URL: https://github.com/vassilisl/freebsd-linux_base-f20
                                contains base Linux emulation enhancements required for 64 bit support URL: https://svnweb.freebsd.org/base/user/dchagin/lemul/

                                Contact: Johannes Meixner <xmj@FreeBSD.org>
                                Contact: Allan Jude <allanjude@FreeBSD.org>
                                Contact: Vassilis Laganakos <vassilis@einval.com >

                                The Linux emulation stack in the ports collection was upgraded to include CentOS 6.6 on November 11. After smoothing out several bugs that had been introduced, we were able to bump the default version of the Linux userland from Fedora 10 to CentOS 6.6 on December 9th. Providing a more modern Linux userland and support libraries allows a large number of Linux applications to be run on FreeBSD.

                                The goal behind providing an updated Fedora-based userland is to support more desktop-oriented applications, which require newer libraries than are provided by CentOS 6. Providing 64-bit versions of the CentOS userland will allow applications that are only available in 64-bit form, such as a number of scientific and math related applications, to be run on FreeBSD. Support for 64-bit binaries also requires the 64-bit Linux kernel emulation layer from the lemul branch, which requires more testing and review before being merged into HEAD.

                                This project was sponsored by Perceivon Hosting Inc., and ScaleEngine Inc..

                                Open tasks:

                                1. Update Allan Jude's 64-bit Linux ports to CentOS 6.6.

                                2. Add Fedora 20 base/userland ports to ports/head.

                                3. Refactor Mk/bsd.linux-*.mk to facilitate the above additions.

                                4. Promote testing and merging of Dmitry Chagin's lemul branch. (Updated Linux kernel emulation, and 64-bit support)

                                The Graphics Stack on FreeBSD

                                Links
                                Graphics stack roadmap and supported hardware matrix URL: https://wiki.freebsd.org/Graphics
                                Graphics stack team blog URL: http://blogs.freebsdish.org/graphics/
                                Ports development tree on GitHub URL: https://github.com/freebsd/freebsd-ports-graphics

                                Contact: FreeBSD Graphics team <freebsd-x11@FreeBSD.org>

                                Mesa was upgraded to 10.3, then 10.4 for FreeBSD 10.1-RELEASE and 11-CURRENT. We test release candidates and therefore this port is now usually updated shortly after a new release. Mesa 10.x brings huge improvements in terms of OpenGL standards support, performance and stability, especially for Radeon owners. Mesa 9.1 is kept for FreeBSD 9.x, but we have plans to fix this; see below.

                                graphics/gbm and devel/libclc are new ports used by Mesa to implement OpenCL. The next step is to finish the port for Mesa's libOpenCL.so, named Clover. This will permit users to run OpenCL programs on Radeon GPUs for now.

                                xserver was upgraded from 1.12 to 1.14. This is the last version of xserver supporting Mesa 9.1. Changes are described in an article on the blog. The most noticeable one is the switch from the input device detection back-end based on HAL to the one based on devd(8). hald(8) is still required by many desktop environments, but the X.Org server itself is free from it.

                                xserver was the last port supporting the WITH_NEW_XORG knob. The knob is now completely removed. This was the occasion to add WITH_NEW_XORG and WITH_KMS to the list of deprecated knobs to help people clean up their make.conf. At the same time, the new-xorg alternate pkg repository was deprecated.

                                After discussion, two options were enabled by default:

                                • TEXTURE_FLOAT in graphics/dri, which allows Mesa to advertise the support for OpenGL 3.0+;
                                • LCD_FILTERING in print/freetype2, which enables the subpixel rendering engine, improving font anti-aliasing.

                                These two packages now provide a better user experience out-of-the-box. Users who are uncomfortable with the options may unset them and rebuild the ports. There is no need to rebuild anything else.

                                On the kernel side, Tijl Coosemans added AGP support back to the TTM memory manager and therefore to the Radeon driver. His work was merged back to stable/10 and will be available in FreeBSD 10.2-RELEASE.

                                We migrated our Ports development tree to Git and GitHub. Tracking changes in the official Ports tree and preparing patches is much easier. Furthermore, we can accept pull requests. All of the reasons behind this change are detailed on the blog and the workflow is described on the wiki.

                                The XDC 2014 (X Developer's Conference) was a great conference. Reviving the relationship with the developers of the graphics stack was a success! A report is available on the blog.

                                Our next items on the roadmap are:

                                1. Provide FreeBSD 10.1-RELEASE's i915 driver to FreeBSD 9.x users through a new port. This is a work in progress, but it would allow us to remove Mesa 9.1 and make Mesa 10.4 available everywhere.
                                2. Once Mesa 9.1 is gone, we can update xserver to 1.16.

                                Open tasks:

                                1. See the "Graphics" wiki page for up-to-date information.

                                Wine/FreeBSD

                                Links
                                Wine wiki page URL: http://wiki.FreeBSD.org/Wine
                                Wine on amd64 wiki page URL: http://wiki.FreeBSD.org/i386-Wine
                                Wine homepage URL: http://www.winehq.org

                                Contact: Gerald Pfeifer <gerald@FreeBSD.org>
                                Contact: David Naylor <dbn@FreeBSD.org>
                                Contact: Kris Moore <kmoore@FreeBSD.org>

                                The Wine on FreeBSD project has been steadily forging ahead for the past three quarters and has updated the ports for the following versions:

                                • Stable releases: 1.6.2 (3 port revisions)
                                • Development releases: 1.7.16 through 1.7.33

                                The ports have packages built for amd64 (available through the ports emulators/i386-wine and i386-wine-devel) for FreeBSD 8.4, 9.1+, 10.0+, and CURRENT.

                                Accomplishments include:

                                • Upstreaming 33 patches to fix Wine on FreeBSD — many thanks to Gerald for this work.
                                • Migrating to the USES framework.
                                • Building Wine with the X compositing extension.
                                • Adding support for MPG123 and V4L.
                                • Backporting changes made to the -devel ports to the stable ones and fixing minutiae here and there.
                                • Creating a new Wine port for the Compholio patches.
                                • Changing i386-wine(-devel) to set the LD_LIBRARY_PATH_RPATH variable.
                                • Improving library bundling for i386-wine(-devel).
                                • Various improvements to the patch-nvidia.sh script for i386-wine(-devel).
                                • Various smaller changes.

                                We would like to thank all the volunteers who contributed feedback or even patches. We would also like to welcome kmoore@ to the Wine team. He has been extensively involved in bringing wine-compholio to the Ports Collection.

                                Future development on Wine will focus on:

                                • Creating a 64-bit capable port of Wine (aka Wine64).
                                • Creating a WoW64 capable port of Wine (aka Wine + Wine64).
                                • Fixing directory listing on FreeBSD 8 and 9.

                                Maintaining and improving Wine is a major undertaking that directly impacts end-users on FreeBSD, including many gamers. If you are interested in helping, please contact us. We will happily accept patches, suggest areas of focus, or have a chat.

                                Open tasks:

                                1. Open Tasks and Known Problems (see the Wine wiki page).

                                2. FreeBSD/amd64 integration (see the i386-Wine wiki page).

                                3. Porting WoW64 and Wine64.

                                Xfce

                                Links
                                URL: https://wiki.freebsd.org/Xfce

                                Contact: FreeBSD Xfce Team <xfce@FreeBSD.org>

                                Xfce is a free software desktop environment for Unix and Unix-like platforms, such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use.

                                During this quarter, the team has kept these applications up-to-date:

                                • misc/xfce4-weather-plugin 0.8.5
                                • science/xfce4-equake-plugin 1.3.6
                                • sysutils/xfce4-netload-plugin 1.2.4
                                • sysutils/xfce4-systemload-plugin 1.1.2
                                • www/midori 0.5.9
                                • x11/xfce4-taskmanager 1.1.0
                                • x11/xfce4-whiskermenu-plugin 1.4.2
                                • x11-wm/xfce4-desktop 4.10.3

                                Two new ports have also been added (taken from our repository):

                                • deskutils/xfce4-volumed-pulse
                                • x11/xfce4-dashboard

                                Moreover, we are working on the next stable release, with these ports being updated:

                                • sysutils/xfce4-power-manager 1.4.2
                                • x11/xfce4-dashboard 0.3.4
                                • x11-wm/xfce4-session 4.11.1

                                We sent some patches to upstream.

                                • bug #11104, to keep 'wallpaper settings' in Ristretto with xfdesktop >= 4.11
                                • bug #11249, add 'Hidden' option in desktop item editor (refused)
                                • bug #11413, to use sysctl(3) and acpi_video(4) for backlight support

                                A FAQ is being written D1305.

                                Open tasks:

                                1. Find a workaround for when acpi_video(4) is not functional (panel crashes); OpenBSD seems to have same problem.

                                2. Clean up patch in order to add new panel plugin in ports tree.

                                3. Continue to work on documentation, especially the Porter's Handbook.


                                Documentation


                                More Michael Lucas Books

                                Links
                                FreeBSD Mastery: Storage Essentials page URL: https://www.michaelwlucas.com/nonfiction/freebsd-mastery-storage-essentials
                                blog URL: http://blather.michaelwlucas.com

                                Contact: Michael Lucas <mwlucas@michaelwlucas.com>

                                The first small FreeBSD Book, "FreeBSD Mastery: Storage Essentials" is available.

                                Lucas is moving on to FreeBSD books on ZFS, Specialty Filesystems, and jails. They will hopefully be available by BSDCan 2015.

                                Get status updates on his blog, or follow @mwlauthor on Twitter.

                                Open tasks:

                                1. Push BSDCan out to June, so he has more time to write the new books.

                                New Translators Mailing List

                                Contact: FreeBSD Translators Mailing List <freebsd-translators@FreeBSD.org>

                                A new mailing list has been created for people translating FreeBSD documents and programs from English into other languages. Discussions can include methods, tools, and techniques. Existing translators are encouraged to join so there is a single point of contact. New translators and those who wish to help with translation are welcome.

                                New members are asked to introduce themselves and mention the languages they are interested in translating.

                                Open tasks:

                                1. Encourage existing translators to join.

                                2. Welcome and educate new volunteers.

                                3. Work on implementing newer and easier translation systems and tools.


                                Miscellaneous


                                Creating Vagrant Images with Packer

                                Links
                                Blog Announcement URL: http://blogs.freebsdish.org/brd/2014/12/22/freebsd-packer-vagrant/
                                Git Repo URL: https://github.com/so14k/packer-freebsd

                                Contact: Brad Davis <brd@FreeBSD.org>

                                We have developed a recipe to use Packer to create FreeBSD Vagrant images to run on VMware and VirtualBox.

                                Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.

                                Vagrant is a tool to create and configure lightweight, reproducible, and portable development environments.

                                To get started, clone the Git repo and follow the directions in the README. More information is available from the Packer and Vagrant websites.

                                FreeBSD Forum Software Migration

                                Contact: FreeBSD Forums Administration Team <forum-admins@>

                                With funding from the FreeBSD Foundation, the FreeBSD forums were migrated to XenForo software. The new software is far more capable and easy to use. While the entire forum team contributed, Daniel Geržo did an excellent job importing existing users and messages and bringing back the often-requested "Thanks" feature. The upgrade was completed in time to be ready for the influx of new users from the release of FreeBSD 10.1, and we have already seen an increase in usage.

                                Developers with an @FreeBSD.org address can contact forum administrators to obtain the highly-desired "@" suffix on their forum user name along with a Developer flag.

                                We want to thank the Foundation for making this possible, and the users for their patience and continued presence on the forums!

                                This project was sponsored by The FreeBSD Foundation.

                                Open tasks:

                                1. Encourage more developers and users to try the new forums.

                                2. Continue getting feedback from users for tuning and improvements.

                                The FreeBSD Foundation

                                Links
                                URL: http://www.FreeBSDFoundation.org/
                                FreeBSD Journal URL: http://freebsdjournal.com/

                                Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                                The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of the funding is used to support FreeBSD development projects, conferences, and developer summits; purchase equipment to grow and improve the FreeBSD infrastructure; and provide legal support for the Project.

                                We ended the year exceeding our fundraising goal, by raising over $2,372,132, from 1670 donors! Thank you to everyone who made a donation in 2014.

                                We produced issues five and six of the FreeBSD Journal, ending the year with over 6300 subscribers, exceeding our first-year goal of 5000 subscribers. We also added the desktop/digital edition, so people can read the magazine from their browsers. We also hosted a meeting with the Journal Editorial Board and worked out the editorial calendar for the next two years. This includes topics and articles for the future issues.

                                We were a gold sponsor of EuroBSDCon 2014, and a sponsor of the preceding Developer Summit. A few of our team members attended, which allowed us to have an informal face-to-face board meeting, with a focus on supporting the European region. Kirk McKusick gave a two-day FreeBSD tutorial and Erwin Lansing helped run the Developer Summit. We sponsored 5 FreeBSD contributors to attend the conference.

                                We were a sponsor of the Grace Hopper Conference. Dru Lavigne gave an "introduction to FreeBSD" presentation, that was well attended. We also sponsored Shteryana Shopova to represent FreeBSD, along with Dru, at our booth.

                                We were a sponsor of MeetBSD. Most of our team members attended this conference. Kirk McKusick gave a talk on BSD history. We also had a booth, and raised over $2,200 in donations. We sponsored one person to attend this conference.

                                George organized and ran the two-day Silicon Valley Vendor and Developer Summit following MeetBSD. A lot of work gets started and accomplished at these summits, for example, Kirk worked with various folks to get the ino64 (64-bit inode numbers) project moving. It started in 2011 as a Summer of Code project and has sputtered since getting pushed into the system.

                                In addition to the above conferences, we helped promote FreeBSD at the following conferences:

                                LISA had a great turnout for Dru Lavigne's FreeBSD BoF talk.

                                We visited a few large FreeBSD users in the Bay Area to discuss their use of FreeBSD, plans, and needs, and help facilitate collaboration between them and the Project.

                                Cheryl Blain joined our board, bringing a strong background in business development and fundraising.

                                We received the largest donation in our history, and our treasurer put together an endowment strategy for us to follow.

                                We increased our FreeBSD marketing efforts to help promote and advocate for FreeBSD, as well as educate people on FreeBSD. Some our FreeBSD marketing highlights include:

                                • Created the FreeBSD 10 brochure
                                • Created the Get Involved brochure for recruiting
                                • Created a testimonial flyer to encourage more companies to write FreeBSD testimonials for us.
                                These flyers are available on the FreeBSD Foundation site for FreeBSD advocates to promote FreeBSD at conferences around the world. We also put ads for the Foundation and FreeBSD in the FreeBSD Journal and USENIX ;login: magazine.

                                We are producing a monthly newsletter to highlight what we did the previous month to support the FreeBSD Project. We also produced our December semi-annual newsletter.

                                We redesigned and launched phase 1 of our website. It should be easier to navigate and find the information you need to get help from or to help the Foundation.

                                Glen Barber visited the Microsoft main campus and worked with Microsoft Hyper-V developers to resolve outstanding issues with providing FreeBSD images for the Microsoft Azure platform.

                                Glen also visited the NYI colocation facility to install and configure new servers purchased by the Foundation.

                                We finished the 10.1-RELEASE cycle.

                                Our project development staff and contractors have been working on various projects to add features to and improve FreeBSD. Some of their reports are included in this overall report. Some projects that were worked on this quarter were adding support for 64-bit ARM architecture to FreeBSD, integration work on the vt(4) updated console and UEFI boot support, Secure Boot, refining the in-kernel iSCSI target and initiator stack, an autofs-based automount daemon, migrating to the ELF Tool Chain, and implementing modern AES modes in FreeBSD's cryptographic framework.

                                To read more about how we helped support the FreeBSD Project and community, read our semi-annual newsletter.

                                News Home | Status Home

                                diff --git a/website/content/en/status/report-2015-01-2015-03.html b/website/content/en/status/report-2015-01-2015-03.html index 3db2012018..9f60d89874 100644 --- a/website/content/en/status/report-2015-01-2015-03.html +++ b/website/content/en/status/report-2015-01-2015-03.html @@ -1,2070 +1,2070 @@ FreeBSD Quarterly Status Report
                                Skip site navigation (1) Skip section navigation (2)

                                Introduction

                                This report covers FreeBSD-related projects between January and March 2015. This is the first of four reports planned for 2015.

                                The first quarter of 2015 was another productive quarter for the FreeBSD project and community. FreeBSD is being used in research projects, and those projects are making their way back into FreeBSD as new and exciting features, bringing improved network performance and security features to the system. Work continues to improve support for more architectures and architecture features, including progress towards the goal of making ARM (32- and 64-bit) a Tier 1 platform in FreeBSD 11. The toolchain is receiving updates, with new versions of clang/LLVM in place, migrations to ELF Tool Chain tools, and updates to the LLDB and gdb debuggers. Work by ports teams and kernel developers is maintaining and improving the state of FreeBSD as a desktop operating system. The pkg team is continuing to make binary packages easier to use and upgrade.

                                Thanks to all the reporters for the excellent work!

                                The deadline for submissions covering the period from April to June 2015 is July 7th, 2015.

                                FreeBSD Team Reports

                                Projects

                                Kernel

                                Architectures

                                Userland Programs

                                Ports

                                Documentation

                                Miscellaneous


                                  FreeBSD Team Reports


                                  FreeBSD Bugmeister

                                  Contact: FreeBSD Bugmeister <bugmeister@FreeBSD.org>

                                  Bugzilla replaced GNATS in June 2014 as the bug management tool of choice for FreeBSD, granting GNATS its well-deserved retirement after more than 20 years of operation. The following months were rough for Bugzilla: a lot of functionality was still missing and several uncertainties caused users and committers to adapt only slowly to the new system.

                                  Over the last six months, a lot of missing features were brought into place to allow users and committers to focus on getting bugs solved. Categories, the status model and many workflow-related knobs were continuously reworked and improved to provide the necessary information without getting in the way.

                                  An auto-assigner for ports issues was implemented, resembling what GNATS successfully did in the past. A dashboard page within Bugzilla provides users and committers with quick access to common queries and overall statistics; many other smaller tweaks, configurations, and extensions were implemented to improve the usability of the system.

                                  An improved reporting system is currently being implemented to provide graphs and statistics for users and committers. Handling MFCs and a better feedback mechanism for requests (flags in Bugzilla) will be the next things to do.

                                  Bugmeister is also working closely with the FreeBSD GitHub team to establish a workflow between GitHub's issue tracker and our Bugzilla system. The technical solution already exists as a proof of concept, but its usage in production will have to wait until Bugzilla 5.0 has been adopted.

                                  Open tasks:

                                  1. Create a solid charting extension for FreeBSD Bugzilla.

                                  2. Improve MFC handling.

                                  3. Do you feel that something important is missing? Let us know!

                                  Ports Collection

                                  Links
                                  URL: http://www.FreeBSD.org/ports/
                                  URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
                                  URL: http://portsmon.freebsd.org/index.html
                                  URL: http://portscout.freebsd.org/
                                  URL: http://www.freebsd.org/portmgr/index.html
                                  URL: http://blogs.freebsdish.org/portmgr/
                                  URL: http://www.twitter.com/freebsd_portmgr/
                                  URL: http://www.facebook.com/portmgr
                                  URL: http://plus.google.com/communities/108335846196454338383

                                  Contact: Frederic Culot <portmgr-secretary@FreeBSD.org>
                                  - Contact: Port Management Team <portmgr@FreeBSD.org> + Contact: Ports Management Team <portmgr@FreeBSD.org>

                                  As of the end of Q1 the ports tree holds almost 25,000 ports, and the PR count is just over 1,500. The tree saw more activity than during the previous quarter, with almost 7,000 commits performed by 163 active committers. The number of problem reports closed also increased by about 20%, with nearly 2,000 PRs closed!

                                  In Q1 two new developers were granted a ports commit bit (jbeich@ and brd@) and one bit was taken in for safekeeping (rafan@, on his request).

                                  On the management side, decke@ decided to step down from his portmgr duties in February. No other changes were made to the team during Q1.

                                  This quarter also saw the release of the first quarterly branch of the year, 2015Q1. On this branch, 140 changes were applied by 35 committers.

                                  On the QA side, 29 exp-runs were performed to validate sensitive updates or cleanups.

                                  Open tasks:

                                  1. As during the previous quarter a tremendous amount of work was done on the tree to update major ports and to close even more PRs than in 2014Q4. However, we sometimes lag behind with regards to documentation, so volunteers are welcome to help on this important task.

                                  The FreeBSD Core Team

                                  Contact: FreeBSD Core Team <core@FreeBSD.org>

                                  The FreeBSD Core Team constitutes the project's "Board of Directors", responsible for deciding the project's overall goals and direction as well as managing specific areas of the FreeBSD project landscape.

                                  January began with members of core dealing with the fallout from the accidental deletion of the Bugzilla database. This incident highlighted the fact that backup and recovery mechanisms in the cluster were not up to the task. Core has discussed what measures are appropriate with clusteradm and is reviewing their implementation.

                                  After a long process of consultation, plans for introducing the new support model with 11.0-RELEASE were finally agreed on and published in early February. This announcement puts the practical detail onto the motion that was adopted at BSDCan 2014, and clarifies the steps needed for implementation.

                                  Also in February core revisited discussions on making the blogs.freebsdish.org blog aggregator an official project service and also providing a blogging platform directly to developers. However, security and man-power are both major concerns. Given the track records of most freely available blogging platforms, core is rightly wary of introducing them into the cluster. Similarly, curating a blogging platform will take a substantial volunteer effort to ensure all posts are appropriate and to remove spam.

                                  March has seen two discussions about potentially divisive topics. Should the ZFS ARC Responsiveness patches be committed and MFC'd as a pragmatic fix to performance problems in 10.1-RELEASE, understanding that this is not an ideal solution to the problem and will need rework? Should we stop maintaining support for older (C89 or earlier) compilers in kernel code, and just code directly to the C11 standard? Broadening out from this last point: should we have a formal mechanism for deciding what has become obsolete in the system and when it should be removed?

                                  During this quarter five new src commit bits were granted and two were taken in for safe-keeping.


                                  Projects


                                  bhyve

                                  Links
                                  bhyve FAQ and talks URL: http://www.bhyve.org

                                  Contact: Peter Grehan <grehan@FreeBSD.org>
                                  Contact: Neel Natu <neel@FreeBSD.org>
                                  Contact: John Baldwin <jhb@FreeBSD.org>
                                  Contact: Tycho Nightingale <tychon@FreeBSD.org>
                                  Contact: Allan Jude <allanjude@FreeBSD.org>
                                  Contact: Alexander Motin <mav@FreeBSD.org>

                                  bhyve is a hypervisor that runs on the FreeBSD/amd64 platform. At present, it runs FreeBSD (8.x or later), Linux i386/x64, OpenBSD i386/amd64, and NetBSD/amd64 guests. Current development is focused on enabling additional guest operating systems and implementing features found in other hypervisors.

                                  Peter Grehan did a status update at bhyvecon 2015 in Tokyo. The slides are available at http://bhyvecon.org/bhyvecon2015-Peter.pdf.

                                  Mihai Carabas presented the results of his GSoC project on implementing instruction caching in bhyve at AsiaBSDCon 2015 in Tokyo. The slides are available at http://people.freebsd.org/~neel/bhyve/bhyve-cache-emul-slides.pdf.

                                  A number of improvements were made to bhyve this quarter:

                                  • The RTC device model can now be instructed to keep UTC time instead of localtime. This is useful for guests like OpenBSD that expect the RTC to keep UTC time.
                                  • The virtio-blk device now does I/O asynchronously without blocking the vcpu thread that initiated the I/O.
                                  • The virtio-blk and ahci-hd devices are now able to execute multiple I/O requests in parallel. This can significantly boost virtual disk throughput.
                                  • The ahci-hd device emulation advertises TRIM to the guest if the backend device supports it (e.g., ZVOL).
                                  • The virtio-blk and ahci-hd devices now advertise the proper logical and physical block size of the backend device or file.

                                  Open tasks:

                                  1. Improve documentation.

                                  2. bhyveucl is a tool for starting bhyve instances based on a UCL formatted config file. More information is at https://github.com/allanjude/bhyveucl

                                  3. Add support for virtio-scsi.

                                  4. Flexible networking backends such as wanproxy and vhost-net.

                                  5. Move to a single process model, instead of bhyveload and bhyve.

                                  6. Support running bhyve as non-root.

                                  7. Add filters for popular VM file formats (VMDK, VHD, QCOW2).

                                  8. Implement an abstraction layer for video (no X11 or SDL in the base system).

                                  9. Suspend/resume support.

                                  10. Live Migration.

                                  11. Nested VT-x support (bhyve in bhyve).

                                  12. Support for other architectures (ARM, MIPS, PPC).

                                  CheriBSD

                                  Links
                                  URL: http://cheri-cpu.org/

                                  Contact: Robert Watson <rwatson@FreeBSD.org>
                                  Contact: Brooks Davis <brooks@FreeBSD.org>
                                  Contact: David Chisnall <theraven@FreeBSD.org>
                                  Contact: Ruslan Bukin <br@FreeBSD.org>

                                  CheriBSD is a fork of FreeBSD to support the CHERI research CPU. We have extended the kernel to provide support for CHERI memory capabilities as well as modifying applications and libraries including tcpdump, libmagic, and libz to take advantage of these capabilities for improved memory safety and compartmentalization. We have also developed custom demo applications and deployment infrastructure for our table demo platform.

                                  As this goes to press, we are finalizing our first open source release of the CHERI CPU which will be available from the CHERI CPU website.

                                  We have been merging support for the BERI CPU platform to FreeBSD since 2012 and continue to do so as new features are developed. Most recently, Ruslan has added support for the Terasis SoCkit board which combines an ARM processor with an FPGA capable of running BERI (and soon CHERI) in a single package.

                                  This project was sponsored by DARPA/AFRL.

                                  Clang, llvm and lldb updated to 3.6.0

                                  Links
                                  LLVM 3.6.0 Release Notes URL: http://llvm.org/releases/3.6.0/docs/ReleaseNotes.html
                                  Clang 3.6.0 Release Notes URL: http://llvm.org/releases/3.6.0/tools/clang/docs/ReleaseNotes.html

                                  Contact: Dimitry Andric <dim@FreeBSD.org>
                                  Contact: Ed Maste <emaste@FreeBSD.org>
                                  Contact: Roman Divacky <rdivacky@FreeBSD.org>
                                  Contact: Davide Italiano <davide@FreeBSD.org>

                                  Just before the end of the quarter, we updated clang, llvm and lldb in the base system to the 3.6.0 release. These all contain numerous improvements; please see the linked release notes for more detailed information.

                                  We have also imported a newer snapshot of compiler-rt, with better support for the Address Sanitizer and the Undefined Behavior Sanitizer, and arm64 runtime support routines. With the updated clang, llvm, and compiler-rt, we now support the Address and Undefined Behavior Sanitizers in the base system toolchain.

                                  As with the 3.5.0 release, these components require C++11 support to build. C++11 support is available in FreeBSD 10.0 and later on the x86 architectures.

                                  It is still unclear whether we will be able to MFC these updates to any of the stable branches, due to the difficulty it will introduce for upgrading from a system without C++11 support, either from older releases or from architectures still using gcc.

                                  In the lld-import branch, we have also imported a recent snapshot of lld, a linker produced by the LLVM project. This is a very preliminary effort of making it available as a system linker.

                                  Thanks to Ed Maste, Roman Divacky, Andrew Turner and Davide Italiano for their help with this import, and thanks to Antoine Brodin for performing a ports exp-run.

                                  Open tasks:

                                  1. After the ports exp-run, a small number of ports turned out to have problems, and for almost all of these, PRs with fixes or workarounds were filed. While most of these PRs have been processed and closed, there are still a few left that need attention, from either the maintainer(s) or other volunteers.

                                  2. Andrew Turner is working on bringing up the arm64 architecture, which is now fully supported in clang and llvm. This will be a very interesting new area for solving challenging problems.

                                  3. There are still issues with the powerpc and sparc64 architectures, and any help in these areas is very much appreciated.

                                  FreeBSD on POWER8

                                  Links
                                  Tyan development reference platform URL: http://www.tyan.com/campaign/openpower/

                                  Contact: Nathan Whitehorn <nwhitehorn@freebsd.org>
                                  Contact: Justin Hibbits <jhibbits@freebsd.org>
                                  Contact: Adrian Chadd <adrian@freebsd.org>

                                  IBM and the OpenPOWER Foundation are pushing for a wider software and hardware ecosystem for POWER8-based systems. Starting in January 2014, we have been doing bringup work on a Tyan GN70-BP010 POWER8 server, a quad-core 3 GHz system with a total of 32 hardware threads.

                                  Updates since the previous report:

                                  • FreeBSD now boots under a hypervisor with the virtual SCSI block device; the issue previously preventing this has been fixed.
                                  • The powerpc64 pmap code was rewritten to be more scalable, as the previous pmap code did not scale beyond a small number of CPUs.
                                  • Initial support for IBM's Vector-Scalar Extensions (VSX) was added.
                                  • The FreeBSD kernel was made completely position independent for powerpc64, and later powerpc32 as well.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Open tasks:

                                  1. Get FreeBSD booting natively, rather than under KVM. This requires writing OPAL drivers for the various hardware devices in the system.

                                  2. Integrate loader(8) with petitboot.

                                  Jenkins Continuous Integration for FreeBSD

                                  Links
                                  The Jenkins CI server in the FreeBSD cluster URL: https://jenkins.freebsd.org
                                  Cloud9ers URL: http://www.cloud9ers.com/
                                  Ahmed Kamal URL: https://wiki.ubuntu.com/AhmedKamal
                                  Ahmed's contributions to SaltStack URL: https://github.com/saltstack/salt/pulls?q=is%3Apr+author%3Akim0
                                  Kyua turns parallel URL: http://julipedia.meroh.net/2015/02/kyua-turns-parallel.html
                                  Jenkins Multiple SCM's plugin fixes URL: https://github.com/jenkinsci/multiple-scms-plugin/commits?author=rodrigc
                                  GCC 4.9 problems URL: https://lists.freebsd.org/pipermail/freebsd-toolchain/2015-March/001545.html
                                  External Toolchain Support URL: https://wiki.freebsd.org/ExternalToolchain

                                  Contact: Craig Rodrigues <rodrigc@FreeBSD.org>
                                  Contact: Jenkins Administrators <jenkins-admin@FreeBSD.org>
                                  Contact: FreeBSD Testing <freebsd-testing@FreeBSD.org>

                                  The Jenkins Continuous Integration and Testing project has been helping to improve the quality of FreeBSD. Since the last status report, we have quickly found commits which caused build breakage or test failures. FreeBSD developers saw these problems and quickly fixed them. Some of the highlights include:

                                  • Ahmed Kamal agreed to join the jenkins-admin team. Even though he is not a FreeBSD committer, he is subscribed to the jenkins-admin alias, and is contributing code via GitHub. Ahmed has contributed multiple SaltStack scripts which are in the freebsd-ci GitHub repository. Ahmed has also found multiple bugs in SaltStack's FreeBSD support. He has fixed these bugs and pushed them back to SaltStack via GitHub pull requests.

                                    Ahmed is a software developer who lives in Cairo, Egypt. He presently works for Cloud9ers, a cloud and devops consulting firm. In the past, he has worked for Canonical as the Ubuntu Cloud and Server community liaison.

                                    Ahmed found out about the Request for Help sent out by Craig Rodrigues for help with Jenkins in FreeBSD via a random web search. Ahmed found FreeBSD to be a very nice project, and was eager to volunteer and help out, and responded to the Request. Ahmed will attend BSDCan, where he will learn more about the BSD Community.

                                  • Julio Merino extended Kyua to support executing test cases in parallel. This should help the scaling of testing in environments with thousands of test cases.
                                  • Craig Rodrigues got a commit bit to the Jenkins Multiple SCM's plugin, and committed fixes to that plugin to help it work with Subversion 1.8
                                  • Craig Rodrigues worked with Dimitry Andric in the freebsd-toolchain team to help identify and fix several compile problems in the FreeBSD src tree when using GCC 4.9. This work will help with the External Toolchain project.

                                  Open tasks:

                                  1. Set up more builds based on different architectures.

                                  2. Improve the maintenance of nodes in the Jenkins cluster using devops frameworks such as Saltstack.

                                  3. People interested in helping out should join the freebsd-testing@FreeBSD.org list.

                                  Lua boot loader

                                  Links
                                  URL: https://svnweb.freebsd.org/base/projects/lua-bootloader/

                                  Contact: Rui Paulo <rpaulo@FreeBSD.org>
                                  Contact: Pedro Souza <pedrosouza@FreeBSD.org>
                                  Contact: Wojciech Koszek <wkoszek@FreeBSD.org>

                                  The Lua boot loader project is in its final stage and it can be used on x86 already. The aim of this project is to replace the Forth boot loader with a Lua boot loader. All the scripts were re-written in Lua and are available in sys/boot/lua. Once all the Forth features have been tested and the boot menus look exactly like in Forth, we will start merging this project to FreeBSD HEAD. Both loaders can co-exist in the source tree with no problems because a pluggable loader was introduced for this purpose.

                                  The project was initially started by Wojciech Koszek, and Pedro Souza wrote most of the Lua code last year in his Google Summer of Code project.

                                  To build a Lua boot loader just use:

                                  WITH_LUA=y
 WITHOUT_FORTH=y

                                  Open tasks:

                                  1. Feature/appearance parity with Forth.

                                  2. Investigate use of floating point by Lua.

                                  3. Test the EFI Lua loader.

                                  4. Test the U-Boot Lua loader.

                                  5. Test the serial console.

                                  Mellanox iSCSI Extensions for RDMA (iSER) Support

                                  Links

                                  Contact: Max Gurtovoy <maxg@mellanox.com>
                                  Contact: Sagi Grimberg <sagig@mellanox.com>

                                  Building on the new in-kernel iSCSI initiator stack released in FreeBSD 10.0, and the recently added iSCSI offload interface, Mellanox Technologies has begun developing iSCSI extensions for RDMA (iSER) initiator support to enable efficient data movement using the hardware offload capabilities of Mellanox's 10, 40, 56, and 100 gigabit IB/Ethernet adapters.

                                  Remote Direct Memory Access (RDMA) has been shown to have a great value for storage applications. RDMA infrastructure provides benefits such as zero-copy, CPU offload, reliable transport, fabric consolidation and many more. The iSER protocol eliminates some of the bottlenecks in the traditional iSCSI/TCP stack, provides low latency and high throughput, and is well suited for latency-aware workloads.

                                  This work includes a new ICL module that implements the iSER initiator. The iSCSI stack is slightly modified to support some extra features such as asynchronous IO completions, unmapped data buffers, and data-transfer offloads. The user will be able to choose iSER as the iSCSI transport with iscsictl(8).

                                  The project is in its initial implementation phase. The code will be released under the BSD license and is expected to be completed later this year.

                                  This project was sponsored by Mellanox Technologies.

                                  Multipath TCP for FreeBSD

                                  Links
                                  URL: http://caia.swin.edu.au/urp/newtcp/mptcp/

                                  Contact: Nigel Williams <njwilliams@swin.edu.au>

                                  Multipath TCP (MPTCP) is an extension to TCP that allows for the use of multiple network interfaces on a standard TCP session. The addition of new addresses and scheduling of data across these occurs transparently from the perspective of the TCP application.

                                  The goal of this project is to deliver an MPTCP kernel patch that interoperates with the reference MPTCP implementation, along with additional enhancements to aid network research.

                                  After a major re-design of the earlier prototype implementation, the patch is again able to establish and carry out multi-path connections that incorporate multiple addresses. Improvements have also been made to path management and to the code handling the addition of subflows to a connection.

                                  Most recently data-level re-transmission support has been added and is being tested. Soon more extensive testing of the patch in different multi-path scenarios will begin, with plans for a public release of v0.5 in May.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Open tasks:

                                  1. Testing of data-level re-transmission.

                                  2. Basic support for per-subflow congestion control algorithm selection.

                                  3. Testing and release of v0.5 patch.

                                  New Automounter

                                  Links
                                  URL: https://wiki.freebsd.org/Automounter
                                  URL: http://people.freebsd.org/~trasz/autofs.pdf
                                  URL: http://freebsdfoundation.blogspot.com/2015/03/freebsd-from-trenches-using-autofs5-to_13.html

                                  Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                                  The new automounter is a cleanroom implementation of functionality available in most other Unix systems, using proper kernel support implemented via an autofs filesystem. The automounter supports a standard map format, and integrates with the Lightweight Directory Access Protocol (LDAP) service.

                                  After shipping in 10.1-RELEASE, most of the work focused on bug fixing, improving documentation, and optimization. The biggest new feature was the addition of a "-media" map, designed to handle removable media, such as flash drives or DVDs, and the necessary elements of infrastructure to support it, namely fstyp(8) and GEOM devd notifications. Also, the "-noauto" map was added, for automatic mounting of filesystems marked "noauto" in fstab(5), instead of having to write an autofs map for them.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Opaque ifnet

                                  Links
                                  Project wiki page URL: https://wiki.freebsd.org/projects/ifnet

                                  Contact: Gleb Smirnoff <glebius@FreeBSD.org>

                                  This project aims to design a new KPI for network drivers that would allow the network stack to evolve without breaking compatibility with older drivers. The core idea is to hide struct ifnet from drivers, giving the project the name "opaque ifnet". However, the project will include more changes than just hiding the struct's definition.

                                  At present, the new KPI has been prototyped, most of the important parts of network stack have been modified appropriately, and several drivers have been converted to the new KPI.

                                  The project needs more manpower, since there are many network drivers in the tree, with a total of 245 sites where a struct ifnet is allocated.

                                  This project was sponsored by Netflix.

                                  Open tasks:

                                  1. Convert more drivers.

                                  pkg

                                  Links
                                  pkg's Github Repo URL: https://github.com/freebsd/pkg
                                  The pkg Mailing List URL: https://lists.freebsd.org/mailman/listinfo/freebsd-pkg

                                  Contact: Baptiste Daroussin <bapt@FreeBSD.org>
                                  Contact: Vsevolod Stakhov <vsevolod@FreeBSD.org>
                                  Contact: Andrej Zverev <az@FreeBSD.org>

                                  Lots of work has been done on the pkg(8) front, which has brought pkg(8) to the 1.5.0 release.

                                  Special attention has been spent on the test suite; the number of tests went from around 20 to more than 70. They are mostly functional tests, each of which tests many different features, with less emphasis on unit tests.

                                  One of the main highlights is initial support for provides/requires. This is still simple but is good enough to allow fixing a lot of situations when dealing with php-related ports: PHP can now safely upgrade from one major version to another. This allows for the pecl/pear packages to be reinstalled each time a minor php upgrade is done.

                                  Some pkg internals have been reworked to allow cross installation of packages without the need for chroot(2) or jail(2) calls.

                                  The plist and keyword parser have been improved to keep simplifying creating new ports:

                                  • Keywords can now have arguments
                                  • A lazy mode is available for setting credentials via the plist
                                  • Flags (immutable and others) can now be specified in the plist

                                  pkg now supports resume for http/ftp downloads.

                                  Open tasks:

                                  1. Populate the ports tree with provides/requires.

                                  2. Make all scripts in the ports tree support cross installation.

                                  3. Improve provides/requires.

                                  4. Continue adding more tests.

                                  Secure Boot

                                  Links
                                  URL: https://wiki.freebsd.org/SecureBoot

                                  Contact: Edward Tomasz Napierała <trasz@FreeBSD.org>

                                  UEFI Secure Boot is a mechanism that requires boot drivers and operating system loaders to be cryptographically signed by an authorized key. It will refuse to execute any software that is not correctly signed, and is intended to secure boot drivers and operating system loaders from malicious tampering or replacement.

                                  The utility to add Authenticode signatures to EFI files, uefisign(8), was committed to 11-CURRENT and will ship in 10.2-RELEASE. Ports for other open source utilities were added to the Ports Collection, as sysutils/pesign, sysutils/sbsigntool, and sysutils/shim. There is a prototype patch that makes boot1 use the Secure Boot shim, and modifies the shim to provide the functionality necessary for a successful bootstrap.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Open tasks:

                                  1. Finalize the shim API extension and get it accepted upstream.

                                  2. Commit boot1 changes.


                                  Kernel


                                  Adding PCIe Hot-plug Support

                                  Links
                                  PCIe Hot-plug Perforce Branch URL: http://p4web.freebsd.org/@md=d&cd=//depot/projects/&c=LQ6@//depot/projects/pciehotplug/?ac=83

                                  Contact: John-Mark Gurney <jmg@FreeBSD.org>

                                  PCI Express (PCIe) hot-plug is used on both laptops and servers to allow peripheral devices to be added or removed while the system is running. Laptops commonly include hot-pluggable PCIe as either an ExpressCard slot or a Thunderbolt interface. ExpressCard has built-in USB support that is already supported by FreeBSD, but ExpressCard PCIe devices like Gigabit Ethernet adapters and eSATA cards are only supported when they are present at boot, and removal may cause a kernel panic.

                                  The goal of this project is to allow these devices to be inserted and removed while FreeBSD is running. The work will provide the basic infrastructure to support adding and removing devices, though it is expected that additional work will be needed to update individual drivers to support hot-plug.

                                  Current testing is focused on getting a simple UART device functional. Basic hot swap is functional.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Open tasks:

                                  1. Get suspend/resume functional by saving/restoring the necessary registers.

                                  2. Make sure that upon suspend, devices are removed so that if they are replaced while the machine is suspended, the new devices will be detected.

                                  3. Improve how state transitions are handled, possibly by using a proper state machine.

                                  Address Space Layout Randomization (ASLR)

                                  Links
                                  HardenedBSD URL: https://hardenedbsd.org/
                                  ASLR Call For Testing URL: https://lists.freebsd.org/pipermail/freebsd-current/2015-February/054669.html
                                  FreeBSD Code Review of ASLR URL: https://reviews.freebsd.org/D473

                                  Contact: Shawn Webb <shawn.webb@hardenedbsd.org>
                                  Contact: Oliver Pinter <oliver.pinter@hardenedbsd.org>

                                  Address Space Layout Randomization (ASLR) is a computer security technique that aids in mitigating low-level vulnerabilities such as buffer overflows. ASLR randomizes the memory layout of running applications to prevent an attacker from knowing where a given exploitable vulnerability lies in memory.

                                  We have been working hard the last few months to ensure the robustness of our ASLR implementation. We have written a manpage and updated the patch on FreeBSD's code review system (Phabricator). Our ASLR implementation is in use by the HardenedBSD team in production environments and is performing robustly.

                                  The next task is to compile the base system applications as Position-Independent Executables (PIEs). For ASLR to be effective, applications must be compiled as PIEs to allow the main binary, as well as shared libraries, to be located at random addresses. It is likely that this part will take a long time to accomplish, given the complexity surrounding building the libraries in the base system. Even if applications are not compiled as PIEs, having ASLR available still helps those applications (like HardenedBSD's secadm) which force compilation as PIE for themselves.

                                  This project was sponsored by SoldierX.

                                  Open tasks:

                                  1. Test our patch against 11-CURRENT.

                                  Modern x86 platform support and VT-d

                                  Contact: Konstantin Belousov <kib@FreeBSD.org>

                                  Modern x86 platforms include a number of architectural enhancements. Work is ongoing to support these features in FreeBSD.

                                  Starting with SandyBridge CPUs, Intel introduced an enhanced local interrupt controller (APIC) mode, called x2APIC. Instead of using a mapped page, registers are now accessed using special Model-Specific Registers (MSR) read and write instructions. This is intended to support virtualization. The access overhead is also reduced by not requiring serialization, and by simplification of Inter-Process Interrupt (IPI) generation. The main commit introducing the feature was r278473, with fixes following on.

                                  End Of Interrupt (EOI) suppression is a mode of EOI delivery to Input/Output Interrupt Controllers (IO-APICs) where the EOI message for a level-triggered interrupt is not broadcast by an EOI write to the local APIC, but instead an explicit EOI command is sent to the source IO-APIC. The optimization reduces the number of APIC messages that must be broadcast; it should be used on all modern Intel systems. Support for EOI suppression was committed in r279319.

                                  VT-d Interrupt Remapping (IR) is provided by hardware with the VT-d feature. It translates interrupt messages on the way from the root complex to the north bridge and allows control of interrupt delivery without reprogramming MSI/MSI-X registers or IO-APICs. The original intent was to allow hypervisors to safely delegate interrupt programming for devices owned by guests to the guest OS. IR is also needed to avoid some limitations in IO-APICs and to make interrupt rebalancing atomic and transparent. Support has been committed as r280260.

                                  Both x2APIC mode and IR are required to send IPIs and device interrupts to processors with LAPIC ID greater then 254. It is believed that the only missing platform code to handle big machines is parsing the "Processor Local x2APIC Structure" and "Local x2APIC NMI Structure" from the ACPI Multiple APIC Description Table (MADT), which report LAPIC IDs > 255, and handling boot on such systems with the x2APIC mode enabled by firmware. The work to complete that is expected to be relatively trivial, and can be done with access to a real high-core-count machine. But an audit of the common machine-independent code must be finished to ensure that large CPU IDs are handled correctly, before such support can safely be enabled.

                                  Additional work remains in progress: split domains and contexts for DMA Remapper Unit (DMAR) driver. Right now, the DMAR driver is only used to implement busdma(9), which is done by assigning a dedicated domain to each translation context. Some devices could issue PCIe Transaction Layer Packets (TLPs) with several originators IDs, e.g., PCIe/PCI bridges, or phantom functions of PCIe devices, or such TLPs could occur just due to hardware bugs. To handle them, a single domain (which shares the translation page tables) must handle several contexts.

                                  Splitting domains and contexts is also required for the DMAR driver to start handling PCI pass-through in bhyve, instead of the less complete implementation which is currently provided by bhyve itself. All PCIe devices passed to the guest must share a domain. The splitting patch is written and is being tested, and external interfaces to manage domains are being formed.

                                  Stability work for the VT-d code is ongoing. In particular, nvme(4) and ixgbe(4)'s use of busdma interfaces was debugged and improved, and tested on a very large-memory machine.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Nanosecond file timestamps

                                  Contact: Jilles Tjoelker <jilles@FreeBSD.org>
                                  Contact: Sergey Kandaurov <pluknet@FreeBSD.org>

                                  Two new system calls, futimens() and utimensat(), were added, making it possible to set file timestamps with nanosecond accuracy. Various utilities like cp, mv and touch were updated to use the new calls to preserve and set timestamps with full precision.

                                  The stat() and related system calls have returned file timestamps with nanosecond accuracy for a long time, but there was no way to set a timestamp more accurately than microseconds.

                                  With these changes, it will be possible to use more accurate timestamps (sysctl vfs.timestamp_precision=3) without anomalies such as a copy of a file (from cp -p) appearing older than the original. This is particularly useful for NFS servers, which use file timestamps for cache invalidation.

                                  Open tasks:

                                  1. Where possible, fix code that still sets inaccurate timestamps on files, typically by calling futimes(), futimesat(), lutimes(), utime() or utimes() with a non-null times pointer. There may be a reason for this such as a limited network protocol or file format, but there is some code left that can be fixed.


                                  Architectures


                                  FreeBSD on newer ARM boards

                                  Links
                                  FreeBSD on Odroid-C1 URL: https://wiki.freebsd.org/FreeBSD/arm/Odroid-C1
                                  URL: https://svnweb.freebsd.org/changeset/base/280905

                                  Contact: John Wehle <john@feith.com>
                                  Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>

                                  We made the changes necessary to support various Amlogic SoC devices, specifically aml8726-m6 and aml8726-m8b SoC-based devices. The aml8726-m6 SoC is used in devices such as the Visson ATV-102, and the Hardkernel ODROID-C1 board uses the aml8726-m8b SoC. The following support is included:

                                  • Basic machdep code
                                  • SMP
                                  • Interrupt controller
                                  • Clock control driver (aka gate)
                                  • Pinctrl
                                  • Timer
                                  • Real time clock
                                  • UART
                                  • GPIO
                                  • I2C
                                  • SD controller
                                  • SDXC controller
                                  • USB
                                  • Watchdog
                                  • Random number generator
                                  • PLL/Clock frequency measurement
                                  • Frame buffer

                                  Open tasks:

                                  1. Get the DWC driver working.

                                  FreeBSD/arm64

                                  Links
                                  FreeBSD arm64 wiki page URL: https://wiki.freebsd.org/arm64
                                  GitHub arm64 development repository URL: https://github.com/FreeBSDFoundation/freebsd/tree/arm64-dev

                                  Contact: Andrew Turner <andrew@FreeBSD.org>
                                  Contact: Ed Maste <emaste@FreeBSD.org>
                                  Contact: Zbigniew Bodek <zbb@semihalf.com>

                                  The collaborative development on the FreeBSD arm64 port made significant progress over the last quarter. The FreeBSD Foundation is collaborating with ARM, Cavium, the Semihalf team, and Andrew Turner to port FreeBSD to the arm64 architecture, also known as ARMv8 and AArch64.

                                  After significant review and refinement, the initial set of changes are being delivered into FreeBSD-HEAD. This initial support targets the QEMU and ARM Foundation Model emulators, and boots to a usable multiuser environment.

                                  Cavium's ThunderX platform is the initial hardware reference target for the FreeBSD arm64 port. The platform currently boots to multiuser, with a root file system mounted over NFS via a PCIe 10 Gbps Ethernet NIC. Reference hardware is installed in the FreeBSD test lab hosted by Sentex Communications and in Semihalf's offices.

                                  This project was sponsored by The FreeBSD Foundation, ARM, and Cavium.

                                  Open tasks:

                                  1. Merge kernel changes to HEAD.

                                  2. Finish remaining userland and kernel support.

                                  3. Produce installable images.

                                  FreeBSD/EC2

                                  Links
                                  URL: http://www.daemonology.net/freebsd-on-ec2/

                                  Contact: Colin Percival <cperciva@freebsd.org>

                                  Support for building Amazon Machine Images ("AMIs") for Amazon EC2 is now in the src tree, via make ec2ami in src/release. The platform is functional and stable, and pre-built images are available in all of the public EC2 regions.

                                  The Amazon Web Services Marketplace reports that approximately 400 users are running approximately 800 FreeBSD EC2 instances. This is an underestimate since it only counts instances launched via the AWS Marketplace.

                                  Open tasks:

                                  1. MFC AMI-building code to stable/10 in time for FreeBSD 10.2-RELEASE.

                                  2. Complete the AMI-building handoff to the release engineering team.

                                  3. Teach the blkfront driver to use indirect segment requests in order to significantly increase I/O performance.

                                  4. Get working SR-IOV driver for the Intel network cards found in EC2 "Enhanced Networking" in order to significantly increase networking performance.

                                  Nested Kernel

                                  Links
                                  Home page for the project that includes links to papers and build instructions. URL: http://nestedkernel.org
                                  Conference publication detailing the problem, design, implementation, and evaluation of our prototype. URL: http://web.engr.illinois.edu/~dautenh1//downloads/publications/asplos200-dautenhahn.pdf
                                  Presentation on the nested kernel URL: http://prezi.com/in6qr3l92ffc/?utm_campaign=share&utm_medium=copy
                                  HardenedBSD branch of the nested kernel being refactored. URL: https://github.com/HardenedBSD/hardenedBSD/tree/hardened/9/kernsep

                                  Contact: Nathan Dautenhahn <dautenh1@illinois.edu>
                                  Contact: Theodoros Kasampalis <kasampa2@illinois.edu>
                                  Contact: Will Dietz <wdietz2@illinois.edu>

                                  This work on a nested kernel architecture is part of Nathan's doctoral thesis work at the University of Illinois at Urbana-Champaign. It attempts to improve upon the traditional monolithic operating system kernel, where a single exploit anywhere in the kernel grants the attacker full superuser privileges. The nested kernel operating system architecture addresses this problem by "nesting" a small, isolated kernel within a traditional monolithic kernel. This "nested kernel" interposes on all updates to virtual memory translations to assert protections on physical memory, thus significantly reducing the trusted computing base for memory access control enforcement.

                                  We incorporated the nested kernel architecture into FreeBSD on x86-64 hardware by write-protecting Memory-Management Unit (MMU) translations and de-privileging the untrusted part of the kernel, thereby enabling the entire operating system, trusted and untrusted components alike, to operate at the highest hardware privilege level. Our implementation inherently enforces kernel code integrity while still allowing dynamically loaded kernel modules, thus defending against code injection attacks. We also demonstrate, by introducing write-mediation and write-logging services, that the nested kernel architecture allows kernel developers to isolate memory in ways not possible in monolithic kernels, though gaining security benefits from this will require adding policies that have not yet been designed.

                                  The performance of the nested kernel prototype shows modest overheads: less than 1% average for Apache, 3.7% average for sshd, and 2.7% average for kernel compilation. Overall, our results and experience show that the nested kernel design can be retrofitted onto existing monolithic kernels, providing defense in depth.

                                  The basic idea is that the nested kernel initializes the system so that all page tables are mapped as read-only. Then all MMU-modifying operations are removed from the untrusted portion of the kernel; runtime code integrity is enforced by write-protecting all code pages, marking all non-code pages as non-executable (NX-bit), and preventing execution of privileged MMU operations located in userspace mappings (Supervisor Mode Execution Prevention, SMEP). Because the nested kernel has control of the page tables it can enforce these integrity properties, leading to virtualization of the MMU.

                                  The links include a recent conference publication that details the design, implementation, and evaluation of our prototype nested kernel architecture on top of FreeBSD 9.0. There is also a link to a presentation on the nested kernel, and a website with information about the project and instructions on how to get the source and build it.

                                  We are very interested in feedback on the design of the nested kernel, and having discussions about how it might get upstreamed.

                                  We are also hoping to gain additional contributors and interest in the project! The nested kernel has the potential to enhance commodity operating system design, and FreeBSD is a major operating system in use today which has high impact. The current implementation is merely a research prototype and requires significant effort to make production-ready (see the list of tasks).

                                  Finally, we have developed an interface to write-protect data structures in the kernel and are soliciting ideas for uses of this service. Section 2.4 in the paper details the interface, and section 4 presents some simple uses of the nested kernel services. We are interested in ways that the nested kernel could be used to protect critical kernel data structures from malware or even just buggy code.

                                  This project was sponsored by University of Illinois at Urbana-Champaign, and ONR via grant number N00014-12-1-0552.

                                  Open tasks:

                                  1. Finish implementing core mechanisms: verify DMAP is properly protected and that we are not using superpages (I think we have this completed but need to fully verify), full NX support for all non-kernel code pages (we might need to specially consider the stack if it is used to execute code), protect IDT and SMM, and add IOMMU protections. We also need to do some optimizations where we batch calls into the nested kernel on process creation (fork) and mmap operations. The motivation for these implementation directives can be reviewed in the paper.

                                  2. Implement SMP functionality and evaluate performance.

                                  3. Port and refactor for FreeBSD-HEAD. The current implementation is a research prototype and requires some refactoring to make it clean and consistent, as well as make it relevant to modern versions of FreeBSD.

                                  4. The nested kernel isolation depends upon certain hardware instructions to be completely removed from a subset of the kernel. Therefore, we need to utilize automated linker/loader techniques to identify and remove privileged MMU operations from untrusted kernel components to make it maintainable in practice.

                                  5. Detailed review on the design and implementation with particular focus on a plan for upstreaming.


                                  Userland Programs


                                  libthr improvements

                                  Contact: Konstantin Belousov <kib@FreeBSD.org>

                                  Historically, dynamic loading of the libthr.so thread library into a single-threaded process did not work in FreeBSD. The longstanding recommendation to work around the problem has been to always link the main binary with -lpthread if there was any chance of a need for threading functionality. This project converted libthr.so into a plugin for libc, which fixed the known issues preventing dynamic loading of libthr.so.

                                  After the fix, linking the main binary with -lpthread is no longer required, but is not harmful. I recommend thoroughly testing before removing libpthread from the library list in favor of dynamic loading, though. Note that potential problems will be subtle and their user-visible manifestations in the affected program even more surprising.

                                  The following issues were present in the old version of libthr with respect to dynamic loading, but are fixed as a result of this work:

                                  • Invalid errno value seen after failed syscalls.
                                  • Broken libthr internal locks and critical sections ignored by signals.
                                  • Hung attempts to lock mutexes.
                                  • Thread cancellation not occurring at guaranteed cancellation points.

                                  The main change was committed as r276630 to HEAD, with many follow ups. It was merged to stable/10 in r277317.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Migration to ELF Tool Chain tools

                                  Links
                                  FreeBSD LLDB wiki page URL: http://elftoolchain.sourceforge.net

                                  Contact: Ed Maste <emaste@FreeBSD.org>

                                  The ELF Tool Chain project provides BSD-licensed implementations of compilation tools and libraries for building and analyzing ELF objects. The project began as part of FreeBSD but later became an independent project to encourage wider participation from others in the open-source developer community.

                                  ELF Tool Chain provides a set of tools equivalent to the GNU Binutils suite. This project's goal is to import these tools into the FreeBSD base system so that we have a set of up-to-date and maintained tools that also provide support for new CPU architectures of interest, such as arm64.

                                  In addition to the libelf and libdwarf libraries, the following tools are now provided by the ELF Tool Chain project:

                                  • addr2line
                                  • nm
                                  • readelf
                                  • size
                                  • strings
                                  • strip (elfcopy)

                                  ELF Tool Chain's elfcopy provides equivalent functionality to Binutils' objcopy, and accepts the same command-line arguments. For it to be a viable replacement for all uses of objcopy in the base system, it must gain support for writing portable executable (PE) format binaries, which are used by UEFI boot code.

                                  The ELF Tool Chain project does not currently provide replacements for as, ld, or objdump. For FreeBSD, these tools will likely be obtained from the LLVM project.

                                  This project was sponsored by The FreeBSD Foundation.

                                  Open tasks:

                                  1. Add missing functionality to elfcopy and migrate the base system build.

                                  2. Fix issues found by fuzzing inputs to the tools.

                                  3. Add automatic support for separate debug files.

                                  The LLDB Debugger

                                  Links
                                  FreeBSD LLDB wiki page URL: https://wiki.freebsd.org/lldb

                                  Contact: Ed Maste <emaste@FreeBSD.org>

                                  LLDB is the debugger project associated with Clang/LLVM. It supports the Mac OS X, Linux, FreeBSD and Windows platforms. It builds on existing components in the larger LLVM project, for example using Clang's expression parser and LLVM's disassembler.

                                  The LLDB in the base system was upgraded to version 3.6.0 as part of the Clang and LLVM upgrade. In the upstream repository, Justin Hibbits added support for live and core file debugging on PowerPC, and Ed Maste added core file support for FreeBSD/arm64.

                                  This project was sponsored by DARP/AFRL, SRI International, and University of Cambridge.

                                  Open tasks:

                                  1. Rework the LLDB build to use LLVM and Clang shared libraries.

                                  2. Port remote debug stub to FreeBSD.

                                  3. Add support for local and core file kernel debugging.

                                  4. Improve support on non-amd64 architectures.

                                  5. Enable by default in the base system.

                                  Updates to GDB

                                  Links
                                  Port of kgdb to gdb 7.9 URL: https://github.com/bsdjhb/gdb/tree/freebsd-7.9.0-kgdb

                                  Contact: John Baldwin <jhb@FreeBSD.org>

                                  Several improvements to GDB have been merged upstream to GDB's master branch over the past few months, including fixes for unwinding across signal trampoline frames on x86, removing the procfs dependency from the gcore command, and support for XSAVE extensions (such as AVX registers) on x86. These fixes are already available in the existing devel/gdb port as patches relative to 7.8.

                                  In addition, progress has been made on porting kgdb to a newer gdb. Currently, only support for the amd64 backend has been ported, but it is functional both for remote debugging and against crash dumps. The current port generally has feature parity with the kgdb in the base system. The plan for kgdb is to fix it to always include all platform targets (so that it always supports cross debugging for remote targets out of the box). At some point it may also include cross debugging support for crash dumps as well (this would require changes to libkvm).

                                  Open tasks:

                                  1. Tidy the amd64 port of kgdb and finish the i386 port. This includes fixing these platform-specific targets to work with cross-debugging for remote targets.

                                  2. Add a KGDB option to the devel/gdb port to include kgdb support.

                                  3. Port the rest of the platform-specific targets for kgdb.

                                  4. Write a new 1:1-only thread target for FreeBSD that can be sent upstream.

                                  5. Add support for debugging powerpc vector registers.


                                  Ports


                                  FreeBSD Ada Ports

                                  Links
                                  URL: http://home.gna.org/ghdl/
                                  URL: http://sourceforge.net/projects/ghdl-updates/

                                  Contact: John Marino <marino@FreeBSD.org>

                                  There are 51 Ada-related ports currently, but two of them are being retired: the GCC 4.7-based lang/gcc47-aux and the BSD->android cross-compiler for ARMv5 (lang/gnatdroid-armv5). The former has no advantage over the newer GCC 4.9-based lang/gcc-aux, and the latter has not built for over a year. Android enthusiasts can still use the the ARMv7 cross-compiler (lang/gnatdroid-armv7).

                                  A new port is lang/gcc5-aux, which includes GNAT from the upcoming release of gcc5. This compiler already builds all Ada ports except gtkada3 (which blocks devel/gps, the GNAT Programming Studio), and gtkada3 should be fixed soon. When GCC5 is released, the Ada framework will switch to using gcc5-aux as the default compiler. For those that cannot wait, it is possible to use it now by putting ADA_DEFAULT=5 in /etc/make.conf, but this requires rebuilding all Ada ports from source.

                                  Open tasks:

                                  1. It is a near-term objective to bring the Ada-based GDHL (VHDL simulator) to ports. The upcoming 0.32 release will be based on GCC 4.9 and the port will be based on this release.

                                  FreeBSD Python Ports

                                  Links
                                  The FreeBSD Python Team Page URL: https://wiki.FreeBSD.org/Python
                                  IRC channel URL: irc://freebsd-python@irc.freenode.net

                                  Contact: FreeBSD Python Team <python@FreeBSD.org>

                                  The FreeBSD Python team continued to improve the overall experience with Python-based software on FreeBSD. A lot of previously deprecated code and option knobs were removed to improve the maintainability of the Python Ports infrastructure.

                                  The CPython interpreters were updated to version 2.7.9 and 3.4.3 and Twisted was updated to version 15.0.0.

                                  Open tasks:

                                  1. Retire the Python 3-specific port duplicates.

                                  2. More tasks can be found on the team's wiki page (see the links).

                                  3. To get involved, interested people can say hello on IRC in #freebsd-python on freenode and let us know their areas of interest!

                                  GNOME on FreeBSD

                                  Links
                                  URL: http://www.freebsd.org/gnome
                                  GNOME development repo URL: https://github.com/freebsd/freebsd-ports-gnome
                                  URL: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD

                                  Contact: FreeBSD GNOME Team <freebsd-gnome@freebsd.org>

                                  The FreeBSD GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for FreeBSD. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel.

                                  At the end of this quarter we updated GNOME and CINNAMON to the latest versions on their branches, 3.14 and 2.4, respectively.

                                  GNOME 3.16 was released February 25th; we ported it to FreeBSD. There are still some showstopper problems that appeared. During testing of the current versions of the 3.16 ports a bug in pkg was uncovered in the multiple repository support, and swiftly fixed in pkg 1.4.99.15.

                                  For the GNOME 3.18 cycle we are going to work closely with the x11 team on porting libinput and testing Wayland. When that is done we need to see if we want to enable Wayland for our stable releases and we probably need XWayland from xorg-server 1.16+ to support X applications. The estimate is that Wayland arriving in ports will have to wait until 8.4-Release is EOL.

                                  Open tasks:

                                  1. The GNOME website is stale. Work is underway, although slowly, on the development section. We could use some help here.

                                  2. MATE 1.10 porting is under way; the latest 1.9 releases are available in the mate-1.10 branch.

                                  KDE on FreeBSD

                                  Links
                                  URL: https://freebsd.kde.org/
                                  URL: https://freebsd.kde.org/area51.php
                                  URL: https://wiki.freebsd.org/KDE
                                  URL: https://mail.kde.org/mailman/listinfo/kde-freebsd
                                  URL: https://github.com/tcberner/kde5

                                  Contact: KDE on FreeBSD team <kde@FreeBSD.org>

                                  The KDE on FreeBSD team focuses on packaging and making sure that the experience of KDE and Qt on FreeBSD is as good as possible.

                                  First of all, we would like to welcome Tobias Berner to the ranks of the area51 (the KDE ports staging area) committers. He has been regularly mentioned in our recent status reports, and has finally received committer privileges to our experimental repository. Becoming an area51 committer is usually the first step towards becoming a kde@ ports committer. We hope that Tobias can fix and update our ports more easily, and start committing his KDE Frameworks 5 ports to area51.

                                  Additionally, this quarter Qt 5.4.1 was committed to the ports tree. This marks the first time ever since Qt 5 was released that we have the latest upstream stable release in our ports tree! This was made possible by all the work we had to put into cleaning up the Qt 5 ports infrastructure for the 5.3 update, mentioned in our previous status report.

                                  Last but not least, Alonso Schaich finally landed an update to our KDE4 ports that had been in our experimental repository for a while, bringing them to the latest 4.14 release, 4.14.3.

                                  Overall, we have updated the following ports in this quarter:

                                  • Calligra 2.9.1 (committed to area51)
                                  • CMake 3.1.0, 3.1.1, 3.1.3 (committed to ports)
                                  • DigiKam 4.2.0 (committed to ports), 4.8.0 (committed to area51)
                                  • PyQt 4.11.3 + QScintilla 2.8.4 + sip 4.16.5 (committed to ports), sip 4.16.7 (committed to area51)
                                  • Qt 5.4.1 (committed to ports)

                                  Open tasks:

                                  1. Put more effort into Qt5-related ports: KDE Frameworks 5 (currently worked on by Tobias Berner) and PyQt 5.

                                  The Graphics stack on FreeBSD

                                  Links
                                  Graphics stack roadmap and supported hardware matrix URL: https://wiki.freebsd.org/Graphics
                                  Graphics stack team blog URL: http://blogs.freebsdish.org/graphics/
                                  Ports development tree on GitHub URL: https://github.com/freebsd/freebsd-ports-graphics

                                  Contact: FreeBSD Graphics team <freebsd-x11@FreeBSD.org>

                                  In the official Ports tree, the Mesa ports (libglapi, libGL, libEGL, libglesv2, gbm, and dri) are kept close to the latest Mesa 10.4.x release.

                                  In the development tree (see the GitHub link), the update to Mesa 10.5 came, along with several improvements and cleanup to the ports themselves. Now all ports share the same configure flags and build dependencies. As Mesa is built from scratch for each port, this ensures that all libraries and drivers are consistent with each other. This fixes at least two problems:

                                  • A long standing bug: the drm EGL platform is now functional, meaning we will be able to enable Glamor (the 2D acceleration engine based on OpenGL) in the X.Org server. This is required to provide 2D acceleration for Radeon HD 7000 and later GPUs, for instance.
                                  • Clover, the Mesa OpenCL implementation, now works; see the next paragraph.

                                  The downside of this unification is that all ports will depend on LLVM. This work is happening in the mesa-10.5 branch.

                                  Progress has been made on OpenCL, thanks to help from Johannes Dieterich. Clover (Mesa's implementation) and Beignet (Intel's implementation) were added as ports to the development tree. They were tested successfully on Radeon and Intel GPUs, but see the wiki for an up-to-date status. Initially developed in the opencl branch, everything has now been merged into the mesa-10.5 branch. This cannot go into the official Ports tree yet because it requires the unification explained above.

                                  A new port, drm-kmod, was added to the official Ports tree. It provides updated drm2, i915kms and radeonkms kernel modules for FreeBSD 9.3-RELEASE and 9.3-STABLE. The only difference from the vanilla modules is the addition of hardware context support to the i915 driver. The xf86-video-radeon and xf86-video-intel drivers were patched to use the drm-kmod port on these versions of FreeBSD. This will allow us to remove the duality of the Mesa ports (libGL/libEGL/dri) and only support one version (as is already the case in the mesa-10.5 branch where Mesa 9.1.7 is gone). There is no ETA yet for when this last part will happen.

                                  In the development Ports tree, the xserver-next branch was updated from xorg-server 1.16 to be tracking 1.17. Again, this depends on the previous step: the removal of Mesa 9.1.7.

                                  Work is finishing up on an update of miscellaneous X.Org components. Apart from updates to several X.Org ports, this update also removes the use of .la files from the X.Org libraries that still have them. Also, the xf86-video-intel driver will receive patches to allow it to compile against a newer xorg-server than 1.14. Most of the X.Org component updates were submitted by Matthew Rezny.

                                  The location where fonts get installed was overhauled and the way to handle fonts from the plist has been simplified. Now all fonts are installed in /usr/local/share/fonts as required by the XDG rules. Furthermore, making a port for fonts should be easier: more aspects, such as calling fc-cache(1), are handled by the Ports framework. Therefore, the font ports' consistency was greatly improved.

                                  In the kernel, the DRM device-independent code was updated to match Linux 3.8. A merge to 10-STABLE is pending. The i915kms kernel driver received an update, too, which is already merged to 10-STABLE.

                                  Having both updates in place enables work on a second update of the i915 driver: this time it will be synchronized with Linux 3.8, like the rest of the DRM subsystem, and include Haswell support. This work was started recently. Our hope is that it will be ready in time for FreeBSD 10.2-RELEASE.

                                  During Q2, we are going to work with the GNOME team on porting libinput and testing Wayland. Currently we know that GTK+3 and GNOME 3 have full support for Wayland. We also need to test Xwayland from xorg-server 1.16+ to support X applications on Wayland desktops. If you know of more software that uses Wayland, we would like to hear about them. At this point there are no plans to port the Weston reference implementation of a Wayland compositor.

                                  Open tasks:

                                  1. See the "Graphics" wiki page for up-to-date information.

                                  Wine/FreeBSD

                                  Links
                                  Wine wiki URL: http://wiki.FreeBSD.org/Wine
                                  Wine on amd64 wiki URL: http://wiki.FreeBSD.org/i386-Wine
                                  Wine homepage URL: http://www.winehq.org

                                  Contact: Gerald Pfeifer <gerald@FreeBSD.org>
                                  Contact: David Naylor <dbn@FreeBSD.org>

                                  This quarter has seen five updates to the wine-devel port that closely tracks upstream development, as well as updates to helper ports (wine-gecko-devel and wine-mono-devel):

                                  • Stable releases: 1.6.2 (1 port revision)
                                  • Development releases: 1.7.34 through 1.7.39

                                  A major development has been the introduction of Wine64 (i.e., the ability to run 64-bit Windows applications). This is currently available through the wine-devel port. At this stage it is currently mutually exclusive with the i386-wine-devel port, however, we have plans to integrate these ports to offer a full Wine experience on amd64. The i386-wine-devel port has packages built for amd64 for FreeBSD 8.4, 9.1+, 10.1+ and CURRENT.

                                  Accomplishments include:

                                  • Upstreaming 8 patches to fix Wine on FreeBSD — many thanks to Gerald and David.
                                  • Optional support for V4L has been added to the stable emulators/wine port.
                                  • Optionally building wine with the X composite extension (if one selects the X11 option).
                                  • Support for alternative toolchains that require LD to be honoured.
                                  • Fixing and tidying up the pkg-plist.
                                  • Wine64 support
                                  • Updating the patch-nvidia.sh script to support arbitrary suffixes.
                                  • Removing support for the old pkg_ tools from patch-nvidia.sh.
                                  • Developing a patch to fix usage of getdirentries(2). This fixes Steam, EVE Online and other applications.

                                  We would like to thank all volunteers who contributed feedback and patches.

                                  Future development on Wine will focus on:

                                  • Rename wine-compholio to wine-staging (to match upstream development).
                                  • Add the getdirentries(2) patch to the wine-devel port.
                                  • Redevelop and upstream the getdirentries(2) patch.
                                  • Redevelop and upstream the kernel32 Makefile patch.
                                  • Add support to the i386-wine port for pkg 1.5 (conflicts with libraries currently prevent such support).
                                  • Add support for WoW64:
                                    • Reduce the i386-wine port to just the components required for WoW64.
                                    • Rename the i386-wine port to wow64.
                                    • Make the wine ports depend on the wow64 ports when built on amd64.
                                    • Investigate and verify the interactions between Wine64 and WoW64.
                                    • Investigate possible update approaches for the wow64 ports (that have to be pre-compiled) and how updating with the wine ports will work.

                                  Maintaining and improving Wine is a major undertaking that directly impacts end-users on FreeBSD (including many gamers). If you are interested in helping, please contact us. We will happily accept patches, suggest areas of focus or have a chat.

                                  Open tasks:

                                  1. FreeBSD/amd64 integration (see the i386-Wine wiki).

                                  2. Porting WoW64.

                                  Xfce on FreeBSD

                                  Links
                                  URL: https://wiki.freebsd.org/Xfce

                                  Contact: FreeBSD Xfce Team <xfce@FreeBSD.org>

                                  Xfce is a free software desktop environment for Unix and Unix-like platforms, such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use.

                                  This quarter was an exciting time for the Xfce Team. We imported version 4.12 of the Xfce desktop environment into the ports tree, after more than two years of development.

                                  Overall, we have updated the following ports:

                                  • Xfce core (4.12)
                                  • audio/xfce4-mpc-plugin (0.4.5)
                                  • deskutils/xfce4-tumbler (0.1.31
                                  • deskutils/xfce4-xkb-plugin (0.7.1)
                                  • editors/mousepad (0.4.0)
                                  • graphics/ristretto (0.8.0)
                                  • multimedia/xfce4-parole (0.8.0)
                                  • sysutils/garcon (0.4.0)
                                  • sysutils/xfce4-diskperf-plugin (2.5.5)
                                  • sysutils/xfce4-fsguard-plugin (1.0.2)
                                  • sysutils/xfce4-power-manager (1.4.4)
                                  • sysutils/xfce4-wavelan-plugin (0.5.12)
                                  • textproc/xfce4-dict-plugin (0.7.1)
                                  • www/xfce4-smartbookmark-plugin (0.4.6)
                                  • x11/libexo (0.10.4)
                                  • x11-clocks/xfce4-timer-out-plugin (1.0.2)
                                  • x11-fm/thunar (1.6.6)
                                  • x11-themes/gtk-xfce-engine (3.2.0)

                                  At the same time we switched to the USES framework, and a new plugin has been added, called audio/xfce4-pulseaudio-plugin.

                                  We also follow the unstable releases (available in our experimental repository) of:

                                  • x11/xfce4-dashboard (0.3.91)
                                  • x11/xfce4-notes-plugin (1.8.0 beta)

                                  The following documentation patches are ready:

                                  Open tasks:

                                  1. Work on support for Compact Disc Digital Audio (CD-DA) in multimedia/xfce4-parole.

                                  2. Add a new property (through xfconf-query) to allow users to change the greyscale value of quicklaunch icons in x11/xfce4-dashboard (this feature is only available in the unstable release).


                                  Documentation


                                  More Michael Lucas FreeBSD books

                                  Links
                                  URL: http://blather.michaelwlucas.com/archives/2352

                                  Contact: Michael Lucas <mwlucas@michaelwlucas.com>

                                  The FreeBSD storage books are proceeding slower than expected. This is a complex project.

                                  It appears that ZFS will be a two-book topic. The first book will cover basic ZFS, while the second will cover advanced cases like live and cold replication, sharing, performance, and using ZFS on top of less common GEOM providers. More details can be found in the links section.

                                  Allan Jude (allanjude@) is co-authoring the ZFS books. Little did he know of the magnitude of the task ahead of him when he signed up....


                                  Miscellaneous


                                  The FreeBSD Foundation

                                  Links
                                  Foundation website URL: http://www.FreeBSDFoundation.org/
                                  FreeBSD Journal URL: http://freebsdjournal.com/
                                  BSDNow PC-BSD Tour URL: http://www.bsdnow.tv/episodes/2015_03_11-the_pcbsd_tour_ii
                                  BSDNow "From the Foundation" URL: http://www.bsdnow.tv/episodes/2015_02_25-from_the_foundation_2

                                  Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

                                  The Foundation turned 15 on March 15th! We kicked off our anniversary celebration by launching a spring fundraising campaign, to bring in 500 new community investors. In conjunction with our anniversary, BSDNow interviewed Justin Gibbs about our history and plans for the future as part of the PC-BSD tour. BSDNow also interviewed Ed Maste about FreeBSD projects and processes in a "From the Foundation" episode.

                                  We were a Platinum Sponsor of AsiaBSDCon and had five team members attend the conference. Kirk McKusick taught a two-day FreeBSD kernel tutorial and gave a talk on Journaled Soft Updates, and George Neville-Neil gave a talk on network performance in FreeBSD; George also taught a two day tutorial (A Look Inside FreeBSD with DTrace). This is from ongoing work with Robert Watson in support of both academic and practitioner educational material for FreeBSD. Dru gave a talk on Advanced OpenSource Storage with FreeNAS 9.3, and Ed Maste gave a talk on the LLDB Debugger in FreeBSD.

                                  We became a Platinum Sponsor for BSDCan, and have approved six travel grants to FreeBSD contributors. We also sponsored Michael Dexter to attend SCALE so he could give a talk on virtualization.

                                  In addition to the above conferences, we helped promote FreeBSD at the following conferences:

                                  We received and published FreeBSD testimonials from Xinuos, Netgate, and Tarsnap.

                                  We launched the "From the Trenches" series to provide stories from FreeBSD contributors on what they are doing with FreeBSD. Glen Barber wrote an article called ZFS and How to Make a Foot Cannon. Glen also investigated a deadlock issue when rebooting after upgrades (PR 195458), and he released weekly 11-CURRENT and 10-STABLE snapshot builds.

                                  The FreeBSD Journal now has over 8300 subscribers and has a 98% renewal rate. We are now publishing a few free FreeBSD Journal articles. We also created landing pages for each Journal issue for easier promotion.

                                  We started work on the Ottawa Vendor and Developer Summits and another one that has not yet been officially announced on the East Coast in the fall.

                                  Our development staff and project grant recipients were responsible for a large number of feature improvements and bug fixes over this past quarter. We have nine individual reports in this quarterly update for Foundation-sponsored projects that demonstrate a number of different ways the Foundation supports the FreeBSD project.

                                  One project is the subject of a research master's project at Swinburne University in Melbourne: the Multipath TCP (MPTCP) implementation for FreeBSD. The PCIe hot plug project is an individual project grant. The FreeBSD/arm64 project represents a collaborative development effort, where the Foundation facilitates a broader project with multiple participants.

                                  There are also a number of projects undertaken directly by Foundation staff. In this quarterly report we have several reports in this category: Secure Boot, the autofs-based automount daemon, dynamically loadable libthr, Intel DMA remapping, and migration to the ELF Tool Chain project tools.

                                  Additionally, one of the benefits of having long-term permanent staff is the ability to continue to maintain projects and contribute improvements beyond a fixed timeline. Over the last quarter, Foundation staff contributed improvements to the UEFI boot process, vt(4) system console, in-kernel iSCSI stack, virtual memory subsystem, and many others.

                                  News Home | Status Home