diff --git a/en_US.ISO8859-1/articles/committers-guide/article.sgml b/en_US.ISO8859-1/articles/committers-guide/article.sgml index 30b283b9ff..d5c4db3b9d 100644 --- a/en_US.ISO8859-1/articles/committers-guide/article.sgml +++ b/en_US.ISO8859-1/articles/committers-guide/article.sgml @@ -1,3622 +1,3622 @@ %articles.ent; ]>
Committer's Guide The FreeBSD Documentation Project $FreeBSD$ 1999 2000 2001 2002 2003 2004 2005 2006 2007 The FreeBSD Documentation Project &tm-attrib.freebsd; &tm-attrib.coverity; &tm-attrib.cvsup; &tm-attrib.ibm; &tm-attrib.intel; &tm-attrib.sparc; &tm-attrib.general; This document provides information for the FreeBSD committer community. All new committers should read this document before they start, and existing committers are strongly encouraged to review it from time to time. Almost all FreeBSD developers have commmit rights to one or more repositories. However, a few developers do not, and some of the information here applies to them as well. (For instance, some people only have rights to work with the Problem Report database). Please see for more information. This document may also be of interest to members of the FreeBSD community who want to learn more about how the project works. Administrative Details Main Repository Host ncvs.FreeBSD.org Login Methods &man.ssh.1;, protocol 2 only Main CVSROOT ncvs.FreeBSD.org:/home/ncvs (although also see ). Main &a.cvsadm; &a.peter; and &a.markm;, as well as &a.joe; and &a.marcus; for ports/ &a.bugmeister; &a.ceri; &a.linimon;, and &a.remko Mailing Lists &a.doc-developers;, &a.doc-committers;; &a.ports-developers;, &a.ports-committers;; &a.src-developers;, &a.src-committers;. (Each project repository has its own -developers and -committers mailing lists. Archives for these lists may be found in files /home/mail/repository-name-developers-archive and /home/mail/repository-name-committers-archive on the FreeBSD.org cluster.) Core Team monthly reports /home/core/public/monthly-reports on the FreeBSD.org cluster. Noteworthy CVS Tags RELENG_5 (5.X-STABLE), RELENG_6 (6.X-STABLE), RELENG_7 (7.X-STABLE), HEAD (-CURRENT) It is required that you use &man.ssh.1; or &man.telnet.1; with Kerberos 5 to connect to the project hosts. For &man.ssh.1; only protocol 2 is allowed. These are generally more secure than plain &man.telnet.1; or &man.rlogin.1; since credential negotiation will always be encrypted. All traffic is encrypted by default with &man.ssh.1;. With utilities like &man.ssh-agent.1; and &man.scp.1; also available, &man.ssh.1; is also far more convenient. If you do not know anything about &man.ssh.1;, please see . Commit Bit Types The FreeBSD CVS repository has a number of components which, when combined, support the basic operating system source, documentation, third party application ports infrastructure, and various maintained utilities. When FreeBSD commit bits are allocated, the areas of the tree where the bit may be used are specified. Generally, the areas associated with a bit reflect who authorized the allocation of the commit bit. Additional areas of authority may be added at a later date: when this occurs, the committer should follow normal commit bit allocation procedures for that area of the tree, seeking approval from the appropriate entity and possibly getting a mentor for that area for some period of time. Committer Type Responsible Tree Components src core@ src/, doc/ subject to appropriate review doc doceng@ doc/, www/, src/ documentation ports portmgr@ ports/ Commit bits allocated prior to the development of the notion of areas of authority may be appropriate for use in many parts of the tree. However, common sense dictates that a committer who has not previously worked in an area of the tree seek review prior to committing, seek approval from the appropriate responsible party, and/or work with a mentor. Since the rules regarding code maintenance differ by area of the tree, this is as much for the benefit of the committer working in an area of less familiarity as it is for others working on the tree. Committers are encouraged to seek review for their work as part of the normal development process, regardless of the area of the tree where the work is occurring. Policy for <filename>doc/</filename> committer activity in <filename>src/</filename> doc committers may commit documentation changes to src files, such as man pages, READMEs, fortune databases, calendar files, and comment fixes without approval from a src committer, subject to the normal care and tending of commits. doc committers may commit minor src changes and fixes, such as build fixes, small features, etc, with an "Approved by" from a src committer. doc committers may seek an upgrade to a src commit bit by acquiring a mentor, who will propose the doc committer to core. When approved, they will be added to 'access' and the normal mentoring period will ensue, which will involve a continuing of Approved by for some period. "Approved by" is only acceptable from non-mentored src committers -- mentored committers can provide a "Reviewed by" but not an "Approved by". CVS Operations It is assumed that you are already familiar with the basic operation of CVS. The &a.cvsadm; are the owners of the CVS repository and are responsible for direct modification of it for the purposes of cleanup or fixing some grievous abuse of CVS by a committer. Should you cause some repository accident, say a bad cvs import or cvs tag operation, mail the responsible part of &a.cvsadm;, as stated in the table below, (or call one of them) and report the problem. For very important issues affecting the entire CVS tree—not just a specific area—you can contact the &a.cvsadm;. Please do not contact the &a.cvsadm; for repocopies or other things that the more specific teams can handle. The only ones able to directly fiddle the repository bits on the repository hosts are the repomeisters. To enforce this, there are no login shells available on the repository machines, except to the repomeisters. Depending on the affected area of the CVS repository, you should send your request for a repocopy to one of the following email addresses. Email sent to these addresses will be forwarded to the appropriate repomeisters. - ncvs@ - regarding + ncvs@ - regarding /home/ncvs, the src repository - pcvs@ - regarding + pcvs@ - regarding /home/pcvs, the ports repository - dcvs@ - regarding + dcvs@ - regarding /home/dcvs, the doc repository - projcvs@ - regarding + projcvs@ - regarding /home/projcvs, the third party projects repository The CVS tree is currently split into four distinct repositories, namely doc, ports, projects and src. These are combined under a single CVSROOT when distributed via CVSup for the convenience of our users. Note that the www module containing sources for the FreeBSD website is contained within the doc repository. The CVS repositories are hosted on the repository machines. Currently, each of the repositories above reside on the same physical machine, ncvs.FreeBSD.org, but to allow for the possibility of placing each on a separate machine in the future, there is a separate hostname for each that committers should use. Additionally, each repository is stored in a separate directory. The following table summarizes the situation. &os; CVS Repositories, Hosts and Directories Repository Host Directory doc dcvs.FreeBSD.org /home/dcvs ports pcvs.FreeBSD.org /home/pcvs projects projcvs.FreeBSD.org /home/projcvs src ncvs.FreeBSD.org /home/ncvs
CVS operations are done remotely by setting the CVSROOT environment variable to the appropriate host and top-level directory (for example, ncvs.FreeBSD.org:/home/ncvs), and doing the appropriate check-out/check-in operations. Many committers define aliases which expand to the correct cvs invocation for the appropriate repository. For example, a &man.tcsh.1; user may add the following to their .cshrc for this purpose: alias dcvs cvs -d user@dcvs.FreeBSD.org:/home/dcvs alias pcvs cvs -d user@pcvs.FreeBSD.org:/home/pcvs alias projcvs cvs -d user@projcvs.FreeBSD.org:/home/projcvs alias scvs cvs -d user@ncvs.FreeBSD.org:/home/ncvs This way they can do all CVS operations locally and use Xcvs commit for committing to the official CVS tree. If you wish to add something which is wholly new (like contrib-ified sources, etc), cvs import should be used. Refer to the &man.cvs.1; manual page for usage. Please do not use cvs checkout or update with the official repository machine set as the CVS Root for keeping your source tree up to date. Remote CVS is not optimized for network distribution and requires a big work/administrative overhead on the server side. Please use our advanced cvsup distribution method for obtaining the repository bits, and only do the actual commit operation on the repository host. We provide an extensive cvsup replication network for this purpose, as well as give access to cvsup-master if you really need to stay current to the latest changes. cvsup-master has got the horsepower to deal with this, the repository master server does not. &a.kuriyama; is in charge of cvsup-master. If you need to use CVS add and delete operations in a manner that is effectively a &man.mv.1; operation, then a repository copy is in order rather than using CVS add and delete. In a repository copy, a repomeister will copy the file(s) to their new name and/or location and let you know when it is done. The purpose of a repository copy is to preserve file change history, or logs. We in the FreeBSD Project greatly value the change history that CVS gives to the project. CVS reference information, tutorials, and FAQs can be found at: . The information in Karl Fogel's chapters from Open Source Development with CVS is also very useful. &a.des; also supplied the following mini primer for CVS. Check out a module with the co or checkout command. &prompt.user; cvs checkout shazam This checks out a copy of the shazam module. If there is no shazam module in the modules file, it looks for a top-level directory named shazam instead. Useful <command>cvs checkout</command> options Do not create empty directories Check out a single level, no subdirectories Check out revision, branch or tag rev Check out the sources as they were on date date
Practical FreeBSD examples: Check out the miscfs module, which corresponds to src/sys/miscfs: &prompt.user; cvs co miscfs You now have a directory named miscfs with subdirectories CVS, deadfs, devfs, and so on. One of these (linprocfs) is empty. Check out the same files, but with full path: &prompt.user; cvs co src/sys/miscfs You now have a directory named src, with subdirectories CVS and sys. The src/sys directory has subdirectories CVS and miscfs, etc. Check out the same files, but prunes empty directories: &prompt.user; cvs co -P miscfs You now have a directory named miscfs with subdirectories CVS, deadfs, devfs... but note that there is no linprocfs subdirectory, because there are no files in it. Check out the directory miscfs, but none of the subdirectories: &prompt.user; cvs co -l miscfs You now have a directory named miscfs with just one subdirectory named CVS. Check out the miscfs module as it is in the 6.X branch: &prompt.user; cvs co -rRELENG_6 miscfs You can modify the sources and commit along this branch. Check out the miscfs module as it was in 6.0-RELEASE. &prompt.user; cvs co -rRELENG_6_0_0_RELEASE miscfs You will not be able to commit modifications, since RELENG_6_0_0_RELEASE is a point in time, not a branch. Check out the miscfs module as it was on Jan 15 2000. &prompt.user; cvs co -D'01/15/2000' miscfs You will not be able to commit modifications. Check out the miscfs module as it was one week ago. &prompt.user; cvs co -D'last week' miscfs You will not be able to commit modifications. Note that cvs stores metadata in subdirectories named CVS. Arguments to and are sticky, which means cvs will remember them later, e.g. when you do a cvs update. Check the status of checked-out files with the status command. &prompt.user; cvs status shazam This displays the status of the file shazam or of every file in the shazam directory. For every file, the status is given as one of: Up-to-date File is up-to-date and unmodified. Needs Patch File is unmodified, but there is a newer revision in the repository. Locally Modified File is up-to-date, but modified. Needs Merge File is modified, and there is a newer revision in the repository. File had conflicts on merge There were conflicts the last time this file was updated, and they have not been resolved yet. You will also see the local revision and date, the revision number of the newest applicable version (newest applicable because if you have a sticky date, tag or branch, it may not be the actual newest revision), and any sticky tags, dates or options. Once you have checked something out, you can update it with the update command. &prompt.user; cvs update shazam This updates the file shazam or the contents of the shazam directory to the latest version along the branch you checked out. If you checked out a point in time, does nothing unless the tags have moved in the repository or some other weird stuff is going on. Useful options, in addition to those listed above for checkout: Check out any additional missing directories. Update to head of main branch. More magic (see below). If you checked out a module with or , running cvs update with a different or argument or with will select a new branch, revision or date. The option clears all sticky tags, dates or revisions whereas and set new ones. Theoretically, specifying HEAD as the argument to will give you the same result as , but that is just theory. The option is useful if: somebody has added subdirectories to the module you have checked out after you checked it out. you checked out with , and later change your mind and want to check out the subdirectories as well. you deleted some subdirectories and want to check them all back out. Watch the output of the cvs update with care. The letter in front of each filename indicates what was done with it: U The file was updated without trouble. P The file was updated without trouble (you will only see this when working against a remote repository). M The file had been modified, and was merged without conflicts. C The file had been modified, and was merged with conflicts. Merging is what happens if you check out a copy of some source code, modify it, then someone else commits a change, and you run cvs update. CVS notices that you have made local changes, and tries to merge your changes with the changes between the version you originally checked out and the one you updated to. If the changes are to separate portions of the file, it will almost always work fine (though the result might not be syntactically or semantically correct). CVS will print an M in front of every locally modified file even if there is no newer version in the repository, so cvs update is handy for getting a summary of what you have changed locally. If you get a C, then your changes conflicted with the changes in the repository (the changes were to the same lines, or neighboring lines, or you changed the local file so much that cvs can not figure out how to apply the repository's changes). You will have to go through the file manually and resolve the conflicts; they will be marked with rows of <, = and > signs. For every conflict, there will be a marker line with seven < signs and the name of the file, followed by a chunk of what your local file contained, followed by a separator line with seven = signs, followed by the corresponding chunk in the repository version, followed by a marker line with seven > signs and the revision number you updated to. The option is slightly voodoo. It updates the local file to the specified revision as if you used , but it does not change the recorded revision number or branch of the local file. It is not really useful except when used twice, in which case it will merge the changes between the two specified versions into the working copy. For instance, say you commit a change to shazam/shazam.c in &os.current; and later want to MFC it. The change you want to MFC was revision 1.15: Check out the &os.stable; version of the shazam module: &prompt.user; cvs co -rRELENG_6 shazam Apply the changes between rev 1.14 and 1.15: &prompt.user; cvs update -j1.14 -j1.15 shazam/shazam.c You will almost certainly get a conflict because - of the $Id: article.sgml,v 1.276 2008-07-22 17:05:47 remko Exp $ (or in FreeBSD's case, + of the $Id: article.sgml,v 1.277 2008-08-06 22:03:48 pgj Exp $ (or in FreeBSD's case, $FreeBSD$) lines, so you will have to edit the file to resolve the conflict - (remove the marker lines and the second $Id: article.sgml,v 1.276 2008-07-22 17:05:47 remko Exp $ line, - leaving the original $Id: article.sgml,v 1.276 2008-07-22 17:05:47 remko Exp $ line intact). + (remove the marker lines and the second $Id: article.sgml,v 1.277 2008-08-06 22:03:48 pgj Exp $ line, + leaving the original $Id: article.sgml,v 1.277 2008-08-06 22:03:48 pgj Exp $ line intact). View differences between the local version and the repository version with the diff command. &prompt.user; cvs diff shazam shows you every modification you have made to the shazam file or module. Useful <command>cvs diff</command> options Uses the unified diff format. Uses the context diff format. Shows missing or added files.
You always want to use , since unified diffs are much easier to read than almost any other diff format (in some circumstances, context diffs generated with the option may be better, but they are much bulkier). A unified diff consists of a series of hunks. Each hunk begins with a line that starts with two @ signs and specifies where in the file the differences are and how many lines they span. This is followed by a number of lines; some (preceded by a blank) are context; some (preceded by a - sign) are outtakes and some (preceded by a +) are additions. You can also diff against a different version than the one you checked out by specifying a version with or as in checkout or update, or even view the diffs between two arbitrary versions (without regard for what you have locally) by specifying two versions with or . View log entries with the log command. &prompt.user; cvs log shazam If shazam is a file, this will print a header with information about this file, such as where in the repository this file is stored, which revision is the HEAD for this file, what branches this file is in, and any tags that are valid for this file. Then, for each revision of this file, a log message is printed. This includes the date and time of the commit, who did the commit, how many lines were added and/or deleted, and finally the log message that the committer who did the change wrote. If shazam is a directory, then the log information described above is printed for each file in the directory in turn. Unless you give the to log, the log for all subdirectories of shazam is printed too, in a recursive manner. Use the log command to view the history of one or more files, as it is stored in the CVS repository. You can even use it to view the log message of a specific revision, if you add the to the log command: &prompt.user; cvs log -r1.2 shazam This will print only the log message for revision 1.2 of file shazam if it is a file, or the log message for revision 1.2 of each file under shazam if it is a directory. See who did what with the annotate command. This command shows you each line of the specified file or files, along with which user most recently changed that line. &prompt.user; cvs annotate shazam Add new files with the add command. Create the file, cvs add it, then cvs commit it. Similarly, you can add new directories by creating them and then cvs adding them. Note that you do not need to commit directories. Remove obsolete files with the remove command. Remove the file, then cvs rm it, then cvs commit it. Commit with the commit or checkin command. Useful <command>cvs commit</command> options Force a commit of an unmodified file. Specify a commit message on the command line rather than invoking an editor.
Use the option if you realize that you left out important information from the commit message. Good commit messages are important. They tell others why you did the changes you did, not just right here and now, but months or years from now when someone wonders why some seemingly illogical or inefficient piece of code snuck into your source file. It is also an invaluable aid to deciding which changes to MFC and which not to MFC. Commit messages should be clear, concise and provide a reasonable summary to give an indication of what was changed and why. Commit messages should provide enough information to enable a third party to decide if the change is relevant to them and if they need to read the change itself. Avoid committing several unrelated changes in one go. It makes merging difficult, and also makes it harder to determine which change is the culprit if a bug crops up. Avoid committing style or whitespace fixes and functionality fixes in one go. It makes merging difficult, and also makes it harder to understand just what functional changes were made. In the case of documentation files, it can make the job of the translation teams more complicated, as it becomes difficult for them to determine exactly what content changes need to be translated. Avoid committing changes to multiple files in one go with a generic, vague message. Instead, commit each file (or small, related groups of files) with tailored commit messages. Before committing, always: verify which branch you are committing to, using cvs status. review your diffs, using cvs diff Also, ALWAYS specify which files to commit explicitly on the command line, so you do not accidentally commit other files than the ones you intended - cvs commit without any arguments will commit every modification in your current working directory and every subdirectory. Additional tips and tricks: You can place commonly used options in your ~/.cvsrc, like this: cvs -z3 diff -Nu update -Pd checkout -P This example says: always use compression level 3 when talking to a remote server. This is a life-saver when working over a slow connection. always use the (show added or removed files) and (unified diff format) options to &man.diff.1;. always use the (prune empty directories) and (check out new directories) options when updating. always use the (prune empty directories) option when checking out. Use Eivind Eklund's cdiff script to view unidiffs. It is a wrapper for &man.less.1; that adds ANSI color codes to make hunk headers, outtakes and additions stand out; context and garbage are unmodified. It also expands tabs properly (tabs often look wrong in diffs because of the extra character in front of each line). textproc/cdiff Simply use it instead of &man.more.1; or &man.less.1;: &prompt.user; cvs diff -Nu shazam | cdiff Alternatively some editors like &man.vim.1; (editors/vim) have color support and when used as a pager with color syntax highlighting switched on will highlight many types of file, including diffs, patches, and CVS/RCS logs. &prompt.user; echo "syn on" >> ~/.vimrc &prompt.user; cvs diff -Nu shazam | vim - &prompt.user; cvs log shazam | vim - CVS is old, arcane, crufty and buggy, and sometimes exhibits non-deterministic behavior which some claim as proof that it is actually merely the Newtonian manifestation of a sentient transdimensional entity. It is not humanly possible to know its every quirk inside out, so do not be afraid to ask the resident AI (&a.cvsadm;) for help. Do not leave the cvs commit command in commit message editing mode for too long (more than 2–3 minutes). It locks the directory you are working with and will prevent other developers from committing into the same directory. If you have to type a long commit message, type it before executing cvs commit and insert it into the commit message or save it in a file before committing and use the option of CVS to read the commit message from that file, i.e. &prompt.user; vi logmsg &prompt.user; cvs ci -F logmsg shazam This is the fastest way of passing a commit message to CVS but you should be careful when editing the logmsg file before the commit, because CVS will not give you a chance to edit the message when you do the actual commit. Speed up your CVS operation considerably by using a persistent ssh connection to the repository machine. First, put this configuration into your ~/.ssh/config: Host ncvs.FreeBSD.org ControlPath /home/user/.ssh/cvs.cpath Host dcvs.FreeBSD.org ControlPath /home/user/.ssh/cvs.cpath Host projcvs.FreeBSD.org ControlPath /home/user/.ssh/cvs.cpath Host pcvs.FreeBSD.org ControlPath /home/user/.ssh/cvs.cpath Now open the persistent connection to the repoman: &prompt.user; ssh -fNM ncvs.FreeBSD.org The CVS commands should now respond faster, as they are reusing existing connection with the repository. Note that all the hostnames are case sensitive. Conventions and Traditions As a new developer there are a number of things you should do first. The first set is specific to committers only. Guidelines For Committers If you have been given commit rights to one or more of the repositories: Add your author entity to doc/en_US.ISO8859-1/share/sgml/authors.ent; this should be done first since an omission of this commit will cause the next commits to break the doc/ build. This is a relatively easy task, but remains a good first test of your CVS skills. Also add your author entity to www/en/developers.sgml. Add yourself to the Developers section of the Contributors List (doc/en_US.ISO8859-1/articles/contributors/contrib.committers.sgml) and remove yourself from the Additional Contributors section (doc/en_US.ISO8859-1/articles/contributors/contrib.additional.sgml). Add an entry for yourself to www/share/sgml/news.xml. Look for the other entries that look like A new committer and follow the format. You should add your PGP or GnuPG key to doc/share/pgpkeys (and if you do not have a key, you should create one). Do not forget to commit the updated doc/share/pgpkeys/pgpkeys.ent and doc/share/pgpkeys/pgpkeys-developers.sgml. &a.des; has written a shell script to make this extremely simple. See the README file for more information. It is important to have an up-to-date PGP/GnuPG key in the Handbook, since the key may be required for positive identification of a committer, e.g. by the &a.admins; for account recovery. A complete keyring of FreeBSD.org users is available for download from http://www.FreeBSD.org/doc/pgpkeyring.txt. Add an entry for yourself to src/share/misc/committers-repository.dot, where repository is either doc, ports or src, depending on the commit privileges you obtained. Some people add an entry for themselves to ports/astro/xearth/files/freebsd.committers.markers. Some people add an entry for themselves to src/usr.bin/calendar/calendars/calendar.freebsd. If you are subscribed to the &a.cvsall;, you will probably want to unsubscribe to avoid receiving duplicate copies of commit messages and their followups. All src commits should go to &os.current; first before being merged to &os.stable;. No major new features or high-risk modifications should be made to the &os.stable; branch. Guidelines For Everyone Whether or not you have commit rights: Introduce yourself to the other developers, otherwise no one will have any idea who you are or what you are working on. You do not have to write a comprehensive biography, just write a paragraph or two about who you are and what you plan to be working on as a developer in FreeBSD. (You should also mention who your mentor will be). Email this to the &a.developers; and you will be on your way! Log into hub.FreeBSD.org and create a /var/forward/user (where user is your username) file containing the e-mail address where you want mail addressed to yourusername@FreeBSD.org to be forwarded. This includes all of the commit messages as well as any other mail addressed to the &a.committers; and the &a.developers;. Really large mailboxes which have taken up permanent residence on hub often get accidentally truncated without warning, so forward it or read it and you will not lose it. Due to the severe load dealing with SPAM places on the central mail servers that do the mailing list processing the front-end server does do some basic checks and will drop some messages based on these checks. At the moment proper DNS information for the connecting host is the only check in place but that may change. Some people blame these checks for bouncing valid email. If you want these checks turned off for your email you can place a file named ~/.spam_lover in your home directory on freefall.FreeBSD.org to disable the checks for your email. If you are a developer but not a committer, you will not be subscribed to the committers or developers mailing lists; the subscriptions are derived from the access rights. Mentors All new developers also have a mentor assigned to them for the first few months. Your mentor is responsible for teaching you the rules and conventions of the project and guiding your first steps in the developer community. He or she is also personally responsible for your actions during this initial period. For committers: until your mentor decides (and announces with a forced commit to access) that you have learned the ropes and are ready to commit on your own, you should not commit anything without first getting your mentor's review and approval, and you should document that approval with an Approved by: line in the commit message. Preferred License for New Files Currently the &os; Project suggests and uses the following text as the preferred license scheme: /*- * Copyright (c) [year] [your name] * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * [id for your version control system, if any] */ The &os; project strongly discourages the so-called "advertising clause" in new code. Due to the large number of contributors to the &os; project, complying with this clause for many commercial vendors has become difficult. If you have code in the tree with the advertising clause, please consider removing it. In fact, please consider using the above license for your code. The &os; project discourages completely new licenses and variations on the standard licenses. New licenses require the approval of core@FreeBSD.org to reside in the main repository. The more different licenses that are used in the tree, the more problems that this causes to those wishing to utilize this code, typically from unintended consequences from a poorly worded license. Project policy dictates that code under some non-BSD licenses must be placed only in specific sections of the repository, and in some cases, compilation must be conditional or even disabled by default. For example, the GENERIC kernel must be compiled under only licenses identical to or substantially similar to the BSD license. GPL, APSL, CDDL, etc, licensed software must not be compiled into GENERIC. Developers are reminded that in open source, getting "open" right is just as important as getting "source" right, as improper handling of intellectual property has serious consequences. Any questions or concerns should immediately be brought to the attention of the core team. Developer Relations If you are working directly on your own code or on code which is already well established as your responsibility, then there is probably little need to check with other committers before jumping in with a commit. If you see a bug in an area of the system which is clearly orphaned (and there are a few such areas, to our shame), the same applies. If, however, you are about to modify something which is clearly being actively maintained by someone else (and it is only by watching the cvs-committers mailing list that you can really get a feel for just what is and is not) then consider sending the change to them instead, just as you would have before becoming a committer. For ports, you should contact the listed MAINTAINER in the Makefile. For other parts of the repository, if you are unsure who the active maintainer might be, it may help to scan the output of cvs log to see who has committed changes in the past. &a.fenner; has written a nice shell script that can help determine who the active maintainer might be. It lists each person who has committed to a given file along with the number of commits each person has made. It can be found on freefall at ~fenner/bin/whodid. If your queries go unanswered or the committer otherwise indicates a lack of proprietary interest in the area affected, go ahead and commit it. If you are unsure about a commit for any reason at all, have it reviewed by -hackers before committing. Better to have it flamed then and there rather than when it is part of the CVS repository. If you do happen to commit something which results in controversy erupting, you may also wish to consider backing the change out again until the matter is settled. Remember – with CVS we can always change it back. Do not impugn the intentions of someone you disagree with. If they see a different solution to a problem than you, or even a different problem, it is not because they are stupid, because they have questionable parentage, or because they are trying to destroy your hard work, personal image, or FreeBSD, but simply because they have a different outlook on the world. Different is good. Disagree honestly. Argue your position from its merits, be honest about any shortcomings it may have, and be open to seeing their solution, or even their vision of the problem, with an open mind. Accept correction. We are all fallible. When you have made a mistake, apologize and get on with life. Do not beat up yourself, and certainly do not beat up others for your mistake. Do not waste time on embarrassment or recrimination, just fix the problem and move on. Ask for help. Seek out (and give) peer reviews. One of the ways open source software is supposed to excel is in the number of eyeballs applied to it; this does not apply if nobody will review code. GNATS The FreeBSD Project utilizes GNATS for tracking bugs and change requests. Be sure that if you commit a fix or suggestion found in a GNATS PR, you use edit-pr pr-number on freefall to close it. It is also considered nice if you take time to close any PRs associated with your commits, if appropriate. You can also make use of &man.send-pr.1; yourself for proposing any change which you feel should probably be made, pending a more extensive peer-review first. You can find out more about GNATS at: http://www.FreeBSD.org/support.html &man.send-pr.1; You can run a local copy of GNATS, and then integrate the FreeBSD GNATS tree in to it using CVSup. Then you can run GNATS commands locally. This lets you query the PR database without needing to be connected to the Internet. Using a local GNATS tree If you are not already downloading the GNATS tree, add this line to your supfile, and re-sup. Note that since GNATS is not under CVS control it has no tag, so if you are adding it to your existing supfile it should appear before any tag= entry as these remain active once set. gnats release=current prefix=/usr This will place the FreeBSD GNATS tree in /usr/gnats. You can use a refuse file to control which categories to receive. For example, to only receive docs PRs, put this line in /usr/local/etc/cvsup/sup/refuse The precise path depends on the *default base setting in your supfile. . gnats/[a-ce-z]* The rest of these examples assume you have only supped the docs category. Adjust them as necessary, depending on the categories you are syncing. Install the GNATS port from ports/databases/gnats. This will place the various GNATS directories under $PREFIX/share/gnats. Symlink the GNATS directories you are supping under the version of GNATS you have installed. &prompt.root; cd /usr/local/share/gnats/gnats-db &prompt.root; ln -s /usr/gnats/docs Repeat as necessary, depending on how many GNATS categories you are syncing. Update the GNATS categories file with these categories. The file is $PREFIX/share/gnats/gnats-db/gnats-adm/categories. # This category is mandatory pending:Category for faulty PRs:gnats-admin: # # FreeBSD categories # docs:Documentation Bug:freebsd-doc: Run $PREFIX/libexec/gnats/gen-index to recreate the GNATS index. The output has to be redirected to $PREFIX/share/gnats/gnats-db/gnats-adm/index. You can do this periodically from &man.cron.8;, or run &man.cvsup.1; from a shell script that does this as well. &prompt.root; /usr/local/libexec/gnats/gen-index \ > /usr/local/share/gnats/gnats-db/gnats-adm/index Test the configuration by querying the PR database. This command shows open docs PRs. &prompt.root; query-pr -c docs -s open Pick a PR and close it. This procedure only works to allow you to view and query the PRs locally. To edit or close them you will still have to log in to freefall and do it from there. Who's Who Besides the repository meisters, there are other FreeBSD project members and teams whom you will probably get to know in your role as a committer. Briefly, and by no means all-inclusively, these are: &a.jhb; John is the manager of the SMPng Project, and has authority over the architectural design and implementation of the move to fine-grained kernel threading and locking. He's also the editor of the SMPng Architecture Document. If you are working on fine-grained SMP and locking, please coordinate with John. You can learn more about the SMPng Project on its home page: &a.doceng; doceng is the group responsible for the documentation build infrastructure, approving new documentation committers, and ensuring that the FreeBSD website and documentation on the FTP site is up to date with respect to the CVS tree. It is not a conflict resolution body. The vast majority of documentation related discussion takes place on the &a.doc;. More details regarding the doceng team can be found in its charter. Committers interested in contributing to the documentation should familiarize themselves with the Documentation Project Primer. &a.ru; Ruslan is Mister &man.mdoc.7;. If you are writing a manual page and need some advice on the structure, or the markup, ask Ruslan. &a.bde; Bruce is the Style Police-Meister. When you do a commit that could have been done better, Bruce will be there to tell you. Be thankful that someone is. Bruce is also very knowledgeable on the various standards applicable to FreeBSD. &a.murray; &a.dwhite; &a.rwatson; &a.kensmith; &a.hrs; &a.mux; &a.bmah; These are the members of the &a.re;. This team is responsible for setting release deadlines and controlling the release process. During code freezes, the release engineers have final authority on all changes to the system for whichever branch is pending release status. If there is something you want merged from &os.current; to &os.stable; (whatever values those may have at any given time), these are the people to talk to about it. Hiroki is also the keeper of the release documentation (src/release/doc/*). If you commit a change that you think is worthy of mention in the release notes, please make sure he knows about it. Better still, send him a patch with your suggested commentary. &a.cperciva; Colin is the FreeBSD Security Officer and oversees the &a.security-officer;. &a.wollman; If you need advice on obscure network internals or are not sure of some potential change to the networking subsystem you have in mind, Garrett is someone to talk to. Garrett is also very knowledgeable on the various standards applicable to FreeBSD. &a.committers; cvs-committers is the entity that CVS uses to send you all your commit messages. You should never send email directly to this list. You should only send replies to this list when they are short and are directly related to a commit. &a.developers; All committers are subscribed to -developers. This list was created to be a forum for the committers community issues. Examples are Core voting, announcements, etc. The &a.developers; is for the exclusive use of FreeBSD committers. In order to develop FreeBSD, committers must have the ability to openly discuss matters that will be resolved before they are publicly announced. Frank discussions of work in progress are not suitable for open publication and may harm FreeBSD. All FreeBSD committers are reminded to obey the copyright of the original author(s) of &a.developers; mail. Do not publish or forward messages from the &a.developers; outside the list membership without permission of all of the authors. Copyright violators will be removed from the &a.developers;, resulting in a suspension of commit privileges. Repeated or flagrant violations may result in permanent revocation of commit privileges. This list is not intended as a place for code reviews or a replacement for the &a.arch; or the &a.audit;. In fact using it as such hurts the FreeBSD Project as it gives a sense of a closed list where general decisions affecting all of the FreeBSD using community are made without being open. Last, but not least never, never ever, email the &a.developers; and CC:/BCC: another FreeBSD list. Never, ever email another FreeBSD email list and CC:/BCC: the &a.developers;. Doing so can greatly diminish the benefits of this list. SSH Quick-Start Guide If you do not wish to type your password in every time you use &man.ssh.1;, and you use RSA or DSA keys to authenticate, &man.ssh-agent.1; is there for your convenience. If you want to use &man.ssh-agent.1;, make sure that you run it before running other applications. X users, for example, usually do this from their .xsession or .xinitrc file. See &man.ssh-agent.1; for details. Generate a key pair using &man.ssh-keygen.1;. The key pair will wind up in your $HOME/.ssh/ directory. Send your public key ($HOME/.ssh/id_dsa.pub or $HOME/.ssh/id_rsa.pub) to the person setting you up as a committer so it can be put into yourlogin file in /etc/ssh-keys/ on freefall. Now you should be able to use &man.ssh-add.1; for authentication once per session. This will prompt you for your private key's pass phrase, and then store it in your authentication agent (&man.ssh-agent.1;). If you no longer wish to have your key stored in the agent, issuing ssh-add -d will remove it. Test by doing something such as ssh freefall.FreeBSD.org ls /usr. For more information, see security/openssh, &man.ssh.1;, &man.ssh-add.1;, &man.ssh-agent.1;, &man.ssh-keygen.1;, and &man.scp.1;. &coverity.prevent; Availability for &os; Committers In January 2006, the &os; Foundation obtained a license for &coverity.prevent; from &coverity Ltd. With this donation, all &os; developers can obtain access to Coverity Prevent analysis results of all &os; Project software. &os; developers who are interested in obtaining access to the analysis results of the automated Coverity Prevent runs, can find out more by logging into freefall and reading the relevant bits of the files: /usr/local/coverity/coverity_license.txt The license terms to which the &os; developers will have to agree in order to use &coverity.prevent; analysis results. /usr/local/coverity/coverity_announcement.txt The announcement posted to the developers' mailing list of the &os; Project. It contains useful information about the &os; Foundation and &coverity; Ltd., as well as signup information for registering with the &coverity.prevent; installation of the &os; Cluster. After reading and understanding the license terms of coverity_license.txt, all &os; developers who are interested in using the analysis results of &coverity.prevent; should read this file. /usr/local/coverity/coverity_readme.txt A short guide about fixes which are committed to the &os; source tree after being detected by &coverity.prevent; and analyzed by an &os; developer. The &os; Wiki includes a mini-guide for developers who are interested in working with the &coverity.prevent; analysis reports: . Please note that this mini-guide is only readable by &os; developers, so if you cannot access this page, you will have to ask someone to add you to the appropriate Wiki access list. Finally, all &os; developers who are going to use &coverity.prevent; are always encouraged to ask for more details and usage information, by posting any questions to the mailing list of the &os; developers. The FreeBSD Committers' Big List of Rules Respect other committers. Respect other contributors. Discuss any significant change before committing. Respect existing maintainers (if listed in the MAINTAINER field in Makefile or in the MAINTAINER file in the top-level directory). Any disputed change must be backed out pending resolution of the dispute if requested by a maintainer. Security related changes may override a maintainer's wishes at the Security Officer's discretion. Changes go to &os.current; before &os.stable; unless specifically permitted by the release engineer or unless they are not applicable to &os.current;. Any non-trivial or non-urgent change which is applicable should also be allowed to sit in &os.current; for at least 3 days before merging so that it can be given sufficient testing. The release engineer has the same authority over the &os.stable; branch as outlined for the maintainer in rule #5. Do not fight in public with other committers; it looks bad. If you must strongly disagree about something, do so only in private. Respect all code freezes and read the committers and developers mailing lists in a timely manner so you know when a code freeze is in effect. When in doubt on any procedure, ask first! Test your changes before committing them. Do not commit to anything under the src/contrib, src/crypto, and src/sys/contrib trees without explicit approval from the respective maintainer(s). As noted, breaking some of these rules can be grounds for suspension or, upon repeated offense, permanent removal of commit privileges. Individual members of core have the power to temporarily suspend commit privileges until core as a whole has the chance to review the issue. In case of an emergency (a committer doing damage to the repository), a temporary suspension may also be done by the repository meisters. Only a 2/3 majority of core has the authority to suspend commit privileges for longer than a week or to remove them permanently. This rule does not exist to set core up as a bunch of cruel dictators who can dispose of committers as casually as empty soda cans, but to give the project a kind of safety fuse. If someone is out of control, it is important to be able to deal with this immediately rather than be paralyzed by debate. In all cases, a committer whose privileges are suspended or revoked is entitled to a hearing by core, the total duration of the suspension being determined at that time. A committer whose privileges are suspended may also request a review of the decision after 30 days and every 30 days thereafter (unless the total suspension period is less than 30 days). A committer whose privileges have been revoked entirely may request a review after a period of 6 months has elapsed. This review policy is strictly informal and, in all cases, core reserves the right to either act on or disregard requests for review if they feel their original decision to be the right one. In all other aspects of project operation, core is a subset of committers and is bound by the same rules. Just because someone is in core this does not mean that they have special dispensation to step outside any of the lines painted here; core's special powers only kick in when it acts as a group, not on an individual basis. As individuals, the core team members are all committers first and core second. Details Respect other committers. This means that you need to treat other committers as the peer-group developers that they are. Despite our occasional attempts to prove the contrary, one does not get to be a committer by being stupid and nothing rankles more than being treated that way by one of your peers. Whether we always feel respect for one another or not (and everyone has off days), we still have to treat other committers with respect at all times, on public forums and in private email. Being able to work together long term is this project's greatest asset, one far more important than any set of changes to the code, and turning arguments about code into issues that affect our long-term ability to work harmoniously together is just not worth the trade-off by any conceivable stretch of the imagination. To comply with this rule, do not send email when you are angry or otherwise behave in a manner which is likely to strike others as needlessly confrontational. First calm down, then think about how to communicate in the most effective fashion for convincing the other person(s) that your side of the argument is correct, do not just blow off some steam so you can feel better in the short term at the cost of a long-term flame war. Not only is this very bad energy economics, but repeated displays of public aggression which impair our ability to work well together will be dealt with severely by the project leadership and may result in suspension or termination of your commit privileges. The project leadership will take into account both public and private communications brought before it. It will not seek the disclosure of private communications, but it will take it into account if it is volunteered by the committers involved in the complaint. All of this is never an option which the project's leadership enjoys in the slightest, but unity comes first. No amount of code or good advice is worth trading that away. Respect other contributors. You were not always a committer. At one time you were a contributor. Remember that at all times. Remember what it was like trying to get help and attention. Do not forget that your work as a contributor was very important to you. Remember what it was like. Do not discourage, belittle, or demean contributors. Treat them with respect. They are our committers in waiting. They are every bit as important to the project as committers. Their contributions are as valid and as important as your own. After all, you made many contributions before you became a committer. Always remember that. Consider the points raised under and apply them also to contributors. Discuss any significant change before committing. The CVS repository is not where changes should be initially submitted for correctness or argued over, that should happen first in the mailing lists and the commit should only happen once something resembling consensus has been reached. This does not mean that you have to ask permission before correcting every obvious syntax error or manual page misspelling, simply that you should try to develop a feel for when a proposed change is not quite such a no-brainer and requires some feedback first. People really do not mind sweeping changes if the result is something clearly better than what they had before, they just do not like being surprised by those changes. The very best way of making sure that you are on the right track is to have your code reviewed by one or more other committers. When in doubt, ask for review! Respect existing maintainers if listed. Many parts of FreeBSD are not owned in the sense that any specific individual will jump up and yell if you commit a change to their area, but it still pays to check first. One convention we use is to put a maintainer line in the Makefile for any package or subtree which is being actively maintained by one or more people; see http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/developers-handbook/policies.html for documentation on this. Where sections of code have several maintainers, commits to affected areas by one maintainer need to be reviewed by at least one other maintainer. In cases where the maintainer-ship of something is not clear, you can also look at the CVS logs for the file(s) in question and see if someone has been working recently or predominantly in that area. Other areas of FreeBSD fall under the control of someone who manages an overall category of FreeBSD evolution, such as internationalization or networking. See http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributors/staff-who.html for more information on this. Any disputed change must be backed out pending resolution of the dispute if requested by a maintainer. Security related changes may override a maintainer's wishes at the Security Officer's discretion. This may be hard to swallow in times of conflict (when each side is convinced that they are in the right, of course) but CVS makes it unnecessary to have an ongoing dispute raging when it is far easier to simply reverse the disputed change, get everyone calmed down again and then try to figure out what is the best way to proceed. If the change turns out to be the best thing after all, it can be easily brought back. If it turns out not to be, then the users did not have to live with the bogus change in the tree while everyone was busily debating its merits. People very rarely call for back-outs in the repository since discussion generally exposes bad or controversial changes before the commit even happens, but on such rare occasions the back-out should be done without argument so that we can get immediately on to the topic of figuring out whether it was bogus or not. Changes go to &os.current; before &os.stable; unless specifically permitted by the release engineer or unless they are not applicable to &os.current;. Any non-trivial or non-urgent change which is applicable should also be allowed to sit in &os.current; for at least 3 days before merging so that it can be given sufficient testing. The release engineer has the same authority over the &os.stable; branch as outlined in rule #5. This is another do not argue about it issue since it is the release engineer who is ultimately responsible (and gets beaten up) if a change turns out to be bad. Please respect this and give the release engineer your full cooperation when it comes to the &os.stable; branch. The management of &os.stable; may frequently seem to be overly conservative to the casual observer, but also bear in mind the fact that conservatism is supposed to be the hallmark of &os.stable; and different rules apply there than in &os.current;. There is also really no point in having &os.current; be a testing ground if changes are merged over to &os.stable; immediately. Changes need a chance to be tested by the &os.current; developers, so allow some time to elapse before merging unless the &os.stable; fix is critical, time sensitive or so obvious as to make further testing unnecessary (spelling fixes to manual pages, obvious bug/typo fixes, etc.) In other words, apply common sense. Changes to the security branches (for example, RELENG_6_0) must be approved by a member of the &a.security-officer;, or in some cases, by a member of the &a.re;. Do not fight in public with other committers; it looks bad. If you must strongly disagree about something, do so only in private. This project has a public image to uphold and that image is very important to all of us, especially if we are to continue to attract new members. There will be occasions when, despite everyone's very best attempts at self-control, tempers are lost and angry words are exchanged. The best thing that can be done in such cases is to minimize the effects of this until everyone has cooled back down. That means that you should not air your angry words in public and you should not forward private correspondence to public mailing lists or aliases. What people say one-to-one is often much less sugar-coated than what they would say in public, and such communications therefore have no place there - they only serve to inflame an already bad situation. If the person sending you a flame-o-gram at least had the grace to send it privately, then have the grace to keep it private yourself. If you feel you are being unfairly treated by another developer, and it is causing you anguish, bring the matter up with core rather than taking it public. Core will do its best to play peace makers and get things back to sanity. In cases where the dispute involves a change to the codebase and the participants do not appear to be reaching an amicable agreement, core may appoint a mutually-agreeable 3rd party to resolve the dispute. All parties involved must then agree to be bound by the decision reached by this 3rd party. Respect all code freezes and read the committers and developers mailing list on a timely basis so you know when a code freeze is in effect. Committing unapproved changes during a code freeze is a really big mistake and committers are expected to keep up-to-date on what is going on before jumping in after a long absence and committing 10 megabytes worth of accumulated stuff. People who abuse this on a regular basis will have their commit privileges suspended until they get back from the FreeBSD Happy Reeducation Camp we run in Greenland. When in doubt on any procedure, ask first! Many mistakes are made because someone is in a hurry and just assumes they know the right way of doing something. If you have not done it before, chances are good that you do not actually know the way we do things and really need to ask first or you are going to completely embarrass yourself in public. There is no shame in asking how in the heck do I do this? We already know you are an intelligent person; otherwise, you would not be a committer. Test your changes before committing them. This may sound obvious, but if it really were so obvious then we probably would not see so many cases of people clearly not doing this. If your changes are to the kernel, make sure you can still compile both GENERIC and LINT. If your changes are anywhere else, make sure you can still make world. If your changes are to a branch, make sure your testing occurs with a machine which is running that code. If you have a change which also may break another architecture, be sure and test on all supported architectures. Please refer to the FreeBSD Internal Page for a list of available resources. As other architectures are added to the FreeBSD supported platforms list, the appropriate shared testing resources will be made available. Do not commit to anything under the src/contrib, src/crypto, and src/sys/contrib trees without explicit approval from the respective maintainer(s). The trees mentioned above are for contributed software usually imported onto a vendor branch. Committing something there, even if it does not take the file off the vendor branch, may cause unnecessary headaches for those responsible for maintaining that particular piece of software. Thus, unless you have explicit approval from the maintainer (or you are the maintainer), do not commit there! Please note that this does not mean you should not try to improve the software in question; you are still more than welcome to do so. Ideally, you should submit your patches to the vendor. If your changes are FreeBSD-specific, talk to the maintainer; they may be willing to apply them locally. But whatever you do, do not commit there by yourself! Contact the &a.core; if you wish to take up maintainership of an unmaintained part of the tree. Policy on Multiple Architectures FreeBSD has added several new arch ports during recent release cycles and is truly no longer an &i386; centric operating system. In an effort to make it easier to keep FreeBSD portable across the platforms we support, core has developed the following mandate:
Our 32 bit reference platform is i386, and our 64 bit reference platform is Sparc64. Major design work (including major API and ABI changes) must prove itself on at least one 32 bit and at least one 64 bit platform, preferably the primary reference platforms, before it may be committed to the source tree.
The i386 and Sparc64 platforms were chosen due to being more readily available to developers and as representatives of more diverse processor and system designs - big vs little endian, register file vs register stack, different DMA and cache implementations, hardware page tables vs software TLB management etc. The ia64 platform has many of the same complications that Sparc64 has, but is still limited in availability to developers. We will continue to re-evaluate this policy as cost and availability of the 64 bit platforms change. Developers should also be aware of our Tier Policy for the long term support of hardware architectures. The rules here are intended to provide guidance during the development process, and are distinct from the requirements for features and architectures listed in that section. The Tier rules for feature support on architectures at release-time are more strict than the rules for changes during the development process. Other Suggestions When committing documentation changes, use a spell checker before committing. For all SGML docs, you should also verify that your formatting directives are correct by running make lint. For all on-line manual pages, run manck (from ports) over the manual page to verify all of the cross references and file references are correct and that the man page has all of the appropriate MLINKs installed. Do not mix style fixes with new functionality. A style fix is any change which does not modify the functionality of the code. Mixing the changes obfuscates the functionality change when using cvs diff, which can hide any new bugs. Do not include whitespace changes with content changes in commits to doc/ or www/. The extra clutter in the diffs makes the translators' job much more difficult. Instead, make any style or whitespace changes in separate commits that are clearly labeled as such in the commit message. Deprecating Features When it is necessary to remove functionality from software in the base system the following guidelines should be followed whenever possible: Mention is made in the manual page and possibly the release notes that the option, utility, or interface is deprecated. Use of the deprecated feature generates a warning. The option, utility, or interface is preserved until the next major (point zero) release. The option, utility, or interface is removed and no longer documented. It is now obsolete. It is also generally a good idea to note its removal in the release notes. Support for Multiple Architectures FreeBSD is a highly portable operating system intended to function on many different types of hardware architectures. Maintaining clean separation of Machine Dependent (MD) and Machine Independent (MI) code, as well as minimizing MD code, is an important part of our strategy to remain agile with regards to current hardware trends. Each new hardware architecture supported by FreeBSD adds substantially to the cost of code maintenance, toolchain support, and release engineering. It also dramatically increases the cost of effective testing of kernel changes. As such, there is strong motivation to differentiate between classes of support for various architectures while remaining strong in a few key architectures that are seen as the FreeBSD "target audience". Statement of General Intent The FreeBSD Project targets "production quality commercial off-the-shelf (COTS) workstation, server, and high-end embedded systems". By retaining a focus on a narrow set of architectures of interest in these environments, the FreeBSD Project is able to maintain high levels of quality, stability, and performance, as well as minimize the load on various support teams on the project, such as the ports team, documentation team, security officer, and release engineering teams. Diversity in hardware support broadens the options for FreeBSD consumers by offering new features and usage opportunities (such as support for 64-bit CPUs, use in embedded environments, etc.), but these benefits must always be carefully considered in terms of the real-world maintenance cost associated with additional platform support. The FreeBSD Project differentiates platform targets into four tiers. Each tier includes a specification of the requirements for an architecture to be in that tier, as well as specifying the obligations of developers with regards to the platform. In addition, a policy is defined regarding the circumstances required to change the tier of an architecture. Tier 1: Fully Supported Architectures Tier 1 platforms are fully supported by the security officer, release engineering, and toolchain maintenance staff. New features added to the operating system must be fully functional across all Tier 1 architectures for every release (features which are inherently architecture-specific, such as support for hardware device drivers, may be exempt from this requirement). In general, all Tier 1 platforms must have build and tinderbox support either in the FreeBSD.org cluster, or easily available for all developers. Embedded platforms may substitute an emulator available in the FreeBSD cluster for actual hardware. Tier 1 architectures are expected to be Production Quality with respects to all aspects of the FreeBSD operating system, including installation and development environments. Tier 1 architectures are expected to be completely integrated into the source tree and have all features necessary to produce an entire system relevant for that target architecture. Tier 1 architectures generally have at least 6 active developers. Tier 1 architectures are expected to be fully supported by the ports system. All the ports should build on a Tier 1 platform, or have the appropriate filters to prevent the inappropriate ones from building there. The packaging system must support all Tier 1 architectures. To ensure an architectures Tier 1 status, proponents of that architecture must show that all relevant packages can be built on that platform. Tier 1 embedded architectures must be able to cross-build packages on at least one other tier 1 architecture. The packages must be the most relevant for the platform, but may be a non-empty subset of those that build natively. Tier 1 architectures must be fully documented. All basic operations need to be covered by the handbook or other documents. All relevant integration documentation must also be integrated into the tree, or readily available. Current Tier 1 platforms are i386, AMD64, and PC98. Tier 2: Developmental Architectures Tier 2 platforms are not supported by the security officer and release engineering teams. Platform maintainers are responsible for toolchain support in the tree. The toolchain maintainer is expected to work with the platform maintainers to refine these changes. Major new toolchain components are allowed to break support for Tier 2 architectures if the FreeBSD-local changes haven't been incorporated upstream. The toolchain maintainers are expected to provide prompt review of any proposed changes and cannot block, through their inaction, changes going into the tree. New features added to FreeBSD should be feasible to implement on these platforms, but an implementation is not required before the feature may be added to the FreeBSD source tree. New features that may be difficult to implement on Tier 2 architectures should provide a means of disabling them on those architectures. The implementation of a Tier 2 architecture may be committed to the main FreeBSD tree as long as it does not interfere with production work on Tier 1 platforms, or substantially with other Tier 2 platforms. Before a Tier 2 platform can be added to the FreeBSD base source tree, the platform must be able to boot multi-user on actual hardware. Generally, there must be at least three active developers working on the platform. Tier 2 architectures are usually systems targeted at Tier 1 support, but that are still under development. Architectures reaching end of life may also be moved from Tier 1 status to Tier 2 status as the availability of resources to continue to maintain the system in a Production Quality state diminishes. Well supported niche architectures may also be tier 2. Tier 2 architectures may have some support for them integrated into the ports infrastructure. They may have cross compilation support added, at the discretion of portmgr. Some ports must built natively, into packages if the package system supports that architecture. If not integrated into the base system, some external patches for the architecture for ports must be available. Tier 2 architectures can be integrated into the FreeBSD handbook. The basics for how to get a system running must be documented, although not necessarily for every single board or system a tier 2 architecture supports. The supported hardware list must exist and should be no more than a couple of months old. It should be integrated into the FreeBSD documentation. Current Tier 2 platforms are ARM, PowerPC, ia64, Sparc64 and sun4v. Tier 3: Experimental Architectures Tier 3 platforms are not supported by the security officer and release engineering teams. At the discretion of the toolchain maintainer, they may be supported in the toolchain. Tier 3 platforms are architectures in the early stages of development, for non-mainstream hardware platforms, or which are considered legacy systems unlikely to see broad future use. New tier 3 systems will not be committed to the base source tree. Support for Tier 3 systems may be worked on in the FreeBSD Perforce Repository, providing source control and easier change integration from the main FreeBSD tree. Platforms that transition to Tier 3 status may be removed from the tree if they are no longer actively supported by the FreeBSD developer community at the discretion of the release engineer. Tier 3 platforms may have ports support, either integrated or external, but do not require it. Tier 3 platforms must have the basics documented for how to build a kernel and how to boot it on at least one target hardware or emulation environment. This documentation need not be integrated into the FreeBSD tree. Current Tier 3 platforms are MIPS and &s390;. Tier 4: Unsupported Architectures Tier 4 systems are not supported in any form by the project. All systems not otherwise classified into a support tier are Tier 4 systems. Policy on Changing the Tier of an Architecture Systems may only be moved from one tier to another by approval of the FreeBSD Core Team, which shall make that decision in collaboration with the Security Officer, Release Engineering, and toolchain maintenance teams. Ports Specific FAQ Adding a New Port How do I add a new port? First, please read the section about repository copies. The easiest way to add a new port is to use the addport script on freefall. It will add a port from the directory you specify, determining the category automatically from the port Makefile. It will also add an entry to the port's category Makefile. It was written by &a.mharo; and &a.will;, and is currently maintained by &a.garga;, so please send questions/patches about addport to him. Any other things I need to know when I add a new port? Check the port, preferably to make sure it compiles and packages correctly. This is the recommended sequence: &prompt.root; make install &prompt.root; make package &prompt.root; make deinstall &prompt.root; pkg_add package you built above &prompt.root; make deinstall &prompt.root; make reinstall &prompt.root; make package The Porters Handbook contains more detailed instructions. Use &man.portlint.1; to check the syntax of the port. You do not necessarily have to eliminate all warnings but make sure you have fixed the simple ones. If the port came from a submitter who has not contributed to the project before, add that person's name to the Additional Contributors section of the FreeBSD Contributors List. Close the PR if the port came in as a PR. To close a PR, just do edit-pr PR# on freefall and change the state from open to closed. You will be asked to enter a log message and then you are done. Removing an Existing Port How do I remove an existing port? First, please read the section about repository copies. Before you remove the port, you have to verify there are no other ports depending on it. Make sure there is no dependency on the port in the ports collection: The port's PKGNAME should appear in exactly one line in a recent INDEX file. No other ports should contain any reference to the port's directory or PKGNAME in their Makefiles Then, remove the port: Remove the port's files via cvs remove. Remove SUBDIR listing of the port in the parent directory Makefile. Add an entry to ports/MOVED. Remove the port from ports/LEGAL if it is there. Alternatively, you can use the rmport - script, from ports/Tools/scripts. + script, from ports/Tools/scripts. This script has been written by &a.vd;, who is also its current maintainer, so please send questions, patches or suggestions about rmport to him. Repository Copies When do we need a repository copy? When you want to add a port that is related to any port that is already in the tree in a separate directory, you have to do a repository copy. Here related means it is a different version or a slightly modified version. Examples are print/ghostscript* (different versions) and x11-wm/windowmaker* (English-only and internationalized version). Another example is when a port is moved from one subdirectory to another, or when you want to change the name of a directory because the author(s) renamed their software even though it is a descendant of a port already in a tree. When do we not need a repository copy? When there is no history to preserve. If a port is added into a wrong category and is moved immediately, it suffices to simply cvs remove the old one and addport the new one. What do I need to do? File a PR in GNATS, listing the reasons for the repository copy request. Assign it to portmgr and set state to repocopy. In a few days, portmgr will do a repository copy from the old to the new location, and reassign the PR back to you. Once everything is done, perform the following: When a port has been repo copied: Do a force commit on the files of the copied port, stating repository copy was performed. Upgrade the copied port to the new version. Remember to change the LATEST_LINK so there are no duplicate ports with the same name. In some rare cases it may be necessary to change the PORTNAME instead of LATEST_LINK, but this should only be done when it is really needed — e.g. using an existing port as the base for a very similar program with a different name, or upgrading a port to a new upstream version which actually changes the distribution name, like the transition from textproc/libxml to textproc/libxml2. In most cases, changing LATEST_LINK should suffice. Add the new subdirectory to the SUBDIR listing in the parent directory Makefile. You can run make checksubdirs in the parent directory to check this. If the port changed categories, modify the CATEGORIES line of the port's Makefile accordingly Add an entry to ports/MOVED, if you remove the original port. When removing a port: Perform a thorough check of the ports collection for any dependencies on the old port location/name, and update them. Running grep on INDEX is not enough because some ports have dependencies enabled by compile-time options. A full grep -r of the ports collection is recommended. Remove the old port, the old SUBDIR entry and the old module entry. Add an entry to ports/MOVED. After repo moves (rename operations where a port is copied and the old location is removed): Follow the same steps that are outlined in the previous two entries, to activate the new location of the port and remove the old one. Ports Freeze What is a ports freeze? Before a release, it is necessary to restrict commits to the ports tree for a short period of time while the packages and the release itself are being built. This is to ensure consistency among the various parts of the release, and is called the ports freeze. For more information on the background and policies surrounding a ports freeze, see the Portmgr Quality Assurance page. How long is a ports freeze? Usually a week or two. What does it mean to me? During the ports freeze, you are not allowed to commit anything to the tree without explicit approval from the ports management team. Explicit approval here means that you send a patch to the ports management team for review and get a reply saying, Go ahead and commit it. Not everything is allowed to be committed during a freeze. Please see the Portmgr Quality Assurance page for more information. Note that you do not have implicit permission to fix a port during the freeze just because it is broken. How do I know when the ports freeze starts? The ports management team will send out warning messages to the &a.ports; and &a.committers; announcing the start of the impending release, usually two or three weeks in advance. The exact starting time will not be determined until a few days before the actual release. This is because the ports freeze has to be synchronized with the release, and it is usually not known until then when exactly the release will be rolled. When the freeze starts, there will be another announcement to the &a.committers;, of course. How do I know when the ports freeze ends? A few hours after the release, the ports management team will send out a mail to the &a.ports; and &a.committers; announcing the end of the ports freeze. Note that the release being cut does not automatically end the freeze. We have to make sure there will not be any last minute snafus that result in an immediate re-rolling of the release. Creating a New Category What is the procedure for creating a new category? Please see Proposing a New Category in the Porter's Handbook. Once that procedure has been followed and the PR has been assigned to &a.portmgr;, it is their decision whether or not to approve it. If they do, it is their responsibility to do the following: Perform any needed repocopies. (This only applies to physical categories.) Update the VALID_CATEGORIES definition in ports/Mk/bsd.port.mk. Assign the PR back to you. What do I need to do to implement a new physical category? The procedure is a strict superset of the one to repocopy individual ports (see above). Upgrade each copied port's Makefile. Do not connect the new category to the build yet. To do this, you will need to: Change the port's CATEGORIES (this was the point of the exercise, remember?) The new category should be listed first. This will help to ensure that the the PKGORIGIN is correct. Run a make describe. Since the top-level make index that you will be running in a few steps is an iteration of make describe over the entire ports hierarchy, catching any errors here will save you having to re-run that step later on. If you want to be really thorough, now might be a good time to run &man.portlint.1;. Check that the PKGORIGINs are correct. The ports system uses each port's CATEGORIES entry to create its PKGORIGIN, which is used to connect installed packages to the port directory they were built from. If this entry is wrong, common port tools like &man.pkg.version.1; and &man.portupgrade.1; fail. To do this, use the chkorigin.sh tool, as follows: env PORTSDIR=/path/to/ports sh -e /path/to/ports/Tools/scripts/chkorigin.sh . This will check every port in the ports tree, even those not connected to the build, so you can run it directly after the repocopy. Hint: do not forget to look at the PKGORIGINs of any slave ports of the ports you just repocopied! On your own local system, test the proposed changes: first, comment out the SUBDIR entries in the old ports' categories' Makefiles; then enable building the new category in ports/Makefile. Run make checksubdirs in the affected category directories to check the SUBDIR entries. Next, in the ports/ directory, run make index. This can take over 40 minutes on even modern systems; however, it is a necessary step to prevent problems for other people. Once this is done, you can commit the updated ports/Makefile to connect the new category to the build and also commit the Makefile changes for the old category or categories. Add appropriate entries to ports/MOVED. Update the instructions for &man.cvsup.1;: add the category to distrib/cvsup/sup/README adding the following files into distrib/cvsup/sup/ports-categoryname: list.cvs and releases. add the category to src/share/examples/cvsup/ports-supfile (Note: these are in the src, not the ports, repository). If you are not a src committer, you will need to submit a PR for this. Update the list of categories used by &man.sysinstall.8; in src/usr.sbin/sysinstall. Update the documentation by modifying the following: the section of the Handbook that lists the cvsup collections. the list of categories in the Porter's Handbook www/en/ports/categories. Note that these are now displayed by sub-groups, as specified in www/en/ports/categories.descriptions. (Note: these are in the docs, not the ports, repository). If you are not a docs committer, you will need to submit a PR for this. Only once all the above have been done, and no one is any longer reporting problems with the new ports, should the old ports be deleted from their previous locations in the repository. It is not necessary to manually update the ports web pages to reflect the new category. This is now done automatically via your change to www/en/ports/categories and the daily automated rebuild of INDEX. What do I need to do to implement a new virtual category? This is much simpler than a physical category. You only need to modify the following: src/usr.sbin/sysinstall the list of categories in the Porter's Handbook www/en/ports/categories Miscellaneous Questions How do I know if my port is building correctly or not? First, go check . There you will find error logs from the latest package building runs on all supported platforms for the most recent branches. However, just because the port does not show up there does not mean it is building correctly. (One of the dependencies may have failed, for instance.) The relevant directories are available on pointyhat under /a/portbuild/<arch>/<major_version> so feel free to dig around. Each architecture and version has the following subdirectories: errors error logs from latest <major_version> run on <arch> logs all logs from latest <major_version> run on <arch> packages packages from latest <major_version> run on <arch> bak/errors error logs from last complete <major_version> run on <arch> bak/logs all logs from last complete <major_version> run on <arch> bak/packages packages from last complete <major_version> run on <arch> Basically, if the port shows up in packages, or it is in logs but not in errors, it built fine. (The errors directories are what you get from the web page.) I added a new port. Do I need to add it to the INDEX? No, INDEX is no longer stored in the CVS repository. The file can either be generated by running make index, or a pre-generated version can be downloaded with make fetchindex. Are there any other files I am not allowed to touch? Any file directly under ports/, or any file under a subdirectory that starts with an uppercase letter (Mk/, Tools/, etc.). In particular, the ports management team is very protective of ports/Mk/bsd.port*.mk so do not commit changes to those files unless you want to face his wra(i)th. What is the proper procedure for updating the checksum for a port's distfile when the file changes without a version change? When the checksum for a port's distfile is updated due to the author updating the file without changing the port's revision, the commit message should include a summary of the relevant diffs between the original and new distfile to ensure that the distfile has not been corrupted or maliciously altered. If the current version of the port has been in the ports tree for a while, a copy of the old distfile will usually be available on the ftp servers; otherwise the author or maintainer should be contacted to find out why the distfile has changed. Issues Specific To Developers Who Are Not Committers A few people who have access to the FreeBSD machines do not have commit bits. For instance, the project is willing to give access to the GNATS database to contributors who have shown interest and dedication in working on Problem Reports. Almost all of this document will apply to these developers as well (except things specific to CVS commits and the mailing list memberships that go with them). In particular, we recommend that you read: Administrative Details Conventions You should get your mentor to add you to the Additional Contributors (doc/en_US.ISO8859-1/articles/contributors/contrib.additional.sgml), if you are not already listed there. Developer Relations SSH Quick-Start Guide The FreeBSD Committers' Big List of Rules Perks of the Job Unfortunately, there are not many perks involved with being a committer. Recognition as a competent software engineer is probably the only thing that will be of benefit in the long run. However, there are at least some perks: Direct access to cvsup-master As a committer, you may apply to &a.kuriyama; for direct access to cvsup-master.FreeBSD.org, providing the public key output from cvpasswd yourusername@FreeBSD.org freefall.FreeBSD.org. Please note: you must specify freefall.FreeBSD.org on the cvpasswd command line even though the actual server is cvsup-master. Access to cvsup-master should not be overused as it is a busy machine. A Free 4-CD Set or DVD Subscription FreeBSD Mall, Inc. offers a free subscription of the 4-CD set or the DVD product to all FreeBSD committers. Information about how to obtain your free media is mailed to developers@FreeBSD.org following each major release. Miscellaneous Questions Why are trivial or cosmetic changes to files on a vendor branch a bad idea? From now on, every new vendor release of that file will need to have patches merged in by hand. From now on, every new vendor release of that file will need to have patches verified by hand. The option does not work very well. Ask &a.obrien; for horror stories. How do I add a new file to a CVS branch? To add a file onto a branch, simply checkout or update to the branch you want to add to and then add the file using cvs add as you normally would. For example, if you wanted to MFC the file src/sys/alpha/include/smp.h from HEAD to RELENG_6 and it does not exist in RELENG_6 yet, you would use the following steps: MFC'ing a New File &prompt.user; cd sys/alpha/include &prompt.user; cvs update -rRELENG_6 cvs update: Updating . U clockvar.h U console.h ... &prompt.user; cvs update -kk -Ap smp.h > smp.h =================================================================== Checking out smp.h RCS: /usr/cvs/src/sys/alpha/include/smp.h,v VERS: 1.1 *************** &prompt.user; cvs add smp.h cvs add: scheduling file `smp.h' for addition on branch `RELENG_6' cvs add: use 'cvs commit' to add this file permanently &prompt.user; cvs commit What meta information should I include in a commit message? As well as including an informative message with each commit you may need to include some additional information as well. This information consists of one or more lines containing the key word or phrase, a colon, tabs for formatting, and then the additional information. The key words or phrases are: PR: The problem report (if any) which is affected (typically, by being closed) by this commit. Submitted by: The name and e-mail address of the person that submitted the fix; for committers, just the username on the FreeBSD cluster. Reviewed by: The name and e-mail address of the person or people that reviewed the change; for committers, just the username on the FreeBSD cluster. If a patch was submitted to a mailing list for review, and the review was favorable, then just include the list name. Approved by: The name and e-mail address of the person or people that approved the change; for committers, just the username on the FreeBSD cluster. It is customary to get prior approval for a commit if it is to an area of the tree to which you do not usually commit. In addition, during the run up to a new release all commits must be approved by the release engineering team. If these are your first commits then you should have passed them past your mentor first, and you should list your mentor, as in ``username-of-mentor (mentor)''. Obtained from: The name of the project (if any) from which the code was obtained. MFC after: If you wish to receive an e-mail reminder to MFC at a later date, specify the number of days, weeks, or months after which an MFC is planned. Security: If the change is related to a security vulnerability or security exposure, include one or more references or a description of the issue. Commit log for a commit based on a PR You want to commit a change based on a PR submitted by John Smith containing a patch. The end of the commit message should look something like this. ... PR: foo/12345 Submitted by: John Smith <John.Smith@example.com> Commit log for a commit needing review You want to change the virtual memory system. You have posted patches to the appropriate mailing list (in this case, freebsd-arch) and the changes have been approved. ... Reviewed by: -arch Commit log for a commit needing approval You want to commit a change to a section of the tree with a MAINTAINER assigned. You have collaborated with the listed MAINTAINER, who has told you to go ahead and commit. ... Approved by: abc Where abc is the account name of the person who approved. Commit log for a commit bringing in code from OpenBSD You want to commit some code based on work done in the OpenBSD project. ... Obtained from: OpenBSD Commit log for a change to &os.current; with a planned commit to &os.stable; to follow at a later date. You want to commit some code which will be merged from &os.current; into the &os.stable; branch after two weeks. ... MFC after: 2 weeks Where 2 is the number of days, weeks, or months after which an MFC is planned. The weeks option may be day, days, week, weeks, month, months, or may be left off (in which case, days will be assumed). In some cases you may need to combine some of these. Consider the situation where a user has submitted a PR containing code from the NetBSD project. You are looking at the PR, but it is not an area of the tree you normally work in, so you have decided to get the change reviewed by the arch mailing list. Since the change is complex, you opt to MFC after one month to allow adequate testing. The extra information to include in the commit would look something like PR: foo/54321 Submitted by: John Smith <John.Smith@example.com> Reviewed by: -arch Obtained from: NetBSD MFC after: 1 month How do I access people.FreeBSD.org to put up personal or project information? people.FreeBSD.org is the same as freefall.FreeBSD.org. Just create a public_html directory. Anything you place in that directory will automatically be visible under . Where are the mailing list archives stored? The mailing lists are archived under /g/mail which will show up as /hub/g/mail with &man.pwd.1;. This location is accessible from any machine on the FreeBSD cluster. I would like to mentor a new committer. What process do I need to follow? See the New Account Creation Procedure document on the internal pages.
diff --git a/en_US.ISO8859-1/articles/cvs-freebsd/article.sgml b/en_US.ISO8859-1/articles/cvs-freebsd/article.sgml index d547edaa69..40c0e0a076 100644 --- a/en_US.ISO8859-1/articles/cvs-freebsd/article.sgml +++ b/en_US.ISO8859-1/articles/cvs-freebsd/article.sgml @@ -1,684 +1,684 @@ %articles.ent; ]>
Setting up a CVS repository - the FreeBSD way Stijn Hoop
stijn@win.tue.nl
 2001 2002 2003 Stijn Hoop $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.general; This article describes the steps I took to set up a CVS repository that uses the same scripts the FreeBSD project uses in their setup. This has several advantages over a stock CVS setup, including more granular access control to the source tree and generation of readable email of every commit. Introduction Most of the open source software projects use CVS as their source code control system. While CVS is pretty good at this, it has its share of flaws and weaknesses. One of these is that sharing a source tree with other developers can quickly lead to a system administration nightmare, especially if one wishes to protect parts of the tree from general access. FreeBSD is one of the projects using CVS. It also has a large base of developers located around the world. They developed some scripts to make management of the repository easier. Recently, these scripts were revisited and normalized by &a.joe; to make it easier to reuse them in other projects. This article describes one method of using the new scripts. To make the most use of the information in this article, you need to be familiar with the basic method of operation of CVS. First setup It might be best to first perform this procedure with an empty test repository, to make sure you understand all consequences. As always, make sure you have recent, readable backups! Initializing the repository The first thing to do when setting up a new repository is to tell CVS to initialize it: &prompt.user; cvs -d path-to-repository init This tells CVS to create the - CVSROOT administrative directory, where all the + CVSROOT administrative directory, where all the customization takes place. The repository group Now we will create the group which will own the repository. All committers need to be in this group, so that they can write to the repository. We will assume the FreeBSD default of ncvs for this group. &prompt.root; pw groupadd ncvs Next, you should &man.chown.8; the directory to the group you just added: &prompt.root; chown -R :ncvs path-to-your-repository This ensures that no one can write to the repository without proper group permissions. Getting the sources - Now you need to obtain the CVSROOT directory + Now you need to obtain the CVSROOT directory from the FreeBSD repository. This is most easily done by checking it out from a FreeBSD anonymous CVS mirror. See the relevant chapter in the handbook for more information. Let us assume that the - sources are stored in CVSROOT-freebsd in the + sources are stored in CVSROOT-freebsd in the current directory. Copying the FreeBSD scripts - Next, we will copy the FreeBSD CVSROOT + Next, we will copy the FreeBSD CVSROOT sources into your own repository. If you are accustomed to CVS, you might be thinking that you can just import the scripts, in an attempt to make synchronizing with later versions easier. However, it turns out that CVS has a deficiency in this area: - when importing sources into the CVSROOT directory, + when importing sources into the CVSROOT directory, it will not update the needed administrative files. In order to make it recognize those, you will need to checkin each file after importing them, losing the value of cvs import. Therefore, the recommended method is to simply copy over the scripts. It does not matter if the above paragraph did not make sense to you—the end result is the same. Simply check out your - CVSROOT and copy the FreeBSD files over your + CVSROOT and copy the FreeBSD files over your local (untouched) copies: &prompt.user; cvs -d path-to-your-repository checkout CVSROOT &prompt.user; cd CVSROOT &prompt.user; cp ../CVSROOT-freebsd/* . &prompt.user; cvs add * Note that you will probably get a few warnings about some directories not being copied; this is normal, you do not need those. The scripts Now you have in your working directory an exact copy of the scripts that the FreeBSD project itself uses for their repository. A summary of what each file is used for is included below. access - this file is not used in the default setup. It is used in the FreeBSD project specific setup, where it controls access to the repository. You can remove this file if you do not wish to use this setup. avail - this file controls access to the repository. In this, you can specify groups of people that are allowed access to the repository, as well as disallow commits on a per-directory or per-file basis. You should tailor it to contain the groups and directories that will be in your repository. cfg.pm - this file parses your configuration, and provides the default configuration. You should not make changes to this file. Instead, put your configuration changes in cfg_local.pm. cfg_local.pm - this file contains all configurable parameters of the system. You should configure all sorts of settings here, such as where commit mail is send, on what hosts people can commit, and others. More information on this below. checkoutlist - this files lists all files under control of CVS in this directory, apart from the standard ones created by cvs init. You should edit this to remove some FreeBSD-specific files. commit_prep.pl - this script performs various pre-commit checks, based on whether you enabled them in your cfg_local.pm. You should not have to touch this. commitcheck - this script is invoked directly from CVS. It first checks if the committer has access to the specified part of the tree using cvs_acls.pl, and then runs commit_prep.pl for the various pre-commit checks. If those are OK, CVS will allow the commit to proceed. You should not have to touch this file. commitinfo - this file is used by CVS to determine which script to run before a commit—in this case commitcheck. You should not have to touch this file. config - the configuration file for this repository. You should change this as needed, but most administrators can probably leave the defaults. More information on the options that can be set here can be found in the CVS manual. cvs_acls.pl - this script determines the committers identity, and whether he/she is allowed access to the tree. It does this based on the avail file. You should not have to touch this file. cvsignore - this file specifies files that CVS should not checkin in the repository. You can edit this as you wish. More information about this file is available in the CVS manual. cvswrappers - this file is used by CVS to enable or disable keyword expansion, or whether a file should be considered binary. You can edit this as you wish. More information about this file is available in the CVS manual. Note that the -t and -f options do not work correctly with client/server CVS. edithook - this file is not used any more, but kept for historic reasons. You can safely remove this file. editinfo - CVS uses this file for editor overrides. FreeBSD does not use this functionality, as parsing the log message is done by verifymsg and logcheck. This is because the editinfo functionality does not work properly with remote commits, or ones that use the -m or -F options. You should not have to touch this file. exclude - this file lists regular expressions that are used by commit_prep.pl to determine files which cannot contain a revision header. In the FreeBSD setup, all files under revision control need to have a revision header (like $FreeBSD$). All filenames that match one of the lines in this file are exempted from this check. You should add expressions to this file as you checkin files that cannot have a revision header. For the purpose of installing the - scripts, it may be best to exclude CVSROOT/ + scripts, it may be best to exclude CVSROOT/ from header checks. log_accum.pl - this is a script that takes the log message as provided by the logcheck script, and appends it to a log file in the repository for backup purposes. It also handles mailing out a message to an email address you provide (in cfg_local.pm). It hooks into CVS via loginfo. You should not have to touch this file. logcheck - this file parses the commit log message that committers provide, and attempts to sanitize it somewhat. It hooks into CVS via verifymsg. You should not have to touch this file. This script depends on a local FreeBSD hack of CVS: this version reads the log message back in after this script has modified it. The stock version of CVS does not do this which makes logcheck unable to clean up the log message, although it is still able to check that it is syntactically OK. CVS 1.11.2 can be configured to have the same behaviour as FreeBSD's version by setting RereadLogAfterVerify=always in the config file. loginfo - this file is used by CVS to control where log information is sent; log_accum.pl hooks in here. You should not have to touch this file. modules - this file retains its traditional meaning in CVS. You should remove the FreeBSD modules from the stock version. You can edit this as you wish. More information about this file is available in the CVS manual. notify - this file is used by CVS in case someone sets a watch on a file. It is not used in the FreeBSD repository. You can edit this as you wish. More information about this file is available in the CVS manual. options - this file is specific to the FreeBSD version of CVS, and is also supported by the Debian version. It contains the keyword to expand in revision headers. You should alter this to match the keyword you specified in cfg_local.pm (if you use that feature, which is FreeBSD specific for now). rcsinfo - this file maps directories in the repository to template files such as rcstemplate. By default, FreeBSD uses one template for the whole repository. You can add others to this file if you wish. rcstemplate - this file is the actual template committers will see when they make a checkin. You should edit this to describe the various extra parameters you defined in cfg_local.pm. tagcheck - this files controls access to tagging in the repository. The stock FreeBSD version disallows tags with names of RELENG*, because of the release engineering process. You should edit this file as desired. taginfo - this file maps tag operations on repository directories to access control scripts such as tagcheck. You should not have to touch this file. unwrap - this script can be used to automatically unwrap binary files (see cvswrappers) on checkout. It is not used in the current FreeBSD setup because the functionality it hooks into does not work well with remote commits. You should not have to touch this file. verifymsg - this file maps repository directories to post processor scripts of log messages such as logcheck. You should not have to touch this file. wrap - this script can be used to automatically wrap binary files (see cvswrappers) on checkin. It is not used in the current FreeBSD setup because the functionality it hooks into does not work well with remote commits. You should not have to touch this file. Customizing the scripts The next step is to set up the scripts so that they work in your environment. You should go over all files in the directory and make your customizations. In particular, you might want to do edit the following files: If you do not wish to use the FreeBSD specific features of the scripts, you can safely remove the access file: &prompt.user; cvs rm -f access Edit avail to contain the various repository directories in which you want to control access. Make sure you retain the avail||CVSROOT line, otherwise you will lock yourself out in the next step. The other thing you can add in this file are committer groups. By default, FreeBSD uses the access file to list all its committers in, but you can use any file you wish. You can also add groups if you want (the syntax is specified at the top of cvs_acls.pl). Edit cfg_local.pm to contain the options you want. In particular, you should take a look at the following configurable items: %TEMPLATE_HEADERS - these get processed by the log scripts, and inserted below the commit mail if present and non-empty in the commit message. You can probably remove the PR and MFC after entries. And of course you can add your own. $MAIL_BRANCH_HDR - if you want to insert a header into each commit mail describing the branch on which the commit was made, define this to match your setup. Or leave it empty if you do not want such a header. @COMMIT_HOSTS - define this to be a list of hosts on which people can commit. $MAILADDRS - set this to the admin or list address that should receive commit mail. @LOG_FILE_MAP - change this array as you wish - each regexp is matched on the directory of the commit, and the commit log message gets stored in - the commitlogs subdirectory in + the commitlogs subdirectory in the filename mentioned. $COMMITCHECK_EXTRA - if you do not want to use the FreeBSD specific access checks, you should remove the definition of $COMMITCHECK_EXTRA from this file. Changing the $IDHEADER parameter is only guaranteed to work on FreeBSD platforms; it depends on FreeBSD specific modifications to CVS. You can check cfg.pm to see which other options can be changed, but the above is a reasonable subset. Edit exclude to remove the FreeBSD specific entries (such as all lines beginning with ^ports/ etc.). Furthermore, comment out the lines beginning with ^CVSROOT/, and add one line with only ^CVSROOT/ on it. After the wrapper is installed, you can add your header to the files in the - CVSROOT directory and restore these lines, + CVSROOT directory and restore these lines, but for now they will only be in the way when you try to commit later on. Edit modules, and delete all FreeBSD stuff. Add your own modules if you wish. This step is only necessary if you specified a value for $IDHEADER in cfg_local.pm (which only works using a FreeBSD modified CVS). Edit options to match the tag you specified in cfg_local.pm. A global search and replace of FreeBSD with your tag should suffice. Edit rcstemplate to contain the same keywords as specified in cfg_local.pm. Optionally remove the FreeBSD checks from tagcheck. You can simply add exit 0 to the top of the file to disable all checks on tagging. The last thing to do before you are finished, is to make sure the commitlogs can be stored. By default these are stored in the repository, in the commitlogs subdirectory - of the CVSROOT directory. This directory + of the CVSROOT directory. This directory needs to be created, so do the following: &prompt.user; mkdir commitlogs &prompt.user; cvs add commitlogs Now, after careful review, you should commit your changes. Be sure that you have granted yourself access to the - CVSROOT directory in your + CVSROOT directory in your avail before you do this, because otherwise you will lock yourself out. So make sure everything is as you intend, and then do the following: &prompt.user; cvs commit -m '- Initial FreeBSD scripts commit' Testing the setup You are ready for the first test: a forced commit to the avail file, to make sure everything works as expected. &prompt.user; cvs commit -f -m 'Forced commit to test the new CVSROOT scripts' avail If everything works, congratulations! You now have a working setup of the FreeBSD scripts for your repository. If CVS still complains about something, go back and recheck if all of the above steps have been performed correctly. FreeBSD specific setup The FreeBSD project itself uses a slightly different setup, which - also uses files from the freebsd subdirectory of - the FreeBSD CVSROOT. The project uses this because + also uses files from the freebsd subdirectory of + the FreeBSD CVSROOT. The project uses this because of the large number of committers, which all would have to be in the same group. So, a simple wrapper was written which ensures that people have the correct credentials to commit, and then sets the group id to that of the repository. If your repository also needs this, the steps to set this up are documented below. But first an overview of the files involved. Files used in the FreeBSD setup access - this file controls access information. You should edit this file to include all members of your project. freebsd/commitmail.pl - this file is not used any more, but kept for historic reasons. You should not have to touch this file. freebsd/cvswrap.c - this is the source to the CVS wrapper that you will need to install to make all access checks actually work. More information on this below. You should edit the paths in the ACCESS and REALCVS macros to match your setup. freebsd/mailsend.c - this file is needed by the FreeBSD setup of the mailing lists. You should not have to touch this file. The procedure Edit the access file to contain only your username. Edit cvswrap.c to contain the correct path for your setup. This is defined in a macro named ACCESS. You should also change the location of the real cvs binary if it is not appropriate to your situation. The stock cvswrap.c expects to be a replacement for the systemwide cvs command, which will be moved to /usr/bin/ncvs. My copy of cvswrap.c has this: #define ACCESS "/local/cvsroot/CVSROOT/access" #define REALCVS "/usr/bin/ncvs" Next up is installing the wrapper to ensure you become the correct group when committing. The sources for this live in cvswrap.c in your - CVSROOT. + CVSROOT. Compile the sources that you edited to include the correct paths: &prompt.user; cc -o cvs cvswrap.c And then install them (you have to be root for this step): &prompt.root; mv /usr/bin/cvs /usr/bin/ncvs &prompt.root; mv cvs /usr/bin/cvs &prompt.root; chown root:ncvs /usr/bin/cvs /usr/bin/ncvs &prompt.root; chmod o-rx /usr/bin/ncvs &prompt.root; chmod u-w,g+s /usr/bin/cvs This installs the wrapper as the default cvs command, making sure that anyone who wants to use the repository has to have the correct access levels. You can now remove everyone from your repository group. All access control is done by your wrapper, and this wrapper will set the correct group for access. Testing the setup Your wrapper should now be setup. You can of course test this by making a forced commit to the access file: &prompt.user; cvs commit -f -m 'Forced commit to test the new CVSROOT scripts' access Again, if this fails, check to see whether all of the above steps have been executed correctly.
diff --git a/en_US.ISO8859-1/articles/mh/article.sgml b/en_US.ISO8859-1/articles/mh/article.sgml index a8a25c4000..ebe99511a6 100644 --- a/en_US.ISO8859-1/articles/mh/article.sgml +++ b/en_US.ISO8859-1/articles/mh/article.sgml @@ -1,815 +1,815 @@ %articles.ent; ]>
An <application>MH</application> Primer Matt Midboe
matt@garply.com
v1.0, 16 January 1996 &tm-attrib.freebsd; &tm-attrib.opengroup; &tm-attrib.general; This document contains an introduction to using MH on FreeBSD Introduction MH started back in 1977 at the RAND Corporation, where the initial philosophies behind MH were developed. MH is not so much a monolithic email program but a philosophy about how best to develop tools for reading email. The MH developers have done a great job adhering to the KISS principle: Keep It Simple Stupid. Rather than have one large program for reading, sending and handling email they have written specialized programs for each part of your email life. One might liken MH to the specialization that one finds in insects and nature. Each tool in MH does one thing, and does it very well. Beyond just the various tools that one uses to handle their email MH has done an excellent job keeping the configuration of each of these tools consistent and uniform. In fact, if you are not quite sure how something is supposed to work or what the arguments for some command are supposed to be, then you can generally guess and be right. Each MH command is consistent about how it handles reading the configuration files and how it takes arguments on the command line. One useful thing to remember is that you can always add a to the command to have it display the options for that command. The first thing that you need to do is to make sure that you have installed the MH package on your FreeBSD machine. If you installed from CDROM you should be able to execute the following to load MH: &prompt.root; pkg_add /cdrom/packages/mh-6.8.3.tgz You will notice that it created a /usr/local/lib/mh directory for you as well as adding several binaries to the /usr/local/bin directory. If you would prefer to compile it yourself then you can anonymous ftp it from ftp.ics.uci.edu or louie.udel.edu. This primer is not a full comprehensive explanation of how MH works. This is just intended to get you started on the road to happier, faster mail reading. You should read the manual pages for the various commands. You might also want to read the comp.mail.mh newsgroup. Also you can read the FAQ for MH. The best resource for MH is Jerry Peek's MH & nmh: Email for Users & Programmers. Reading Mail This section covers how to use inc, show, scan, next, prev, rmm, rmf, and msgchk. One of the best things about MH is the consistent interface between programs. One thing to keep in mind when using these commands is how to specify message lists. In the case of inc this does not really make any sense but with commands like show it is useful to know. A message list can consist of something like 23 20 16 which will act on messages 23, 20 and 16. This is fairly simple but you can do more useful things like 23-30 which will act on all the messages between 23 and 30. You can also specify something like cur:10 which will act on the current message and the next 9 messages. The cur, last, and first messages are special messages that refer to the current, last or first message in the folder. <command>inc</command>, <command>msgchk</command>—read in your new email or check it If you just type in inc and hit return you will be well on your way to getting started with MH. The first time you run inc it will set up your account to use all the MH defaults and ask you about creating a Mail directory under your HOME directory. If you have mail waiting to be downloaded you will see something that looks like: 29 01/15 Doug White Re: Another Failed to boot problem<<On Mon, 15 J 30 01/16 "Jordan K. Hubbar Re: FBSD 2.1<<> Do you want a library instead of 31 01/16 Bruce Evans Re: location of bad144 table<<>> >It would appea 32 01/16 "Jordan K. Hubbar Re: video is up<<> Anyway, mrouted won't run, ev 33 01/16 Michael Smith Re: FBSD 2.1<<Nate Williams stands accused of sa This is the same thing you will see from a scan (see ). If you just run inc with no arguments it will look on your computer for email that is supposed to be coming to you. A lot of people like to use POP for grabbing their email. MH can do POP to grab your email. You will need to give inc a few command line arguments. &prompt.user; inc -host mail.pop.org -user username -norpop That tells inc to go to mail.pop.org to download your email, and that your username on their system is username. The option tells inc to use plain POP3 for downloading your email. MH has support for a few different dialects of POP. More than likely you will never ever need to use them though. While you can do more complex things with inc such as audit files and scan format files this will get you going. The msgchk command is used to get information on whether or not you have new email. msgchk takes the same and options that inc takes. <command>show</command>, <command>next</command> and <command>prev</command>—displaying and moving through email show is to show a letter in your current folder. Like inc, show is a fairly straightforward command. If you just type show and hit return then it displays the current message. You can also give specific message numbers to show: &prompt.user; show 32 45 56 This would display message numbers 32, 45 and 56 right after each other. Unless you change the default behavior show basically just does a more on the email message. next is used to move onto the next message and prev will go to the previous message. Both commands have an implied show command so that when you go to the next message it automatically displays it. <command>scan</command>—shows you a scan of your messages scan will display a brief listing of the messages in your current folder. This is an example of what the scan command will give you. 30+ 01/16 Jordan K. Hubbar Re: FBSD 2.1<<> Do you want a library instead of 31 01/16 Bruce Evans Re: location of bad144 table<<>> >It would appea 32 01/16 Jordan K. Hubbar Re: video is up<<> Anyway, mrouted won't run, ev 33 01/16 Michael Smith Re: FBSD 2.1<<Nate Williams stands accused of sa Like just about everything in MH this display is very configurable. This is the typical default display. It gives you the message number, the date on the email, the sender, the subject line, and a sentence fragment from the very beginning of the email if it can fit it. The + means that message is the current message, so if you do a show it will display that message. One useful option for scan is the option. This will list your messages with the highest message number first and lowest message number last. Another useful option with scan is to have it read from a file. If you want to scan your incoming mailbox on FreeBSD without having to inc it you can do scan -file /var/mail/username. This can be used with any file that is in the mbox format. <command>rmm</command> and <command>rmf</command>—remove the current message or folder rmm is used to remove a mail message. The default is typically to not actually remove the message but to rename the file to one that is ignored by the MH commands. You will periodically need to go through and physically delete the removed messages. The rmf command is used to remove folders. This does not just rename the files but actually removes the from the hard drive so you should be careful when you use this command. A typical session of reading with MH The first thing that you will want to do is inc your new mail. So at a shell prompt just type in inc and hit return. &prompt.user; inc Incorporating new mail into inbox... 36+ 01/19 Stephen L. Lange Request...<<Please remove me as contact for pind 37 01/19 Matt Thomas Re: kern/950: Two PCI bridge chips fail (multipl 38 01/19 Amancio Hasty Jr Re: FreeBSD and VAT<<>>> Bill Fenner said: > In &prompt.user; This shows you the new email that has been added to your mailbox. So the next thing to do is show the email and move around. &prompt.user; show Received: by sashimi.wwa.com (Smail3.1.29.1 #2) id m0tdMZ2-001W2UC; Fri, 19 Jan 96 13:33 CST Date: Fri, 19 Jan 1996 13:33:31 -0600 (CST) From: "Stephen L. Lange" <stvlange@wwa.com> To: matt@garply.com Subject: Request... Message-Id: <Pine.BSD.3.91.960119133211.824A-100000@sashimi.wwa.com> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Please remove me as contact for pindat.com &prompt.user; rmm &prompt.user; next Received: from localhost (localhost [127.0.0.1]) by whydos.lkg.dec.com (8.6.11/8 .6.9) with SMTP id RAA24416; Fri, 19 Jan 1996 17:56:48 GMT Message-Id: <199601191756.RAA24416@whydos.lkg.dec.com> X-Authentication-Warning: whydos.lkg.dec.com: Host localhost didn't use HELO pro tocol To: hsu@clinet.fi Cc: hackers@FreeBSD.org Subject: Re: kern/950: Two PCI bridge chips fail (multiple multiport ethernet boards) In-Reply-To: Your message of "Fri, 19 Jan 1996 00:18:36 +0100." <199601182318.AA11772@Sysiphos> X-Mailer: exmh version 1.5omega 10/6/94 Date: Fri, 19 Jan 1996 17:56:40 +0000 From: Matt Thomas <matt@lkg.dec.com> Sender: owner-hackers@FreeBSD.org Precedence: bulk This is due to a typo in pcireg.h (to which I am probably the guilty party). The rmm removed the current message and the next command moved me on to the next message. Now if I wanted to look at ten most recent messages so I could read one of them here is what I would do: &prompt.user; scan last:10 26 01/16 maddy Re: Testing some stuff<<yeah, well, Trinity has 27 01/17 Automatic digest NET-HAPPENINGS Digest - 16 Jan 1996 to 17 Jan 19 28 01/17 Evans A Criswell Re: Hey dude<<>From matt@tempest.garply.com Tue 29 01/16 Karl Heuer need configure/make volunteers<<The FSF is looki 30 01/18 Paul Stephanouk Re: [alt.religion.scientology] Raw Meat (humor)< 31 01/18 Bill Lenherr Re: Linux NIS Solaris<<--- On Thu, 18 Jan 1996 1 34 01/19 John Fieber Re: Stuff for the email section?<<On Fri, 19 Jan 35 01/19 support@foo.garpl [garply.com #1138] parlor<<Hello. This is the Ne 37+ 01/19 Matt Thomas Re: kern/950: Two PCI bridge chips fail (multipl 38 01/19 Amancio Hasty Jr Re: FreeBSD and VAT<<>>> Bill Fenner said: > In &prompt.user; Then if I wanted to read message number 27 I would do a show 27 and it would be displayed. As you can probably tell from this sample session MH is pretty easy to use and looking through emails and displaying them is fairly intuitive and easy. Folders and Mail Searching Anybody who gets lots of email definitely wants to be able to prioritize, stamp, brief, de-brief, and number their emails in a variety of different ways. MH can do this better than just about anything. One thing that we have not really talked about is the concept of folders. You have undoubtedly come across the folders concept using other email programs. MH has folders too. MH can even do sub-folders of a folder. One thing you should keep in mind with MH is that when you ran inc for the first time and it asked you if it could create a Mail directory it began storing everything in that directory. If you look at that directory you will find a directory named inbox. The inbox directory houses all of your incoming mail that has not been thrown anywhere else. Whenever you create a new folder a new directory is going to be created underneath your MH Mail directory, and messages in that folder are going to be stored in that directory. When a new email message comes, it is thrown into your inbox directory with a file name that is equivalent to the message number. So even if you did not have any of the MH tools to read your email you could still use standard &unix; commands to munge around in those directories and just more your files. It is this simplicity that really gives you a lot of power with what you can do with your email. Just as you can use message lists like 23 16 42 with most MH commands there is a folder option you can specify with just about every MH command. If you do a scan +freebsd it will scan your freebsd folder, and your current folder will be changed to freebsd. If you do a show +freebsd 23 16 42, show is going to switch to your freebsd folder and display messages 23, 16 and 42. So remember that syntax. You will need to make sure you use it to make commands process different folders. Remember you default folder for mail is inbox so doing a folder +inbox should always get you back to your mail. Of course, in MH's infinite flexibility this can be changed but most places have probably left it as inbox. <command>pick</command>—search email that matches certain criteria pick is one of the more complex commands in the MH system. So you might want to read the pick1 man page for a more thorough understanding. At its simplest level you can do something like &prompt.user; pick -search pci 15 42 55 56 57 This will tell pick to look through every single line in every message in your current folder and tell you which message numbers it found the word pci in. You can then show those messages and read them if you wish or rmm them. You would have to specify something like show 15 42 55-57 to display them though. A slightly more useful thing to do is this: &prompt.user; pick -search pci -seq pick 5 hits &prompt.user; show pick This will show you the same messages you just did not have to work as hard to do it. The option is really an abbreviation of and pick is just a sequence which contains the message numbers that matched. You can use sequences with just about any MH command. So you could have done an rmm pick and all those messages would be removed instead. You sequence can be named anything. If you run pick again it will overwrite the old sequence if you use the same name. Doing a pick -search can be a bit more time consuming than just searching for message from someone, or to someone. So pick allows you to use the following predefined search criteria: search based upon who the message is to search based on who is in the Cc: list search for who sent the message search for emails with this subject find emails with a matching date search for any other component in the header. (i.e. to find all emails with a certain reply-to in the header) This allows you to do things like &prompt.user; pick -to freebsd-hackers@FreeBSD.org -seq hackers to get a list of all the email send to the FreeBSD hackers mailing list. pick also allows you to group these criteria in different ways using the following options: … … … … … … These commands allow you to do things like &prompt.user; pick -to freebsd-hackers -or -cc freebsd-hackers That will grab all the email in your inbox that was sent to + class="directory">inbox that was sent to freebsd-hackers or cc'd to that list. The brace options allow you to group search criteria together. This is sometimes very necessary as in the following example &prompt.user; pick -lbrace -to freebsd-hackers -and -not -cc freebsd-questions -rbrace -and -subject pci Basically this says pick (to freebsd-hackers and not cc'd on freebsd-questions) and the subject is pci. It should look through your folder and find all messages sent to the freebsd-hackers list that are not cc'd to the freebsd-questions list and contain pci in the subject line. Ordinarily you might have to worry about something called operator precedence. Remember in math how you evaluate from left to right and you do multiplication and division first and addition and subtraction second? MH has the same type of rules for pick. It is fairly complex so you might want to study the manual page. This document is just to help you get acquainted with MH. <command>folder</command>, <command>folders</command>, <command>refile</command>—three useful programs for folder maintenance There are three programs which are primarily just for manipulating your folders. The folder program is used to switch between folders, pack them, and list them. At its simplest level you can do a folder +newfolder and you will be switched into newfolder. From there on out all your MH commands like comp, repl, scan, and show will act on that newfolder folder. Sometimes when you are reading and deleting messages you will develop holes in your folders. If you do a scan you might just see messages 34, 35, 36, 43, 55, 56, 57, 80. If you do a folder -pack this will renumber all your messages so that there are no holes. It does not actually delete any messages though. So you may need to periodically go through and physically delete rmm'd messages. If you need statistics on your folders you can do a folders or folder -all to list all your folders, how many messages they have, what the current message is in each one and so on. This line of stats it displays for all your folders is the same one you get when you change to a folder with folder +foldername. A folders command looks like this: Folder # of messages ( range ); cur msg (other files) announce has 1 message ( 1- 1). drafts has no messages. f-hackers has 43 messages ( 1- 43). f-questions has 16 messages ( 1- 16). inbox+ has 35 messages ( 1- 38); cur= 37. lists has 8 messages ( 1- 8). netfuture has 1 message ( 1- 1). out has 31 messages ( 1- 31). personal has 6 messages ( 1- 6). todo has 58 messages ( 1- 58); cur= 1. TOTAL= 199 messages in 13 folders. The refile command is what you use to move messages between folders. When you do something like refile 23 +netfuture message number 23 is moved into the netfuture folder. You could also do something like refile 23 +netfuture/latest which would put message number 23 in a subfolder called latest under the netfuture folder. If you want to keep a message in the current folder and link it you can do a refile -link 23 +netfuture which would keep 23 in your current inbox but also list in your netfuture folder. You are probably beginning to realize some of the really powerful things you can do with MH. Sending Mail Email is a two way street for most people so you want to be able to send something back. The way MH handles sending mail can be a bit difficult to follow at first, but it allows for incredible flexibility. The first thing MH does is to copy a components file into your outgoing email. A components file is basically a skeleton email letter with stuff like the To: and Subject: headers already in it. You are then sent into your editor where you fill in the header information and then type the body of your message below the dashed lines in the message. When you leave the editor, the whatnow program is run. When you are at the What now? prompt you can tell it to send, list, edit, push, and quit. Most of these commands are self-explanatory. So the message sending process involves copying a component file, editing your email, and then telling the whatnow program what to do with your email. <command>comp</command>, <command>forw</command>, <command>reply</command>—compose, forward or reply to a message to someone The comp program has a few useful command line options. The most important one to know right now is the option. When MH is installed the default editor is usually a program called prompter which comes with MH. It is not a very exciting editor and basically just gets the job done. So when you go to compose a message to someone you might want to use comp -editor /usr/bin/vi or comp -editor /usr/local/bin/pico instead. Once you have run comp you are in your editor and you see something that looks like this: To: cc: Subject: -------- You need to put the person you are sending the mail to after the To: line. It works the same way for the other headers also, so you would need to put your subject after the Subject: line. Then you would just put the body of your message after the dashed lines. It may seem a bit simplistic since a lot of email programs have special requesters that ask you for this information but there really is no point to that. Plus this really gives you excellent flexibility. To:freebsd-rave@FreeBSD.org cc: Subject:And on the 8th day God created the FreeBSD core team -------- Wow this is an amazing operating system. Thanks! You can now save this message and exit your editor. You will see the What now? prompt and you can type in send or s and hit return. Then the FreeBSD core team will receive their just rewards. As I mentioned earlier, you can also use other commands at the What now? prompt. For example you can use quit, if you do not want to send the message. The forw command is stunningly similar. The big difference being that the message you are forwarding is automatically included in the outgoing message. When you run forw it will forward your current message. You can always tell it to forward something else by doing something like forw 23 and then message number 23 will be put in your outgoing message instead of the current message. Beyond those small differences forw functions exactly the same as comp. You go through the exact same message sending process. The repl command will reply to the current message, unless you give it a different message to reply to. repl will do its best to go ahead and fill in some of the email headers already. So you will notice that the To: header already has the address of the recipient in there. Also the Subject: line will already be filled in. You then go about the normal message composition process and you are done. One useful command line option to know here is the option. You can use all, to, cc, me after the option to have repl automatically add the various addresses to the Cc: list in the message. You have probably noticed that the original message is not included. This is because most MH setups are configured to do this from the start. <filename>components</filename>, and <filename>replcomps</filename>—components files for <command>comp</command> and <command>repl</command> The components file is usually in /usr/local/lib/mh. You can copy that file into your MH Mail directory and edit to contain what you want it to contain. It is a fairly basic file. You have various email headers at the top, a dashed line and then nothing. The comp command just copies this components file and then edits it. You can add any kind of valid RFC822 header you want. For instance you could have something like this in your components file: To: Fcc: out Subject: X-Mailer: MH 6.8.3 X-Home-Page: http://www.FreeBSD.org/ ------- MH would then copy this components file and throw you into your editor. The components file is fairly simple. If you wanted to have a signature on those messages you would just put your signature in that components file. The replcomps file is a bit more complex. The default replcomps looks like this: %(lit)%(formataddr %<{reply-to}%?{from}%?{sender}%?{return-path}%>)\ %<(nonnull)%(void(width))%(putaddr To: )\n%>\ %(lit)%(formataddr{to})%(formataddr{cc})%(formataddr(me))\ %<(nonnull)%(void(width))%(putaddr cc: )\n%>\ %<{fcc}Fcc: %{fcc}\n%>\ %<{subject}Subject: Re: %{subject}\n%>\ %<{date}In-reply-to: Your message of "\ %<(nodate{date})%{date}%|%(pretty{date})%>."%<{message-id} %{message-id}%>\n%>\ -------- It is in the same basic format as the components file but it contains quite a few extra formatting codes. The %(lit) command makes room for the address. The %(formataddr) is a function that returns a proper email address. The next part is %< which means if and the {reply-to} means the reply-to field in the original message. So that might be translated this way: %<if {reply-to} the original message has a reply-to then give that to formataddr, %? else {from} take the from address, %? else {sender} take the sender address, %? else {return-path} take the return-path from the original message, %> endif. As you can tell MH formatting can get rather involved. You can probably decipher what most of the other functions and variables mean. All of the information on writing these format strings is in the MH-Format manual page. The really nice thing is that once you have built your customized replcomps file you will not need to touch it again. No other email program really gives you the power and flexibility that MH gives you.
diff --git a/en_US.ISO8859-1/articles/nanobsd/article.sgml b/en_US.ISO8859-1/articles/nanobsd/article.sgml index 4bf9c3bcc5..c2340547e0 100644 --- a/en_US.ISO8859-1/articles/nanobsd/article.sgml +++ b/en_US.ISO8859-1/articles/nanobsd/article.sgml @@ -1,506 +1,506 @@ %articles.ent; ]>
Introduction to NanoBSD Daniel Gerzo $FreeBSD$ 2006 The FreeBSD Documentation Project &tm-attrib.freebsd; &tm-attrib.general; This document provides information about the NanoBSD tools, which can be used to create &os; system images for embedded applications, suitable for use on a Compact Flash card (or other mass storage medium). Introduction to NanoBSD NanoBSD NanoBSD is a tool currently developed by &a.phk;. It creates a &os; system image for embedded applications, suitable for use on a Compact Flash card (or other mass storage medium). It can be used to build specialized install images, designed for easy installation and maintenance of systems commonly called computer appliances. Computer appliances have their hardware and software bundled in the product, which means all applications are pre-installed. The appliance is plugged into an existing network and can begin working (almost) immediately. The features of NanoBSD include: Ports and packages work as in &os; — Every single application can be installed and used in a NanoBSD image, the same way as in &os;. No missing functionality — If it is possible to do something with &os;, it is possible to do the same thing with NanoBSD, unless the specific feature or features were explicitly removed from the NanoBSD image when it was created. Everything is read-only at run-time — It is safe to pull the power-plug. There is no necessity to run &man.fsck.8; after a non-graceful shutdown of the system. Easy to build and customize — Making use of just one shell script and one configuration file it is possible to build reduced and customized images satisfying any arbitrary set of requirements. NanoBSD Howto The design of NanoBSD Once the image is present on the medium, it is possible to boot NanoBSD. The mass storage medium is divided into three parts by default: Two image partitions: code#1 and code#2. The configuration file partition, which can be mounted - under the /cfg directory + under the /cfg directory at run time. These partitions are normally mounted read-only. - The /etc and - /var directories are + The /etc and + /var directories are &man.md.4; (malloc) disks. The configuration file partition persists under the - /cfg directory. It - contains files for /etc + /cfg directory. It + contains files for /etc directory and is briefly mounted read-only right after the system boot, therefore it is required to copy modified files - from /etc back to the - /cfg directory if changes + from /etc back to the + /cfg directory if changes are expected to persist after the system restarts. Making persistent changes to <filename>/etc/resolv.conf</filename> &prompt.root; vi /etc/resolv.conf [...] &prompt.root; mount /cfg &prompt.root; cp /etc/resolv.conf /cfg &prompt.root; umount /cfg The partition containing - /cfg should be mounted + /cfg should be mounted only at boot time and while overriding the configuration files. - Keeping /cfg mounted at + Keeping /cfg mounted at all times is not a good idea, especially if the NanoBSD system runs off a mass storage medium that may be adversely affected by a large number of writes to the partition (i.e. when the filesystem syncer flushes data to the system disks). Building a NanoBSD image A NanoBSD image is built using a simple nanobsd.sh shell script, which can be found in the - /usr/src/tools/tools/nanobsd + /usr/src/tools/tools/nanobsd directory. This script creates an image, which can be copied on the storage medium using the &man.dd.1; utility. The necessary commands to build a NanoBSD image are: &prompt.root; cd /usr/src/tools/tools/nanobsd &prompt.root; sh nanobsd.sh &prompt.root; cd /usr/obj/nanobsd.full &prompt.root; dd if=_.disk.full of=/dev/da0 bs=64k Change the current directory to the base directory of the NanoBSD build script. Start the build process. Change the current directory to the place where the built images are located. Install NanoBSD onto the storage medium. Customizing a NanoBSD image This is probably the most important and most interesting feature of NanoBSD. This is also where you will be spending most of the time when developing with NanoBSD. Invocation of the following command will force the nanobsd.sh to read its configuration from the myconf.nano file located in the current directory: &prompt.root; sh nanobsd.sh -c myconf.nano Customization is done in two ways: Configuration options Custom functions Configuration options With configuration settings, it is possible to configure options passed to both the buildworld and installworld stages of the NanoBSD build process, as well as internal options passed to the main build process of NanoBSD. Through these options it is possible to cut the system down, so it will fit on as little as 64MB. You can use the configuration options to trim down &os; even more, until it will consists of just the kernel and two or three files in the userland. The configuration file consists of configuration options, which override the default values. The most important directives are: NANO_NAME — Name of build (used to construct the workdir names). NANO_SRC — Path to the source tree used to build the image. NANO_KERNEL — Name of kernel configuration file used to build kernel. CONF_BUILD — Options passed to the buildworld stage of the build. CONF_INSTALL — Options passed to the installworld stage of the build. CONF_WORLD — Options passed to both the buildworld and the installworld stage of the build. FlashDevice — Defines what type of media to use. Check the FlashDevice.sub file for more details. Custom functions It is possible to fine-tune NanoBSD using shell functions in the configuration file. The following example illustrates the basic model of custom functions: cust_foo () ( echo "bar=topless" > \ ${NANO_WORLDDIR}/etc/foo ) customize_cmd cust_foo A more useful example of a customization function is the following, which changes the default size of the - /etc directory + /etc directory from 5MB to 30MB: cust_etc_size () ( cd ${NANO_WORLDDIR}/conf echo 30000 > default/etc/md_size ) customize_cmd cust_etc_size There are a few default pre-defined customization functions ready for use: cust_comconsole — Disables &man.getty.8; on the VGA devices (the /dev/ttyv* device nodes) and enables the use of the COM1 serial port as the system console. cust_allow_ssh_root — Allow root to login via &man.sshd.8;. cust_install_files — Installs files from the - nanobsd/Files + nanobsd/Files directory, which contains some useful scripts for system administration. Adding packages Packages can be added to a NanoBSD image using a custom function. The following fuction will install all the packages located in /usr/src/tools/tools/nanobsd/packages: install_packages () ( mkdir -p ${NANO_WORLDDIR}/packages cp /usr/src/tools/tools/nanobsd/packages/* ${NANO_WORLDDIR}/packages chroot ${NANO_WORLDDIR} sh -c 'cd packages; pkg_add -v *;cd ..;' rm -rf ${NANO_WORLDDIR}/packages ) customize_cmd install_packages Configuration file example A complete example of a configuration file for building a custom NanoBSD image can be: NANO_NAME=custom NANO_SRC=/usr/src NANO_KERNEL=MYKERNEL NANO_IMAGES=2 CONF_BUILD=' NO_KLDLOAD=YES NO_NETGRAPH=YES NO_PAM=YES ' CONF_INSTALL=' NO_ACPI=YES NO_BLUETOOTH=YES NO_CVS=YES NO_FORTRAN=YES NO_HTML=YES NO_LPR=YES NO_MAN=YES NO_SENDMAIL=YES NO_SHAREDOCS=YES NO_EXAMPLES=YES NO_INSTALLLIB=YES NO_CALENDAR=YES NO_MISC=YES NO_SHARE=YES ' CONF_WORLD=' NO_BIND=YES NO_MODULES=YES NO_KERBEROS=YES NO_GAMES=YES NO_RESCUE=YES NO_LOCALES=YES NO_SYSCONS=YES NO_INFO=YES ' FlashDevice SanDisk 1G cust_nobeastie() ( touch ${NANO_WORLDDIR}/boot/loader.conf echo "beastie_disable=\"YES\"" >> ${NANO_WORLDDIR}/boot/loader.conf ) customize_cmd cust_comconsole customize_cmd cust_install_files customize_cmd cust_allow_ssh_root customize_cmd cust_nobeastie Updating NanoBSD The update process of NanoBSD is relatively simple: Build a new NanoBSD image, as usual. Upload the new image into an unused partition of a running NanoBSD appliance. The most important difference of this step from the initial NanoBSD installation is that now instead of using the _.disk.full file (which contains an image of the entire disk), the _.disk.image image is installed (which contains an image of a single system partition). Reboot, and start the system from the newly installed partition. If all goes well, the upgrade is finished. If anything goes wrong, reboot back into the previous partition (which contains the old, working image), to restore system functionality as fast as possible. Fix any problems of the new build, and repeat the process. To install new image onto the running NanoBSD system, it is possible to use either the updatep1 or updatep2 script located in the - /root directory, depending + /root directory, depending from which partition is running the current system. According to which services are available on host serving new NanoBSD image and what type of transfer is preferred, it is possible to examine one of these three ways: Using &man.ftp.1; If the transfer speed is in first place, use this example: &prompt.root; ftp myhost get _.disk.image "| sh updatep1" Using &man.ssh.1; If a secure transfer is preferred, consider using this example: &prompt.root; ssh myhost cat _.disk.image.gz | zcat | sh updatep1 Using &man.nc.1; Try this example if the remote host is not running neither &man.ftp.1; or &man.sshd.8; service: At first, open a TCP listener on host serving the image and make it send the image to client: myhost&prompt.root; nc -l 2222 < _.disk.image Make sure that the used port is not blocked to receive incoming connections from NanoBSD host by firewall. Connect to the host serving new image and execute updatep1 script: &prompt.root; nc myhost 2222 | sh updatep1
diff --git a/en_US.ISO8859-1/articles/p4-primer/article.sgml b/en_US.ISO8859-1/articles/p4-primer/article.sgml index d0d2455896..164072eb55 100644 --- a/en_US.ISO8859-1/articles/p4-primer/article.sgml +++ b/en_US.ISO8859-1/articles/p4-primer/article.sgml @@ -1,890 +1,890 @@ %articles.ent; ]>
Perforce in &os; Development Scott Long
scottl@FreeBSD.org
$FreeBSD$ &tm-attrib.freebsd; &tm-attrib.cvsup; &tm-attrib.general; Introduction The &os; project uses the Perforce version control system to manage experimental projects that are not ready for the main CVS repository. Availability, Documentation, and Resources While Perforce is a commercial product, the client software for interacting with the server is freely available from Perforce. It can be easily installed on &os; via the devel/perforce port or can be downloaded from the Perforce web site at , which also offers client applications for other OS's. While there is a GUI client available, most people use the command line application called p4. This document is written from the point of view of using this command. Detailed documentation is available online at . Reading the Perforce User's Guide and Perforce Command Reference is highly recommended. The p4 application also contains an extensive amount of online help accessible via the p4 help command. The &os; Perforce server is hosted on perforce.freebsd.org, port 1666. The repository is browsable online at . Some portions of the repository are also automatically exported to a number of CVSup servers. Getting Started The first step to using Perforce is to obtain an account on the server. If you already have a FreeBSD.org account, log into freefall, run the following command, and enter a password that is not the same as your &os; login or any other SSH passphrase: &prompt.user; /usr/local/bin/p4newuser Of course if you do not have a FreeBSD.org account, you will need to coordinate with your sponsor. An email will be sent to your &os; address that contains the password you specified above in cleartext. Be sure to change the password once your Perforce account has been created! The next step is to set the environment variables that p4 needs, and verify that it can connect to the server. The P4PORT variable is required to be set for all operations, and specifies the appropriate Perforce server to talk to. For the &os; project, set it like so: &prompt.user; export P4PORT=perforce.freebsd.org:1666 Users with shell access on the FreeBSD.org cluster may wish to tunnel the Perforce client-server protocol via an SSH tunnel, in which case the above string should be set to localhost. The &os; server also requires that the P4USER and P4PASSWD variables be set. Use the username and password from above, like so: &prompt.user; export P4USER=username &prompt.user; export P4PASSWD=password Test that this works by running the following command: &prompt.user; p4 info This should return a list of information about the server. If it does not, check that you have the P4PORT variable set correctly. Clients Perforce provides access to the repository and tracks state on a per-client basis. In Perforce terms, a client is a specification that maps files and directories from the repository to the local machine. Each user can have multiple clients, and each client can access different or overlapping parts of the repository. The client also specifies the root directory of the file tree that it maps, and it specifies the machine that the tree lives on. Thus, working on multiple machines requires that multiple clients be used. Clients may be accessed via the p4 client command. Running this command with no arguments will bring up a client template in an editor, allowing you to create a new client for your work. The important fields in this template are explained below: Client: This is the name of the client spec. It can be anything you want, but it must be unique within the Perforce server. A naming convention that is commonly used is username_machinename, which makes it easy to identify clients when browsing them. A default name will be filled in that is just the machine name. Description: This can contain a simple text description to help identify the client. Root: This is the local directory that will serve as the root directory of all the files in the client mapping. This should be a unique location in your filesystem that does not overlap with other files or Perforce clients. Options: Most of the default options are fine, though it is usually a good idea to make sure that the and options are present and do not have a no prefix on them. Details about each option are in the Perforce docs. LineEnd: This handles CR-LF conversions and should be left to the default unless you have special needs for it. View: This is where the server-to-local file mappings go. The default is //depot/... //client/... This will map the entire Perforce repository to the - Root directory of your + Root directory of your client. DO NOT USE THIS DEFAULT! The &os; repo is huge, and trying to map and sync it all will take an enormous amount of resources. Instead, only map the section of the repo that you intend to work on. For example, there is the smpng project tree at //depot/projects/smpng. A + class="directory">//depot/projects/smpng. A mapping for this might look like: //depot/projects/smpng/... //client/... The ... should be taken literally. It is a Perforce idiom for saying this directory and all files and directories below it. A Perforce view can contain multiple mappings. Let's say you want to map in both the SMPng tree and the NFS tree. Your View might look like: //depot/projects/smpng/... //client/smpng/... //depot/projects/nfs/... //client/nfs/... Remember that the client is the name of the client that was specified in the Client section, but in the View it also resolves to the directory that was specified in the Root section. Also note that the same file or directory cannot be mapped multiple times in a single view. The following is illegal and will produce undefined results: //depot/projects/smpng/... //client/smpng-foo/... //depot/projects/smpng/... //client/smpng-bar/... Views are a tricky part of the learning experience with Perforce, so do not be afraid to ask questions. Existing clients can be listed via the p4 clients command. They can be viewed without being modified via the p4 client -o clientname command. Whenever you are interacting with files in Perforce, the P4CLIENT environment variable must be set to the name of the client that you are using, like so: &prompt.user; export P4CLIENT=myclientname Note that client mappings in the repository are not exclusive; multiple clients can map in the same part of the repository. This allows multiple people to access and modify the same parts of the repository, allowing a team of people to work together on the same code. Syncing Once you have a client specification defined and the P4CLIENT variable set, the next step is to pull the files for that client down to your local machine. This is done with the p4 sync command, which instructs Perforce to synchronize the local files in your client with the repository. The first time it runs, it will download all of the files. Subsequent runs will only download files that have changed since the previous run. This allows you to stay in sync with others whom you might be working with. Sync operations only work on files that the Perforce server knows has changed. If you change or delete a file locally without informing the server, doing a sync will not bring it back. However, doing a p4 sync -f will unconditionally sync all files, regardless of their state. This is useful for resolving problems where you think that your tree might be corrupt. You can sync a subset of your tree or client by specifying a relative path to the sync command. For example, to only sync the - ufs directory of the + ufs directory of the smpng project, you might do the following: &prompt.user; cd projectroot/smpng &prompt.user; p4 sync src/sys/ufs/... Specifying a local relative path works for many other p4 commands. Branches One of the strongest features of Perforce is branching. Branches are very cheap to create, and moving changes between related branches is very easy (as will be explained later). Branches also allow you to do very experimental work in a sandbox-like environment, without having to worry about colliding with others or destabilizing the main tree. They also provide insulation against mistakes while learning the Perforce system. With all of these benefits, it makes sense for each project to have its own branch, and we strongly encourage that with &os;. Frequent submits of changes to the server are also encouraged. The Perforce repository (the depot) is a single flat tree. Every file, whether a unique creation or a derivative from a branch, is accessible via a simple path under the server //depot directory. When you create a + class="directory">//depot directory. When you create a branch, all you are doing is creating a new path under the - //depot. This is in sharp + //depot. This is in sharp contrast to systems like CVS, where each branch lives in the same path as its parent. With Perforce, the server tracks the relationship between the files in the parent and child, but the files themselves live under their own paths. The first step to creating a branch is to create a branch specification. This is similar to a client specification, but is created via the command p4 branch branchname. The following important fields are explained: Branch The name of the branch. It can be any name, but must be unique within the repository. The common convention in &os; is to use username_projectname. Description This can hold a simple text description to describe the branch. View This is the branch mapping. Instead of mapping from the depot to the local machine like a client map, it maps between the branch parent and branch child in the depot. For example, you might want to create a branch of the smpng project. The mapping might look like: //depot/projects/smpng/... //depot/projects/my-super-smpng/... Or, you might want to create a brand new branch off of the stock &os; sources: //depot/vendor/freebsd/... //depot/projects/my-new-project/... This will map the &os; HEAD tree to your new branch. Creating the branch spec only saves the spec itself in the server, it does not modify the depot or change any files. The directory that you specified in the branch is empty on the server until you populate it. To populate your branch, first edit your client with the p4 client command and make sure that the branch directory is mapped in your client. You might need to add a View line like: //depot/projects/my-new-project/... //myclient/my-new-project/... The next step is to run the p4 integrate command, as described in the next section. Integrations Integration is the term used by Perforce to describe the action of moving changes from one part of the depot to another. It is most commonly done in conjunction with creating and maintaining branches. An integration is done when you want to initially populate a branch, and it is done when you want to move subsequent changes in the branch from the parent to the child, or from the child to the parent. A common example of this is periodically integrating changes from the vendor &os; tree to your child branch tree, allowing you to keep up to date with changes in the &os; tree. The Perforce server tracks the changes in each tree and knows when there are changes that can be integrated from one tree to another. The common way to do an integration is with the following command: &prompt.user; p4 integrate -b branchname branchname is the name given to a branch spec, as discussed in the previous section. This command will instruct Perforce to look for changes in the branch parent that are not yet in the child. From those changes it will prepare a list of diffs to move. If the integration is being done for the first time on a branch (i.e. doing an initial population operation), then the parent files will simply be copied to the child location on the local machine. Once the integration operation is done, you must run p4 resolve to accept the changes and resolve possible conflicts. Conflicts can arise from overlapping changes that happened in both the parent and child copy of a file. Usually, however, there are no conflicts, and Perforce can quickly figure out how to merge the changes together. Use the following commands to do a resolve operation: &prompt.user; p4 resolve -as &prompt.user; p4 resolve The first invocation will instruct Perforce to automatically merge the changes together and accept files that have no conflicts. The second invocation will allow you to inspect each file that has a possible conflict and resolve it by hand if needed. Once all of the integrated files have been resolved, they need to be committed back to the repository. This is done via the p4 submit command, explained in the next section. Submit Changes that are made locally should be committed back to the Perforce server for safe keeping and so that others can access them. This is done via the p4 submit command. When you run this command, it will open up a submit template in an editor. &os; has a custom template, and the important fields are described below: Description: <enter description here> PR: Submitted by: Reviewed by: Approved by: Obtained from: MFP4 after: It is good practice to provide at least 2-3 sentences that describe what the changes are that you are submitting. You should say what the change does, why it was done that way or what problem is solves, and what APIs it might change or other side effects it might have. This text should replace the <enter description here> line in the template. You should wrap your lines and start each line with a TAB. The tags below it are &os;-specific and can be removed if not needed. Files: This is automatically populated with all of the files in your client that were marked in the add, delete, integrate, or edit states on the server. It is always a very good idea to review this list and remove files that might not be ready yet. Once you save the editor session, the submit will happen to the server. This also means that the local copies of the submitted files will be copied back to the server. If anything goes wrong during this process, the submit will be aborted, and you will be notified that the submit has been turned into a changelist that must be corrected and re-submitted. Submits are atomic, so if one file fails, the entire submit is aborted. Submits cannot be reverted, but they can be aborted while in the editor by exiting the editor without changing the Description text. Perforce will complain about this the first time you do it and will put you back in the editor. Exiting the editor the second time will abort the operation. Reverting a submitted change is very difficult and is best handled on a case-by-case basis. Editing The state of each file in the client is tracked and saved on the server. In order to avoid collisions from multiple people working on the same file at once, Perforce tracks which files are opened for edit, and uses this to help with submit, sync, and integration operations later on. To open a file for editing, use the p4 edit command like so: &prompt.user; p4 edit filename This marks the file on the server as being in the edit state, which then allows it to be submitted after changes are made, or marks it for special handling when doing an integration or sync operation. Note that editing is not exclusive in Perforce. Multiple people can have the same file in the edit state (you will be informed of others when you run the edit command), and you can submit your changes even when others are still editing the file. When someone else submits a change to a file that you are editing, you will need to resolve his changes with yours before your submit will succeed. The easiest way to do this is to either run a p4 sync or p4 submit and let it fail with the conflict, then run p4 resolve to manually resolve and accept his changes into your copy, then run p4 submit to commit your changes to the repository. If you have a file open for edit and you want to throw away your changes and revert it to its original state, run the p4 revert command like so: &prompt.user; p4 revert filename This resyncs the file to the contents of the server, and removes the edit attribute from the server. Any local changes that you had will be lost. This is quite useful when you have a made changes to a file but later decide that you do not want to keep them. When a file is synced, it is marked read-only in the filesystem. When you tell the server to open it for editing, it is changed to read-write on the filesystem. While these permissions can easily be overridden by hand, they are meant to gently remind you that you should being using the p4 edit command. Files that have local changes but are not in the edit state may get overwritten when doing a p4 sync. Changes, Descriptions, and History Changes to the Perforce depot can be listed via the p4 changes command. This will provide a brief description of each change, who made the change, and what its change number was. A change can be examined in detail via the p4 describe changenumber command. This will provide the submit log and the diffs of the actual change. Commonly, the p4 describe command is used in one of three ways: p4 describe -s CHANGE List a short description of changeset CHANGE, including the commit log of the particular changeset and a list of the files it affected. p4 describe -du CHANGE List a description of changeset CHANGE, including the commit log of the particular changeset, a list of the files it affected and a patch for each modified file, in a format similar to unified diff patches (but not exactly the same). p4 describe -dc CHANGE List a description of changeset CHANGE, including the commit log of the particular changeset, a list of the files it affected and a patch for each modified file, in a format similar to context diff patches (but not exactly the same). The p4 filelog filename command will show the history of a file, including all submits, integrations, and branches of it. Diffs There are two methods of producing file diffs in Perforce, either against local changes that have not been submitted yet, or between two trees (or within a branch) in the depot. These are done with different commands, and : p4 diff This generates a diff of the local changes to files in the edit state. The and flags can be used to create unified or context diffs, respectively, or the P4DIFF environment variable can be set to a local diff command to be used instead. It is a very good idea to use this command to review your changes before submitting them. p4 diff2 This creates a diff between arbitrary files in the depot, or between files specified in a branch spec. The diff operation takes place on the server, so P4DIFF variable has no effect, though the and flags do work. The two forms of this command are: &prompt.user; p4 diff2 -b branchname and &prompt.user; p4 diff2 //depot/path1 //depot/path2 In all cases the diff will be written to the standard output. Unfortunately, Perforce produces a diff format that is slightly incompatible with the traditional Unix diff and patch tools. Using the P4DIFF variable to point to the real &man.diff.1; tool can help this, but only for the p4 diff command. The output of command must be post-processed to be useful (the flag of will produce unified diffs that are somewhat compatible, but it does not include files that have been added or deleted). There is a post-processing script at: . Adding and Removing Files Integrating a branch will bring existing files into your tree, but you may still want to add new files or remove existing ones. Adding files is easily done be creating the file and then running the p4 add command like so: &prompt.user; p4 add filename If you want to add a whole tree of files, run a command like: &prompt.user; find . -type f | xargs p4 add Perforce can track UNIX symlinks too, so you can probably use \! -type d as the matching expression in &man.find.1; above. We don't commit symlinks into the source tree of &os; though, so this should not be necessary. Doing a p4 submit will then copy the file to the depot on the server. It is very important to only add files, not directories. Explicitly adding a directory will cause Perforce to treat it like a file, which is not what you want. Removing a file is just as easy with the p4 delete command like so: &prompt.user; p4 delete filename This will mark the file for deletion from the depot the next time that a submit is run. It will also remove the local copy of the file, so beware. Of course, deleting a file does not actually remove it from the repository. Deleted files can be resurrected by syncing them to a prior version. The only way to permanently remove a file is to use the p4 obliterate command. This command is irreversible and expensive, so it is only available to those with admin access. Working with diffs Sometimes you might need to apply a diff from another source to a tree under Perforce control. If it is a large diff that affects lots of files, it might be inconvenient to manually run p4 edit on each file. There is a trick for making this easier. First, make sure that no files are open on your client and that your tree is synced and up to date. Then apply the diff using the normal tools, and coerce the permissions on the files if needed. Then run the following commands: &prompt.user; p4 diff -se ... | xargs p4 edit &prompt.user; p4 diff -sd ... | xargs p4 delete &prompt.user; find . -type f | xargs p4 add The first command tells Perforce to look for files that have changed, even if they are not open. The second command tells Perforce to look for files that no longer exist on the local machine but do exist on the server. The third command then attempts to add all of the files that it can find locally. This is a very brute-force method, but it works because Perforce will only add the files that it does not already know about. The result of running these commands will be a set of files that are opened for edit, removal, or add, as appropriate. Verify the active changelist with: &prompt.user; p4 changelist &prompt.user; p4 diff -du and just do a p4 submit after that. Renaming files Perforce does not have a built-in way of renaming files or moving them to a different part of the tree. Simply copying a file to the new location, doing a p4 add on it, and a p4 delete on the old copy, works, but does not preserve change history of the file. This can make future integrations with parents and children very bumpy, in fact. A better method of dealing with this is to do a one-time, in-tree integration, like so: &prompt.user; p4 integrate -i oldfile newfile &prompt.user; p4 resolve &prompt.user; p4 delete oldfile &prompt.user; p4 submit The integration will force Perforce to keep a record of the relationship between the old and new names, which will assist it in future integrations. The flag tells it that it is a baseless integration, meaning that there is no branch history available for it to use in the integration. That is perfect for an integration like this, but should not be used for normal branch-based integrations. Interactions between &os; CVS and Perforce The &os; Perforce and CVS repositories are completely separate. However, changes to CVS are tracked at near-real-time in Perforce. Every 2 minutes, the CVS server is polled for updates in the HEAD branch, and those updates are committed to Perforce in the //depot/vendor/freebsd/... tree. This + class="directory">//depot/vendor/freebsd/... tree. This tree is then available for branching and integrating to derivative projects. Any project that directly modifies that &os; source code should have this tree as its branch parent (or grandparent, depending on the needs), and periodic integrations and syncs should be done so that your tree stays up to date and avoids conflicts with mainline development. The bridge between CVS and Perforce is one-way; changes to CVS will be reflected in Perforce, but changes in Perforce will not be reflected in CVS. On request, some parts of the Perforce repo can be exported to CVSup and made available for distribution that way. Contact the &os; Perforce administrators if this is something that you are interested in. Offline Operation One weakness of Perforce is that it assumes that network access to the server is always available. Most state, history, and metadata is saved on the server, and there is no provision for replicating the server like there is with CVS/CVSup. It is possible to run a proxy server, but it only provides very limited utility for offline operation. The best way to work offline is to make sure that your client has no open files and is fully synced before going offline. Then when editing a file, manually change the permissions to read-write. When you get back online, run the commands listed in the to automatically identify files that have been edited, added, and removed. It is quite common to be surprised by Perforce overwriting a locally changed file that was not opened for edit, so be extra vigilant with this. Notes for Google Summer of Code Most &os; projects under the Google Summer of Code program are located on the &os; Perforce server under one of the following locations: //depot/projects/soc2005/project-name/... + class="directory">//depot/projects/soc2005/project-name/... //depot/projects/soc2006/project-name/... + class="directory">//depot/projects/soc2006/project-name/... //depot/projects/soc2007/project-name/... + class="directory">//depot/projects/soc2007/project-name/... //depot/projects/soc2008/project-name/... + class="directory">//depot/projects/soc2008/project-name/... The project mentor is responsible for choosing a suitable project name and getting the student going with Perforce. Access to the &os; Perforce server does not imply membership in the &os; CVS committer community, though we happily encourage all students to consider joining the project when the time is appropriate.
diff --git a/en_US.ISO8859-1/articles/relaydelay/article.sgml b/en_US.ISO8859-1/articles/relaydelay/article.sgml index 1d2cacaee3..8f28605cca 100644 --- a/en_US.ISO8859-1/articles/relaydelay/article.sgml +++ b/en_US.ISO8859-1/articles/relaydelay/article.sgml @@ -1,262 +1,262 @@ %articles.ent; ]>
Using Greylist with &os; Tom Rhodes
trhodes@FreeBSD.org
 2004 The &os; Documentation Project An article written for the sole purpose of explaining the relaydelay system on a &os; mail server. A relaydelay or greylisting server cuts down on spam simply by issuing a TEMPFAIL error message to every incoming email. The purpose behind this idea is that most spammers use their personal computers with software to do their spamming. A real mail server should queue the message and try to send it later. Thus the spammer most likely moves on to the next host in place of trying to send the email again. This is an excellent idea; at least until the spammers begin to use software that offers to try again. But how does this work exactly? Well, when an email is received the message ID is stored in a database and the TEMPFAIL is returned along with the email. If the email is resent, the message ID will be checked against the message IDs currently stored in the database. If it exists in the database then the email is permitted to reach its intended recipient. Otherwise, the ID will be stored and a TEMPFAIL will be issued. This cycle will repeat with every email which comes into the server. From my personal experience, this really does cut out 90% of the spam. Basic Configuration We need to install the threaded perl. Install lang/perl5.8 with the USE_THREADS=yes variable set. The current version of perl may need to be removed first; errors will be reported by the install process if this is necessary. This will require all ports which require perl to be rebuilt and reinstalled; ports-mgmt/portupgrade is perfect for this. At least it will point out which ports have been removed and which will need to be reinstalled. Now for the database server; MySQL is perfect for this sort of work. Install the databases/mysql40-server along with databases/p5-DBD-mysql40. The previous port should imply the installation of databases/p5-DBI-137 so that knocks off another step. Install the perl based portable server plugin, net/p5-Net-Daemon port. Most of these port installations should have been straight forward. The next step will be more involved. Now install the mail/p5-Sendmail-Milter port. As of this writing the Makefile contains a line beginning with BROKEN, just remove it or comment it out. It is only marked this way because &os; neither has nor installs a threaded perl package by default. Once that line is removed it should build and install perfectly fine. Create a directory to hold temporary configuration files: &prompt.root; mkdir /tmp/relaydelay &prompt.root; cd /tmp/relaydelay Now that we have a temporary directory to work in, the following URLs should be sent to the fetch command: &prompt.root; fetch http://projects.puremagic.com/greylisting/releases/relaydelay-0.04.tgz &prompt.root; fetch http://lists.puremagic.com/pipermail/greylist-users/attachments/20030904/b8dafed9/relaydelay-0.04.bin The source code should now be unpacked: &prompt.root; gunzip -c relaydelay-0.04.tgz | tar xvf - There should now be several files into the temporary directory by this point. The appropriate information can now be passed to the database server by importing it from the mysql.sql file: &prompt.root; mysql < relaydelay-0.04/mysql.sql And patch the other files with the relaydelay.bin by running: &prompt.root; patch -d /tmp/relaydelay/relaydelay-0.04 < relaydelay.bin Edit the relaydelay.conf and the db_maintenance.pl file to append the correct username and password for the MySQL database. If the database was built and installed like the above then no users or passwords exist. This should be altered before putting this into production, that is covered in the database documentation and is beyond the scope of this document. Change the working directory to the - relaydelay-0.04 + relaydelay-0.04 directory: &prompt.root; cd relaydelay-0.04 Copy or move the configuration files to their respective directories: &prompt.root; mv db_maintenance.pl relaydelay.pl /usr/local/sbin &prompt.root; mv relaydelay.conf /etc/mail &prompt.root; mv relaydelay.sh /usr/local/etc/rc.d/ Test the current configuration by running: &prompt.root; sh /usr/local/etc/rc.d/relaydelay.sh start This file will not exist if the previous &man.mv.1; commands were neglected. If everything worked correctly a new file, relaydelay.log, should exist in - /var/log. It should + /var/log. It should contain something similar to the following text: Loaded Config File: /etc/mail/relaydelay.conf Using connection 'local:/var/run/relaydelay.sock' for filter relaydelay DBI Connecting to DBI:mysql:database=relaydelay:host=localhost:port=3306 Spawned relaydelay daemon process 38277. Starting Sendmail::Milter 0.18 engine. If this does not appear then something went wrong, review the screen output or look for anything new in the messages log file. Glue everything together by adding the following line to /etc/mail/sendmail.mc or the customized site specific mc file: INPUT_MAIL_FILTER(`relaydelay', `S=local:/var/run/relaydelay.sock, T=S:1m;R:2m;E:3m')dnl Rebuild and reinstall the files in the /etc/mail directory and restart sendmail. A quick make restart should do the trick. Obtain the perl script located at http://lists.puremagic.com/pipermail/greylist-users/2003-November/000327.html and save it in the - relaydelay-0.04 + relaydelay-0.04 directory. In the following examples this script is referred to as addlist.pl. Edit the whitelist_ip.txt file and modify it to include IP addresses of servers which should have the explicit abilities to bypass the relaydelay filters. i.e., domains from which email will not be issued a TEMPFAIL when received. Some examples could include: 192.168. # My internal network. 66.218.66 # Yahoo groups has unique senders. The blacklist_ip.txt file should be treated similarly but with reversed rules. List within this file IPs which should be denied without being issued a TEMPFAIL. This list of domains will never have the opportunity to prove that they are legitimate email servers. These files should now be imported into the database with the addlist.pl script obtained a few lines ago: &prompt.root; perl addlist.pl -whitelist 9999-12-31 23:59:59 < whitelist_ip.txt &prompt.root; perl addlist.pl -blacklist 9999-12-31 23:59:59 < blacklist_ip.txt To have relaydelay start with every system boot, add the to the /etc/rc.conf file. The /var/log/relaydelay.log log file should slowly fill up with success stories. Lines like the following should appear after a short time, depending on how busy the mail server is. === 2004-05-24 21:03:22 === Stored Sender: <someasshole@flawed-example.com> Passed Recipient: <local_user@pittgoth.com> Relay: example.net [XXX.XX.XXX.XX] - If_Addr: MY_IP_ADDRESS RelayIP: XX.XX.XX.XX - RelayName: example.net - RelayIdent: - PossiblyForged: 0 From: someasshole@flawed-example.com - To: local_user InMailer: esmtp - OutMailer: local - QueueID: i4P13Lo6000701111 Email is known but block has not expired. Issuing a tempfail. rowid: 51 IN ABORT CALLBACK - PrivData: 0<someasshole@flawed-example.com> The following line may now be added to /etc/newsyslog.conf to cause for relaydelay.log rotation at every 100 Kb: /var/log/relaydelay.log 644 3 100 * Z At some point there was an error about improper perl variables in the /etc/mail/relaydelay.conf. If those two variables are commented out then configuration may proceed as normal. Just remember to uncomment them before starting the relaydelay process.
diff --git a/en_US.ISO8859-1/articles/remote-install/article.sgml b/en_US.ISO8859-1/articles/remote-install/article.sgml index e213c17a12..7b979cc304 100644 --- a/en_US.ISO8859-1/articles/remote-install/article.sgml +++ b/en_US.ISO8859-1/articles/remote-install/article.sgml @@ -1,562 +1,562 @@ %articles.ent; ]>
Remote Installation of the &os; Operating System without a Remote Console Daniel Gerzo
danger@FreeBSD.org
 $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.general; 2008 The &os; Documentation Project This article documents the remote installation of the &os; operating system when the console of the remote system is unavailable. The main idea behind this article is the result of a collaboration with &a.mm; with valuable input provided by &a.pjd;. Background There are many server hosting providers in the world, but very few of them are officially supporting &os;. They usually provide support for a &linux; distribution to be installed on the servers they offer. In some cases, these companies will install your preferred &linux; distribution if you request it. Using this option, we will attempt to install &os;. In other cases, they may offer a rescue system which would be used in an emergency. It's possible to use this for our purposes as well. This article covers the basic installation and configuration steps required to bootstrap a remote installation of &os; with RAID-1 and ZFS capabilities. Introduction This section will summarize the purpose of this article and better explain what is covered herein. The instructions included in this article will benefit those using services provided by colocation facilities not supporting &os;. As we have mentioned in the Background section, many of the reputable server hosting companies provide some kind of rescue system, which is booted from their LAN and accessible over SSH. They usually provide this support in order to help their customers fix broken operating systems. As this article will explain, it is possible to install &os; with the help of these rescue systems. The next section of this article will describe how to configure, and build minimalistic &os; on the local machine. That version will eventually be running on the remote machine from a ramdisk, which will allow us to install a complete &os; operating system from an FTP mirror using the sysinstall utility. The rest of this article will describe the installation procedure itself, as well as the configuration of the ZFS file system. Requirements To continue successfully, you must: Have a network accessible operating system with SSH access Understand the &os; installation process Be familiar with the &man.sysinstall.8; utility Have the &os; installation ISO image or CD handy Preparation - <application>mfsBSD</application> Before &os; may be installed on the target system, it is necessary to build the minimal &os; operating system image which will boot from the hard drive. This way the new system can be accessed from the network, and the rest of the installation can be done without remote access to the system console. The mfsBSD tool-set can be used to build a tiny &os; image. As the name of mfsBSD suggests (mfs means memory file system), the resulting image runs entirely from a ramdisk. Thanks to this feature, the manipulation of hard drives will not be limited, therefore it will be possible to install a complete &os; operating system. The home page of mfsBSD, at , includes pointers to the latest release of the toolset. Please note that the internals of mfsBSD and how it all fits together is beyond the scope of this article. The interested reader should consult the original documentation of mfsBSD for more details. Download and extract the latest mfsBSD release and change your working directory to the directory where the mfsBSD scripts will reside: &prompt.root; fetch http://people.freebsd.org/~mm/mfsbsd/mfsbsd-latest.tar.gz &prompt.root; tar xvzf mfsbsd-1.0-beta1.tar.gz &prompt.root; cd mfsbsd-1.0-beta1/ Configuration of <application>mfsBSD</application> Before booting mfsBSD, a few important configuration options have to be set. The most important that we have to get right is, naturally, the network setup. The most suitable method to configure networking options depends on whether we know beforehand the type of the network interface we will use, and the network interface driver to be loaded for our hardware. We will see how mfsBSD can be configured in either case. Another important thing to set is the root password. This can be done by editing the conf/rootpw.conf file. Please keep in mind that the file will contain your password in the plain text, thus we do not recommend to use real password here. Nevertheless, this is just a temporary one-time password which can be later changed in a live system. The <filename>conf/interfaces.conf</filename> method When the installed network interface card is unknown, we can use the auto-detection features of mfsBSD. The startup scripts of mfsBSD can detect the correct driver to use, based on the MAC address of the interface, if we set the following options in conf/interfaces.conf: initconf_interfaces="ext1" initconf_mac_ext1="00:00:00:00:00:00" initconf_ip_ext1="192.168.0.2" initconf_netmask_ext1="255.255.255.0" Do not forget to add the defaultrouter information to the conf/rc.conf file: defaultrouter="192.168.0.1" The <filename>conf/rc.conf</filename> method When the network interface driver is known, it is more convenient to use the conf/rc.conf file for networking options. The syntax of this file is the same as the one used in the standard &man.rc.conf.5; file of &os;. For example, if you know that a &man.re.4; network interface is going to be available, you can set the following options in conf/rc.conf: defaultrouter="192.168.0.1" ifconfig_re0="inet 192.168.0.2 netmask 255.255.255.0" Building an <application>mfsBSD</application> image The process of building an mfsBSD image is pretty straightforward. The first step is to mount the &os; installation CD, or the installation ISO image to /cdrom. For the sake of example, + class="directory">/cdrom. For the sake of example, in this article we will assume that you have downloaded the &os; 7.0-RELEASE ISO. Mounting this ISO image to - the /cdrom directory is + the /cdrom directory is easy with the &man.mdconfig.8; utility: &prompt.root; mdconfig -a -t vnode -u 10 -f 7.0-RELEASE-amd64-disc1.iso &prompt.root; mount_cd9660 /dev/md10 /cdrom Next, build the bootable mfsBSD image: &prompt.root; make BASE=/cdrom/7.0-RELEASE The above make command has to be run from the top level of the mfsBSD directory tree, i.e. ~/mfsbsd-1.0-beta1/. + class="directory">~/mfsbsd-1.0-beta1/. Booting <application>mfsBSD</application> Now that the mfsBSD image is ready, it must be uploaded to the remote system running a live rescue system or pre-installed &linux; distribution. The most suitable tool for this task is scp: &prompt.root; scp disk.img root@192.168.0.2:. To boot mfsBSD image properly, it must be placed on the first (bootable) device of the given machine. This may be accomplished using this example providing that sda is the first bootable disk device: &prompt.root; dd if=/root/disk.img of=/dev/sda bs=1m If all went well, the image should now be in the MBR of the first device and the machine can be rebooted. Watch for the machine to boot up properly with the &man.ping.8; tool. Once it has came back on-line, it should be possible to access it over &man.ssh.1; as user root with the configured password. Installation of The &os; Operating System The mfsBSD has been successfully booted and it should be possible to log in through &man.ssh.1;. This section will describe how to create and label slices, set up gmirror for RAID-1, and how to use sysinstall to install a minimal distribution of the &os; operating system. Preparation of Hard Drives The first task is to allocate disk space for &os;, i.e.: to create slices and partitions. Obviously, the currently running system is fully loaded in system memory and therefore there will be no problems with manipulating hard drives. To complete this task, it is possible to use either sysinstall or &man.fdisk.8; in conjunction to &man.bsdlabel.8;. At the start, mark all system disks as empty. Repeat the following command for each hard drive: &prompt.root; dd if=/dev/zero of=/dev/ad0 count=2 Next, create slices and label them with your preferred tool. While it is considered easier to use sysinstall, a powerful and also probably less buggy method will be to use standard text-based &unix; tools, such as &man.fdisk.8; and &man.bsdlabel.8;, which will also be covered in this section. The former option is well documented in the Installing &os; chapter of the &os; Handbook. As it was mentioned in the introduction, this article will present how to set up a system with RAID-1 and ZFS capabilities. Our set up will consist of a small &man.gmirror.8; mirrored - / (root), /usr and /var file systems, and the rest of + / (root), /usr and /var file systems, and the rest of the disk space will be allocated for a &man.zpool.8; mirrored ZFS file system. Please note, that the ZFS file system will be configured after the &os; operating system is successfully installed and booted. The following example will describe how to create slices and labels, initialize &man.gmirror.8; on each partition and how to create a UFS2 file system in each mirrored partition: &prompt.root; fdisk -BI /dev/ad0 &prompt.root; fdisk -BI /dev/ad1 &prompt.root; bsdlabel -wB /dev/ad0s1 &prompt.root; bsdlabel -wB /dev/ad1s1 &prompt.root; bsdlabel -e /dev/ad0s1 &prompt.root; bsdlabel /dev/ad0s1 > /tmp/bsdlabel.txt && bsdlabel -R /dev/ad1s1 /tmp/bsdlabel.txt &prompt.root; gmirror label root /dev/ad[01]s1a &prompt.root; gmirror label var /dev/ad[01]s1d &prompt.root; gmirror label usr /dev/ad[01]s1e &prompt.root; gmirror label -F swap /dev/ad[01]s1b &prompt.root; newfs /dev/mirror/root &prompt.root; newfs /dev/mirror/var &prompt.root; newfs /dev/mirror/usr Create a slice covering the entire disk and initialize the boot code contained in sector 0 of the given disk. Repeat this command for all hard drives in the system. Write a standard label for each disk including the bootstrap code. Now, manually edit the label of the given disk. Refer to the &man.bsdlabel.8; manual page in order to find out how to create partitions. Create partitions a for / (root) file system, + class="directory">/ (root) file system, b for swap, d for - /var, + /var, e for /usr and finally + class="directory">/usr and finally f which will later be used for ZFS. Import the recently created label for the second hard drive, so both hard drives will be labeled in the same way. Initialize &man.gmirror.8; on each partition. Note the option used for swap partition. This instructs &man.gmirror.8; to assume that the device is in the consistent state after the power/system failure. Create a UFS2 file system on each mirrored partition. System Installation This is the most important part. This section will describe how to actually install the minimal distribution of &os; on the hard drives that we have prepared in the previous section. To accomplish this goal, all file systems need to be mounted so sysinstall may write the contents of &os; to the hard drives: &prompt.root; mount /dev/mirror/root /mnt &prompt.root; mkdir /mnt/var /mnt/usr &prompt.root; mount /dev/mirror/var /mnt/var &prompt.root; mount /dev/mirror/usr /mnt/usr When you are done, start &man.sysinstall.8;. Select the Custom installation from the main menu. Select Options and press Enter. With the help of arrow keys, move the cursor on the Install Root item, press Space and change it to /mnt. Press + class="directory">/mnt. Press Enter to submit your changes and exit the Options menu by pressing q. Note that this step is very important and if skipped, sysinstall will be unable to install &os;. Go to the Distributions menu, move the cursor with the arrow keys on the option, and check it by pressing Space. This article uses the Minimal distribution in order to save network traffic, because the system itself will be installed over ftp. Exit this menu by choosing option. The Partition and Label menus will be skipped, as these are useless now. In the Media menu, select . Select the nearest mirror and let sysinstall assume that the network is already configured. You will be returned back to the Custom menu. Finally, perform the system installation by selecting the last option, Commit. Exit sysinstall when it finishes the installation. Post Installation Steps The &os; operating system should be installed now; however, the process is not finished yet. It is necessary to perform some post installation steps in order to allow &os; to boot in the future and to be able to log in to the system. You must now &man.chroot.8; into the freshly installed system in order to finish the installation. Use the following command: &prompt.root; chroot /mnt To complete our goal, perform these steps: Copy the GENERIC kernel to the - /boot/kernel + /boot/kernel directory: &prompt.root; cp -Rp /boot/GENERIC/* /boot/kernel Create the /etc/rc.conf, /etc/resolv.conf and /etc/fstab files. Do not forget to properly set the network information and to enable sshd in the /etc/rc.conf file. The contents of the /etc/fstab file will be similar to the following: # Device Mountpoint FStype Options Dump Pass# /dev/mirror/swap none swap sw 0 0 /dev/mirror/root / ufs rw 1 1 /dev/mirror/usr /usr ufs rw 2 2 /dev/mirror/var /var ufs rw 2 2 /dev/cd0 /cdrom cd9660 ro,noauto 0 0 Create the /boot/loader.conf file, with the following contents: geom_mirror_load="YES" zfs_load="YES" Perform the following command, which will make ZFS available on the next boot: &prompt.root; echo 'zfs_enable="YES"' >> /etc/rc.conf Add additional users to the system using the &man.adduser.8; tool. Do not forget to add a user to the wheel group so you may obtain root access after the reboot. Double-check all your settings. The system should now be ready for the next boot. Use the &man.reboot.8; command to reboot your system. ZFS If your system survived the reboot, it should now be possible to log in. Welcome to the fresh &os; installation, performed remotely without the use of a remote console! The only remaining step is to configure &man.zpool.8; and create some &man.zfs.8; file systems. Creating and administering ZFS is very straightforward. First, create a mirrored pool: &prompt.root; zpool create tank mirror /dev/ad[01]s1f Next, create some file systems: &prompt.root; zfs create tank/ports &prompt.root; zfs create tank/src &prompt.root; zfs set compression=gzip tank/ports &prompt.root; zfs set compression=on tank/src &prompt.root; zfs set mountpoint=/usr/ports tank/ports &prompt.root; zfs set mountpoint=/usr/src tank/src That's all. If you are interested in more details about ZFS on &os;, please refer to the ZFS section of the &os; Wiki.
diff --git a/en_US.ISO8859-1/books/developers-handbook/kernelbuild/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/kernelbuild/chapter.sgml index bb61477d2a..8a569c66dd 100644 --- a/en_US.ISO8859-1/books/developers-handbook/kernelbuild/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/kernelbuild/chapter.sgml @@ -1,88 +1,88 @@ Building and Installing a &os; Kernel Being a kernel developer requires understanding of the kernel build process. To debug the &os; kernel it is required to be able to build one. There are two known ways to do so: The Traditional Way The New Way It is supposed that the reader of this chapter is familiar with the information described in the Building and Installing a Custom Kernel chapter of the &os; Handbook. If this is not the case, please read through the above mentioned chapter to understand how the build process works. Building a Kernel the <quote>Traditional</quote> Way Up to version 4.X of &os; this was the recommended way to build a new kernel. It can still be used on newer versions (instead of the buildkernel target of the toplevel - /usr/src/ makefiles). + /usr/src/ makefiles). Building the kernel this way may be useful when working on the kernel code and it may actually be faster than the New procedure when only a single option or two were tweaked in the kernel configuration file. On the other hand, it might lead to unexpected kernel build breakage when used by beginners on newer versions of &os;. Run &man.config.8; to generate the kernel source code: &prompt.root; /usr/sbin/config MYKERNEL Change into the build directory. &man.config.8; will print the name of this directory after being run as above. &prompt.root; cd ../compile/MYKERNEL Compile the kernel: &prompt.root; make depend &prompt.root; make Install the new kernel: &prompt.root; make install Building a Kernel the <quote>New</quote> Way This procedure is well supported and recommended under the latest &os; releases and is documented in the Building and Installing a Custom Kernel chapter of the &os; Handbook. diff --git a/en_US.ISO8859-1/books/fdp-primer/the-website/chapter.sgml b/en_US.ISO8859-1/books/fdp-primer/the-website/chapter.sgml index bfef1cb72a..2bd46eb90d 100644 --- a/en_US.ISO8859-1/books/fdp-primer/the-website/chapter.sgml +++ b/en_US.ISO8859-1/books/fdp-primer/the-website/chapter.sgml @@ -1,390 +1,390 @@ The Website Preparation Use a disk with sufficient free space. You may need anything from 200 MB to over 500 MB, depending on the method you choose. This space will hold the SGML tools, a subset of the CVS tree, temporary build space and the installed web pages. Make sure your documentation ports are up to date! When in doubt, remove the old ports using &man.pkg.delete.1; command before installing the port. For example, we currently depend on jade-1.2 and if you have installed jade-1.1, please do: &prompt.root; pkg_delete jade-1.1 There are two methods to get the files required for the website build: Use csup to get a local copy of the files from a CVSup server. This is the easiest method, and does not require installation of additional software. The supfile presented in the next section will always checkout the latest version of the required files. This is sufficient if you are simply rebuilding the website and do not intend to commit any changes. &man.csup.1; became part of the base system in &os; 6.2-RELEASE. If you are using an earlier version of &os; you will need to install net/csup from the Ports Collection. Use cvsup in cvs mode to create and maintain a local CVS repository with the required files. This will require you to install a program like net/cvsup-without-gui, but it is a more flexible method if you need to have quick access to different revisions of the doc/www files, revision histories, or if you intend to commit changes to the central &os; CVS repository. Simple method: Using <command>csup</command> The csup command is part of the base system and already used extensively by most people for updating the Ports Collection. The following sample supfile can be used to obtain a checkout of the files required for the website build: # # This file checks out all collections required to rebuild # the FreeBSD website # # Use the nearest CVSup mirror # listed at http://www.freebsd.org/doc/handbook/mirrors.html. *default host=cvsup10.FreeBSD.org *default base=/var/db *default prefix=/usr/build *default release=cvs tag=. *default delete use-rel-suffix *default compress # This will retrieve the entire doc branch of the FreeBSD repository. doc-all # This will retrieve the files required for the website www # This will retrieve some basic ports info required for the build ports-base You should, of course, change the default host entry to a CVSup mirror near your location, and the default prefix entry to the location where you intend to store the checked out files. Save this file as e.g. doc-www-supfile, and then execute the following command: &prompt.root; csup doc-www-supfile When this command finishes, you will find the directories - doc/, - www/ and - ports/ under the directory you + doc/, + www/ and + ports/ under the directory you specified in default prefix (/usr/build + class="directory">/usr/build in our example). We will use this same directory for the build process itself, so it would be better to use a filesystem with sufficient free space. That's it! You can now proceed with the website build. Advanced method: Maintaining a local <application>CVS</application> doc/www repository This method will give you more advanced options, but will require you to install the net/cvsup-without-gui port or package. The net/cvsup-without-gui port has a build dependency on lang/ezm3, a Modula 3 compiler. This compiler takes quite some time to build, and since most people will not need it for anything else, it is perhaps best to use a package to install CVSup. The CVSup utility has a special cvs mode that allows the retrieval of the ,v files that make up a CVS repository. This function is not currently available in csup. For detailed information on CVSup, please read the CVSup introduction in the &os; Handbook. The supfile shown below will fetch the cvs collections required for the website build, and create a local CVS repository: # # This file will create a local CVS repository # with the collections required for a complete # FreeBSD website rebuild. It should be used with # cvsup *only* (csup will not work) *default host=cvsup10.FreeBSD.org *default base=/var/db *default prefix=/usr/dcvs *default release=cvs *default delete use-rel-suffix *default compress # The following collections are needed # for the website build ports-base doc-all www # These collections are needed # for CVS functionality cvsroot-common cvsroot-ports cvsroot-doc You should, of course, change the default host entry to a CVSup mirror near your location, and the default prefix entry to the location where you intend to store the repository files. Save this file as e.g. doc-www-cvsfile, and then execute the following command: &prompt.root; cvsup doc-www-cvsfile It is also advisable to set the CVSROOT environment variable in your shell's startup files. For example, use the following entry in your ~/.cshrc file: setenv CVSROOT /usr/dcvs If you set this variable, you may omit the argument (shown below) when performing repository operations using the cvs command. Currently, you will need more than 400 MB of free space to host the repository files. An additional 200 MB will be needed for the temporary build space. Once the cvsup command completes, you are ready to check out the files to your build directory: &prompt.root; mkdir /usr/build &prompt.root; cd /usr/build &prompt.root; cvs /usr/dcvs co doc www ports The above command is consistent with the way csup checks out the files from the CVSup servers. When it completes, you will have a build directory with similar contents to the one used in the simple csup method. You can continue to use the cvsup command shown above, to update your local CVS repository on a regular basis. After the initial somewhat lengthy download, regular updates will only take a few minutes. Build the web pages from scratch Having completed either of the two methods, you will be ready to start the website build. In our example, the build directory is /usr/build + class="directory">/usr/build and all the required files are already in place. Change into the build directory: &prompt.root; cd /usr/build The website build starts from the - www/en directory by executing + www/en directory by executing the &man.make.1; all target, to create the web pages. &prompt.root; cd www/en &prompt.root; make all Install the web pages into your web server If you have moved out of the - en directory, change back to + en directory, change back to it. &prompt.root; cd /usr/build/www/en Run the &man.make.1; install target, setting the DESTDIR variable to the name of the directory you want to install the files to. &prompt.root; env DESTDIR=/usr/local/www make install If you have previously installed the web pages into the same directory the install process will not have deleted any old or outdated pages. For example, if you build and install a new copy of the site every day, this command will find and delete all files that have not been updated in three days. &prompt.root; find /usr/local/www 3 | xargs rm Environment variables CVSROOT Location of the CVS tree. We suggest you set this variable, if you use the CVSup method: &prompt.root; CVSROOT=/usr/dcvs; export CVSROOT CVSROOT is an environment variable. You must set it on the command line or in your dot files (e.g., ~/.profile). The exact syntax will differ depending on your shell (the above example is for bash and bash-like shells). ENGLISH_ONLY If set and not empty, the makefiles will build and install only the English documents. All translations will be ignored. E.g.: &prompt.root; make ENGLISH_ONLY=YES all install If you want to unset the variable ENGLISH_ONLY and build all pages, including translations, set the variable ENGLISH_ONLY to an empty value: &prompt.root; make ENGLISH_ONLY="" all install clean WEB_ONLY If set and not empty, the makefiles will build and install only the HTML pages from the www directory. All documents from - the doc directory (Handbook, + the doc directory (Handbook, FAQ, Tutorials) will be ignored. E.g.: &prompt.root; make WEB_ONLY=YES all install NOPORTSCVS If set, the makefiles will not checkout files from the ports cvs repository. Instead, it will copy the files from - /usr/ports (or where the + /usr/ports (or where the variable PORTSBASE points to). WEB_ONLY, ENGLISH_ONLY and NOPORTSCVS are make variables. You can set the variables in /etc/make.conf, Makefile.inc, as environment variables on the command line, or in your dot files. diff --git a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml index 5b1f404fac..8292d632db 100644 --- a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml @@ -1,5626 +1,5626 @@ Advanced Networking Synopsis This chapter will cover a number of advanced networking topics. After reading this chapter, you will know: The basics of gateways and routes. How to set up IEEE 802.11 and &bluetooth; devices. How to make FreeBSD act as a bridge. How to set up network booting on a diskless machine. How to set up network address translation. How to connect two computers via PLIP. How to set up IPv6 on a FreeBSD machine. How to configure ATM. How to enable and utilize the features of CARP, the Common Access Redundancy Protocol in &os; Before reading this chapter, you should: Understand the basics of the /etc/rc scripts. Be familiar with basic network terminology. Know how to configure and install a new FreeBSD kernel (). Know how to install additional third-party software (). Coranth Gryphon Contributed by Gateways and Routes routing gateway subnet For one machine to be able to find another over a network, there must be a mechanism in place to describe how to get from one to the other. This is called routing. A route is a defined pair of addresses: a destination and a gateway. The pair indicates that if you are trying to get to this destination, communicate through this gateway. There are three types of destinations: individual hosts, subnets, and default. The default route is used if none of the other routes apply. We will talk a little bit more about default routes later on. There are also three types of gateways: individual hosts, interfaces (also called links), and Ethernet hardware addresses (MAC addresses). An Example To illustrate different aspects of routing, we will use the following example from netstat: &prompt.user; netstat -r Routing tables Destination Gateway Flags Refs Use Netif Expire default outside-gw UGSc 37 418 ppp0 localhost localhost UH 0 181 lo0 test0 0:e0:b5:36:cf:4f UHLW 5 63288 ed0 77 10.20.30.255 link#1 UHLW 1 2421 example.com link#1 UC 0 0 host1 0:e0:a8:37:8:1e UHLW 3 4601 lo0 host2 0:e0:a8:37:8:1e UHLW 0 5 lo0 => host2.example.com link#1 UC 0 0 224 link#1 UC 0 0 default route The first two lines specify the default route (which we will cover in the next section) and the localhost route. loopback device The interface (Netif column) that this routing table specifies to use for localhost is lo0, also known as the loopback device. This says to keep all traffic for this destination internal, rather than sending it out over the LAN, since it will only end up back where it started. Ethernet MAC address The next thing that stands out are the addresses beginning with 0:e0:. These are Ethernet hardware addresses, which are also known as MAC addresses. FreeBSD will automatically identify any hosts (test0 in the example) on the local Ethernet and add a route for that host, directly to it over the Ethernet interface, ed0. There is also a timeout (Expire column) associated with this type of route, which is used if we fail to hear from the host in a specific amount of time. When this happens, the route to this host will be automatically deleted. These hosts are identified using a mechanism known as RIP (Routing Information Protocol), which figures out routes to local hosts based upon a shortest path determination. subnet FreeBSD will also add subnet routes for the local subnet (10.20.30.255 is the broadcast address for the subnet 10.20.30, and example.com is the domain name associated with that subnet). The designation link#1 refers to the first Ethernet card in the machine. You will notice no additional interface is specified for those. Both of these groups (local network hosts and local subnets) have their routes automatically configured by a daemon called routed. If this is not run, then only routes which are statically defined (i.e. entered explicitly) will exist. The host1 line refers to our host, which it knows by Ethernet address. Since we are the sending host, FreeBSD knows to use the loopback interface (lo0) rather than sending it out over the Ethernet interface. The two host2 lines are an example of what happens when we use an &man.ifconfig.8; alias (see the section on Ethernet for reasons why we would do this). The => symbol after the lo0 interface says that not only are we using the loopback (since this address also refers to the local host), but specifically it is an alias. Such routes only show up on the host that supports the alias; all other hosts on the local network will simply have a link#1 line for such routes. The final line (destination subnet 224) deals with multicasting, which will be covered in another section. Finally, various attributes of each route can be seen in the Flags column. Below is a short table of some of these flags and their meanings: U Up: The route is active. H Host: The route destination is a single host. G Gateway: Send anything for this destination on to this remote system, which will figure out from there where to send it. S Static: This route was configured manually, not automatically generated by the system. C Clone: Generates a new route based upon this route for machines we connect to. This type of route is normally used for local networks. W WasCloned: Indicated a route that was auto-configured based upon a local area network (Clone) route. L Link: Route involves references to Ethernet hardware. Default Routes default route When the local system needs to make a connection to a remote host, it checks the routing table to determine if a known path exists. If the remote host falls into a subnet that we know how to reach (Cloned routes), then the system checks to see if it can connect along that interface. If all known paths fail, the system has one last option: the default route. This route is a special type of gateway route (usually the only one present in the system), and is always marked with a c in the flags field. For hosts on a local area network, this gateway is set to whatever machine has a direct connection to the outside world (whether via PPP link, DSL, cable modem, T1, or another network interface). If you are configuring the default route for a machine which itself is functioning as the gateway to the outside world, then the default route will be the gateway machine at your Internet Service Provider's (ISP) site. Let us look at an example of default routes. This is a common configuration: [Local2] <--ether--> [Local1] <--PPP--> [ISP-Serv] <--ether--> [T1-GW] The hosts Local1 and Local2 are at your site. Local1 is connected to an ISP via a dial up PPP connection. This PPP server computer is connected through a local area network to another gateway computer through an external interface to the ISPs Internet feed. The default routes for each of your machines will be: Host Default Gateway Interface Local2 Local1 Ethernet Local1 T1-GW PPP A common question is Why (or how) would we set the T1-GW to be the default gateway for Local1, rather than the ISP server it is connected to?. Remember, since the PPP interface is using an address on the ISP's local network for your side of the connection, routes for any other machines on the ISP's local network will be automatically generated. Hence, you will already know how to reach the T1-GW machine, so there is no need for the intermediate step of sending traffic to the ISP server. It is common to use the address X.X.X.1 as the gateway address for your local network. So (using the same example), if your local class-C address space was 10.20.30 and your ISP was using 10.9.9 then the default routes would be: Host Default Route Local2 (10.20.30.2) Local1 (10.20.30.1) Local1 (10.20.30.1, 10.9.9.30) T1-GW (10.9.9.1) You can easily define the default route via the /etc/rc.conf file. In our example, on the Local2 machine, we added the following line in /etc/rc.conf: defaultrouter="10.20.30.1" It is also possible to do it directly from the command line with the &man.route.8; command: &prompt.root; route add default 10.20.30.1 For more information on manual manipulation of network routing tables, consult &man.route.8; manual page. Dual Homed Hosts dual homed hosts There is one other type of configuration that we should cover, and that is a host that sits on two different networks. Technically, any machine functioning as a gateway (in the example above, using a PPP connection) counts as a dual-homed host. But the term is really only used to refer to a machine that sits on two local-area networks. In one case, the machine has two Ethernet cards, each having an address on the separate subnets. Alternately, the machine may only have one Ethernet card, and be using &man.ifconfig.8; aliasing. The former is used if two physically separate Ethernet networks are in use, the latter if there is one physical network segment, but two logically separate subnets. Either way, routing tables are set up so that each subnet knows that this machine is the defined gateway (inbound route) to the other subnet. This configuration, with the machine acting as a router between the two subnets, is often used when we need to implement packet filtering or firewall security in either or both directions. If you want this machine to actually forward packets between the two interfaces, you need to tell FreeBSD to enable this ability. See the next section for more details on how to do this. Building a Router router A network router is simply a system that forwards packets from one interface to another. Internet standards and good engineering practice prevent the FreeBSD Project from enabling this by default in FreeBSD. You can enable this feature by changing the following variable to YES in &man.rc.conf.5;: gateway_enable=YES # Set to YES if this host will be a gateway This option will set the &man.sysctl.8; variable net.inet.ip.forwarding to 1. If you should need to stop routing temporarily, you can reset this to 0 temporarily. Your new router will need routes to know where to send the traffic. If your network is simple enough you can use static routes. FreeBSD also comes with the standard BSD routing daemon &man.routed.8;, which speaks RIP (both version 1 and version 2) and IRDP. Support for BGP v4, OSPF v2, and other sophisticated routing protocols is available with the net/zebra package. Commercial products such as &gated; are also available for more complex network routing solutions. BGP RIP OSPF Al Hoang Contributed by Setting Up Static Routes Manual Configuration Let us assume we have a network as follows: INTERNET | (10.0.0.1/24) Default Router to Internet | |Interface xl0 |10.0.0.10/24 +------+ | | RouterA | | (FreeBSD gateway) +------+ | Interface xl1 | 192.168.1.1/24 | +--------------------------------+ Internal Net 1 | 192.168.1.2/24 | +------+ | | RouterB | | +------+ | 192.168.2.1/24 | Internal Net 2 In this scenario, RouterA is our &os; machine that is acting as a router to the rest of the Internet. It has a default route set to 10.0.0.1 which allows it to connect with the outside world. We will assume that RouterB is already configured properly and knows how to get wherever it needs to go. (This is simple in this picture. Just add a default route on RouterB using 192.168.1.1 as the gateway.) If we look at the routing table for RouterA we would see something like the following: &prompt.user; netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 10.0.0.1 UGS 0 49378 xl0 127.0.0.1 127.0.0.1 UH 0 6 lo0 10.0.0/24 link#1 UC 0 0 xl0 192.168.1/24 link#2 UC 0 0 xl1 With the current routing table RouterA will not be able to reach our Internal Net 2. It does not have a route for 192.168.2.0/24. One way to alleviate this is to manually add the route. The following command would add the Internal Net 2 network to RouterA's routing table using 192.168.1.2 as the next hop: &prompt.root; route add -net 192.168.2.0/24 192.168.1.2 Now RouterA can reach any hosts on the 192.168.2.0/24 network. Persistent Configuration The above example is perfect for configuring a static route on a running system. However, one problem is that the routing information will not persist if you reboot your &os; machine. The way to handle the addition of a static route is to put it in your /etc/rc.conf file: # Add Internal Net 2 as a static route static_routes="internalnet2" route_internalnet2="-net 192.168.2.0/24 192.168.1.2" The static_routes configuration variable is a list of strings separated by a space. Each string references to a route name. In our above example we only have one string in static_routes. This string is internalnet2. We then add a configuration variable called route_internalnet2 where we put all of the configuration parameters we would give to the &man.route.8; command. For our example above we would have used the command: &prompt.root; route add -net 192.168.2.0/24 192.168.1.2 so we need "-net 192.168.2.0/24 192.168.1.2". As said above, we can have more than one string in static_routes. This allows us to create multiple static routes. The following lines shows an example of adding static routes for the 192.168.0.0/24 and 192.168.1.0/24 networks on an imaginary router: static_routes="net1 net2" route_net1="-net 192.168.0.0/24 192.168.0.1" route_net2="-net 192.168.1.0/24 192.168.1.1" Routing Propagation routing propagation We have already talked about how we define our routes to the outside world, but not about how the outside world finds us. We already know that routing tables can be set up so that all traffic for a particular address space (in our examples, a class-C subnet) can be sent to a particular host on that network, which will forward the packets inbound. When you get an address space assigned to your site, your service provider will set up their routing tables so that all traffic for your subnet will be sent down your PPP link to your site. But how do sites across the country know to send to your ISP? There is a system (much like the distributed DNS information) that keeps track of all assigned address-spaces, and defines their point of connection to the Internet Backbone. The Backbone are the main trunk lines that carry Internet traffic across the country, and around the world. Each backbone machine has a copy of a master set of tables, which direct traffic for a particular network to a specific backbone carrier, and from there down the chain of service providers until it reaches your network. It is the task of your service provider to advertise to the backbone sites that they are the point of connection (and thus the path inward) for your site. This is known as route propagation. Troubleshooting traceroute Sometimes, there is a problem with routing propagation, and some sites are unable to connect to you. Perhaps the most useful command for trying to figure out where routing is breaking down is the &man.traceroute.8; command. It is equally useful if you cannot seem to make a connection to a remote machine (i.e. &man.ping.8; fails). The &man.traceroute.8; command is run with the name of the remote host you are trying to connect to. It will show the gateway hosts along the path of the attempt, eventually either reaching the target host, or terminating because of a lack of connection. For more information, see the manual page for &man.traceroute.8;. Multicast Routing multicast routing kernel options MROUTING FreeBSD supports both multicast applications and multicast routing natively. Multicast applications do not require any special configuration of FreeBSD; applications will generally run out of the box. Multicast routing requires that support be compiled into the kernel: options MROUTING In addition, the multicast routing daemon, &man.mrouted.8; must be configured to set up tunnels and DVMRP via /etc/mrouted.conf. More details on multicast configuration may be found in the manual page for &man.mrouted.8;. As of &os; 7.0 the &man.mrouted.8; multicast routing daemon has been removed from the base system. It implements the DVMRP multicast routing protocol, which has largely been replaced by &man.pim.4; in many multicast installations. The related &man.map-mbone.8; and &man.mrinfo.8; utilities have also been removed. These programs are now available in the &os; Ports Collection as net/mrouted. Loader Marc Fonvieille Murray Stokely Wireless Networking wireless networking 802.11 wireless networking Wireless Networking Basics Most wireless networks are based on the IEEE 802.11 standards. A basic wireless network consists of multiple stations communicating with radios that broadcast in either the 2.4GHz or 5GHz band (though this varies according to the locale and is also changing to enable communication in the 2.3GHz and 4.9GHz ranges). 802.11 networks are organized in two ways: in infrastructure mode one station acts as a master with all the other stations associating to it; the network is known as a BSS and the master station is termed an access point (AP). In a BSS all communication passes through the AP; even when one station wants to communicate with another wireless station messages must go through the AP. In the second form of network there is no master and stations communicate directly. This form of network is termed an IBSS and is commonly known as an ad-hoc network. 802.11 networks were first deployed in the 2.4GHz band using protocols defined by the IEEE 802.11 and 802.11b standard. These specifications include the operating frequencies, MAC layer characteristics including framing and transmission rates (communication can be done at various rates). Later the 802.11a standard defined operation in the 5GHz band, including different signalling mechanisms and higher transmission rates. Still later the 802.11g standard was defined to enable use of 802.11a signalling and transmission mechanisms in the 2.4GHz band in such a way as to be backwards compatible with 802.11b networks. Separate from the underlying transmission techniques 802.11 networks have a variety of security mechanisms. The original 802.11 specifications defined a simple security protocol called WEP. This protocol uses a fixed pre-shared key and the RC4 cryptographic cipher to encode data transmitted on a network. Stations must all agree on the fixed key in order to communicate. This scheme was shown to be easily broken and is now rarely used except to discourage transient users from joining networks. Current security practice is given by the IEEE 802.11i specification that defines new cryptographic ciphers and an additional protocol to authenticate stations to an access point and exchange keys for doing data communication. Further, cryptographic keys are periodically refreshed and there are mechanisms for detecting intrusion attempts (and for countering intrusion attempts). Another security protocol specification commonly used in wireless networks is termed WPA. This was a precursor to 802.11i defined by an industry group as an interim measure while waiting for 802.11i to be ratified. WPA specifies a subset of the requirements found in 802.11i and is designed for implementation on legacy hardware. Specifically WPA requires only the TKIP cipher that is derived from the original WEP cipher. 802.11i permits use of TKIP but also requires support for a stronger cipher, AES-CCM, for encrypting data. (The AES cipher was not required in WPA because it was deemed too computationally costly to be implemented on legacy hardware.) Other than the above protocol standards the other important standard to be aware of is 802.11e. This defines protocols for deploying multi-media applications such as streaming video and voice over IP (VoIP) in an 802.11 network. Like 802.11i, 802.11e also has a precursor specification termed WME (later renamed WMM) that has been defined by an industry group as a subset of 802.11e that can be deployed now to enable multi-media applications while waiting for the final ratification of 802.11e. The most important thing to know about 802.11e and WME/WMM is that it enables prioritized traffic use of a wireless network through Quality of Service (QoS) protocols and enhanced media access protocols. Proper implementation of these protocols enable high speed bursting of data and prioritized traffic flow. Since the 6.0 version, &os; supports networks that operate using 802.11a, 802.11b, and 802.11g. The WPA and 802.11i security protocols are likewise supported (in conjunction with any of 11a, 11b, and 11g) and QoS and traffic prioritization required by the WME/WMM protocols are supported for a limited set of wireless devices. Basic Setup Kernel Configuration To use wireless networking you need a wireless networking card and to configure the kernel with the appropriate wireless networking support. The latter is separated into multiple modules so that you only need to configure the software you are actually going to use. The first thing you need is a wireless device. The most commonly used devices are those that use parts made by Atheros. These devices are supported by the &man.ath.4; driver and require the following line to be added to the /boot/loader.conf file: if_ath_load="YES" The Atheros driver is split up into three separate pieces: the driver proper (&man.ath.4;), the hardware support layer that handles chip-specific functions (&man.ath.hal.4;), and an algorithm for selecting which of several possible rates for transmitting frames (ath_rate_sample here). When you load this support as modules these dependencies are automatically handled for you. If instead of an Atheros device you had another device you would select the module for that device; e.g.: if_wi_load="YES" for devices based on the Intersil Prism parts (&man.wi.4; driver). In the rest of this document, we will use an &man.ath.4; device, the device name in the examples must be changed according to your configuration. A list of available wireless drivers can be found at the beginning of the &man.wlan.4; manual page. If a native &os; driver for your wireless device does not exist, it may be possible to directly use the &windows; driver with the help of the NDIS driver wrapper. With a device driver configured you need to also bring in the 802.11 networking support required by the driver. For the &man.ath.4; driver this is at least the &man.wlan.4; module; this module is automatically loaded with the wireless device driver. With that you will need the modules that implement cryptographic support for the security protocols you intend to use. These are intended to be dynamically loaded on demand by the &man.wlan.4; module but for now they must be manually configured. The following modules are available: &man.wlan.wep.4;, &man.wlan.ccmp.4; and &man.wlan.tkip.4;. Both &man.wlan.ccmp.4; and &man.wlan.tkip.4; drivers are only needed if you intend to use the WPA and/or 802.11i security protocols. If your network is to run totally open (i.e., with no encryption) then you do not even need the &man.wlan.wep.4; support. To load these modules at boot time, add the following lines to /boot/loader.conf: wlan_wep_load="YES" wlan_ccmp_load="YES" wlan_tkip_load="YES" With this information in the system bootstrap configuration file (i.e., /boot/loader.conf), you have to reboot your &os; box. If you do not want to reboot your machine for the moment, you can just load the modules by hand using &man.kldload.8;. If you do not want to use modules, it is possible to compile these drivers into the kernel by adding the following lines to your kernel configuration file: device ath # Atheros IEEE 802.11 wireless network driver device ath_hal # Atheros Hardware Access Layer device ath_rate_sample # John Bicket's SampleRate control algorithm. device wlan # 802.11 support (Required) device wlan_wep # WEP crypto support for 802.11 devices device wlan_ccmp # AES-CCMP crypto support for 802.11 devices device wlan_tkip # TKIP and Michael crypto support for 802.11 devices With this information in the kernel configuration file, recompile the kernel and reboot your &os; machine. When the system is up, we could find some information about the wireless device in the boot messages, like this: ath0: <Atheros 5212> mem 0xff9f0000-0xff9fffff irq 17 at device 2.0 on pci2 ath0: Ethernet address: 00:11:95:d5:43:62 ath0: mac 7.9 phy 4.5 radio 5.6 Infrastructure Mode The infrastructure mode or BSS mode is the mode that is typically used. In this mode, a number of wireless access points are connected to a wired network. Each wireless network has its own name, this name is called the SSID of the network. Wireless clients connect to the wireless access points. &os; Clients How to Find Access Points To scan for networks, use the ifconfig command. This request may take a few moments to complete as it requires that the system switches to each available wireless frequency and probes for available access points. Only the super-user can initiate such a scan: &prompt.root; ifconfig ath0 up scan SSID BSSID CHAN RATE S:N INT CAPS dlinkap 00:13:46:49:41:76 6 54M 29:3 100 EPS WPA WME freebsdap 00:11:95:c3:0d:ac 1 54M 22:1 100 EPS WPA You must mark the interface before you can scan. Subsequent scan requests do not require you to mark the interface up again. The output of a scan request lists each BSS/IBSS network found. Beside the name of the network, SSID, we find the BSSID which is the MAC address of the access point. The CAPS field identifies the type of each network and the capabilities of the stations operating there: E Extended Service Set (ESS). Indicates that the station is part of an infrastructure network (in contrast to an IBSS/ad-hoc network). I IBSS/ad-hoc network. Indicates that the station is part of an ad-hoc network (in contrast to an ESS network). P Privacy. Data confidentiality is required for all data frames exchanged within the BSS. This means that this BSS requires the station to use cryptographic means such as WEP, TKIP or AES-CCMP to encrypt/decrypt data frames being exchanged with others. S Short Preamble. Indicates that the network is using short preambles (defined in 802.11b High Rate/DSSS PHY, short preamble utilizes a 56 bit sync field in contrast to a 128 bit field used in long preamble mode). s Short slot time. Indicates that the 802.11g network is using a short slot time because there are no legacy (802.11b) stations present. One can also display the current list of known networks with: &prompt.root; ifconfig ath0 list scan This information may be updated automatically by the adapter or manually with a request. Old data is automatically removed from the cache, so over time this list may shrink unless more scans are done. Basic Settings This section provides a simple example of how to make the wireless network adapter work in &os; without encryption. After you are familiar with these concepts, we strongly recommend using WPA to set up your wireless network. There are three basic steps to configure a wireless network: selecting an access point, authenticating your station, and configuring an IP address. The following sections discuss each step. Selecting an Access Point Most of time it is sufficient to let the system choose an access point using the builtin heuristics. This is the default behaviour when you mark an interface up or otherwise configure an interface by listing it in /etc/rc.conf, e.g.: ifconfig_ath0="DHCP" If there are multiple access points and you want to select a specific one, you can select it by its SSID: ifconfig_ath0="ssid your_ssid_here DHCP" In an environment where there are multiple access points with the same SSID (often done to simplify roaming) it may be necessary to associate to one specific device. In this case you can also specify the BSSID of the access point (you can also leave off the SSID): ifconfig_ath0="ssid your_ssid_here bssid xx:xx:xx:xx:xx:xx DHCP" There are other ways to constrain the choice of an access point such as limiting the set of frequencies the system will scan on. This may be useful if you have a multi-band wireless card as scanning all the possible channels can be time-consuming. To limit operation to a specific band you can use the parameter; e.g.: ifconfig_ath0="mode 11g ssid your_ssid_here DHCP" will force the card to operate in 802.11g which is defined only for 2.4GHz frequencies so any 5GHz channels will not be considered. Other ways to do this are the parameter, to lock operation to one specific frequency, and the parameter, to specify a list of channels for scanning. More information about these parameters can be found in the &man.ifconfig.8; manual page. Authentication Once you have selected an access point your station needs to authenticate before it can pass data. Authentication can happen in several ways. The most common scheme used is termed open authentication and allows any station to join the network and communicate. This is the authentication you should use for test purpose the first time you set up a wireless network. Other schemes require cryptographic handshakes be completed before data traffic can flow; either using pre-shared keys or secrets, or more complex schemes that involve backend services such as RADIUS. Most users will use open authentication which is the default setting. Next most common setup is WPA-PSK, also known as WPA Personal, which is described below. If you have an &apple; &airport; Extreme base station for an access point you may need to configure shared-key authentication together with a WEP key. This can be done in the /etc/rc.conf file or using the &man.wpa.supplicant.8; program. If you have a single &airport; base station you can setup access with something like: ifconfig_ath0="authmode shared wepmode on weptxkey 1 wepkey 01234567 DHCP" In general shared key authentication is to be avoided because it uses the WEP key material in a highly-constrained manner making it even easier to crack the key. If WEP must be used (e.g., for compatibility with legacy devices) it is better to use WEP with open authentication. More information regarding WEP can be found in the . Getting an IP Address with DHCP Once you have selected an access point and set the authentication parameters, you will have to get an IP address to communicate. Most of time you will obtain your wireless IP address via DHCP. To achieve that, simply edit /etc/rc.conf and add DHCP to the configuration for your device as shown in various examples above: ifconfig_ath0="DHCP" At this point, you are ready to bring up the wireless interface: &prompt.root; /etc/rc.d/netif start Once the interface is running, use ifconfig to see the status of the interface ath0: &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (OFDM/54Mbps) status: associated ssid dlinkap channel 6 bssid 00:13:46:49:41:76 authmode OPEN privacy OFF txpowmax 36 protmode CTS bintval 100 The status: associated means you are connected to the wireless network (to the dlinkap network in our case). The bssid 00:13:46:49:41:76 part is the MAC address of your access point; the authmode line informs you that the communication is not encrypted (OPEN). Static IP Address In the case you cannot obtain an IP address from a DHCP server, you can set a fixed IP address. Replace the DHCP keyword shown above with the address information. Be sure to retain any other parameters you have set up for selecting an access point: ifconfig_ath0="ssid your_ssid_here inet 192.168.1.100 netmask 255.255.255.0" WPA WPA (Wi-Fi Protected Access) is a security protocol used together with 802.11 networks to address the lack of proper authentication and the weakness of WEP. WPA leverages the 802.1X authentication protocol and uses one of several ciphers instead of WEP for data integrity. The only cipher required by WPA is TKIP (Temporary Key Integrity Protocol) which is a cipher that extends the basic RC4 cipher used by WEP by adding integrity checking, tamper detection, and measures for responding to any detected intrusions. TKIP is designed to work on legacy hardware with only software modification; it represents a compromise that improves security but is still not entirely immune to attack. WPA also specifies the AES-CCMP cipher as an alternative to TKIP and that is preferred when possible; for this specification the term WPA2 (or RSN) is commonly used. WPA defines authentication and encryption protocols. Authentication is most commonly done using one of two techniques: by 802.1X and a backend authentication service such as RADIUS, or by a minimal handshake between the station and the access point using a pre-shared secret. The former is commonly termed WPA Enterprise with the latter known as WPA Personal. Since most people will not set up a RADIUS backend server for wireless network, WPA-PSK is by far the most commonly encountered configuration for WPA. The control of the wireless connection and the authentication (key negotiation or authentication with a server) is done with the &man.wpa.supplicant.8; utility. This program requires a configuration file, /etc/wpa_supplicant.conf, to run. More information regarding this file can be found in the &man.wpa.supplicant.conf.5; manual page. WPA-PSK WPA-PSK also known as WPA-Personal is based on a pre-shared key (PSK) generated from a given password and that will be used as the master key in the wireless network. This means every wireless user will share the same key. WPA-PSK is intended for small networks where the use of an authentication server is not possible or desired. Always use strong passwords that are sufficiently long and made from a rich alphabet so they will not be guessed and/or attacked. The first step is the configuration of the /etc/wpa_supplicant.conf file with the SSID and the pre-shared key of your network: network={ ssid="freebsdap" psk="freebsdmall" } Then, in /etc/rc.conf, we indicate that the wireless device configuration will be done with WPA and the IP address will be obtained with DHCP: ifconfig_ath0="WPA DHCP" Then, we can bring up the interface: &prompt.root; /etc/rc.d/netif start Starting wpa_supplicant. DHCPDISCOVER on ath0 to 255.255.255.255 port 67 interval 5 DHCPDISCOVER on ath0 to 255.255.255.255 port 67 interval 6 DHCPOFFER from 192.168.0.1 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.1 bound to 192.168.0.254 -- renewal in 300 seconds. ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (OFDM/36Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 Or you can try to configure it manually using the same /etc/wpa_supplicant.conf above, and run: &prompt.root; wpa_supplicant -i ath0 -c /etc/wpa_supplicant.conf Trying to associate with 00:11:95:c3:0d:ac (SSID='freebsdap' freq=2412 MHz) Associated with 00:11:95:c3:0d:ac WPA: Key negotiation completed with 00:11:95:c3:0d:ac [PTK=TKIP GTK=TKIP] The next operation is the launch of the dhclient command to get the IP address from the DHCP server: &prompt.root; dhclient ath0 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.1 bound to 192.168.0.254 -- renewal in 300 seconds. &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (OFDM/48Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 If the /etc/rc.conf is set up with the line ifconfig_ath0="DHCP" then it is no need to run the dhclient command manually, dhclient will be launched after wpa_supplicant plumbs the keys. In the case where the use of DHCP is not possible, you can set a static IP address after wpa_supplicant has authenticated the station: &prompt.root; ifconfig ath0 inet 192.168.0.100 netmask 255.255.255.0 &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (OFDM/36Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 When DHCP is not used, you also have to manually set up the default gateway and the nameserver: &prompt.root; route add default your_default_router &prompt.root; echo "nameserver your_DNS_server" >> /etc/resolv.conf WPA with EAP-TLS The second way to use WPA is with an 802.1X backend authentication server, in this case WPA is called WPA-Enterprise to make difference with the less secure WPA-Personal with its pre-shared key. The authentication in WPA-Enterprise is based on EAP (Extensible Authentication Protocol). EAP does not come with an encryption method, it was decided to embed EAP inside an encrypted tunnel. Many types of EAP authentication methods have been designed, the most common methods are EAP-TLS, EAP-TTLS and EAP-PEAP. EAP-TLS (EAP with Transport Layer Security) is a very well-supported authentication protocol in the wireless world since it was the first EAP method to be certified by the Wi-Fi alliance. EAP-TLS will require three certificates to run: the CA certificate (installed on all machines), the server certificate for your authentication server, and one client certificate for each wireless client. In this EAP method, both authentication server and wireless client authenticate each other in presenting their respective certificates, and they verify that these certificates were signed by your organization's certificate authority (CA). As previously, the configuration is done via /etc/wpa_supplicant.conf: network={ ssid="freebsdap" proto=RSN key_mgmt=WPA-EAP eap=TLS identity="loader" ca_cert="/etc/certs/cacert.pem" client_cert="/etc/certs/clientcert.pem" private_key="/etc/certs/clientkey.pem" private_key_passwd="freebsdmallclient" } This field indicates the network name (SSID). Here, we use RSN (IEEE 802.11i) protocol, i.e., WPA2. The key_mgmt line refers to the key management protocol we use. In our case it is WPA using EAP authentication: WPA-EAP. In this field, we mention the EAP method for our connection. The identity field contains the identity string for EAP. The ca_cert field indicates the pathname of the CA certificate file. This file is needed to verify the server certificat. The client_cert line gives the pathname to the client certificate file. This certificate is unique to each wireless client of the network. The private_key field is the pathname to the client certificate private key file. The private_key_passwd field contains the passphrase for the private key. Then add the following line to /etc/rc.conf: ifconfig_ath0="WPA DHCP" The next step is to bring up the interface with the help of the rc.d facility: &prompt.root; /etc/rc.d/netif start Starting wpa_supplicant. DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.20 bound to 192.168.0.254 -- renewal in 300 seconds. ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (DS/11Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA2/802.11i privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 As previously shown, it is also possible to bring up the interface manually with both wpa_supplicant and ifconfig commands. WPA with EAP-TTLS With EAP-TLS both the authentication server and the client need a certificate, with EAP-TTLS (EAP-Tunneled Transport Layer Security) a client certificate is optional. This method is close to what some secure web sites do , where the web server can create a secure SSL tunnel even if the visitors do not have client-side certificates. EAP-TTLS will use the encrypted TLS tunnel for safe transport of the authentication data. The configuration is done via the /etc/wpa_supplicant.conf file: network={ ssid="freebsdap" proto=RSN key_mgmt=WPA-EAP eap=TTLS identity="test" password="test" ca_cert="/etc/certs/cacert.pem" phase2="auth=MD5" } In this field, we mention the EAP method for our connection. The identity field contains the identity string for EAP authentication inside the encrypted TLS tunnel. The password field contains the passphrase for the EAP authentication. The ca_cert field indicates the pathname of the CA certificate file. This file is needed to verify the server certificat. In this field, we mention the authentication method used in the encrypted TLS tunnel. In our case, EAP with MD5-Challenge has been used. The inner authentication phase is often called phase2. You also have to add the following line to /etc/rc.conf: ifconfig_ath0="WPA DHCP" The next step is to bring up the interface: &prompt.root; /etc/rc.d/netif start Starting wpa_supplicant. DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.20 bound to 192.168.0.254 -- renewal in 300 seconds. ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (DS/11Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA2/802.11i privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 WPA with EAP-PEAP PEAP (Protected EAP) has been designed as an alternative to EAP-TTLS. There are two types of PEAP methods, the most common one is PEAPv0/EAP-MSCHAPv2. In the rest of this document, we will use the PEAP term to refer to that EAP method. PEAP is the most used EAP standard after EAP-TLS, in other words if you have a network with mixed OSes, PEAP should be the most supported standard after EAP-TLS. PEAP is similar to EAP-TTLS: it uses a server-side certificate to authenticate clients by creating an encrypted TLS tunnel between the client and the authentication server, which protects the ensuing exchange of authentication information. In term of security the difference between EAP-TTLS and PEAP is that PEAP authentication broadcasts the username in clear, only the password is sent in the encrypted TLS tunnel. EAP-TTLS will use the TLS tunnel for both username and password. We have to edit the /etc/wpa_supplicant.conf file and add the EAP-PEAP related settings: network={ ssid="freebsdap" proto=RSN key_mgmt=WPA-EAP eap=PEAP identity="test" password="test" ca_cert="/etc/certs/cacert.pem" phase1="peaplabel=0" phase2="auth=MSCHAPV2" } In this field, we mention the EAP method for our connection. The identity field contains the identity string for EAP authentication inside the encrypted TLS tunnel. The password field contains the passphrase for the EAP authentication. The ca_cert field indicates the pathname of the CA certificate file. This file is needed to verify the server certificat. This field contains the parameters for the first phase of the authentication (the TLS tunnel). According to the authentication server used, you will have to specify a specific label for the authentication. Most of time, the label will be client EAP encryption which is set by using peaplabel=0. More information can be found in the &man.wpa.supplicant.conf.5; manual page. In this field, we mention the authentication protocol used in the encrypted TLS tunnel. In the case of PEAP, it is auth=MSCHAPV2. The following must be added to /etc/rc.conf: ifconfig_ath0="WPA DHCP" Then, we can bring up the interface: &prompt.root; /etc/rc.d/netif start Starting wpa_supplicant. DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPREQUEST on ath0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.20 bound to 192.168.0.254 -- renewal in 300 seconds. ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.254 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (DS/11Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA2/802.11i privacy ON deftxkey UNDEF TKIP 2:128-bit txpowmax 36 protmode CTS roaming MANUAL bintval 100 WEP WEP (Wired Equivalent Privacy) is part of the original 802.11 standard. There is no authentication mechanism, only a weak form of access control, and it is easily to be cracked. WEP can be set up with ifconfig: &prompt.root; ifconfig ath0 ssid my_net wepmode on weptxkey 3 wepkey 3:0x3456789012 \ inet 192.168.1.100 netmask 255.255.255.0 The weptxkey means which WEP key will be used in the transmission. Here we used the third key. This must match the setting in the access point. The wepkey means setting the selected WEP key. It should in the format index:key, if the index is not given, key 1 is set. That is to say we need to set the index if we use keys other than the first key. You must replace the 0x3456789012 with the key configured for use on the access point. You are encouraged to read &man.ifconfig.8; manual page for further information. The wpa_supplicant facility also can be used to configure your wireless interface with WEP. The example above can be set up by adding the following lines to /etc/wpa_supplicant.conf: network={ ssid="my_net" key_mgmt=NONE wep_key3=3456789012 wep_tx_keyidx=3 } Then: &prompt.root; wpa_supplicant -i ath0 -c /etc/wpa_supplicant.conf Trying to associate with 00:13:46:49:41:76 (SSID='dlinkap' freq=2437 MHz) Associated with 00:13:46:49:41:76 Ad-hoc Mode IBSS mode, also called ad-hoc mode, is designed for point to point connections. For example, to establish an ad-hoc network between the machine A and the machine B we will just need to choose two IP adresses and a SSID. On the box A: &prompt.root; ifconfig ath0 ssid freebsdap mediaopt adhoc inet 192.168.0.1 netmask 255.255.255.0 &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 inet6 fe80::211:95ff:fec3:dac%ath0 prefixlen 64 scopeid 0x4 ether 00:11:95:c3:0d:ac media: IEEE 802.11 Wireless Ethernet autoselect <adhoc> (autoselect <adhoc>) status: associated ssid freebsdap channel 2 bssid 02:11:95:c3:0d:ac authmode OPEN privacy OFF txpowmax 36 protmode CTS bintval 100 The adhoc parameter indicates the interface is running in the IBSS mode. On B, we should be able to detect A: &prompt.root; ifconfig ath0 up scan SSID BSSID CHAN RATE S:N INT CAPS freebsdap 02:11:95:c3:0d:ac 2 54M 19:3 100 IS The I in the output confirms the machine A is in ad-hoc mode. We just have to configure B with a different IP address: &prompt.root; ifconfig ath0 ssid freebsdap mediaopt adhoc inet 192.168.0.2 netmask 255.255.255.0 &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect <adhoc> (autoselect <adhoc>) status: associated ssid freebsdap channel 2 bssid 02:11:95:c3:0d:ac authmode OPEN privacy OFF txpowmax 36 protmode CTS bintval 100 Both A and B are now ready to exchange informations. &os; Host Access Points &os; can act as an Access Point (AP) which eliminates the need to buy a hardware AP or run an ad-hoc network. This can be particularly useful when your &os; machine is acting as a gateway to another network (e.g., the Internet). Basic Settings Before configuring your &os; machine as an AP, the kernel must be configured with the appropriate wireless networking support for your wireless card. You also have to add the support for the security protocols you intend to use. For more details, see . The use of the NDIS driver wrapper and the &windows; drivers do not allow currently the AP operation. Only native &os; wireless drivers support AP mode. Once the wireless networking support is loaded, you can check if your wireless device supports the host-based access point mode (also know as hostap mode): &prompt.root; ifconfig ath0 list caps ath0=783ed0f<WEP,TKIP,AES,AES_CCM,IBSS,HOSTAP,AHDEMO,TXPMGT,SHSLOT,SHPREAMBLE,MONITOR,TKIPMIC,WPA1,WPA2,BURST,WME> This output displays the card capabilities; the HOSTAP word confirms this wireless card can act as an Access Point. Various supported ciphers are also mentioned: WEP, TKIP, WPA2, etc., these informations are important to know what security protocols could be set on the Access Point. The wireless device can now be put into hostap mode and configured with the correct SSID and IP address: &prompt.root; ifconfig ath0 ssid freebsdap mode 11g mediaopt hostap inet 192.168.0.1 netmask 255.255.255.0 Use again ifconfig to see the status of the ath0 interface: &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 inet6 fe80::211:95ff:fec3:dac%ath0 prefixlen 64 scopeid 0x4 ether 00:11:95:c3:0d:ac media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap> status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode OPEN privacy OFF txpowmax 38 bmiss 7 protmode CTS burst dtimperiod 1 bintval 100 The hostap parameter indicates the interface is running in the host-based access point mode. The interface configuration can be done automatically at boot time by adding the following line to /etc/rc.conf: ifconfig_ath0="ssid freebsdap mode 11g mediaopt hostap inet 192.168.0.1 netmask 255.255.255.0" Host-based Access Point without Authentication or Encryption Although it is not recommended to run an AP without any authentication or encryption, this is a simple way to check if your AP is working. This configuration is also important for debugging client issues. Once the AP configured as previously shown, it is possible from another wireless machine to initiate a scan to find the AP: &prompt.root; ifconfig ath0 up scan SSID BSSID CHAN RATE S:N INT CAPS freebsdap 00:11:95:c3:0d:ac 1 54M 22:1 100 ES The client machine found the Access Point and can be associated with it: &prompt.root; ifconfig ath0 ssid freebsdap inet 192.168.0.2 netmask 255.255.255.0 &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::211:95ff:fed5:4362%ath0 prefixlen 64 scopeid 0x1 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:11:95:d5:43:62 media: IEEE 802.11 Wireless Ethernet autoselect (OFDM/54Mbps) status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode OPEN privacy OFF txpowmax 36 protmode CTS bintval 100 WPA Host-based Access Point This section will focus on setting up &os; Access Point using the WPA security protocol. More details regarding WPA and the configuration of WPA-based wireless clients can be found in the . The hostapd daemon is used to deal with client authentication and keys management on the WPA enabled Access Point. In the following, all the configuration operations will be performed on the &os; machine acting as AP. Once the AP is correctly working, hostapd should be automatically enabled at boot with the following line in /etc/rc.conf: hostapd_enable="YES" Before trying to configure hostapd, be sure you have done the basic settings introduced in the . WPA-PSK WPA-PSK is intended for small networks where the use of an backend authentication server is not possible or desired. The configuration is done in the /etc/hostapd.conf file: interface=ath0 debug=1 ctrl_interface=/var/run/hostapd ctrl_interface_group=wheel ssid=freebsdap wpa=1 wpa_passphrase=freebsdmall wpa_key_mgmt=WPA-PSK wpa_pairwise=CCMP TKIP This field indicates the wireless interface used for the Access Point. This field sets the level of verbosity during the execution of hostapd. A value of 1 represents the minimal level. The ctrl_interface field gives the pathname of the directory used by hostapd to stores its domain socket files for the communication with external programs such as &man.hostapd.cli.8;. The default value is used here. The ctrl_interface_group line sets the group (here, it is the wheel group) allowed to access to the control interface files. This field sets the network name. The wpa field enables WPA and specifies which WPA authentication protocol will be required. A value of 1 configures the AP for WPA-PSK. The wpa_passphrase field contains the ASCII passphrase for the WPA authentication. Always use strong passwords that are sufficiently long and made from a rich alphabet so they will not be guessed and/or attacked. The wpa_key_mgmt line refers to the key management protocol we use. In our case it is WPA-PSK. The wpa_pairwise field indicates the set of accepted encryption algorithms by the Access Point. Here both TKIP (WPA) and CCMP (WPA2) ciphers are accepted. CCMP cipher is an alternative to TKIP and that is strongly preferred when possible; TKIP should be used solely for stations incapable of doing CCMP. The next step is to start hostapd: &prompt.root /etc/rc.d/hostapd forcestart &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2290 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 inet6 fe80::211:95ff:fec3:dac%ath0 prefixlen 64 scopeid 0x4 ether 00:11:95:c3:0d:ac media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap> status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode WPA2/802.11i privacy MIXED deftxkey 2 TKIP 2:128-bit txpowmax 36 protmode CTS dtimperiod 1 bintval 100 The Access Point is running, the clients can now be associated with it, see for more details. It is possible to see the stations associated with the AP using the ifconfig ath0 list sta command. WEP Host-based Access Point It is not recommended to use WEP for setting up an Access Point since there is no authentication mechanism and it is easily to be cracked. Some legacy wireless cards only support WEP as security protocol, these cards will only allow to set up AP without authentication or encryption or using the WEP protocol. The wireless device can now be put into hostap mode and configured with the correct SSID and IP address: &prompt.root; ifconfig ath0 ssid freebsdap wepmode on weptxkey 3 wepkey 3:0x3456789012 mode 11g mediaopt hostap \ inet 192.168.0.1 netmask 255.255.255.0 The weptxkey means which WEP key will be used in the transmission. Here we used the third key (note that the key numbering starts with 1). This parameter must be specified to really encrypt the data. The wepkey means setting the selected WEP key. It should in the format index:key, if the index is not given, key 1 is set. That is to say we need to set the index if we use keys other than the first key. Use again ifconfig to see the status of the ath0 interface: &prompt.root; ifconfig ath0 ath0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255 inet6 fe80::211:95ff:fec3:dac%ath0 prefixlen 64 scopeid 0x4 ether 00:11:95:c3:0d:ac media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap> status: associated ssid freebsdap channel 1 bssid 00:11:95:c3:0d:ac authmode OPEN privacy ON deftxkey 3 wepkey 3:40-bit txpowmax 36 protmode CTS dtimperiod 1 bintval 100 From another wireless machine, it is possible to initiate a scan to find the AP: &prompt.root; ifconfig ath0 up scan SSID BSSID CHAN RATE S:N INT CAPS freebsdap 00:11:95:c3:0d:ac 1 54M 22:1 100 EPS The client machine found the Access Point and can be associated with it using the correct parameters (key, etc.), see for more details. Troubleshooting If you are having trouble with wireless networking, there are a number of steps you can take to help troubleshoot the problem. If you do not see the access point listed when scanning be sure you have not configured your wireless device to a limited set of channels. If you cannot associate to an access point verify the configuration of your station matches the one of the access point. This includes the authentication scheme and any security protocols. Simplify your configuration as much as possible. If you are using a security protocol such as WPA or WEP configure the access point for open authentication and no security to see if you can get traffic to pass. Once you can associate to the access point diagnose any security configuration using simple tools like &man.ping.8;. The wpa_supplicant has much debugging support; try running it manually with the option and look at the system logs. There are also many lower-level debugging tools. You can enable debugging messages in the 802.11 protocol support layer using the wlandebug program found in /usr/src/tools/tools/net80211. For example: &prompt.root; wlandebug -i ath0 +scan+auth+debug+assoc net.wlan.0.debug: 0 => 0xc80000<assoc,auth,scan> can be used to enable console messages related to scanning for access points and doing the 802.11 protocol handshakes required to arrange communication. There are also many useful statistics maintained by the 802.11 layer; the wlanstats tool will dump these informations. These statistics should identify all errors identified by the 802.11 layer. Beware however that some errors are identified in the device drivers that lie below the 802.11 layer so they may not show up. To diagnose device-specific problems you need to refer to the drivers' documentation. If the above information does not help to clarify the problem, please submit a problem report and include output from the above tools. Pav Lucistnik Written by
pav@FreeBSD.org
 Bluetooth Bluetooth Introduction Bluetooth is a wireless technology for creating personal networks operating in the 2.4 GHz unlicensed band, with a range of 10 meters. Networks are usually formed ad-hoc from portable devices such as cellular phones, handhelds and laptops. Unlike the other popular wireless technology, Wi-Fi, Bluetooth offers higher level service profiles, e.g. FTP-like file servers, file pushing, voice transport, serial line emulation, and more. The Bluetooth stack in &os; is implemented using the Netgraph framework (see &man.netgraph.4;). A broad variety of Bluetooth USB dongles is supported by the &man.ng.ubt.4; driver. The Broadcom BCM2033 chip based Bluetooth devices are supported via the &man.ubtbcmfw.4; and &man.ng.ubt.4; drivers. The 3Com Bluetooth PC Card 3CRWB60-A is supported by the &man.ng.bt3c.4; driver. Serial and UART based Bluetooth devices are supported via &man.sio.4;, &man.ng.h4.4; and &man.hcseriald.8;. This section describes the use of the USB Bluetooth dongle. Plugging in the Device By default Bluetooth device drivers are available as kernel modules. Before attaching a device, you will need to load the driver into the kernel: &prompt.root; kldload ng_ubt If the Bluetooth device is present in the system during system startup, load the module from /boot/loader.conf: ng_ubt_load="YES" Plug in your USB dongle. The output similar to the following will appear on the console (or in syslog): ubt0: vendor 0x0a12 product 0x0001, rev 1.10/5.25, addr 2 ubt0: Interface 0 endpoints: interrupt=0x81, bulk-in=0x82, bulk-out=0x2 ubt0: Interface 1 (alt.config 5) endpoints: isoc-in=0x83, isoc-out=0x3, wMaxPacketSize=49, nframes=6, buffer size=294 Copy /usr/share/examples/netgraph/bluetooth/rc.bluetooth into some convenient place, like /etc/rc.bluetooth. This script is used to start and stop the Bluetooth stack. It is a good idea to stop the stack before unplugging the device, but it is not (usually) fatal. When starting the stack, you will receive output similar to the following: &prompt.root; /etc/rc.bluetooth start ubt0 BD_ADDR: 00:02:72:00:d4:1a Features: 0xff 0xff 0xf 00 00 00 00 00 <3-Slot> <5-Slot> <Encryption> <Slot offset> <Timing accuracy> <Switch> <Hold mode> <Sniff mode> <Park mode> <RSSI> <Channel quality> <SCO link> <HV2 packets> <HV3 packets> <u-law log> <A-law log> <CVSD> <Paging scheme> <Power control> <Transparent SCO data> Max. ACL packet size: 192 bytes Number of ACL packets: 8 Max. SCO packet size: 64 bytes Number of SCO packets: 8 HCI Host Controller Interface (HCI) Host Controller Interface (HCI) provides a command interface to the baseband controller and link manager, and access to hardware status and control registers. This interface provides a uniform method of accessing the Bluetooth baseband capabilities. HCI layer on the Host exchanges data and commands with the HCI firmware on the Bluetooth hardware. The Host Controller Transport Layer (i.e. physical bus) driver provides both HCI layers with the ability to exchange information with each other. A single Netgraph node of type hci is created for a single Bluetooth device. The HCI node is normally connected to the Bluetooth device driver node (downstream) and the L2CAP node (upstream). All HCI operations must be performed on the HCI node and not on the device driver node. Default name for the HCI node is devicehci. For more details refer to the &man.ng.hci.4; manual page. One of the most common tasks is discovery of Bluetooth devices in RF proximity. This operation is called inquiry. Inquiry and other HCI related operations are done with the &man.hccontrol.8; utility. The example below shows how to find out which Bluetooth devices are in range. You should receive the list of devices in a few seconds. Note that a remote device will only answer the inquiry if it put into discoverable mode. &prompt.user; hccontrol -n ubt0hci inquiry Inquiry result, num_responses=1 Inquiry result #0 BD_ADDR: 00:80:37:29:19:a4 Page Scan Rep. Mode: 0x1 Page Scan Period Mode: 00 Page Scan Mode: 00 Class: 52:02:04 Clock offset: 0x78ef Inquiry complete. Status: No error [00] BD_ADDR is unique address of a Bluetooth device, similar to MAC addresses of a network card. This address is needed for further communication with a device. It is possible to assign human readable name to a BD_ADDR. The /etc/bluetooth/hosts file contains information regarding the known Bluetooth hosts. The following example shows how to obtain human readable name that was assigned to the remote device: &prompt.user; hccontrol -n ubt0hci remote_name_request 00:80:37:29:19:a4 BD_ADDR: 00:80:37:29:19:a4 Name: Pav's T39 If you perform an inquiry on a remote Bluetooth device, it will find your computer as your.host.name (ubt0). The name assigned to the local device can be changed at any time. The Bluetooth system provides a point-to-point connection (only two Bluetooth units involved), or a point-to-multipoint connection. In the point-to-multipoint connection the connection is shared among several Bluetooth devices. The following example shows how to obtain the list of active baseband connections for the local device: &prompt.user; hccontrol -n ubt0hci read_connection_list Remote BD_ADDR Handle Type Mode Role Encrypt Pending Queue State 00:80:37:29:19:a4 41 ACL 0 MAST NONE 0 0 OPEN A connection handle is useful when termination of the baseband connection is required. Note, that it is normally not required to do it by hand. The stack will automatically terminate inactive baseband connections. &prompt.root; hccontrol -n ubt0hci disconnect 41 Connection handle: 41 Reason: Connection terminated by local host [0x16] Refer to hccontrol help for a complete listing of available HCI commands. Most of the HCI commands do not require superuser privileges. L2CAP Logical Link Control and Adaptation Protocol (L2CAP) Logical Link Control and Adaptation Protocol (L2CAP) provides connection-oriented and connectionless data services to upper layer protocols with protocol multiplexing capability and segmentation and reassembly operation. L2CAP permits higher level protocols and applications to transmit and receive L2CAP data packets up to 64 kilobytes in length. L2CAP is based around the concept of channels. Channel is a logical connection on top of baseband connection. Each channel is bound to a single protocol in a many-to-one fashion. Multiple channels can be bound to the same protocol, but a channel cannot be bound to multiple protocols. Each L2CAP packet received on a channel is directed to the appropriate higher level protocol. Multiple channels can share the same baseband connection. A single Netgraph node of type l2cap is created for a single Bluetooth device. The L2CAP node is normally connected to the Bluetooth HCI node (downstream) and Bluetooth sockets nodes (upstream). Default name for the L2CAP node is devicel2cap. For more details refer to the &man.ng.l2cap.4; manual page. A useful command is &man.l2ping.8;, which can be used to ping other devices. Some Bluetooth implementations might not return all of the data sent to them, so 0 bytes in the following example is normal. &prompt.root; l2ping -a 00:80:37:29:19:a4 0 bytes from 0:80:37:29:19:a4 seq_no=0 time=48.633 ms result=0 0 bytes from 0:80:37:29:19:a4 seq_no=1 time=37.551 ms result=0 0 bytes from 0:80:37:29:19:a4 seq_no=2 time=28.324 ms result=0 0 bytes from 0:80:37:29:19:a4 seq_no=3 time=46.150 ms result=0 The &man.l2control.8; utility is used to perform various operations on L2CAP nodes. This example shows how to obtain the list of logical connections (channels) and the list of baseband connections for the local device: &prompt.user; l2control -a 00:02:72:00:d4:1a read_channel_list L2CAP channels: Remote BD_ADDR SCID/ DCID PSM IMTU/ OMTU State 00:07:e0:00:0b:ca 66/ 64 3 132/ 672 OPEN &prompt.user; l2control -a 00:02:72:00:d4:1a read_connection_list L2CAP connections: Remote BD_ADDR Handle Flags Pending State 00:07:e0:00:0b:ca 41 O 0 OPEN Another diagnostic tool is &man.btsockstat.1;. It does a job similar to as &man.netstat.1; does, but for Bluetooth network-related data structures. The example below shows the same logical connection as &man.l2control.8; above. &prompt.user; btsockstat Active L2CAP sockets PCB Recv-Q Send-Q Local address/PSM Foreign address CID State c2afe900 0 0 00:02:72:00:d4:1a/3 00:07:e0:00:0b:ca 66 OPEN Active RFCOMM sessions L2PCB PCB Flag MTU Out-Q DLCs State c2afe900 c2b53380 1 127 0 Yes OPEN Active RFCOMM sockets PCB Recv-Q Send-Q Local address Foreign address Chan DLCI State c2e8bc80 0 250 00:02:72:00:d4:1a 00:07:e0:00:0b:ca 3 6 OPEN RFCOMM RFCOMM Protocol The RFCOMM protocol provides emulation of serial ports over the L2CAP protocol. The protocol is based on the ETSI standard TS 07.10. RFCOMM is a simple transport protocol, with additional provisions for emulating the 9 circuits of RS-232 (EIATIA-232-E) serial ports. The RFCOMM protocol supports up to 60 simultaneous connections (RFCOMM channels) between two Bluetooth devices. For the purposes of RFCOMM, a complete communication path involves two applications running on different devices (the communication endpoints) with a communication segment between them. RFCOMM is intended to cover applications that make use of the serial ports of the devices in which they reside. The communication segment is a Bluetooth link from one device to another (direct connect). RFCOMM is only concerned with the connection between the devices in the direct connect case, or between the device and a modem in the network case. RFCOMM can support other configurations, such as modules that communicate via Bluetooth wireless technology on one side and provide a wired interface on the other side. In &os; the RFCOMM protocol is implemented at the Bluetooth sockets layer. pairing Pairing of Devices By default, Bluetooth communication is not authenticated, and any device can talk to any other device. A Bluetooth device (for example, cellular phone) may choose to require authentication to provide a particular service (for example, Dial-Up service). Bluetooth authentication is normally done with PIN codes. A PIN code is an ASCII string up to 16 characters in length. User is required to enter the same PIN code on both devices. Once user has entered the PIN code, both devices will generate a link key. After that the link key can be stored either in the devices themselves or in a persistent storage. Next time both devices will use previously generated link key. The described above procedure is called pairing. Note that if the link key is lost by any device then pairing must be repeated. The &man.hcsecd.8; daemon is responsible for handling of all Bluetooth authentication requests. The default configuration file is /etc/bluetooth/hcsecd.conf. An example section for a cellular phone with the PIN code arbitrarily set to 1234 is shown below: device { bdaddr 00:80:37:29:19:a4; name "Pav's T39"; key nokey; pin "1234"; } There is no limitation on PIN codes (except length). Some devices (for example Bluetooth headsets) may have a fixed PIN code built in. The switch forces the &man.hcsecd.8; daemon to stay in the foreground, so it is easy to see what is happening. Set the remote device to receive pairing and initiate the Bluetooth connection to the remote device. The remote device should say that pairing was accepted, and request the PIN code. Enter the same PIN code as you have in hcsecd.conf. Now your PC and the remote device are paired. Alternatively, you can initiate pairing on the remote device. On &os; 5.5, 6.1 and newer, the following line can be added to the /etc/rc.conf file to have hcsecd started automatically on system start: hcsecd_enable="YES" The following is a sample of the hcsecd daemon output: hcsecd[16484]: Got Link_Key_Request event from 'ubt0hci', remote bdaddr 0:80:37:29:19:a4 hcsecd[16484]: Found matching entry, remote bdaddr 0:80:37:29:19:a4, name 'Pav's T39', link key doesn't exist hcsecd[16484]: Sending Link_Key_Negative_Reply to 'ubt0hci' for remote bdaddr 0:80:37:29:19:a4 hcsecd[16484]: Got PIN_Code_Request event from 'ubt0hci', remote bdaddr 0:80:37:29:19:a4 hcsecd[16484]: Found matching entry, remote bdaddr 0:80:37:29:19:a4, name 'Pav's T39', PIN code exists hcsecd[16484]: Sending PIN_Code_Reply to 'ubt0hci' for remote bdaddr 0:80:37:29:19:a4 SDP Service Discovery Protocol (SDP) The Service Discovery Protocol (SDP) provides the means for client applications to discover the existence of services provided by server applications as well as the attributes of those services. The attributes of a service include the type or class of service offered and the mechanism or protocol information needed to utilize the service. SDP involves communication between a SDP server and a SDP client. The server maintains a list of service records that describe the characteristics of services associated with the server. Each service record contains information about a single service. A client may retrieve information from a service record maintained by the SDP server by issuing a SDP request. If the client, or an application associated with the client, decides to use a service, it must open a separate connection to the service provider in order to utilize the service. SDP provides a mechanism for discovering services and their attributes, but it does not provide a mechanism for utilizing those services. Normally, a SDP client searches for services based on some desired characteristics of the services. However, there are times when it is desirable to discover which types of services are described by an SDP server's service records without any a priori information about the services. This process of looking for any offered services is called browsing. The Bluetooth SDP server &man.sdpd.8; and command line client &man.sdpcontrol.8; are included in the standard &os; installation. The following example shows how to perform a SDP browse query. &prompt.user; sdpcontrol -a 00:01:03:fc:6e:ec browse Record Handle: 00000000 Service Class ID List: Service Discovery Server (0x1000) Protocol Descriptor List: L2CAP (0x0100) Protocol specific parameter #1: u/int/uuid16 1 Protocol specific parameter #2: u/int/uuid16 1 Record Handle: 0x00000001 Service Class ID List: Browse Group Descriptor (0x1001) Record Handle: 0x00000002 Service Class ID List: LAN Access Using PPP (0x1102) Protocol Descriptor List: L2CAP (0x0100) RFCOMM (0x0003) Protocol specific parameter #1: u/int8/bool 1 Bluetooth Profile Descriptor List: LAN Access Using PPP (0x1102) ver. 1.0 ... and so on. Note that each service has a list of attributes (RFCOMM channel for example). Depending on the service you might need to make a note of some of the attributes. Some Bluetooth implementations do not support service browsing and may return an empty list. In this case it is possible to search for the specific service. The example below shows how to search for the OBEX Object Push (OPUSH) service: &prompt.user; sdpcontrol -a 00:01:03:fc:6e:ec search OPUSH Offering services on &os; to Bluetooth clients is done with the &man.sdpd.8; server. On &os; 5.5, 6.1 and newer, the following line can be added to the /etc/rc.conf file: sdpd_enable="YES" Then the sdpd daemon can be started with: &prompt.root; /etc/rc.d/sdpd start The local server application that wants to provide Bluetooth service to the remote clients will register service with the local SDP daemon. The example of such application is &man.rfcomm.pppd.8;. Once started it will register Bluetooth LAN service with the local SDP daemon. The list of services registered with the local SDP server can be obtained by issuing SDP browse query via local control channel: &prompt.root; sdpcontrol -l browse Dial-Up Networking (DUN) and Network Access with PPP (LAN) Profiles The Dial-Up Networking (DUN) profile is mostly used with modems and cellular phones. The scenarios covered by this profile are the following: use of a cellular phone or modem by a computer as a wireless modem for connecting to a dial-up Internet access server, or using other dial-up services; use of a cellular phone or modem by a computer to receive data calls. Network Access with PPP (LAN) profile can be used in the following situations: LAN access for a single Bluetooth device; LAN access for multiple Bluetooth devices; PC to PC (using PPP networking over serial cable emulation). In &os; both profiles are implemented with &man.ppp.8; and &man.rfcomm.pppd.8; - a wrapper that converts RFCOMM Bluetooth connection into something PPP can operate with. Before any profile can be used, a new PPP label in the /etc/ppp/ppp.conf must be created. Consult &man.rfcomm.pppd.8; manual page for examples. In the following example &man.rfcomm.pppd.8; will be used to open RFCOMM connection to remote device with BD_ADDR 00:80:37:29:19:a4 on DUN RFCOMM channel. The actual RFCOMM channel number will be obtained from the remote device via SDP. It is possible to specify RFCOMM channel by hand, and in this case &man.rfcomm.pppd.8; will not perform SDP query. Use &man.sdpcontrol.8; to find out RFCOMM channel on the remote device. &prompt.root; rfcomm_pppd -a 00:80:37:29:19:a4 -c -C dun -l rfcomm-dialup In order to provide Network Access with PPP (LAN) service the &man.sdpd.8; server must be running. A new entry for LAN clients must be created in the /etc/ppp/ppp.conf file. Consult &man.rfcomm.pppd.8; manual page for examples. Finally, start RFCOMM PPP server on valid RFCOMM channel number. The RFCOMM PPP server will automatically register Bluetooth LAN service with the local SDP daemon. The example below shows how to start RFCOMM PPP server. &prompt.root; rfcomm_pppd -s -C 7 -l rfcomm-server OBEX OBEX Object Push (OPUSH) Profile OBEX is a widely used protocol for simple file transfers between mobile devices. Its main use is in infrared communication, where it is used for generic file transfers between notebooks or PDAs, and for sending business cards or calendar entries between cellular phones and other devices with PIM applications. The OBEX server and client are implemented as a third-party package obexapp, which is available as comms/obexapp port. OBEX client is used to push and/or pull objects from the OBEX server. An object can, for example, be a business card or an appointment. The OBEX client can obtain RFCOMM channel number from the remote device via SDP. This can be done by specifying service name instead of RFCOMM channel number. Supported service names are: IrMC, FTRN and OPUSH. It is possible to specify RFCOMM channel as a number. Below is an example of an OBEX session, where device information object is pulled from the cellular phone, and a new object (business card) is pushed into the phone's directory. &prompt.user; obexapp -a 00:80:37:29:19:a4 -C IrMC obex> get telecom/devinfo.txt devinfo-t39.txt Success, response: OK, Success (0x20) obex> put new.vcf Success, response: OK, Success (0x20) obex> di Success, response: OK, Success (0x20) In order to provide OBEX Object Push service, &man.sdpd.8; server must be running. A root folder, where all incoming objects will be stored, must be created. The default path to the root folder is /var/spool/obex. Finally, start OBEX server on valid RFCOMM channel number. The OBEX server will automatically register OBEX Object Push service with the local SDP daemon. The example below shows how to start OBEX server. &prompt.root; obexapp -s -C 10 Serial Port Profile (SPP) The Serial Port Profile (SPP) allows Bluetooth devices to perform RS232 (or similar) serial cable emulation. The scenario covered by this profile deals with legacy applications using Bluetooth as a cable replacement, through a virtual serial port abstraction. The &man.rfcomm.sppd.1; utility implements the Serial Port profile. A pseudo tty is used as a virtual serial port abstraction. The example below shows how to connect to a remote device Serial Port service. Note that you do not have to specify a RFCOMM channel - &man.rfcomm.sppd.1; can obtain it from the remote device via SDP. If you would like to override this, specify a RFCOMM channel on the command line. &prompt.root; rfcomm_sppd -a 00:07:E0:00:0B:CA -t /dev/ttyp6 rfcomm_sppd[94692]: Starting on /dev/ttyp6... Once connected, the pseudo tty can be used as serial port: &prompt.root; cu -l ttyp6 Troubleshooting A remote device cannot connect Some older Bluetooth devices do not support role switching. By default, when &os; is accepting a new connection, it tries to perform a role switch and become master. Devices, which do not support this will not be able to connect. Note that role switching is performed when a new connection is being established, so it is not possible to ask the remote device if it does support role switching. There is a HCI option to disable role switching on the local side: &prompt.root; hccontrol -n ubt0hci write_node_role_switch 0 Something is going wrong, can I see what exactly is happening? Yes, you can. Use the third-party package hcidump, which is available as comms/hcidump port. The hcidump utility is similar to &man.tcpdump.1;. It can be used to display the content of the Bluetooth packets on the terminal and to dump the Bluetooth packets to a file. Andrew Thompson Written by Bridging Introduction IP subnet bridge It is sometimes useful to divide one physical network (such as an Ethernet segment) into two separate network segments without having to create IP subnets and use a router to connect the segments together. A device that connects two networks together in this fashion is called a bridge. A FreeBSD system with two network interface cards can act as a bridge. The bridge works by learning the MAC layer addresses (Ethernet addresses) of the devices on each of its network interfaces. It forwards traffic between two networks only when its source and destination are on different networks. In many respects, a bridge is like an Ethernet switch with very few ports. Situations Where Bridging Is Appropriate There are many common situations in which a bridge is used today. Connecting Networks The basic operation of a bridge is to join two or more network segments together. There are many reasons to use a host based bridge over plain networking equipment such as cabling constraints, firewalling or connecting pseudo networks such as a Virtual Machine interface. A bridge can also connect a wireless interface running in hostap mode to a wired network and act as an access point. Filtering/Traffic Shaping Firewall firewall NAT A common situation is where firewall functionality is needed without routing or network address translation (NAT). An example is a small company that is connected via DSL or ISDN to their ISP. They have a 13 globally-accessible IP addresses from their ISP and have 10 PCs on their network. In this situation, using a router-based firewall is difficult because of subnetting issues. router DSL ISDN A bridge-based firewall can be configured and dropped into the path just downstream of their DSL/ISDN router without any IP numbering issues. Network Tap A bridge can join two network segments and be used to inspect all Ethernet frames that pass between them. This can either be from using &man.bpf.4;/&man.tcpdump.1; on the bridge interface or by sending a copy of all frames out an additional interface (span port). Layer 2 VPN Two Ethernet networks can be joined across an IP link by bridging the networks to an EtherIP tunnel or a &man.tap.4; based solution such as OpenVPN. Layer 2 Redundancy A network can be connected together with multiple links and use the Spanning Tree Protocol to block redundant paths. For an Ethernet network to function properly only one active path can exist between two devices, Spanning Tree will detect loops and put the redundant links into a blocked state. Should one of the active links fail then the protocol will calculate a different tree and reenable one of the blocked paths to restore connectivity to all points in the network. Kernel Configuration This section covers &man.if.bridge.4; bridge implementation, a netgraph bridging driver is also available, for more information see &man.ng.bridge.4; manual page. The bridge driver is a kernel module and will be automatically loaded by &man.ifconfig.8; when creating a bridge interface. It is possible to compile the bridge in to the kernel by adding device if_bridge to your kernel configuration file. Packet filtering can be used with any firewall package that hooks in via the &man.pfil.9; framework. The firewall can be loaded as a module or compiled into the kernel. The bridge can be used as a traffic shaper with &man.altq.4; or &man.dummynet.4;. Enabling the Bridge The bridge is created using interface cloning. To create a bridge use &man.ifconfig.8;, if the bridge driver is not present in the kernel then it will be loaded automatically. &prompt.root; ifconfig bridge create bridge0 &prompt.root; ifconfig bridge0 bridge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 96:3d:4b:f1:79:7a id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0 A bridge interface is created and is automatically assigned a randomly generated Ethernet address. The maxaddr and timeout parameters control how many MAC addresses the bridge will keep in its forwarding table and how many seconds before each entry is removed after it is last seen. The other parameters control how Spanning Tree operates. Add the member network interfaces to the bridge. For the bridge to forward packets all member interfaces and the bridge need to be up: &prompt.root; ifconfig bridge0 addm fxp0 addm fxp1 up &prompt.root; ifconfig fxp0 up &prompt.root; ifconfig fxp1 up The bridge is now forwarding Ethernet frames between fxp0 and fxp1. The equivalent configuration in /etc/rc.conf so the bridge is created at startup is: cloned_interfaces="bridge0" ifconfig_bridge0="addm fxp0 addm fxp1 up" ifconfig_fxp0="up" ifconfig_fxp1="up" If the bridge host needs an IP address then the correct place to set this is on the bridge interface itself rather than one of the member interfaces. This can be set statically or via DHCP: &prompt.root; ifconfig bridge0 inet 192.168.0.1/24 It is also possible to assign an IPv6 address to a bridge interface. Firewalling firewall When packet filtering is enabled, bridged packets will pass through the filter inbound on the originating interface, on the bridge interface and outbound on the appropriate interfaces. Either stage can be disabled. When direction of the packet flow is important it is best to firewall on the member interfaces rather than the bridge itself. The bridge has several configurable settings for passing non-IP and ARP packets, and layer2 firewalling with IPFW. See &man.if.bridge.4; for more information. Spanning Tree The bridge driver implements the Rapid Spanning Tree Protocol (RSTP or 802.1w) with backwards compatibility with the legacy Spanning Tree Protocol (STP). Spanning Tree is used to detect and remove loops in a network topology. RSTP provides faster Spanning Tree convergence than legacy STP, the protocol will exchange information with neighbouring switches to quickly transition to forwarding without creating loops. The following table shows the supported operating modes: OS Version STP Modes Default Mode &os; 5.4—&os; 6.2 STP STP &os; 6.3+ RSTP or STP STP &os; 7.0+ RSTP or STP RSTP Spanning Tree can be enabled on member interfaces using the stp command. For a bridge with fxp0 and fxp1 as the current interfaces, enable STP with the following: &prompt.root; ifconfig bridge0 stp fxp0 stp fxp1 bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether d6:cf:d5:a0:94:6d id 00:01:02:4b:d4:50 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:01:02:4b:d4:50 priority 32768 ifcost 0 port 0 member: fxp0 flags=1c7<LEARNING,DISCOVER,STP,AUTOEDGE,PTP,AUTOPTP> port 3 priority 128 path cost 200000 proto rstp role designated state forwarding member: fxp1 flags=1c7<LEARNING,DISCOVER,STP,AUTOEDGE,PTP,AUTOPTP> port 4 priority 128 path cost 200000 proto rstp role designated state forwarding This bridge has a spanning tree ID of 00:01:02:4b:d4:50 and a priority of 32768. As the root id is the same it indicates that this is the root bridge for the tree. Another bridge on the network also has spanning tree enabled: bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 96:3d:4b:f1:79:7a id 00:13:d4:9a:06:7a priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:01:02:4b:d4:50 priority 32768 ifcost 400000 port 4 member: fxp0 flags=1c7<LEARNING,DISCOVER,STP,AUTOEDGE,PTP,AUTOPTP> port 4 priority 128 path cost 200000 proto rstp role root state forwarding member: fxp1 flags=1c7<LEARNING,DISCOVER,STP,AUTOEDGE,PTP,AUTOPTP> port 5 priority 128 path cost 200000 proto rstp role designated state forwarding The line root id 00:01:02:4b:d4:50 priority 32768 ifcost 400000 port 4 shows that the root bridge is 00:01:02:4b:d4:50 as above and has a path cost of 400000 from this bridge, the path to the root bridge is via port 4 which is fxp0. Advanced Bridging Reconstruct Traffic Flows The bridge supports monitor mode, where the packets are discarded after &man.bpf.4; processing, and are not processed or forwarded further. This can be used to multiplex the input of two or more interfaces into a single &man.bpf.4; stream. This is useful for reconstructing the traffic for network taps that transmit the RX/TX signals out through two separate interfaces. To read the input from four network interfaces as one stream: &prompt.root; ifconfig bridge0 addm fxp0 addm fxp1 addm fxp2 addm fxp3 monitor up &prompt.root; tcpdump -i bridge0 Span Ports A copy of every Ethernet frame received by the bridge will be transmitted out a designated span port. The number of span ports configured on a bridge is unlimited, if an interface is designated as a span port then it may not also be used as a regular bridge port. This is most useful for snooping a bridged network passively on another host connected to one of the span ports of the bridge. To send a copy of all frames out the interface named fxp4: &prompt.root; ifconfig bridge0 span fxp4 Private Interfaces A private interface does not forward any traffic to any other port that is also a private interface. The traffic is blocked unconditionally so no Ethernet frames will be forwarded, including ARP. If traffic needs to be selectively blocked then a firewall should be used instead. Sticky Interfaces If a bridge member interface is marked as sticky then dynamically learned address entries are treated at static once entered into the forwarding cache. Sticky entries are never aged out of the cache or replaced, even if the address is seen on a different interface. This gives the benefit of static address entries without the need to pre-populate the forwarding table, clients learnt on a particular segment of the bridge can not roam to another segment. Another example of using sticky addresses would be to combine the bridge with VLANs to create a router where customer networks are isolated without wasting IP address space. Consider that CustomerA is on vlan100 and CustomerB is on vlan101. The bridge has the address 192.168.0.1 and is also an internet router. &prompt.root; ifconfig bridge0 addm vlan100 sticky vlan100 addm vlan101 sticky vlan101 &prompt.root; ifconfig bridge0 inet 192.168.0.1/24 Both clients see 192.168.0.1 as their default gateway and since the bridge cache is sticky they can not spoof the MAC address of the other customer to intercept their traffic. Any communication between the VLANs can be blocked using private interfaces (or a firewall): &prompt.root; ifconfig bridge0 private vlan100 private vlan101 The customers are completely isolated from each other, the full /24 address range can be allocated without subnetting. Address limits The number of unique source MAC addresses behind an interface can limited. Once the limit is reached packets with unknown source addresses are dropped until an existing host cache entry expires or is removed. The following example sets the maximum number of Ethernet devices for CustomerA on vlan100 to 10. &prompt.root; ifconfig bridge0 ifmaxaddr vlan100 10 SNMP Monitoring The bridge interface and STP parameters can be monitored via the SNMP daemon which is included in the &os; base system. The exported bridge MIBs conform to the IETF standards so any SNMP client or monitoring package can be used to retrieve the data. On the bridge machine uncomment the begemotSnmpdModulePath."bridge" = "/usr/lib/snmp_bridge.so" line from /etc/snmp.config and start the bsnmpd daemon. Other configuration such as community names and access lists may need to be modified. See &man.bsnmpd.1; and &man.snmp.bridge.3; for more information. The following examples use the Net-SNMP software (net-mgmt/net-snmp) to query a bridge, the net-mgmt/bsnmptools port can also be used. From the SNMP client host add to $HOME/.snmp/snmp.conf the following lines to import the bridge MIB definitions in to Net-SNMP: mibdirs +/usr/share/snmp/mibs mibs +BRIDGE-MIB:RSTP-MIB:BEGEMOT-MIB:BEGEMOT-BRIDGE-MIB To monitor a single bridge via the IETF BRIDGE-MIB (RFC4188) do &prompt.user; snmpwalk -v 2c -c public bridge1.example.com mib-2.dot1dBridge BRIDGE-MIB::dot1dBaseBridgeAddress.0 = STRING: 66:fb:9b:6e:5c:44 BRIDGE-MIB::dot1dBaseNumPorts.0 = INTEGER: 1 ports BRIDGE-MIB::dot1dStpTimeSinceTopologyChange.0 = Timeticks: (189959) 0:31:39.59 centi-seconds BRIDGE-MIB::dot1dStpTopChanges.0 = Counter32: 2 BRIDGE-MIB::dot1dStpDesignatedRoot.0 = Hex-STRING: 80 00 00 01 02 4B D4 50 ... BRIDGE-MIB::dot1dStpPortState.3 = INTEGER: forwarding(5) BRIDGE-MIB::dot1dStpPortEnable.3 = INTEGER: enabled(1) BRIDGE-MIB::dot1dStpPortPathCost.3 = INTEGER: 200000 BRIDGE-MIB::dot1dStpPortDesignatedRoot.3 = Hex-STRING: 80 00 00 01 02 4B D4 50 BRIDGE-MIB::dot1dStpPortDesignatedCost.3 = INTEGER: 0 BRIDGE-MIB::dot1dStpPortDesignatedBridge.3 = Hex-STRING: 80 00 00 01 02 4B D4 50 BRIDGE-MIB::dot1dStpPortDesignatedPort.3 = Hex-STRING: 03 80 BRIDGE-MIB::dot1dStpPortForwardTransitions.3 = Counter32: 1 RSTP-MIB::dot1dStpVersion.0 = INTEGER: rstp(2) The dot1dStpTopChanges.0 value is two which means that the STP bridge topology has changed twice, a topology change means that one or more links in the network have changed or failed and a new tree has been calculated. The dot1dStpTimeSinceTopologyChange.0 value will show when this happened. To monitor multiple bridge interfaces one may use the private BEGEMOT-BRIDGE-MIB: &prompt.user; snmpwalk -v 2c -c public bridge1.example.com enterprises.fokus.begemot.begemotBridge BEGEMOT-BRIDGE-MIB::begemotBridgeBaseName."bridge0" = STRING: bridge0 BEGEMOT-BRIDGE-MIB::begemotBridgeBaseName."bridge2" = STRING: bridge2 BEGEMOT-BRIDGE-MIB::begemotBridgeBaseAddress."bridge0" = STRING: e:ce:3b:5a:9e:13 BEGEMOT-BRIDGE-MIB::begemotBridgeBaseAddress."bridge2" = STRING: 12:5e:4d:74:d:fc BEGEMOT-BRIDGE-MIB::begemotBridgeBaseNumPorts."bridge0" = INTEGER: 1 BEGEMOT-BRIDGE-MIB::begemotBridgeBaseNumPorts."bridge2" = INTEGER: 1 ... BEGEMOT-BRIDGE-MIB::begemotBridgeStpTimeSinceTopologyChange."bridge0" = Timeticks: (116927) 0:19:29.27 centi-seconds BEGEMOT-BRIDGE-MIB::begemotBridgeStpTimeSinceTopologyChange."bridge2" = Timeticks: (82773) 0:13:47.73 centi-seconds BEGEMOT-BRIDGE-MIB::begemotBridgeStpTopChanges."bridge0" = Counter32: 1 BEGEMOT-BRIDGE-MIB::begemotBridgeStpTopChanges."bridge2" = Counter32: 1 BEGEMOT-BRIDGE-MIB::begemotBridgeStpDesignatedRoot."bridge0" = Hex-STRING: 80 00 00 40 95 30 5E 31 BEGEMOT-BRIDGE-MIB::begemotBridgeStpDesignatedRoot."bridge2" = Hex-STRING: 80 00 00 50 8B B8 C6 A9 To change the bridge interface being monitored via the mib-2.dot1dBridge subtree do: &prompt.user; snmpset -v 2c -c private bridge1.example.com BEGEMOT-BRIDGE-MIB::begemotBridgeDefaultBridgeIf.0 s bridge2 Andrew Thompson Written by Link Aggregation and Failover lagg failover fec lacp loadbalance roundrobin Introduction The &man.lagg.4; interface allows aggregation of multiple network interfaces as one virtual interface for the purpose of providing fault-tolerance and high-speed links. Operating Modes failover Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port; any interfaces added after that are used as failover devices. fec Supports Cisco EtherChannel. This is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link, if the switch supports LACP then that should be used instead. Balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address. lacp Supports the IEEE 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups. Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports. In the event of changes in physical connectivity, Link Aggregation will quickly converge to a new configuration. Balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address. loadbalance This is an alias of fec mode. roundrobin Distributes outgoing traffic using a round-robin scheduler through all active ports and accepts incoming traffic from any active port. This mode will violate Ethernet frame ordering and should be used with caution. Examples LACP aggregation with a Cisco switch This example connects two interfaces on a &os; machine to the switch as a single load balanced and fault tolerant link. More interfaces can be added to increase throughput and fault tolerance. Since frame ordering is mandatory on Ethernet links then any traffic between two stations always flows over the same physical link limiting the maximum speed to that of one interface. The transmit algorithm attempts to use as much information as it can to distinguish different traffic flows and balance across the available interfaces. On the Cisco switch add the interfaces to the channel group. interface FastEthernet0/1 channel-group 1 mode active channel-protocol lacp ! interface FastEthernet0/2 channel-group 1 mode active channel-protocol lacp ! On the &os; machine create the lagg interface. &prompt.root; ifconfig lagg0 create &prompt.root; ifconfig lagg0 up laggproto lacp laggport fxp0 laggport fxp1 View the interface status from ifconfig; ports marked as ACTIVE are part of the active aggregation group that has been negotiated with the remote switch and traffic will be transmitted and received. Use the verbose output of &man.ifconfig.8; to view the LAG identifiers. lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 00:05:5d:71:8d:b8 media: Ethernet autoselect status: active laggproto lacp laggport: fxp1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport: fxp0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> The switch will show which ports are active. For more detail use show lacp neighbor detail. switch# show lacp neighbor Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 neighbors Partner's information: LACP port Oper Port Port Port Flags Priority Dev ID Age Key Number State Fa0/1 SA 32768 0005.5d71.8db8 29s 0x146 0x3 0x3D Fa0/2 SA 32768 0005.5d71.8db8 29s 0x146 0x4 0x3D Failover mode Failover mode can be used to switch over to another interface if the link is lost on the master. &prompt.root; ifconfig lagg0 create &prompt.root; ifconfig lagg0 up laggproto failover laggport fxp0 laggport fxp1 lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 00:05:5d:71:8d:b8 media: Ethernet autoselect status: active laggproto failover laggport: fxp1 flags=0<> laggport: fxp0 flags=5<MASTER,ACTIVE> Traffic will be transmitted and received on fxp0. If the link is lost on fxp0 then fxp1 will become the active link. If the link is restored on the master interface then it will once again become the active link. Jean-François Dockès Updated by Alex Dupre Reorganized and enhanced by Diskless Operation diskless workstation diskless operation A FreeBSD machine can boot over the network and operate without a local disk, using file systems mounted from an NFS server. No system modification is necessary, beyond standard configuration files. Such a system is relatively easy to set up because all the necessary elements are readily available: There are at least two possible methods to load the kernel over the network: PXE: The &intel; Preboot eXecution Environment system is a form of smart boot ROM built into some networking cards or motherboards. See &man.pxeboot.8; for more details. The Etherboot port (net/etherboot) produces ROM-able code to boot kernels over the network. The code can be either burnt into a boot PROM on a network card, or loaded from a local floppy (or hard) disk drive, or from a running &ms-dos; system. Many network cards are supported. A sample script (/usr/share/examples/diskless/clone_root) eases the creation and maintenance of the workstation's root file system on the server. The script will probably require a little customization but it will get you started very quickly. Standard system startup files exist in /etc to detect and support a diskless system startup. Swapping, if needed, can be done either to an NFS file or to a local disk. There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local taste. The following will describe variations on the setup of a complete system, emphasizing simplicity and compatibility with the standard FreeBSD startup scripts. The system described has the following characteristics: The diskless workstations use a shared read-only / file system, and a shared read-only /usr. The root file system is a copy of a standard FreeBSD root (typically the server's), with some configuration files overridden by ones specific to diskless operation or, possibly, to the workstation they belong to. The parts of the root which have to be writable are overlaid with &man.md.4; file systems. Any changes will be lost when the system reboots. The kernel is transferred and loaded either with Etherboot or PXE as some situations may mandate the use of either method. As described, this system is insecure. It should live in a protected area of a network, and be untrusted by other hosts. All the information in this section has been tested using &os; 5.2.1-RELEASE. Background Information Setting up diskless workstations is both relatively straightforward and prone to errors. These are sometimes difficult to diagnose for a number of reasons. For example: Compile time options may determine different behaviors at runtime. Error messages are often cryptic or totally absent. In this context, having some knowledge of the background mechanisms involved is very useful to solve the problems that may arise. Several operations need to be performed for a successful bootstrap: The machine needs to obtain initial parameters such as its IP address, executable filename, server name, root path. This is done using the DHCP or BOOTP protocols. DHCP is a compatible extension of BOOTP, and uses the same port numbers and basic packet format. It is possible to configure a system to use only BOOTP. The &man.bootpd.8; server program is included in the base &os; system. However, DHCP has a number of advantages over BOOTP (nicer configuration files, possibility of using PXE, plus many others not directly related to diskless operation), and we will describe mainly a DHCP configuration, with equivalent examples using &man.bootpd.8; when possible. The sample configuration will use the ISC DHCP software package (release 3.0.1.r12 was installed on the test server). The machine needs to transfer one or several programs to local memory. Either TFTP or NFS are used. The choice between TFTP and NFS is a compile time option in several places. A common source of error is to specify filenames for the wrong protocol: TFTP typically transfers all files from a single directory on the server, and would expect filenames relative to this directory. NFS needs absolute file paths. The possible intermediate bootstrap programs and the kernel need to be initialized and executed. There are several important variations in this area: PXE will load &man.pxeboot.8;, which is a modified version of the &os; third stage loader. The &man.loader.8; will obtain most parameters necessary to system startup, and leave them in the kernel environment before transferring control. It is possible to use a GENERIC kernel in this case. Etherboot, will directly load the kernel, with less preparation. You will need to build a kernel with specific options. PXE and Etherboot work equally well; however, because kernels normally let the &man.loader.8; do more work for them, PXE is the preferred method. If your BIOS and network cards support PXE, you should probably use it. Finally, the machine needs to access its file systems. NFS is used in all cases. See also &man.diskless.8; manual page. Setup Instructions Configuration Using <application>ISC DHCP</application> DHCP diskless operation The ISC DHCP server can answer both BOOTP and DHCP requests. ISC DHCP 3.0 is not part of the base system. You will first need to install the net/isc-dhcp3-server port or the corresponding package. Once ISC DHCP is installed, it needs a configuration file to run (normally named /usr/local/etc/dhcpd.conf). Here follows a commented example, where host margaux uses Etherboot and host corbieres uses PXE: default-lease-time 600; max-lease-time 7200; authoritative; option domain-name "example.com"; option domain-name-servers 192.168.4.1; option routers 192.168.4.1; subnet 192.168.4.0 netmask 255.255.255.0 { use-host-decl-names on; option subnet-mask 255.255.255.0; option broadcast-address 192.168.4.255; host margaux { hardware ethernet 01:23:45:67:89:ab; fixed-address margaux.example.com; next-server 192.168.4.4; filename "/data/misc/kernel.diskless"; option root-path "192.168.4.4:/data/misc/diskless"; } host corbieres { hardware ethernet 00:02:b3:27:62:df; fixed-address corbieres.example.com; next-server 192.168.4.4; filename "pxeboot"; option root-path "192.168.4.4:/data/misc/diskless"; } } This option tells dhcpd to send the value in the host declarations as the hostname for the diskless host. An alternate way would be to add an option host-name margaux inside the host declarations. The next-server directive designates the TFTP or NFS server to use for loading loader or kernel file (the default is to use the same host as the DHCP server). The filename directive defines the file that Etherboot or PXE will load for the next execution step. It must be specified according to the transfer method used. Etherboot can be compiled to use NFS or TFTP. The &os; port configures NFS by default. PXE uses TFTP, which is why a relative filename is used here (this may depend on the TFTP server configuration, but would be fairly typical). Also, PXE loads pxeboot, not the kernel. There are other interesting possibilities, like loading pxeboot from a &os; CD-ROM - /boot directory (as + /boot directory (as &man.pxeboot.8; can load a GENERIC kernel, this makes it possible to use PXE to boot from a remote CD-ROM). The root-path option defines the path to the root file system, in usual NFS notation. When using PXE, it is possible to leave off the host's IP as long as you do not enable the kernel option BOOTP. The NFS server will then be the same as the TFTP one. Configuration Using BOOTP BOOTP diskless operation Here follows an equivalent bootpd configuration (reduced to one client). This would be found in /etc/bootptab. Please note that Etherboot must be compiled with the non-default option NO_DHCP_SUPPORT in order to use BOOTP, and that PXE needs DHCP. The only obvious advantage of bootpd is that it exists in the base system. .def100:\ :hn:ht=1:sa=192.168.4.4:vm=rfc1048:\ :sm=255.255.255.0:\ :ds=192.168.4.1:\ :gw=192.168.4.1:\ :hd="/tftpboot":\ :bf="/kernel.diskless":\ :rp="192.168.4.4:/data/misc/diskless": margaux:ha=0123456789ab:tc=.def100 Preparing a Boot Program with <application>Etherboot</application> Etherboot Etherboot's Web site contains extensive documentation mainly intended for Linux systems, but nonetheless containing useful information. The following will just outline how you would use Etherboot on a FreeBSD system. You must first install the net/etherboot package or port. You can change the Etherboot configuration (i.e. to use TFTP instead of NFS) by editing the Config file in the Etherboot source directory. For our setup, we shall use a boot floppy. For other methods (PROM, or &ms-dos; program), please refer to the Etherboot documentation. To make a boot floppy, insert a floppy in the drive on the machine where you installed Etherboot, then change your current directory to the src directory in the Etherboot tree and type: &prompt.root; gmake bin32/devicetype.fd0 devicetype depends on the type of the Ethernet card in the diskless workstation. Refer to the NIC file in the same directory to determine the right devicetype. Booting with <acronym>PXE</acronym> By default, the &man.pxeboot.8; loader loads the kernel via NFS. It can be compiled to use TFTP instead by specifying the LOADER_TFTP_SUPPORT option in /etc/make.conf. See the comments in /usr/share/examples/etc/make.conf for instructions. There are two other make.conf options which may be useful for setting up a serial console diskless machine: BOOT_PXELDR_PROBE_KEYBOARD, and BOOT_PXELDR_ALWAYS_SERIAL. To use PXE when the machine starts, you will usually need to select the Boot from network option in your BIOS setup, or type a function key during the PC initialization. Configuring the <acronym>TFTP</acronym> and <acronym>NFS</acronym> Servers TFTP diskless operation NFS diskless operation If you are using PXE or Etherboot configured to use TFTP, you need to enable tftpd on the file server: Create a directory from which tftpd will serve the files, e.g. /tftpboot. Add this line to your /etc/inetd.conf: tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /tftpboot It appears that at least some PXE versions want the TCP version of TFTP. In this case, add a second line, replacing dgram udp with stream tcp. Tell inetd to reread its configuration file. The must be in the /etc/rc.conf file for this command to execute correctly: &prompt.root; /etc/rc.d/inetd restart You can place the tftpboot directory anywhere on the server. Make sure that the location is set in both inetd.conf and dhcpd.conf. In all cases, you also need to enable NFS and export the appropriate file system on the NFS server. Add this to /etc/rc.conf: nfs_server_enable="YES" Export the file system where the diskless root directory is located by adding the following to /etc/exports (adjust the volume mount point and replace margaux corbieres with the names of the diskless workstations): /data/misc -alldirs -ro margaux corbieres Tell mountd to reread its configuration file. If you actually needed to enable NFS in /etc/rc.conf at the first step, you probably want to reboot instead. &prompt.root; /etc/rc.d/mountd restart Building a Diskless Kernel diskless operation kernel configuration If using Etherboot, you need to create a kernel configuration file for the diskless client with the following options (in addition to the usual ones): options BOOTP # Use BOOTP to obtain IP address/hostname options BOOTP_NFSROOT # NFS mount root file system using BOOTP info You may also want to use BOOTP_NFSV3, BOOT_COMPAT and BOOTP_WIRED_TO (refer to NOTES). These option names are historical and slightly misleading as they actually enable indifferent use of DHCP and BOOTP inside the kernel (it is also possible to force strict BOOTP or DHCP use). Build the kernel (see ), and copy it to the place specified in dhcpd.conf. When using PXE, building a kernel with the above options is not strictly necessary (though suggested). Enabling them will cause more DHCP requests to be issued during kernel startup, with a small risk of inconsistency between the new values and those retrieved by &man.pxeboot.8; in some special cases. The advantage of using them is that the host name will be set as a side effect. Otherwise you will need to set the host name by another method, for example in a client-specific rc.conf file. In order to be loadable with Etherboot, a kernel needs to have the device hints compiled in. You would typically set the following option in the configuration file (see the NOTES configuration comments file): hints "GENERIC.hints" Preparing the Root Filesystem root file system diskless operation You need to create a root file system for the diskless workstations, in the location listed as root-path in dhcpd.conf. Using <command>make world</command> to populate root This method is quick and will install a complete virgin system (not only the root file system) into DESTDIR. All you have to do is simply execute the following script: #!/bin/sh export DESTDIR=/data/misc/diskless mkdir -p ${DESTDIR} cd /usr/src; make buildworld && make buildkernel cd /usr/src/etc; make distribution Once done, you may need to customize your /etc/rc.conf and /etc/fstab placed into DESTDIR according to your needs. Configuring Swap If needed, a swap file located on the server can be accessed via NFS. <acronym>NFS</acronym> Swap The kernel does not support enabling NFS swap at boot time. Swap must be enabled by the startup scripts, by mounting a writable file system and creating and enabling a swap file. To create a swap file of appropriate size, you can do like this: &prompt.root; dd if=/dev/zero of=/path/to/swapfile bs=1k count=1 oseek=100000 To enable it you have to add the following line to your rc.conf: swapfile=/path/to/swapfile Miscellaneous Issues Running with a Read-only <filename>/usr</filename> diskless operation /usr read-only If the diskless workstation is configured to run X, you will have to adjust the XDM configuration file, which puts the error log on /usr by default. Using a Non-FreeBSD Server When the server for the root file system is not running FreeBSD, you will have to create the root file system on a FreeBSD machine, then copy it to its destination, using tar or cpio. In this situation, there are sometimes problems with the special files in /dev, due to differing major/minor integer sizes. A solution to this problem is to export a directory from the non-FreeBSD server, mount this directory onto a FreeBSD machine, and use &man.devfs.5; to allocate device nodes transparently for the user. ISDN ISDN A good resource for information on ISDN technology and hardware is Dan Kegel's ISDN Page. A quick simple road map to ISDN follows: If you live in Europe you might want to investigate the ISDN card section. If you are planning to use ISDN primarily to connect to the Internet with an Internet Provider on a dial-up non-dedicated basis, you might look into Terminal Adapters. This will give you the most flexibility, with the fewest problems, if you change providers. If you are connecting two LANs together, or connecting to the Internet with a dedicated ISDN connection, you might consider the stand alone router/bridge option. Cost is a significant factor in determining what solution you will choose. The following options are listed from least expensive to most expensive. Hellmuth Michaelis Contributed by ISDN Cards ISDN cards FreeBSD's ISDN implementation supports only the DSS1/Q.931 (or Euro-ISDN) standard using passive cards. Some active cards are supported where the firmware also supports other signaling protocols; this also includes the first supported Primary Rate (PRI) ISDN card. The isdn4bsd software allows you to connect to other ISDN routers using either IP over raw HDLC or by using synchronous PPP: either by using kernel PPP with isppp, a modified &man.sppp.4; driver, or by using userland &man.ppp.8;. By using userland &man.ppp.8;, channel bonding of two or more ISDN B-channels is possible. A telephone answering machine application is also available as well as many utilities such as a software 300 Baud modem. Some growing number of PC ISDN cards are supported under FreeBSD and the reports show that it is successfully used all over Europe and in many other parts of the world. The passive ISDN cards supported are mostly the ones with the Infineon (formerly Siemens) ISAC/HSCX/IPAC ISDN chipsets, but also ISDN cards with chips from Cologne Chip (ISA bus only), PCI cards with Winbond W6692 chips, some cards with the Tiger300/320/ISAC chipset combinations and some vendor specific chipset based cards such as the AVM Fritz!Card PCI V.1.0 and the AVM Fritz!Card PnP. Currently the active supported ISDN cards are the AVM B1 (ISA and PCI) BRI cards and the AVM T1 PCI PRI cards. For documentation on isdn4bsd, have a look at /usr/share/examples/isdn/ directory on your FreeBSD system or at the homepage of isdn4bsd which also has pointers to hints, erratas and much more documentation such as the isdn4bsd handbook. In case you are interested in adding support for a different ISDN protocol, a currently unsupported ISDN PC card or otherwise enhancing isdn4bsd, please get in touch with &a.hm;. For questions regarding the installation, configuration and troubleshooting isdn4bsd, a &a.isdn.name; mailing list is available. ISDN Terminal Adapters Terminal adapters (TA), are to ISDN what modems are to regular phone lines. modem Most TA's use the standard Hayes modem AT command set, and can be used as a drop in replacement for a modem. A TA will operate basically the same as a modem except connection and throughput speeds will be much faster than your old modem. You will need to configure PPP exactly the same as for a modem setup. Make sure you set your serial speed as high as possible. PPP The main advantage of using a TA to connect to an Internet Provider is that you can do Dynamic PPP. As IP address space becomes more and more scarce, most providers are not willing to provide you with a static IP anymore. Most stand-alone routers are not able to accommodate dynamic IP allocation. TA's completely rely on the PPP daemon that you are running for their features and stability of connection. This allows you to upgrade easily from using a modem to ISDN on a FreeBSD machine, if you already have PPP set up. However, at the same time any problems you experienced with the PPP program and are going to persist. If you want maximum stability, use the kernel PPP option, not the userland PPP. The following TA's are known to work with FreeBSD: Motorola BitSurfer and Bitsurfer Pro Adtran Most other TA's will probably work as well, TA vendors try to make sure their product can accept most of the standard modem AT command set. The real problem with external TA's is that, like modems, you need a good serial card in your computer. You should read the FreeBSD Serial Hardware tutorial for a detailed understanding of serial devices, and the differences between asynchronous and synchronous serial ports. A TA running off a standard PC serial port (asynchronous) limits you to 115.2 Kbs, even though you have a 128 Kbs connection. To fully utilize the 128 Kbs that ISDN is capable of, you must move the TA to a synchronous serial card. Do not be fooled into buying an internal TA and thinking you have avoided the synchronous/asynchronous issue. Internal TA's simply have a standard PC serial port chip built into them. All this will do is save you having to buy another serial cable and find another empty electrical socket. A synchronous card with a TA is at least as fast as a stand-alone router, and with a simple 386 FreeBSD box driving it, probably more flexible. The choice of synchronous card/TA v.s. stand-alone router is largely a religious issue. There has been some discussion of this in the mailing lists. We suggest you search the archives for the complete discussion. Stand-alone ISDN Bridges/Routers ISDN stand-alone bridges/routers ISDN bridges or routers are not at all specific to FreeBSD or any other operating system. For a more complete description of routing and bridging technology, please refer to a networking reference book. In the context of this section, the terms router and bridge will be used interchangeably. As the cost of low end ISDN routers/bridges comes down, it will likely become a more and more popular choice. An ISDN router is a small box that plugs directly into your local Ethernet network, and manages its own connection to the other bridge/router. It has built in software to communicate via PPP and other popular protocols. A router will allow you much faster throughput than a standard TA, since it will be using a full synchronous ISDN connection. The main problem with ISDN routers and bridges is that interoperability between manufacturers can still be a problem. If you are planning to connect to an Internet provider, you should discuss your needs with them. If you are planning to connect two LAN segments together, such as your home LAN to the office LAN, this is the simplest lowest maintenance solution. Since you are buying the equipment for both sides of the connection you can be assured that the link will work. For example to connect a home computer or branch office network to a head office network the following setup could be used: Branch Office or Home Network 10 base 2 Network uses a bus based topology with 10 base 2 Ethernet (thinnet). Connect router to network cable with AUI/10BT transceiver, if necessary. ---Sun workstation | ---FreeBSD box | ---Windows 95 | Stand-alone router | ISDN BRI line 10 Base 2 Ethernet If your home/branch office is only one computer you can use a twisted pair crossover cable to connect to the stand-alone router directly. Head Office or Other LAN 10 base T Network uses a star topology with 10 base T Ethernet (Twisted Pair). -------Novell Server | H | | ---Sun | | | U ---FreeBSD | | | ---Windows 95 | B | |___---Stand-alone router | ISDN BRI line ISDN Network Diagram One large advantage of most routers/bridges is that they allow you to have 2 separate independent PPP connections to 2 separate sites at the same time. This is not supported on most TA's, except for specific (usually expensive) models that have two serial ports. Do not confuse this with channel bonding, MPP, etc. This can be a very useful feature if, for example, you have an dedicated ISDN connection at your office and would like to tap into it, but do not want to get another ISDN line at work. A router at the office location can manage a dedicated B channel connection (64 Kbps) to the Internet and use the other B channel for a separate data connection. The second B channel can be used for dial-in, dial-out or dynamically bonding (MPP, etc.) with the first B channel for more bandwidth. IPX/SPX An Ethernet bridge will also allow you to transmit more than just IP traffic. You can also send IPX/SPX or whatever other protocols you use. Chern Lee Contributed by Network Address Translation Overview natd FreeBSD's Network Address Translation daemon, commonly known as &man.natd.8; is a daemon that accepts incoming raw IP packets, changes the source to the local machine and re-injects these packets back into the outgoing IP packet stream. &man.natd.8; does this by changing the source IP address and port such that when data is received back, it is able to determine the original location of the data and forward it back to its original requester. Internet connection sharing NAT The most common use of NAT is to perform what is commonly known as Internet Connection Sharing. Setup Due to the diminishing IP space in IPv4, and the increased number of users on high-speed consumer lines such as cable or DSL, people are increasingly in need of an Internet Connection Sharing solution. The ability to connect several computers online through one connection and IP address makes &man.natd.8; a reasonable choice. Most commonly, a user has a machine connected to a cable or DSL line with one IP address and wishes to use this one connected computer to provide Internet access to several more over a LAN. To do this, the FreeBSD machine on the Internet must act as a gateway. This gateway machine must have two NICs—one for connecting to the Internet router, the other connecting to a LAN. All the machines on the LAN are connected through a hub or switch. There are many ways to get a LAN connected to the Internet through a &os; gateway. This example will only cover a gateway with at least two NICs. _______ __________ ________ | | | | | | | Hub |-----| Client B |-----| Router |----- Internet |_______| |__________| |________| | ____|_____ | | | Client A | |__________| Network Layout A setup like this is commonly used to share an Internet connection. One of the LAN machines is connected to the Internet. The rest of the machines access the Internet through that gateway machine. kernel configuration Configuration The following options must be in the kernel configuration file: options IPFIREWALL options IPDIVERT Additionally, at choice, the following may also be suitable: options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_VERBOSE The following must be in /etc/rc.conf: gateway_enable="YES" firewall_enable="YES" firewall_type="OPEN" natd_enable="YES" natd_interface="fxp0" natd_flags="" Sets up the machine to act as a gateway. Running sysctl net.inet.ip.forwarding=1 would have the same effect. Enables the firewall rules in /etc/rc.firewall at boot. This specifies a predefined firewall ruleset that allows anything in. See /etc/rc.firewall for additional types. Indicates which interface to forward packets through (the interface connected to the Internet). Any additional configuration options passed to &man.natd.8; on boot. Having the previous options defined in /etc/rc.conf would run natd -interface fxp0 at boot. This can also be run manually. It is also possible to use a configuration file for &man.natd.8; when there are too many options to pass. In this case, the configuration file must be defined by adding the following line to /etc/rc.conf: natd_flags="-f /etc/natd.conf" The /etc/natd.conf file will contain a list of configuration options, one per line. For example the next section case would use the following file: redirect_port tcp 192.168.0.2:6667 6667 redirect_port tcp 192.168.0.3:80 80 For more information about the configuration file, consult the &man.natd.8; manual page about the option. Each machine and interface behind the LAN should be assigned IP address numbers in the private network space as defined by RFC 1918 and have a default gateway of the natd machine's internal IP address. For example, client A and B behind the LAN have IP addresses of 192.168.0.2 and 192.168.0.3, while the natd machine's LAN interface has an IP address of 192.168.0.1. Client A and B's default gateway must be set to that of the natd machine, 192.168.0.1. The natd machine's external, or Internet interface does not require any special modification for &man.natd.8; to work. Port Redirection The drawback with &man.natd.8; is that the LAN clients are not accessible from the Internet. Clients on the LAN can make outgoing connections to the world but cannot receive incoming ones. This presents a problem if trying to run Internet services on one of the LAN client machines. A simple way around this is to redirect selected Internet ports on the natd machine to a LAN client. For example, an IRC server runs on client A, and a web server runs on client B. For this to work properly, connections received on ports 6667 (IRC) and 80 (web) must be redirected to the respective machines. The must be passed to &man.natd.8; with the proper options. The syntax is as follows: -redirect_port proto targetIP:targetPORT[-targetPORT] [aliasIP:]aliasPORT[-aliasPORT] [remoteIP[:remotePORT[-remotePORT]]] In the above example, the argument should be: -redirect_port tcp 192.168.0.2:6667 6667 -redirect_port tcp 192.168.0.3:80 80 This will redirect the proper tcp ports to the LAN client machines. The argument can be used to indicate port ranges over individual ports. For example, tcp 192.168.0.2:2000-3000 2000-3000 would redirect all connections received on ports 2000 to 3000 to ports 2000 to 3000 on client A. These options can be used when directly running &man.natd.8;, placed within the natd_flags="" option in /etc/rc.conf, or passed via a configuration file. For further configuration options, consult &man.natd.8; Address Redirection address redirection Address redirection is useful if several IP addresses are available, yet they must be on one machine. With this, &man.natd.8; can assign each LAN client its own external IP address. &man.natd.8; then rewrites outgoing packets from the LAN clients with the proper external IP address and redirects all traffic incoming on that particular IP address back to the specific LAN client. This is also known as static NAT. For example, the IP addresses 128.1.1.1, 128.1.1.2, and 128.1.1.3 belong to the natd gateway machine. 128.1.1.1 can be used as the natd gateway machine's external IP address, while 128.1.1.2 and 128.1.1.3 are forwarded back to LAN clients A and B. The syntax is as follows: -redirect_address localIP publicIP localIP The internal IP address of the LAN client. publicIP The external IP address corresponding to the LAN client. In the example, this argument would read: -redirect_address 192.168.0.2 128.1.1.2 -redirect_address 192.168.0.3 128.1.1.3 Like , these arguments are also placed within the natd_flags="" option of /etc/rc.conf, or passed via a configuration file. With address redirection, there is no need for port redirection since all data received on a particular IP address is redirected. The external IP addresses on the natd machine must be active and aliased to the external interface. Look at &man.rc.conf.5; to do so. Parallel Line IP (PLIP) PLIP Parallel Line IP PLIP PLIP lets us run TCP/IP between parallel ports. It is useful on machines without network cards, or to install on laptops. In this section, we will discuss: Creating a parallel (laplink) cable. Connecting two computers with PLIP. Creating a Parallel Cable You can purchase a parallel cable at most computer supply stores. If you cannot do that, or you just want to know how it is done, the following table shows how to make one out of a normal parallel printer cable. Wiring a Parallel Cable for Networking A-name A-End B-End Descr. Post/Bit DATA0 -ERROR 2 15 15 2 Data 0/0x01 1/0x08 DATA1 +SLCT 3 13 13 3 Data 0/0x02 1/0x10 DATA2 +PE 4 12 12 4 Data 0/0x04 1/0x20 DATA3 -ACK 5 10 10 5 Strobe 0/0x08 1/0x40 DATA4 BUSY 6 11 11 6 Data 0/0x10 1/0x80 GND 18-25 18-25 GND -
Setting Up PLIP First, you have to get a laplink cable. Then, confirm that both computers have a kernel with &man.lpt.4; driver support: &prompt.root; grep lp /var/run/dmesg.boot lpt0: <Printer> on ppbus0 lpt0: Interrupt-driven port The parallel port must be an interrupt driven port, you should have lines similar to the following in your in the /boot/device.hints file: hint.ppc.0.at="isa" hint.ppc.0.irq="7" Then check if the kernel configuration file has a device plip line or if the plip.ko kernel module is loaded. In both cases the parallel networking interface should appear when you use the &man.ifconfig.8; command to display it: &prompt.root; ifconfig plip0 plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> mtu 1500 Plug the laplink cable into the parallel interface on both computers. Configure the network interface parameters on both sites as root. For example, if you want to connect the host host1 with another machine host2: host1 <-----> host2 IP Address 10.0.0.1 10.0.0.2 Configure the interface on host1 by doing: &prompt.root; ifconfig plip0 10.0.0.1 10.0.0.2 Configure the interface on host2 by doing: &prompt.root; ifconfig plip0 10.0.0.2 10.0.0.1 You now should have a working connection. Please read the manual pages &man.lp.4; and &man.lpt.4; for more details. You should also add both hosts to /etc/hosts: 127.0.0.1 localhost.my.domain localhost 10.0.0.1 host1.my.domain host1 10.0.0.2 host2.my.domain To confirm the connection works, go to each host and ping the other. For example, on host1: &prompt.root; ifconfig plip0 plip0: flags=8851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 10.0.0.1 --> 10.0.0.2 netmask 0xff000000 &prompt.root; netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire host2 host1 UH 0 0 plip0 &prompt.root; ping -c 4 host2 PING host2 (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=255 time=2.774 ms 64 bytes from 10.0.0.2: icmp_seq=1 ttl=255 time=2.530 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=255 time=2.556 ms 64 bytes from 10.0.0.2: icmp_seq=3 ttl=255 time=2.714 ms --- host2 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.530/2.643/2.774/0.103 ms Aaron Kaplan Originally Written by Tom Rhodes Restructured and Added by Brad Davis Extended by IPv6 IPv6 (also known as IPng IP next generation) is the new version of the well known IP protocol (also known as IPv4). Like the other current *BSD systems, FreeBSD includes the KAME IPv6 reference implementation. So your FreeBSD system comes with all you will need to experiment with IPv6. This section focuses on getting IPv6 configured and running. In the early 1990s, people became aware of the rapidly diminishing address space of IPv4. Given the expansion rate of the Internet there were two major concerns: Running out of addresses. Today this is not so much of a concern anymore since RFC1918 private address space (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16) and Network Address Translation (NAT) are being employed. Router table entries were getting too large. This is still a concern today. IPv6 deals with these and many other issues: 128 bit address space. In other words theoretically there are 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses available. This means there are approximately 6.67 * 10^27 IPv6 addresses per square meter on our planet. Routers will only store network aggregation addresses in their routing tables thus reducing the average space of a routing table to 8192 entries. There are also lots of other useful features of IPv6 such as: Address autoconfiguration (RFC2462) Anycast addresses (one-out-of many) Mandatory multicast addresses IPsec (IP security) Simplified header structure Mobile IP IPv6-to-IPv4 transition mechanisms For more information see: IPv6 overview at playground.sun.com KAME.net Background on IPv6 Addresses There are different types of IPv6 addresses: Unicast, Anycast and Multicast. Unicast addresses are the well known addresses. A packet sent to a unicast address arrives exactly at the interface belonging to the address. Anycast addresses are syntactically indistinguishable from unicast addresses but they address a group of interfaces. The packet destined for an anycast address will arrive at the nearest (in router metric) interface. Anycast addresses may only be used by routers. Multicast addresses identify a group of interfaces. A packet destined for a multicast address will arrive at all interfaces belonging to the multicast group. The IPv4 broadcast address (usually xxx.xxx.xxx.255) is expressed by multicast addresses in IPv6. Reserved IPv6 addresses IPv6 address Prefixlength (Bits) Description Notes :: 128 bits unspecified cf. 0.0.0.0 in IPv4 ::1 128 bits loopback address cf. 127.0.0.1 in IPv4 ::00:xx:xx:xx:xx 96 bits embedded IPv4 The lower 32 bits are the IPv4 address. Also called IPv4 compatible IPv6 address ::ff:xx:xx:xx:xx 96 bits IPv4 mapped IPv6 address The lower 32 bits are the IPv4 address. For hosts which do not support IPv6. fe80:: - feb:: 10 bits link-local cf. loopback address in IPv4 fec0:: - fef:: 10 bits site-local   ff:: 8 bits multicast   001 (base 2) 3 bits global unicast All global unicast addresses are assigned from this pool. The first 3 bits are 001.
Reading IPv6 Addresses The canonical form is represented as: x:x:x:x:x:x:x:x, each x being a 16 Bit hex value. For example FEBC:A574:382B:23C1:AA49:4592:4EFE:9982 Often an address will have long substrings of all zeros therefore one such substring per address can be abbreviated by ::. Also up to three leading 0s per hexquad can be omitted. For example fe80::1 corresponds to the canonical form fe80:0000:0000:0000:0000:0000:0000:0001. A third form is to write the last 32 Bit part in the well known (decimal) IPv4 style with dots . as separators. For example 2002::10.0.0.1 corresponds to the (hexadecimal) canonical representation 2002:0000:0000:0000:0000:0000:0a00:0001 which in turn is equivalent to writing 2002::a00:1. By now the reader should be able to understand the following: &prompt.root; ifconfig rl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 inet 10.0.0.10 netmask 0xffffff00 broadcast 10.0.0.255 inet6 fe80::200:21ff:fe03:8e1%rl0 prefixlen 64 scopeid 0x1 ether 00:00:21:03:08:e1 media: Ethernet autoselect (100baseTX ) status: active fe80::200:21ff:fe03:8e1%rl0 is an auto configured link-local address. It is generated from the MAC address as part of the auto configuration. For further information on the structure of IPv6 addresses see RFC3513. Getting Connected Currently there are four ways to connect to other IPv6 hosts and networks: Contact your Internet Service Provider to see if they offer IPv6 yet. SixXS offers tunnels with end-points all around the globe. Tunnel via 6-to-4 (RFC3068) Use the net/freenet6 port if you are on a dial-up connection. DNS in the IPv6 World There used to be two types of DNS records for IPv6. The IETF has declared A6 records obsolete. AAAA records are the standard now. Using AAAA records is straightforward. Assign your hostname to the new IPv6 address you just received by adding: MYHOSTNAME AAAA MYIPv6ADDR To your primary zone DNS file. In case you do not serve your own DNS zones ask your DNS provider. Current versions of bind (version 8.3 and 9) and dns/djbdns (with the IPv6 patch) support AAAA records. Applying the needed changes to <filename>/etc/rc.conf</filename> IPv6 Client Settings These settings will help you configure a machine that will be on your LAN and act as a client, not a router. To have &man.rtsol.8; autoconfigure your interface on boot all you need to add is: ipv6_enable="YES" To statically assign an IP address such as 2001:471:1f11:251:290:27ff:fee0:2093, to your fxp0 interface, add: ipv6_ifconfig_fxp0="2001:471:1f11:251:290:27ff:fee0:2093" To assign a default router of 2001:471:1f11:251::1 add the following to /etc/rc.conf: ipv6_defaultrouter="2001:471:1f11:251::1" IPv6 Router/Gateway Settings This will help you take the directions that your tunnel provider has given you and convert it into settings that will persist through reboots. To restore your tunnel on startup use something like the following in /etc/rc.conf: List the Generic Tunneling interfaces that will be configured, for example gif0: gif_interfaces="gif0" To configure the interface with a local endpoint of MY_IPv4_ADDR to a remote endpoint of REMOTE_IPv4_ADDR: gifconfig_gif0="MY_IPv4_ADDR REMOTE_IPv4_ADDR" To apply the IPv6 address you have been assigned for use as your IPv6 tunnel endpoint, add: ipv6_ifconfig_gif0="MY_ASSIGNED_IPv6_TUNNEL_ENDPOINT_ADDR" Then all you have to do is set the default route for IPv6. This is the other side of the IPv6 tunnel: ipv6_defaultrouter="MY_IPv6_REMOTE_TUNNEL_ENDPOINT_ADDR" IPv6 Tunnel Settings If the server is to route IPv6 between the rest of your network and the world, the following /etc/rc.conf setting will also be needed: ipv6_gateway_enable="YES" Router Advertisement and Host Auto Configuration This section will help you setup &man.rtadvd.8; to advertise the IPv6 default route. To enable &man.rtadvd.8; you will need the following in your /etc/rc.conf: rtadvd_enable="YES" It is important that you specify the interface on which to do IPv6 router solicitation. For example to tell &man.rtadvd.8; to use fxp0: rtadvd_interfaces="fxp0" Now we must create the configuration file, /etc/rtadvd.conf. Here is an example: fxp0:\ :addrs#1:addr="2001:471:1f11:246::":prefixlen#64:tc=ether: Replace fxp0 with the interface you are going to be using. Next, replace 2001:471:1f11:246:: with the prefix of your allocation. If you are dedicated a /64 subnet you will not need to change anything else. Otherwise, you will need to change the prefixlen# to the correct value. Harti Brandt Contributed by Asynchronous Transfer Mode (ATM) Configuring classical IP over ATM (PVCs) Classical IP over ATM (CLIP) is the simplest method to use Asynchronous Transfer Mode (ATM) with IP. It can be used with switched connections (SVCs) and with permanent connections (PVCs). This section describes how to set up a network based on PVCs. Fully meshed configurations The first method to set up a CLIP with PVCs is to connect each machine to each other machine in the network via a dedicated PVC. While this is simple to configure it tends to become impractical for a larger number of machines. The example supposes that we have four machines in the network, each connected to the ATM network with an ATM adapter card. The first step is the planning of the IP addresses and the ATM connections between the machines. We use the following: Host IP Address hostA 192.168.173.1 hostB 192.168.173.2 hostC 192.168.173.3 hostD 192.168.173.4 To build a fully meshed net we need one ATM connection between each pair of machines: Machines VPI.VCI couple hostA - hostB 0.100 hostA - hostC 0.101 hostA - hostD 0.102 hostB - hostC 0.103 hostB - hostD 0.104 hostC - hostD 0.105 The VPI and VCI values at each end of the connection may of course differ, but for simplicity we assume that they are the same. Next we need to configure the ATM interfaces on each host: hostA&prompt.root; ifconfig hatm0 192.168.173.1 up hostB&prompt.root; ifconfig hatm0 192.168.173.2 up hostC&prompt.root; ifconfig hatm0 192.168.173.3 up hostD&prompt.root; ifconfig hatm0 192.168.173.4 up assuming that the ATM interface is hatm0 on all hosts. Now the PVCs need to be configured on hostA (we assume that they are already configured on the ATM switches, you need to consult the manual for the switch on how to do this). hostA&prompt.root; atmconfig natm add 192.168.173.2 hatm0 0 100 llc/snap ubr hostA&prompt.root; atmconfig natm add 192.168.173.3 hatm0 0 101 llc/snap ubr hostA&prompt.root; atmconfig natm add 192.168.173.4 hatm0 0 102 llc/snap ubr hostB&prompt.root; atmconfig natm add 192.168.173.1 hatm0 0 100 llc/snap ubr hostB&prompt.root; atmconfig natm add 192.168.173.3 hatm0 0 103 llc/snap ubr hostB&prompt.root; atmconfig natm add 192.168.173.4 hatm0 0 104 llc/snap ubr hostC&prompt.root; atmconfig natm add 192.168.173.1 hatm0 0 101 llc/snap ubr hostC&prompt.root; atmconfig natm add 192.168.173.2 hatm0 0 103 llc/snap ubr hostC&prompt.root; atmconfig natm add 192.168.173.4 hatm0 0 105 llc/snap ubr hostD&prompt.root; atmconfig natm add 192.168.173.1 hatm0 0 102 llc/snap ubr hostD&prompt.root; atmconfig natm add 192.168.173.2 hatm0 0 104 llc/snap ubr hostD&prompt.root; atmconfig natm add 192.168.173.3 hatm0 0 105 llc/snap ubr Of course other traffic contracts than UBR can be used given the ATM adapter supports those. In this case the name of the traffic contract is followed by the parameters of the traffic. Help for the &man.atmconfig.8; tool can be obtained with: &prompt.root; atmconfig help natm add or in the &man.atmconfig.8; manual page. The same configuration can also be done via /etc/rc.conf. For hostA this would look like: network_interfaces="lo0 hatm0" ifconfig_hatm0="inet 192.168.173.1 up" natm_static_routes="hostB hostC hostD" route_hostB="192.168.173.2 hatm0 0 100 llc/snap ubr" route_hostC="192.168.173.3 hatm0 0 101 llc/snap ubr" route_hostD="192.168.173.4 hatm0 0 102 llc/snap ubr" The current state of all CLIP routes can be obtained with: hostA&prompt.root; atmconfig natm show Tom Rhodes Contributed by Common Access Redundancy Protocol (CARP) CARP Common Access Redundancy Protocol The Common Access Redundancy Protocol, or CARP allows multiple hosts to share the same IP address. In some configurations, this may be used for availability or load balancing. Hosts may use separate IP addresses as well, as in the example provided here. To enable support for CARP, the &os; kernel must be rebuilt with the following option: device carp CARP functionality should now be available and may be tuned via several sysctl OIDs: OID Description net.inet.carp.allow Accept incoming CARP packets. Enabled by default. net.inet.carp.preempt This option downs all of the CARP interfaces on the host when one of them goes down. Disabled by default net.inet.carp.log A value of 0 disables any logging. A Value of 1 enables logging of bad CARP packets. Values greater than 1 enables logging of state changes for the CARP interfaces. The default value is 1. net.inet.carp.arpbalance Balance local network traffic using ARP. Disabled by default. net.inet.carp.suppress_preempt A read only OID showing the status of preemption suppression. Preemption can be suppressed if link on an interface is down. A value of 0, means that preemption is not suppressed. Every problem increments this OID. The CARP devices themselves may be created via the ifconfig command: &prompt.root; ifconfig carp0 create In a real environment, these interfaces will need unique identification numbers known as a VHID. This VHID or Virtual Host Identification will be used to distinguish the host on the network. Using CARP For Server Availability (CARP) One use of CARP, as noted above, is for server availability. This example will provide failover support for three hosts, all with unique IP addresses and providing the same web content. These machines will act in conjunction with a Round Robin DNS configuration. The failover machine will have two additional CARP interfaces, one for each of the content server's IPs. When a failure occurs, the failover server should pick up the failed machine's IP address. This means the failure should go completely unnoticed to the user. The failover server requires identical content and services as the other content servers it is expected to pick up load for. The two machines should be configured identically other than their issued hostnames and VHIDs. This example calls these machines hosta.example.org and hostb.example.org respectively. First, the required lines for a CARP configuration have to be added to rc.conf. For hosta.example.org, the rc.conf file should contain the following lines: hostname="hosta.example.org" ifconfig_fxp0="inet 192.168.1.3 netmask 255.255.255.0" cloned_interfaces="carp0" ifconfig_carp0="vhid 1 pass testpass 192.168.1.50/24" On hostb.example.org the following lines should be in rc.conf: hostname="hostb.example.org" ifconfig_fxp0="inet 192.168.1.4 netmask 255.255.255.0" cloned_interfaces="carp0" ifconfig_carp0="vhid 2 pass testpass 192.168.1.51/24" It is very important that the passwords, specified by the option to ifconfig, are identical. The carp devices will only listen to and accept advertisements from machines with the correct password. The VHID must also be different for each machine. The third machine, provider.example.org, should be prepared so that it may handle failover from either host. This machine will require two carp devices, one to handle each host. The appropriate rc.conf configuration lines will be similar to the following: hostname="provider.example.org" ifconfig_fxp0="inet 192.168.1.5 netmask 255.255.255.0" cloned_interfaces="carp0 carp1" ifconfig_carp0="vhid 1 advskew 100 pass testpass 192.168.1.50/24" ifconfig_carp1="vhid 2 advskew 100 pass testpass 192.168.1.51/24" Having the two carp devices will allow provider.example.org to notice and pick up the IP address of either machine should it stop responding. The default &os; kernel may have preemption enabled. If so, provider.example.org may not relinquish the IP address back to the original content server. In this case, an administrator may have to manually force the IP back to the master. The following command should be issued on provider.example.org: &prompt.root; ifconfig carp0 down && ifconfig carp0 up This should be done on the carp interface which corresponds to the correct host. At this point, CARP should be completely enabled and available for testing. For testing, either networking has to be restarted or the machines need to be rebooted. More information is always available in the &man.carp.4; manual page. diff --git a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml index 92b7a42103..32ac60fbda 100644 --- a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml @@ -1,4135 +1,4135 @@ Storage Synopsis This chapter covers the use of disks in FreeBSD. This includes memory-backed disks, network-attached disks, standard SCSI/IDE storage devices, and devices using the USB interface. After reading this chapter, you will know: The terminology FreeBSD uses to describe the organization of data on a physical disk (partitions and slices). How to add additional hard disks to your system. How to configure &os; to use USB storage devices. How to set up virtual file systems, such as memory disks. How to use quotas to limit disk space usage. How to encrypt disks to secure them against attackers. How to create and burn CDs and DVDs on FreeBSD. The various storage media options for backups. How to use backup programs available under FreeBSD. How to backup to floppy disks. What file system snapshots are and how to use them efficiently. Before reading this chapter, you should: Know how to configure and install a new FreeBSD kernel (). Device Names The following is a list of physical storage devices supported in FreeBSD, and the device names associated with them. Physical Disk Naming Conventions Drive type Drive device name IDE hard drives ad IDE CDROM drives acd SCSI hard drives and USB Mass storage devices da SCSI CDROM drives cd Assorted non-standard CDROM drives mcd for Mitsumi CD-ROM and scd for Sony CD-ROM devices Floppy drives fd SCSI tape drives sa IDE tape drives ast Flash drives fla for &diskonchip; Flash device RAID drives aacd for &adaptec; AdvancedRAID, mlxd and mlyd for &mylex;, amrd for AMI &megaraid;, idad for Compaq Smart RAID, twed for &tm.3ware; RAID.
David O'Brien Originally contributed by Adding Disks disks adding Lets say we want to add a new SCSI disk to a machine that currently only has a single drive. First turn off the computer and install the drive in the computer following the instructions of the computer, controller, and drive manufacturer. Due to the wide variations of procedures to do this, the details are beyond the scope of this document. Login as user root. After you have installed the drive, inspect /var/run/dmesg.boot to ensure the new disk was found. Continuing with our example, the newly added drive will be da1 and we want to mount it on /1 (if you are adding an IDE drive, the device name will be ad1). partitions slices fdisk FreeBSD runs on IBM-PC compatible computers, therefore it must take into account the PC BIOS partitions. These are different from the traditional BSD partitions. A PC disk has up to four BIOS partition entries. If the disk is going to be truly dedicated to FreeBSD, you can use the dedicated mode. Otherwise, FreeBSD will have to live within one of the PC BIOS partitions. FreeBSD calls the PC BIOS partitions slices so as not to confuse them with traditional BSD partitions. You may also use slices on a disk that is dedicated to FreeBSD, but used in a computer that also has another operating system installed. This is a good way to avoid confusing the fdisk utility of other, non-FreeBSD operating systems. In the slice case the drive will be added as /dev/da1s1e. This is read as: SCSI disk, unit number 1 (second SCSI disk), slice 1 (PC BIOS partition 1), and e BSD partition. In the dedicated case, the drive will be added simply as /dev/da1e. Due to the use of 32-bit integers to store the number of sectors, &man.bsdlabel.8; is limited to 2^32-1 sectors per disk or 2TB in most cases. The &man.fdisk.8; format allows a starting sector of no more than 2^32-1 and a length of no more than 2^32-1, limiting partitions to 2TB and disks to 4TB in most cases. The &man.sunlabel.8; format is limited to 2^32-1 sectors per partition and 8 partitions for a total of 16TB. For larger disks, &man.gpt.8; partitions may be used. Using &man.sysinstall.8; sysinstall adding disks su Navigating <application>Sysinstall</application> You may use sysinstall to partition and label a new disk using its easy to use menus. Either login as user root or use the su command. Run sysinstall and enter the Configure menu. Within the FreeBSD Configuration Menu, scroll down and select the Fdisk option. <application>fdisk</application> Partition Editor Once inside fdisk, pressing A will use the entire disk for FreeBSD. When asked if you want to remain cooperative with any future possible operating systems, answer YES. Write the changes to the disk using W. Now exit the FDISK editor by pressing Q. Next you will be asked about the Master Boot Record. Since you are adding a disk to an already running system, choose None. Disk Label Editor BSD partitions Next, you need to exit sysinstall and start it again. Follow the directions above, although this time choose the Label option. This will enter the Disk Label Editor. This is where you will create the traditional BSD partitions. A disk can have up to eight partitions, labeled a-h. A few of the partition labels have special uses. The a partition is used for the root partition (/). Thus only your system disk (e.g, the disk you boot from) should have an a partition. The b partition is used for swap partitions, and you may have many disks with swap partitions. The c partition addresses the entire disk in dedicated mode, or the entire FreeBSD slice in slice mode. The other partitions are for general use. sysinstall's Label editor favors the e partition for non-root, non-swap partitions. Within the Label editor, create a single file system by pressing C. When prompted if this will be a FS (file system) or swap, choose FS and type in a mount point (e.g, /mnt). When adding a disk in post-install mode, sysinstall will not create entries in /etc/fstab for you, so the mount point you specify is not important. You are now ready to write the new label to the disk and create a file system on it. Do this by pressing W. Ignore any errors from sysinstall that it could not mount the new partition. Exit the Label Editor and sysinstall completely. Finish The last step is to edit /etc/fstab to add an entry for your new disk. Using Command Line Utilities Using Slices This setup will allow your disk to work correctly with other operating systems that might be installed on your computer and will not confuse other operating systems' fdisk utilities. It is recommended to use this method for new disk installs. Only use dedicated mode if you have a good reason to do so! &prompt.root; dd if=/dev/zero of=/dev/da1 bs=1k count=1 &prompt.root; fdisk -BI da1 #Initialize your new disk &prompt.root; bsdlabel -B -w da1s1 auto #Label it. &prompt.root; bsdlabel -e da1s1 # Edit the bsdlabel just created and add any partitions. &prompt.root; mkdir -p /1 &prompt.root; newfs /dev/da1s1e # Repeat this for every partition you created. &prompt.root; mount /dev/da1s1e /1 # Mount the partition(s) &prompt.root; vi /etc/fstab # Add the appropriate entry/entries to your /etc/fstab. If you have an IDE disk, substitute ad for da. Dedicated OS/2 If you will not be sharing the new drive with another operating system, you may use the dedicated mode. Remember this mode can confuse Microsoft operating systems; however, no damage will be done by them. IBM's &os2; however, will appropriate any partition it finds which it does not understand. &prompt.root; dd if=/dev/zero of=/dev/da1 bs=1k count=1 &prompt.root; bsdlabel -Bw da1 auto &prompt.root; bsdlabel -e da1 # create the `e' partition &prompt.root; newfs /dev/da1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 An alternate method is: &prompt.root; dd if=/dev/zero of=/dev/da1 count=2 &prompt.root; bsdlabel /dev/da1 | bsdlabel -BR da1 /dev/stdin &prompt.root; newfs /dev/da1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 RAID Software RAID Christopher Shumway Original work by Jim Brown Revised by RAIDsoftware RAIDCCD Concatenated Disk Driver (CCD) Configuration When choosing a mass storage solution the most important factors to consider are speed, reliability, and cost. It is rare to have all three in balance; normally a fast, reliable mass storage device is expensive, and to cut back on cost either speed or reliability must be sacrificed. In designing the system described below, cost was chosen as the most important factor, followed by speed, then reliability. Data transfer speed for this system is ultimately constrained by the network. And while reliability is very important, the CCD drive described below serves online data that is already fully backed up on CD-R's and can easily be replaced. Defining your own requirements is the first step in choosing a mass storage solution. If your requirements prefer speed or reliability over cost, your solution will differ from the system described in this section. Installing the Hardware In addition to the IDE system disk, three Western Digital 30GB, 5400 RPM IDE disks form the core of the CCD disk described below providing approximately 90GB of online storage. Ideally, each IDE disk would have its own IDE controller and cable, but to minimize cost, additional IDE controllers were not used. Instead the disks were configured with jumpers so that each IDE controller has one master, and one slave. Upon reboot, the system BIOS was configured to automatically detect the disks attached. More importantly, FreeBSD detected them on reboot: ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33 ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave UDMA33 ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33 ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave UDMA33 If FreeBSD does not detect all the disks, ensure that you have jumpered them correctly. Most IDE drives also have a Cable Select jumper. This is not the jumper for the master/slave relationship. Consult the drive documentation for help in identifying the correct jumper. Next, consider how to attach them as part of the file system. You should research both &man.vinum.8; () and &man.ccd.4;. In this particular configuration, &man.ccd.4; was chosen. Setting Up the CCD The &man.ccd.4; driver allows you to take several identical disks and concatenate them into one logical file system. In order to use &man.ccd.4;, you need a kernel with &man.ccd.4; support built in. Add this line to your kernel configuration file, rebuild, and reinstall the kernel: device ccd The &man.ccd.4; support can also be loaded as a kernel loadable module. To set up &man.ccd.4;, you must first use &man.bsdlabel.8; to label the disks: bsdlabel -w ad1 auto bsdlabel -w ad2 auto bsdlabel -w ad3 auto This creates a bsdlabel for ad1c, ad2c and ad3c that spans the entire disk. The next step is to change the disk label type. You can use &man.bsdlabel.8; to edit the disks: bsdlabel -e ad1 bsdlabel -e ad2 bsdlabel -e ad3 This opens up the current disk label on each disk with the editor specified by the EDITOR environment variable, typically &man.vi.1;. An unmodified disk label will look something like this: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) Add a new e partition for &man.ccd.4; to use. This can usually be copied from the c partition, but the must be 4.2BSD. The disk label should now look something like this: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) e: 60074784 0 4.2BSD 0 0 0 # (Cyl. 0 - 59597) Building the File System Now that you have all the disks labeled, you must build the &man.ccd.4;. To do that, use &man.ccdconfig.8;, with options similar to the following: ccdconfig ccd0 32 0 /dev/ad1e /dev/ad2e /dev/ad3e The use and meaning of each option is shown below: The first argument is the device to configure, in this case, /dev/ccd0c. The /dev/ portion is optional. The interleave for the file system. The interleave defines the size of a stripe in disk blocks, each normally 512 bytes. So, an interleave of 32 would be 16,384 bytes. Flags for &man.ccdconfig.8;. If you want to enable drive mirroring, you can specify a flag here. This configuration does not provide mirroring for &man.ccd.4;, so it is set at 0 (zero). The final arguments to &man.ccdconfig.8; are the devices to place into the array. Use the complete pathname for each device. After running &man.ccdconfig.8; the &man.ccd.4; is configured. A file system can be installed. Refer to &man.newfs.8; for options, or simply run: newfs /dev/ccd0c Making it All Automatic Generally, you will want to mount the &man.ccd.4; upon each reboot. To do this, you must configure it first. Write out your current configuration to /etc/ccd.conf using the following command: ccdconfig -g > /etc/ccd.conf During reboot, the script /etc/rc runs ccdconfig -C if /etc/ccd.conf exists. This automatically configures the &man.ccd.4; so it can be mounted. If you are booting into single user mode, before you can &man.mount.8; the &man.ccd.4;, you need to issue the following command to configure the array: ccdconfig -C To automatically mount the &man.ccd.4;, place an entry for the &man.ccd.4; in /etc/fstab so it will be mounted at boot time: /dev/ccd0c /media ufs rw 2 2 The Vinum Volume Manager RAIDsoftware RAID Vinum The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. &man.vinum.8; implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination. See for more information about &man.vinum.8;. Hardware RAID RAID hardware FreeBSD also supports a variety of hardware RAID controllers. These devices control a RAID subsystem without the need for FreeBSD specific software to manage the array. Using an on-card BIOS, the card controls most of the disk operations itself. The following is a brief setup description using a Promise IDE RAID controller. When this card is installed and the system is started up, it displays a prompt requesting information. Follow the instructions to enter the card's setup screen. From here, you have the ability to combine all the attached drives. After doing so, the disk(s) will look like a single drive to FreeBSD. Other RAID levels can be set up accordingly. Rebuilding ATA RAID1 Arrays FreeBSD allows you to hot-replace a failed disk in an array. This requires that you catch it before you reboot. You will probably see something like the following in /var/log/messages or in the &man.dmesg.8; output: ad6 on monster1 suffered a hard error. ad6: READ command timeout tag=0 serv=0 - resetting ad6: trying fallback to PIO mode ata3: resetting devices .. done ad6: hard error reading fsbn 1116119 of 0-7 (ad6 bn 1116119; cn 1107 tn 4 sn 11)\\ status=59 error=40 ar0: WARNING - mirror lost Using &man.atacontrol.8;, check for further information: &prompt.root; atacontrol list ATA channel 0: Master: no device present Slave: acd0 <HL-DT-ST CD-ROM GCR-8520B/1.00> ATA/ATAPI rev 0 ATA channel 1: Master: no device present Slave: no device present ATA channel 2: Master: ad4 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present ATA channel 3: Master: ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present &prompt.root; atacontrol status ar0 ar0: ATA RAID1 subdisks: ad4 ad6 status: DEGRADED You will first need to detach the ata channel with the failed disk so you can safely remove it: &prompt.root; atacontrol detach ata3 Replace the disk. Reattach the ata channel: &prompt.root; atacontrol attach ata3 Master: ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present Add the new disk to the array as a spare: &prompt.root; atacontrol addspare ar0 ad6 Rebuild the array: &prompt.root; atacontrol rebuild ar0 It is possible to check on the progress by issuing the following command: &prompt.root; dmesg | tail -10 [output removed] ad6: removed from configuration ad6: deleted from ar0 disk1 ad6: inserted into ar0 disk1 as spare &prompt.root; atacontrol status ar0 ar0: ATA RAID1 subdisks: ad4 ad6 status: REBUILDING 0% completed Wait until this operation completes. Marc Fonvieille Contributed by USB Storage Devices USB disks A lot of external storage solutions, nowadays, use the Universal Serial Bus (USB): hard drives, USB thumbdrives, CD-R burners, etc. &os; provides support for these devices. Configuration The USB mass storage devices driver, &man.umass.4;, provides the support for USB storage devices. If you use the GENERIC kernel, you do not have to change anything in your configuration. If you use a custom kernel, be sure that the following lines are present in your kernel configuration file: device scbus device da device pass device uhci device ohci device usb device umass The &man.umass.4; driver uses the SCSI subsystem to access to the USB storage devices, your USB device will be seen as a SCSI device by the system. Depending on the USB chipset on your motherboard, you only need either device uhci or device ohci, however having both in the kernel configuration file is harmless. Do not forget to compile and install the new kernel if you added any lines. If your USB device is a CD-R or DVD burner, the SCSI CD-ROM driver, &man.cd.4;, must be added to the kernel via the line: device cd Since the burner is seen as a SCSI drive, the driver &man.atapicam.4; should not be used in the kernel configuration. Support for USB 2.0 controllers is provided on &os;; however, you must add: device ehci to your configuration file for USB 2.0 support. Note &man.uhci.4; and &man.ohci.4; drivers are still needed if you want USB 1.X support. Testing the Configuration The configuration is ready to be tested: plug in your USB device, and in the system message buffer (&man.dmesg.8;), the drive should appear as something like: umass0: USB Solid state disk, rev 1.10/1.00, addr 2 GEOM: create disk da0 dp=0xc2d74850 da0 at umass-sim0 bus 0 target 0 lun 0 da0: <Generic Traveling Disk 1.11> Removable Direct Access SCSI-2 device da0: 1.000MB/s transfers da0: 126MB (258048 512 byte sectors: 64H 32S/T 126C) Of course, the brand, the device node (da0) and other details can differ according to your configuration. Since the USB device is seen as a SCSI one, the camcontrol command can be used to list the USB storage devices attached to the system: &prompt.root; camcontrol devlist <Generic Traveling Disk 1.11> at scbus0 target 0 lun 0 (da0,pass0) If the drive comes with a file system, you should be able to mount it. The will help you to format and create partitions on the USB drive if needed. To make this device mountable as a normal user, certain steps have to be taken. First, the devices that are created when a USB storage device is connected need to be accessible by the user. A solution is to make all users of these devices a member of the operator group. This is done with &man.pw.8;. Second, when the devices are created, the operator group should be able to read and write them. This is accomplished by adding these lines to /etc/devfs.rules: [localrules=1] add path 'da*' mode 0660 group operator If there already are SCSI disks in the system, it must be done a bit different. E.g., if the system already contains disks da0 through da2 attached to the system, change the second line as follows: add path 'da[3-9]*' mode 0660 group operator This will exclude the already existing disks from belonging to the operator group. You also have to enable your &man.devfs.rules.5; ruleset in your /etc/rc.conf file: devfs_system_ruleset="localrules" Next, the kernel has to be configured to allow regular users to mount file systems. The easiest way is to add the following line to /etc/sysctl.conf: vfs.usermount=1 Note that this only takes effect after the next reboot. Alternatively, one can also use &man.sysctl.8; to set this variable. The final step is to create a directory where the file system is to be mounted. This directory needs to be owned by the user that is to mount the file system. One way to do that is for root to create a subdirectory owned by that user as /mnt/username (replace username by the login name of the actual user): &prompt.root; mkdir /mnt/username &prompt.root; chown user:user /mnt/user Suppose a USB thumbdrive is plugged in, and a device /dev/da0s1 appears. Since these devices usually come preformatted with a FAT file system, one can mount them like this: &prompt.user; mount -t msdosfs -m 644 -M 755 /dev/da0s1 /mnt/username If you unplug the device (the disk must be unmounted before), you should see, in the system message buffer, something like the following: umass0: at uhub0 port 1 (addr 2) disconnected (da0:umass-sim0:0:0:0): lost device (da0:umass-sim0:0:0:0): removing device entry GEOM: destroy disk da0 dp=0xc2d74850 umass0: detached Further Reading Beside the Adding Disks and Mounting and Unmounting File Systems sections, reading various manual pages may be also useful: &man.umass.4;, &man.camcontrol.8;, and &man.usbdevs.8;. Mike Meyer Contributed by Creating and Using Optical Media (CDs) CDROMs creating Introduction CDs have a number of features that differentiate them from conventional disks. Initially, they were not writable by the user. They are designed so that they can be read continuously without delays to move the head between tracks. They are also much easier to transport between systems than similarly sized media were at the time. CDs do have tracks, but this refers to a section of data to be read continuously and not a physical property of the disk. To produce a CD on FreeBSD, you prepare the data files that are going to make up the tracks on the CD, then write the tracks to the CD. ISO 9660 file systems ISO 9660 The ISO 9660 file system was designed to deal with these differences. It unfortunately codifies file system limits that were common then. Fortunately, it provides an extension mechanism that allows properly written CDs to exceed those limits while still working with systems that do not support those extensions. sysutils/cdrtools The sysutils/cdrtools port includes &man.mkisofs.8;, a program that you can use to produce a data file containing an ISO 9660 file system. It has options that support various extensions, and is described below. CD burner ATAPI Which tool to use to burn the CD depends on whether your CD burner is ATAPI or something else. ATAPI CD burners use the burncd program that is part of the base system. SCSI and USB CD burners should use cdrecord from the sysutils/cdrtools port. It is also possible to use cdrecord and other tools for SCSI drives on ATAPI hardware with the ATAPI/CAM module. If you want CD burning software with a graphical user interface, you may wish to take a look at either X-CD-Roast or K3b. These tools are available as packages or from the sysutils/xcdroast and sysutils/k3b ports. X-CD-Roast and K3b require the ATAPI/CAM module with ATAPI hardware. mkisofs The &man.mkisofs.8; program, which is part of the sysutils/cdrtools port, produces an ISO 9660 file system that is an image of a directory tree in the &unix; file system name space. The simplest usage is: &prompt.root; mkisofs -o imagefile.iso /path/to/tree file systems ISO 9660 This command will create an imagefile.iso containing an ISO 9660 file system that is a copy of the tree at /path/to/tree. In the process, it will map the file names to names that fit the limitations of the standard ISO 9660 file system, and will exclude files that have names uncharacteristic of ISO file systems. file systems HFS file systems Joliet A number of options are available to overcome those restrictions. In particular, enables the Rock Ridge extensions common to &unix; systems, enables Joliet extensions used by Microsoft systems, and can be used to create HFS file systems used by &macos;. For CDs that are going to be used only on FreeBSD systems, can be used to disable all filename restrictions. When used with , it produces a file system image that is identical to the FreeBSD tree you started from, though it may violate the ISO 9660 standard in a number of ways. CDROMs creating bootable The last option of general use is . This is used to specify the location of the boot image for use in producing an El Torito bootable CD. This option takes an argument which is the path to a boot image from the top of the tree being written to the CD. By default, &man.mkisofs.8; creates an ISO image in the so-called floppy disk emulation mode, and thus expects the boot image to be exactly 1200, 1440 or 2880 KB in size. Some boot loaders, like the one used by the FreeBSD distribution disks, do not use emulation mode; in this case, the option should be used. So, if /tmp/myboot holds a bootable FreeBSD system with the boot image in /tmp/myboot/boot/cdboot, you could produce the image of an ISO 9660 file system in /tmp/bootable.iso like so: &prompt.root; mkisofs -R -no-emul-boot -b boot/cdboot -o /tmp/bootable.iso /tmp/myboot Having done that, if you have md configured in your kernel, you can mount the file system with: &prompt.root; mdconfig -a -t vnode -f /tmp/bootable.iso -u 0 &prompt.root; mount -t cd9660 /dev/md0 /mnt At which point you can verify that /mnt and /tmp/myboot are identical. There are many other options you can use with &man.mkisofs.8; to fine-tune its behavior. In particular: modifications to an ISO 9660 layout and the creation of Joliet and HFS discs. See the &man.mkisofs.8; manual page for details. burncd CDROMs burning If you have an ATAPI CD burner, you can use the burncd command to burn an ISO image onto a CD. burncd is part of the base system, installed as /usr/sbin/burncd. Usage is very simple, as it has few options: &prompt.root; burncd -f cddevice data imagefile.iso fixate Will burn a copy of imagefile.iso on cddevice. The default device is /dev/acd0. See &man.burncd.8; for options to set the write speed, eject the CD after burning, and write audio data. cdrecord If you do not have an ATAPI CD burner, you will have to use cdrecord to burn your CDs. cdrecord is not part of the base system; you must install it from either the port at sysutils/cdrtools or the appropriate package. Changes to the base system can cause binary versions of this program to fail, possibly resulting in a coaster. You should therefore either upgrade the port when you upgrade your system, or if you are tracking -STABLE, upgrade the port when a new version becomes available. While cdrecord has many options, basic usage is even simpler than burncd. Burning an ISO 9660 image is done with: &prompt.root; cdrecord dev=device imagefile.iso The tricky part of using cdrecord is finding the to use. To find the proper setting, use the flag of cdrecord, which might produce results like this: CDROMs burning &prompt.root; cdrecord -scanbus Cdrecord-Clone 2.01 (i386-unknown-freebsd7.0) Copyright (C) 1995-2004 Jörg Schilling Using libscg version 'schily-0.1' scsibus0: 0,0,0 0) 'SEAGATE ' 'ST39236LW ' '0004' Disk 0,1,0 1) 'SEAGATE ' 'ST39173W ' '5958' Disk 0,2,0 2) * 0,3,0 3) 'iomega ' 'jaz 1GB ' 'J.86' Removable Disk 0,4,0 4) 'NEC ' 'CD-ROM DRIVE:466' '1.26' Removable CD-ROM 0,5,0 5) * 0,6,0 6) * 0,7,0 7) * scsibus1: 1,0,0 100) * 1,1,0 101) * 1,2,0 102) * 1,3,0 103) * 1,4,0 104) * 1,5,0 105) 'YAMAHA ' 'CRW4260 ' '1.0q' Removable CD-ROM 1,6,0 106) 'ARTEC ' 'AM12S ' '1.06' Scanner 1,7,0 107) * This lists the appropriate value for the devices on the list. Locate your CD burner, and use the three numbers separated by commas as the value for . In this case, the CRW device is 1,5,0, so the appropriate input would be . There are easier ways to specify this value; see &man.cdrecord.1; for details. That is also the place to look for information on writing audio tracks, controlling the speed, and other things. Duplicating Audio CDs You can duplicate an audio CD by extracting the audio data from the CD to a series of files, and then writing these files to a blank CD. The process is slightly different for ATAPI and SCSI drives. SCSI Drives Use cdda2wav to extract the audio. &prompt.user; cdda2wav -v255 -D2,0 -B -Owav Use cdrecord to write the .wav files. &prompt.user; cdrecord -v dev=2,0 -dao -useinfo *.wav Make sure that 2,0 is set appropriately, as described in . ATAPI Drives The ATAPI CD driver makes each track available as /dev/acddtnn, where d is the drive number, and nn is the track number written with two decimal digits, prefixed with zero as needed. So the first track on the first disk is /dev/acd0t01, the second is /dev/acd0t02, the third is /dev/acd0t03, and so on. Make sure the appropriate files exist in /dev. If the entries are missing, force the system to retaste the media: &prompt.root; dd if=/dev/acd0 of=/dev/null count=1 Extract each track using &man.dd.1;. You must also use a specific block size when extracting the files. &prompt.root; dd if=/dev/acd0t01 of=track1.cdr bs=2352 &prompt.root; dd if=/dev/acd0t02 of=track2.cdr bs=2352 ... Burn the extracted files to disk using burncd. You must specify that these are audio files, and that burncd should fixate the disk when finished. &prompt.root; burncd -f /dev/acd0 audio track1.cdr track2.cdr ... fixate Duplicating Data CDs You can copy a data CD to a image file that is functionally equivalent to the image file created with &man.mkisofs.8;, and you can use it to duplicate any data CD. The example given here assumes that your CDROM device is acd0. Substitute your correct CDROM device. &prompt.root; dd if=/dev/acd0 of=file.iso bs=2048 Now that you have an image, you can burn it to CD as described above. Using Data CDs Now that you have created a standard data CDROM, you probably want to mount it and read the data on it. By default, &man.mount.8; assumes that a file system is of type ufs. If you try something like: &prompt.root; mount /dev/cd0 /mnt you will get a complaint about Incorrect super block, and no mount. The CDROM is not a UFS file system, so attempts to mount it as such will fail. You just need to tell &man.mount.8; that the file system is of type ISO9660, and everything will work. You do this by specifying the option &man.mount.8;. For example, if you want to mount the CDROM device, /dev/cd0, under /mnt, you would execute: &prompt.root; mount -t cd9660 /dev/cd0 /mnt Note that your device name (/dev/cd0 in this example) could be different, depending on the interface your CDROM uses. Also, the option just executes &man.mount.cd9660.8;. The above example could be shortened to: &prompt.root; mount_cd9660 /dev/cd0 /mnt You can generally use data CDROMs from any vendor in this way. Disks with certain ISO 9660 extensions might behave oddly, however. For example, Joliet disks store all filenames in two-byte Unicode characters. The FreeBSD kernel does not speak Unicode, but the &os; CD9660 driver is able to convert Unicode characters on the fly. If some non-English characters show up as question marks you will need to specify the local charset you use with the option. For more information, consult the &man.mount.cd9660.8; manual page. To be able to do this character conversion with the help of the option, the kernel will require the cd9660_iconv.ko module to be loaded. This can be done either by adding this line to loader.conf: cd9660_iconv_load="YES" and then rebooting the machine, or by directly loading the module with &man.kldload.8;. Occasionally, you might get Device not configured when trying to mount a CDROM. This usually means that the CDROM drive thinks that there is no disk in the tray, or that the drive is not visible on the bus. It can take a couple of seconds for a CDROM drive to realize that it has been fed, so be patient. Sometimes, a SCSI CDROM may be missed because it did not have enough time to answer the bus reset. If you have a SCSI CDROM please add the following option to your kernel configuration and rebuild your kernel. options SCSI_DELAY=15000 This tells your SCSI bus to pause 15 seconds during boot, to give your CDROM drive every possible chance to answer the bus reset. Burning Raw Data CDs You can choose to burn a file directly to CD, without creating an ISO 9660 file system. Some people do this for backup purposes. This runs more quickly than burning a standard CD: &prompt.root; burncd -f /dev/acd1 -s 12 data archive.tar.gz fixate In order to retrieve the data burned to such a CD, you must read data from the raw device node: &prompt.root; tar xzvf /dev/acd1 You cannot mount this disk as you would a normal CDROM. Such a CDROM cannot be read under any operating system except FreeBSD. If you want to be able to mount the CD, or share data with another operating system, you must use &man.mkisofs.8; as described above. Marc Fonvieille Contributed by CD burner ATAPI/CAM driver Using the ATAPI/CAM Driver This driver allows ATAPI devices (CD-ROM, CD-RW, DVD drives etc...) to be accessed through the SCSI subsystem, and so allows the use of applications like sysutils/cdrdao or &man.cdrecord.1;. To use this driver, you will need to add the following line to the /boot/loader.conf file: atapicam_load="YES" then, reboot your machine. If you prefer to statically compile the &man.atapicam.4; support in your kernel, you will have to add this line to your kernel configuration file: device atapicam You also need the following lines in your kernel configuration file: device ata device scbus device cd device pass which should already be present. Then rebuild, install your new kernel, and reboot your machine. During the boot process, your burner should show up, like so: acd0: CD-RW <MATSHITA CD-RW/DVD-ROM UJDA740> at ata1-master PIO4 cd0 at ata1 bus 0 target 0 lun 0 cd0: <MATSHITA CDRW/DVD UJDA740 1.00> Removable CD-ROM SCSI-0 device cd0: 16.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed The drive could now be accessed via the /dev/cd0 device name, for example to mount a CD-ROM on /mnt, just type the following: &prompt.root; mount -t cd9660 /dev/cd0 /mnt As root, you can run the following command to get the SCSI address of the burner: &prompt.root; camcontrol devlist <MATSHITA CDRW/DVD UJDA740 1.00> at scbus1 target 0 lun 0 (pass0,cd0) So 1,0,0 will be the SCSI address to use with &man.cdrecord.1; and other SCSI application. For more information about ATAPI/CAM and SCSI system, refer to the &man.atapicam.4; and &man.cam.4; manual pages. Marc Fonvieille Contributed by Andy Polyakov With inputs from Creating and Using Optical Media (DVDs) DVD burning Introduction Compared to the CD, the DVD is the next generation of optical media storage technology. The DVD can hold more data than any CD and is nowadays the standard for video publishing. Five physical recordable formats can be defined for what we will call a recordable DVD: DVD-R: This was the first DVD recordable format available. The DVD-R standard is defined by the DVD Forum. This format is write once. DVD-RW: This is the rewritable version of the DVD-R standard. A DVD-RW can be rewritten about 1000 times. DVD-RAM: This is also a rewritable format supported by the DVD Forum. A DVD-RAM can be seen as a removable hard drive. However, this media is not compatible with most DVD-ROM drives and DVD-Video players; only a few DVD writers support the DVD-RAM format. Read the for more information on DVD-RAM use. DVD+RW: This is a rewritable format defined by the DVD+RW Alliance. A DVD+RW can be rewritten about 1000 times. DVD+R: This format is the write once variation of the DVD+RW format. A single layer recordable DVD can hold up to 4,700,000,000 bytes which is actually 4.38 GB or 4485 MB (1 kilobyte is 1024 bytes). A distinction must be made between the physical media and the application. For example, a DVD-Video is a specific file layout that can be written on any recordable DVD physical media: DVD-R, DVD+R, DVD-RW etc. Before choosing the type of media, you must be sure that both the burner and the DVD-Video player (a standalone player or a DVD-ROM drive on a computer) are compatible with the media under consideration. Configuration The program &man.growisofs.1; will be used to perform DVD recording. This command is part of the dvd+rw-tools utilities (sysutils/dvd+rw-tools). The dvd+rw-tools support all DVD media types. These tools use the SCSI subsystem to access to the devices, therefore the ATAPI/CAM support must be added to your kernel. If your burner uses the USB interface this addition is useless, and you should read the for more details on USB devices configuration. You also have to enable DMA access for ATAPI devices, this can be done in adding the following line to the /boot/loader.conf file: hw.ata.atapi_dma="1" Before attempting to use the dvd+rw-tools you should consult the dvd+rw-tools' hardware compatibility notes for any information related to your DVD burner. If you want a graphical user interface, you should have a look to K3b (sysutils/k3b) which provides a user friendly interface to &man.growisofs.1; and many other burning tools. Burning Data DVDs The &man.growisofs.1; command is a frontend to mkisofs, it will invoke &man.mkisofs.8; to create the file system layout and will perform the write on the DVD. This means you do not need to create an image of the data before the burning process. To burn onto a DVD+R or a DVD-R the data from the /path/to/data directory, use the following command: &prompt.root; growisofs -dvd-compat -Z /dev/cd0 -J -R /path/to/data The options are passed to &man.mkisofs.8; for the file system creation (in this case: an ISO 9660 file system with Joliet and Rock Ridge extensions), consult the &man.mkisofs.8; manual page for more details. The option is used for the initial session recording in any case: multiple sessions or not. The DVD device, /dev/cd0, must be changed according to your configuration. The parameter will close the disk, the recording will be unappendable. In return this should provide better media compatibility with DVD-ROM drives. It is also possible to burn a pre-mastered image, for example to burn the image imagefile.iso, we will run: &prompt.root; growisofs -dvd-compat -Z /dev/cd0=imagefile.iso The write speed should be detected and automatically set according to the media and the drive being used. If you want to force the write speed, use the parameter. For more information, read the &man.growisofs.1; manual page. DVD DVD-Video Burning a DVD-Video A DVD-Video is a specific file layout based on ISO 9660 and the micro-UDF (M-UDF) specifications. The DVD-Video also presents a specific data structure hierarchy, it is the reason why you need a particular program such as multimedia/dvdauthor to author the DVD. If you already have an image of the DVD-Video file system, just burn it in the same way as for any image, see the previous section for an example. If you have made the DVD authoring and the result is in, for example, the directory /path/to/video, the following command should be used to burn the DVD-Video: &prompt.root; growisofs -Z /dev/cd0 -dvd-video /path/to/video The option will be passed down to &man.mkisofs.8; and will instruct it to create a DVD-Video file system layout. Beside this, the option implies &man.growisofs.1; option. DVD DVD+RW Using a DVD+RW Unlike CD-RW, a virgin DVD+RW needs to be formatted before first use. The &man.growisofs.1; program will take care of it automatically whenever appropriate, which is the recommended way. However you can use the dvd+rw-format command to format the DVD+RW: &prompt.root; dvd+rw-format /dev/cd0 You need to perform this operation just once, keep in mind that only virgin DVD+RW medias need to be formatted. Then you can burn the DVD+RW in the way seen in previous sections. If you want to burn new data (burn a totally new file system not append some data) onto a DVD+RW, you do not need to blank it, you just have to write over the previous recording (in performing a new initial session), like this: &prompt.root; growisofs -Z /dev/cd0 -J -R /path/to/newdata DVD+RW format offers the possibility to easily append data to a previous recording. The operation consists in merging a new session to the existing one, it is not multisession writing, &man.growisofs.1; will grow the ISO 9660 file system present on the media. For example, if we want to append data to our previous DVD+RW, we have to use the following: &prompt.root; growisofs -M /dev/cd0 -J -R /path/to/nextdata The same &man.mkisofs.8; options we used to burn the initial session should be used during next writes. You may want to use the option if you want better media compatibility with DVD-ROM drives. In the DVD+RW case, this will not prevent you from adding data. If for any reason you really want to blank the media, do the following: &prompt.root; growisofs -Z /dev/cd0=/dev/zero DVD DVD-RW Using a DVD-RW A DVD-RW accepts two disc formats: the incremental sequential one and the restricted overwrite. By default DVD-RW discs are in sequential format. A virgin DVD-RW can be directly written without the need of a formatting operation, however a non-virgin DVD-RW in sequential format needs to be blanked before to be able to write a new initial session. To blank a DVD-RW in sequential mode, run: &prompt.root; dvd+rw-format -blank=full /dev/cd0 A full blanking () will take about one hour on a 1x media. A fast blanking can be performed using the option if the DVD-RW will be recorded in Disk-At-Once (DAO) mode. To burn the DVD-RW in DAO mode, use the command: &prompt.root; growisofs -use-the-force-luke=dao -Z /dev/cd0=imagefile.iso The option should not be required since &man.growisofs.1; attempts to detect minimally (fast blanked) media and engage DAO write. In fact one should use restricted overwrite mode with any DVD-RW, this format is more flexible than the default incremental sequential one. To write data on a sequential DVD-RW, use the same instructions as for the other DVD formats: &prompt.root; growisofs -Z /dev/cd0 -J -R /path/to/data If you want to append some data to your previous recording, you will have to use the &man.growisofs.1; option. However, if you perform data addition on a DVD-RW in incremental sequential mode, a new session will be created on the disc and the result will be a multi-session disc. A DVD-RW in restricted overwrite format does not need to be blanked before a new initial session, you just have to overwrite the disc with the option, this is similar to the DVD+RW case. It is also possible to grow an existing ISO 9660 file system written on the disc in a same way as for a DVD+RW with the option. The result will be a one-session DVD. To put a DVD-RW in the restricted overwrite format, the following command must be used: &prompt.root; dvd+rw-format /dev/cd0 To change back to the sequential format use: &prompt.root; dvd+rw-format -blank=full /dev/cd0 Multisession Very few DVD-ROM drives support multisession DVDs, they will most of time, hopefully, only read the first session. DVD+R, DVD-R and DVD-RW in sequential format can accept multiple sessions, the notion of multiple sessions does not exist for the DVD+RW and the DVD-RW restricted overwrite formats. Using the following command after an initial (non-closed) session on a DVD+R, DVD-R, or DVD-RW in sequential format, will add a new session to the disc: &prompt.root; growisofs -M /dev/cd0 -J -R /path/to/nextdata Using this command line with a DVD+RW or a DVD-RW in restricted overwrite mode, will append data in merging the new session to the existing one. The result will be a single-session disc. This is the way used to add data after an initial write on these medias. Some space on the media is used between each session for end and start of sessions. Therefore, one should add sessions with large amount of data to optimize media space. The number of sessions is limited to 154 for a DVD+R, about 2000 for a DVD-R, and 127 for a DVD+R Double Layer. For More Information To obtain more information about a DVD, the dvd+rw-mediainfo /dev/cd0 command can be ran with the disc in the drive. More information about the dvd+rw-tools can be found in the &man.growisofs.1; manual page, on the dvd+rw-tools web site and in the cdwrite mailing list archives. The dvd+rw-mediainfo output of the resulting recording or the media with issues is mandatory for any problem report. Without this output, it will be quite impossible to help you. Using a DVD-RAM DVD DVD-RAM Configuration DVD-RAM writers come with either SCSI or ATAPI interface. DMA access for ATAPI devices has to be enabled, this can be done by adding the following line to the /boot/loader.conf file: hw.ata.atapi_dma="1" Preparing the Medium As previously mentioned in the chapter introduction, a DVD-RAM can be seen as a removable hard drive. As any other hard drive the DVD-RAM must be prepared before the first use. In the example, the whole disk space will be used with a standard UFS2 file system: &prompt.root; dd if=/dev/zero of=/dev/acd0 bs=2k count=1 &prompt.root; bsdlabel -Bw acd0 &prompt.root; newfs /dev/acd0 The DVD device, acd0, must be changed according to the configuration. Using the Medium Once the previous operations have been performed on the DVD-RAM, it can be mounted as a normal hard drive: &prompt.root; mount /dev/acd0 /mnt After this the DVD-RAM will be both readable and writeable. Julio Merino Original work by Martin Karlsson Rewritten by Creating and Using Floppy Disks Storing data on floppy disks is sometimes useful, for example when one does not have any other removable storage media or when one needs to transfer small amounts of data to another computer. This section will explain how to use floppy disks in FreeBSD. It will primarily cover formatting and usage of 3.5inch DOS floppies, but the concepts are similar for other floppy disk formats. Formatting Floppies The Device Floppy disks are accessed through entries in /dev, just like other devices. To access the raw floppy disk, simply use /dev/fdN. Formatting A floppy disk needs to be low-level formated before it can be used. This is usually done by the vendor, but formatting is a good way to check media integrity. Although it is possible to force larger (or smaller) disk sizes, 1440kB is what most floppy disks are designed for. To low-level format the floppy disk you need to use &man.fdformat.1;. This utility expects the device name as an argument. Make note of any error messages, as these can help determine if the disk is good or bad. Formatting Floppy Disks Use the /dev/fdN devices to format the floppy. Insert a new 3.5inch floppy disk in your drive and issue: &prompt.root; /usr/sbin/fdformat -f 1440 /dev/fd0 The Disk Label After low-level formatting the disk, you will need to place a disk label on it. This disk label will be destroyed later, but it is needed by the system to determine the size of the disk and its geometry later. The new disk label will take over the whole disk, and will contain all the proper information about the geometry of the floppy. The geometry values for the disk label are listed in /etc/disktab. You can run now &man.bsdlabel.8; like so: &prompt.root; /sbin/bsdlabel -B -w /dev/fd0 fd1440 The File System Now the floppy is ready to be high-level formated. This will place a new file system on it, which will let FreeBSD read and write to the disk. After creating the new file system, the disk label is destroyed, so if you want to reformat the disk, you will have to recreate the disk label. The floppy's file system can be either UFS or FAT. FAT is generally a better choice for floppies. To put a new file system on the floppy, issue: &prompt.root; /sbin/newfs_msdos /dev/fd0 The disk is now ready for use. Using the Floppy To use the floppy, mount it with &man.mount.msdosfs.8;. One can also use emulators/mtools from the ports collection. Creating and Using Data Tapes tape media The major tape media are the 4mm, 8mm, QIC, mini-cartridge and DLT. 4mm (DDS: Digital Data Storage) tape media DDS (4mm) tapes tape media QIC tapes 4mm tapes are replacing QIC as the workstation backup media of choice. This trend accelerated greatly when Conner purchased Archive, a leading manufacturer of QIC drives, and then stopped production of QIC drives. 4mm drives are small and quiet but do not have the reputation for reliability that is enjoyed by 8mm drives. The cartridges are less expensive and smaller (3 x 2 x 0.5 inches, 76 x 51 x 12 mm) than 8mm cartridges. 4mm, like 8mm, has comparatively short head life for the same reason, both use helical scan. Data throughput on these drives starts ~150 kB/s, peaking at ~500 kB/s. Data capacity starts at 1.3 GB and ends at 2.0 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. Multi-drive tape library units can have 6 drives in a single cabinet with automatic tape changing. Library capacities reach 240 GB. The DDS-3 standard now supports tape capacities up to 12 GB (or 24 GB compressed). 4mm drives, like 8mm drives, use helical-scan. All the benefits and drawbacks of helical-scan apply to both 4mm and 8mm drives. Tapes should be retired from use after 2,000 passes or 100 full backups. 8mm (Exabyte) tape media Exabyte (8mm) tapes 8mm tapes are the most common SCSI tape drives; they are the best choice of exchanging tapes. Nearly every site has an Exabyte 2 GB 8mm tape drive. 8mm drives are reliable, convenient and quiet. Cartridges are inexpensive and small (4.8 x 3.3 x 0.6 inches; 122 x 84 x 15 mm). One downside of 8mm tape is relatively short head and tape life due to the high rate of relative motion of the tape across the heads. Data throughput ranges from ~250 kB/s to ~500 kB/s. Data sizes start at 300 MB and go up to 7 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. These drives are available as single units or multi-drive tape libraries with 6 drives and 120 tapes in a single cabinet. Tapes are changed automatically by the unit. Library capacities reach 840+ GB. The Exabyte Mammoth model supports 12 GB on one tape (24 GB with compression) and costs approximately twice as much as conventional tape drives. Data is recorded onto the tape using helical-scan, the heads are positioned at an angle to the media (approximately 6 degrees). The tape wraps around 270 degrees of the spool that holds the heads. The spool spins while the tape slides over the spool. The result is a high density of data and closely packed tracks that angle across the tape from one edge to the other. QIC tape media QIC-150 QIC-150 tapes and drives are, perhaps, the most common tape drive and media around. QIC tape drives are the least expensive serious backup drives. The downside is the cost of media. QIC tapes are expensive compared to 8mm or 4mm tapes, up to 5 times the price per GB data storage. But, if your needs can be satisfied with a half-dozen tapes, QIC may be the correct choice. QIC is the most common tape drive. Every site has a QIC drive of some density or another. Therein lies the rub, QIC has a large number of densities on physically similar (sometimes identical) tapes. QIC drives are not quiet. These drives audibly seek before they begin to record data and are clearly audible whenever reading, writing or seeking. QIC tapes measure 6 x 4 x 0.7 inches (152 x 102 x 17 mm). Data throughput ranges from ~150 kB/s to ~500 kB/s. Data capacity ranges from 40 MB to 15 GB. Hardware compression is available on many of the newer QIC drives. QIC drives are less frequently installed; they are being supplanted by DAT drives. Data is recorded onto the tape in tracks. The tracks run along the long axis of the tape media from one end to the other. The number of tracks, and therefore the width of a track, varies with the tape's capacity. Most if not all newer drives provide backward-compatibility at least for reading (but often also for writing). QIC has a good reputation regarding the safety of the data (the mechanics are simpler and more robust than for helical scan drives). Tapes should be retired from use after 5,000 backups. DLT tape media DLT DLT has the fastest data transfer rate of all the drive types listed here. The 1/2" (12.5mm) tape is contained in a single spool cartridge (4 x 4 x 1 inches; 100 x 100 x 25 mm). The cartridge has a swinging gate along one entire side of the cartridge. The drive mechanism opens this gate to extract the tape leader. The tape leader has an oval hole in it which the drive uses to hook the tape. The take-up spool is located inside the tape drive. All the other tape cartridges listed here (9 track tapes are the only exception) have both the supply and take-up spools located inside the tape cartridge itself. Data throughput is approximately 1.5 MB/s, three times the throughput of 4mm, 8mm, or QIC tape drives. Data capacities range from 10 GB to 20 GB for a single drive. Drives are available in both multi-tape changers and multi-tape, multi-drive tape libraries containing from 5 to 900 tapes over 1 to 20 drives, providing from 50 GB to 9 TB of storage. With compression, DLT Type IV format supports up to 70 GB capacity. Data is recorded onto the tape in tracks parallel to the direction of travel (just like QIC tapes). Two tracks are written at once. Read/write head lifetimes are relatively long; once the tape stops moving, there is no relative motion between the heads and the tape. AIT tape media AIT AIT is a new format from Sony, and can hold up to 50 GB (with compression) per tape. The tapes contain memory chips which retain an index of the tape's contents. This index can be rapidly read by the tape drive to determine the position of files on the tape, instead of the several minutes that would be required for other tapes. Software such as SAMS:Alexandria can operate forty or more AIT tape libraries, communicating directly with the tape's memory chip to display the contents on screen, determine what files were backed up to which tape, locate the correct tape, load it, and restore the data from the tape. Libraries like this cost in the region of $20,000, pricing them a little out of the hobbyist market. Using a New Tape for the First Time The first time that you try to read or write a new, completely blank tape, the operation will fail. The console messages should be similar to: sa0(ncr1:4:0): NOT READY asc:4,1 sa0(ncr1:4:0): Logical unit is in process of becoming ready The tape does not contain an Identifier Block (block number 0). All QIC tape drives since the adoption of QIC-525 standard write an Identifier Block to the tape. There are two solutions: mt fsf 1 causes the tape drive to write an Identifier Block to the tape. Use the front panel button to eject the tape. Re-insert the tape and dump data to the tape. dump will report DUMP: End of tape detected and the console will show: HARDWARE FAILURE info:280 asc:80,96. rewind the tape using: mt rewind. Subsequent tape operations are successful. Backups to Floppies Can I Use Floppies for Backing Up My Data? backup floppies floppy disks Floppy disks are not really a suitable media for making backups as: The media is unreliable, especially over long periods of time. Backing up and restoring is very slow. They have a very limited capacity (the days of backing up an entire hard disk onto a dozen or so floppies has long since passed). However, if you have no other method of backing up your data then floppy disks are better than no backup at all. If you do have to use floppy disks then ensure that you use good quality ones. Floppies that have been lying around the office for a couple of years are a bad choice. Ideally use new ones from a reputable manufacturer. So How Do I Backup My Data to Floppies? The best way to backup to floppy disk is to use &man.tar.1; with the (multi volume) option, which allows backups to span multiple floppies. To backup all the files in the current directory and sub-directory use this (as root): &prompt.root; tar Mcvf /dev/fd0 * When the first floppy is full &man.tar.1; will prompt you to insert the next volume (because &man.tar.1; is media independent it refers to volumes; in this context it means floppy disk). Prepare volume #2 for /dev/fd0 and hit return: This is repeated (with the volume number incrementing) until all the specified files have been archived. Can I Compress My Backups? tar gzip compression Unfortunately, &man.tar.1; will not allow the option to be used for multi-volume archives. You could, of course, &man.gzip.1; all the files, &man.tar.1; them to the floppies, then &man.gunzip.1; the files again! How Do I Restore My Backups? To restore the entire archive use: &prompt.root; tar Mxvf /dev/fd0 There are two ways that you can use to restore only specific files. First, you can start with the first floppy and use: &prompt.root; tar Mxvf /dev/fd0 filename The utility &man.tar.1; will prompt you to insert subsequent floppies until it finds the required file. Alternatively, if you know which floppy the file is on then you can simply insert that floppy and use the same command as above. Note that if the first file on the floppy is a continuation from the previous one then &man.tar.1; will warn you that it cannot restore it, even if you have not asked it to! Lowell Gilbert Original work by Backup Strategies The first requirement in devising a backup plan is to make sure that all of the following problems are covered: Disk failure Accidental file deletion Random file corruption Complete machine destruction (e.g. fire), including destruction of any on-site backups. It is perfectly possible that some systems will be best served by having each of these problems covered by a completely different technique. Except for strictly personal systems with very low-value data, it is unlikely that one technique would cover all of them. Some of the techniques in the toolbox are: Archives of the whole system, backed up onto permanent media offsite. This actually provides protection against all of the possible problems listed above, but is slow and inconvenient to restore from. You can keep copies of the backups onsite and/or online, but there will still be inconveniences in restoring files, especially for non-privileged users. Filesystem snapshots. This is really only helpful in the accidental file deletion scenario, but it can be very helpful in that case, and is quick and easy to deal with. Copies of whole filesystems and/or disks (e.g. periodic &man.rsync.1; of the whole machine). This is generally most useful in networks with unique requirements. For general protection against disk failure, it is usually inferior to RAID. For restoring accidentally deleted files, it can be comparable to UFS snapshots, but that depends on your preferences. RAID. Minimizes or avoids downtime when a disk fails. At the expense of having to deal with disk failures more often (because you have more disks), albeit at a much lower urgency. Checking fingerprints of files. The &man.mtree.8; utility is very useful for this. Although it is not a backup technique, it helps guarantee that you will notice when you need to resort to your backups. This is particularly important for offline backups, and should be checked periodically. It is quite easy to come up with even more techniques, many of them variations on the ones listed above. Specialized requirements will usually lead to specialized techniques (for example, backing up a live database usually requires a method particular to the database software as an intermediate step). The important thing is to know what dangers you want to protect against, and how you will handle each. Backup Basics The three major backup programs are &man.dump.8;, &man.tar.1;, and &man.cpio.1;. Dump and Restore backup software dump / restore dump restore The traditional &unix; backup programs are dump and restore. They operate on the drive as a collection of disk blocks, below the abstractions of files, links and directories that are created by the file systems. dump backs up an entire file system on a device. It is unable to backup only part of a file system or a directory tree that spans more than one file system. dump does not write files and directories to tape, but rather writes the raw data blocks that comprise files and directories. If you use dump on your root directory, you would not back up /home, /usr or many other directories since these are typically mount points for other file systems or symbolic links into those file systems. dump has quirks that remain from its early days in Version 6 of AT&T UNIX (circa 1975). The default parameters are suitable for 9-track tapes (6250 bpi), not the high-density media available today (up to 62,182 ftpi). These defaults must be overridden on the command line to utilize the capacity of current tape drives. .rhosts It is also possible to backup data across the network to a tape drive attached to another computer with rdump and rrestore. Both programs rely upon &man.rcmd.3; and &man.ruserok.3; to access the remote tape drive. Therefore, the user performing the backup must be listed in the .rhosts file on the remote computer. The arguments to rdump and rrestore must be suitable to use on the remote computer. When rdumping from a FreeBSD computer to an Exabyte tape drive connected to a Sun called komodo, use: &prompt.root; /sbin/rdump 0dsbfu 54000 13000 126 komodo:/dev/nsa8 /dev/da0a 2>&1 Beware: there are security implications to allowing .rhosts authentication. Evaluate your situation carefully. It is also possible to use dump and restore in a more secure fashion over ssh. Using <command>dump</command> over <application>ssh</application> &prompt.root; /sbin/dump -0uan -f - /usr | gzip -2 | ssh -c blowfish \ targetuser@targetmachine.example.com dd of=/mybigfiles/dump-usr-l0.gz Or using dump's built-in method, setting the environment variable RSH: Using <command>dump</command> over <application>ssh</application> with <envar>RSH</envar> set &prompt.root; RSH=/usr/bin/ssh /sbin/dump -0uan -f targetuser@targetmachine.example.com:/dev/sa0 /usr <command>tar</command> backup software tar &man.tar.1; also dates back to Version 6 of AT&T UNIX (circa 1975). tar operates in cooperation with the file system; it writes files and directories to tape. tar does not support the full range of options that are available from &man.cpio.1;, but it does not require the unusual command pipeline that cpio uses. tar On FreeBSD 5.3 and later, both GNU tar and the default bsdtar are available. The GNU version can be invoked with gtar. It supports remote devices using the same syntax as rdump. To tar to an Exabyte tape drive connected to a Sun called komodo, use: &prompt.root; /usr/bin/gtar cf komodo:/dev/nsa8 . 2>&1 The same could be accomplished with bsdtar by using a pipeline and rsh to send the data to a remote tape drive. &prompt.root; tar cf - . | rsh hostname dd of=tape-device obs=20b If you are worried about the security of backing up over a network you should use the ssh command instead of rsh. <command>cpio</command> backup software cpio &man.cpio.1; is the original &unix; file interchange tape program for magnetic media. cpio has options (among many others) to perform byte-swapping, write a number of different archive formats, and pipe the data to other programs. This last feature makes cpio an excellent choice for installation media. cpio does not know how to walk the directory tree and a list of files must be provided through stdin. cpio cpio does not support backups across the network. You can use a pipeline and rsh to send the data to a remote tape drive. &prompt.root; for f in directory_list; do find $f >> backup.list done &prompt.root; cpio -v -o --format=newc < backup.list | ssh user@host "cat > backup_device" Where directory_list is the list of directories you want to back up, user@host is the user/hostname combination that will be performing the backups, and backup_device is where the backups should be written to (e.g., /dev/nsa0). <command>pax</command> backup software pax pax POSIX IEEE &man.pax.1; is IEEE/&posix;'s answer to tar and cpio. Over the years the various versions of tar and cpio have gotten slightly incompatible. So rather than fight it out to fully standardize them, &posix; created a new archive utility. pax attempts to read and write many of the various cpio and tar formats, plus new formats of its own. Its command set more resembles cpio than tar. <application>Amanda</application> backup software Amanda Amanda Amanda (Advanced Maryland Network Disk Archiver) is a client/server backup system, rather than a single program. An Amanda server will backup to a single tape drive any number of computers that have Amanda clients and a network connection to the Amanda server. A common problem at sites with a number of large disks is that the length of time required to backup to data directly to tape exceeds the amount of time available for the task. Amanda solves this problem. Amanda can use a holding disk to backup several file systems at the same time. Amanda creates archive sets: a group of tapes used over a period of time to create full backups of all the file systems listed in Amanda's configuration file. The archive set also contains nightly incremental (or differential) backups of all the file systems. Restoring a damaged file system requires the most recent full backup and the incremental backups. The configuration file provides fine control of backups and the network traffic that Amanda generates. Amanda will use any of the above backup programs to write the data to tape. Amanda is available as either a port or a package, it is not installed by default. Do Nothing Do nothing is not a computer program, but it is the most widely used backup strategy. There are no initial costs. There is no backup schedule to follow. Just say no. If something happens to your data, grin and bear it! If your time and your data is worth little to nothing, then Do nothing is the most suitable backup program for your computer. But beware, &unix; is a useful tool, you may find that within six months you have a collection of files that are valuable to you. Do nothing is the correct backup method for /usr/obj and other directory trees that can be exactly recreated by your computer. An example is the files that comprise the HTML or &postscript; version of this Handbook. These document formats have been created from SGML input files. Creating backups of the HTML or &postscript; files is not necessary. The SGML files are backed up regularly. Which Backup Program Is Best? LISA &man.dump.8; Period. Elizabeth D. Zwicky torture tested all the backup programs discussed here. The clear choice for preserving all your data and all the peculiarities of &unix; file systems is dump. Elizabeth created file systems containing a large variety of unusual conditions (and some not so unusual ones) and tested each program by doing a backup and restore of those file systems. The peculiarities included: files with holes, files with holes and a block of nulls, files with funny characters in their names, unreadable and unwritable files, devices, files that change size during the backup, files that are created/deleted during the backup and more. She presented the results at LISA V in Oct. 1991. See torture-testing Backup and Archive Programs. Emergency Restore Procedure Before the Disaster There are only four steps that you need to perform in preparation for any disaster that may occur. bsdlabel First, print the bsdlabel from each of your disks (e.g. bsdlabel da0 | lpr), your file system table (/etc/fstab) and all boot messages, two copies of each. fix-it floppies Second, determine that the boot and fix-it floppies (boot.flp and fixit.flp) have all your devices. The easiest way to check is to reboot your machine with the boot floppy in the floppy drive and check the boot messages. If all your devices are listed and functional, skip on to step three. Otherwise, you have to create two custom bootable floppies which have a kernel that can mount all of your disks and access your tape drive. These floppies must contain: fdisk, bsdlabel, newfs, mount, and whichever backup program you use. These programs must be statically linked. If you use dump, the floppy must contain restore. Third, create backup tapes regularly. Any changes that you make after your last backup may be irretrievably lost. Write-protect the backup tapes. Fourth, test the floppies (either boot.flp and fixit.flp or the two custom bootable floppies you made in step two.) and backup tapes. Make notes of the procedure. Store these notes with the bootable floppy, the printouts and the backup tapes. You will be so distraught when restoring that the notes may prevent you from destroying your backup tapes (How? In place of tar xvf /dev/sa0, you might accidentally type tar cvf /dev/sa0 and over-write your backup tape). For an added measure of security, make bootable floppies and two backup tapes each time. Store one of each at a remote location. A remote location is NOT the basement of the same office building. A number of firms in the World Trade Center learned this lesson the hard way. A remote location should be physically separated from your computers and disk drives by a significant distance. A Script for Creating a Bootable Floppy /mnt/sbin/init gzip -c -best /sbin/fsck > /mnt/sbin/fsck gzip -c -best /sbin/mount > /mnt/sbin/mount gzip -c -best /sbin/halt > /mnt/sbin/halt gzip -c -best /sbin/restore > /mnt/sbin/restore gzip -c -best /bin/sh > /mnt/bin/sh gzip -c -best /bin/sync > /mnt/bin/sync cp /root/.profile /mnt/root chmod 500 /mnt/sbin/init chmod 555 /mnt/sbin/fsck /mnt/sbin/mount /mnt/sbin/halt chmod 555 /mnt/bin/sh /mnt/bin/sync chmod 6555 /mnt/sbin/restore # # create minimum file system table # cat > /mnt/etc/fstab < /mnt/etc/passwd < /mnt/etc/master.passwd < After the Disaster The key question is: did your hardware survive? You have been doing regular backups so there is no need to worry about the software. If the hardware has been damaged, the parts should be replaced before attempting to use the computer. If your hardware is okay, check your floppies. If you are using a custom boot floppy, boot single-user (type -s at the boot: prompt). Skip the following paragraph. If you are using the boot.flp and fixit.flp floppies, keep reading. Insert the boot.flp floppy in the first floppy drive and boot the computer. The original install menu will be displayed on the screen. Select the Fixit--Repair mode with CDROM or floppy. option. Insert the fixit.flp when prompted. restore and the other programs that you need are located in /mnt2/rescue (/mnt2/stand for &os; versions older than 5.2). Recover each file system separately. mount root partition bsdlabel newfs Try to mount (e.g. mount /dev/da0a /mnt) the root partition of your first disk. If the bsdlabel was damaged, use bsdlabel to re-partition and label the disk to match the label that you printed and saved. Use newfs to re-create the file systems. Re-mount the root partition of the floppy read-write (mount -u -o rw /mnt). Use your backup program and backup tapes to recover the data for this file system (e.g. restore vrf /dev/sa0). Unmount the file system (e.g. umount /mnt). Repeat for each file system that was damaged. Once your system is running, backup your data onto new tapes. Whatever caused the crash or data loss may strike again. Another hour spent now may save you from further distress later. * I Did Not Prepare for the Disaster, What Now? ]]> Marc Fonvieille Reorganized and enhanced by Network, Memory, and File-Backed File Systems virtual disks disks virtual Aside from the disks you physically insert into your computer: floppies, CDs, hard drives, and so forth; other forms of disks are understood by FreeBSD - the virtual disks. NFS Coda disks memory These include network file systems such as the Network File System and Coda, memory-based file systems and file-backed file systems. According to the FreeBSD version you run, you will have to use different tools for creation and use of file-backed and memory-based file systems. Use &man.devfs.5; to allocate device nodes transparently for the user. File-Backed File System disks file-backed The utility &man.mdconfig.8; is used to configure and enable memory disks, &man.md.4;, under FreeBSD. To use &man.mdconfig.8;, you have to load &man.md.4; module or to add the support in your kernel configuration file: device md The &man.mdconfig.8; command supports three kinds of memory backed virtual disks: memory disks allocated with &man.malloc.9;, memory disks using a file or swap space as backing. One possible use is the mounting of floppy or CD images kept in files. To mount an existing file system image: Using <command>mdconfig</command> to Mount an Existing File System Image &prompt.root; mdconfig -a -t vnode -f diskimage -u 0 &prompt.root; mount /dev/md0 /mnt To create a new file system image with &man.mdconfig.8;: Creating a New File-Backed Disk with <command>mdconfig</command> &prompt.root; dd if=/dev/zero of=newimage bs=1k count=5k 5120+0 records in 5120+0 records out &prompt.root; mdconfig -a -t vnode -f newimage -u 0 &prompt.root; bsdlabel -w md0 auto &prompt.root; newfs md0a /dev/md0a: 5.0MB (10224 sectors) block size 16384, fragment size 2048 using 4 cylinder groups of 1.25MB, 80 blks, 192 inodes. super-block backups (for fsck -b #) at: 160, 2720, 5280, 7840 &prompt.root; mount /dev/md0a /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0a 4710 4 4330 0% /mnt If you do not specify the unit number with the option, &man.mdconfig.8; will use the &man.md.4; automatic allocation to select an unused device. The name of the allocated unit will be output on stdout like md4. For more details about &man.mdconfig.8;, please refer to the manual page. The utility &man.mdconfig.8; is very useful, however it asks many command lines to create a file-backed file system. FreeBSD also comes with a tool called &man.mdmfs.8;, this program configures a &man.md.4; disk using &man.mdconfig.8;, puts a UFS file system on it using &man.newfs.8;, and mounts it using &man.mount.8;. For example, if you want to create and mount the same file system image as above, simply type the following: Configure and Mount a File-Backed Disk with <command>mdmfs</command> &prompt.root; dd if=/dev/zero of=newimage bs=1k count=5k 5120+0 records in 5120+0 records out &prompt.root; mdmfs -F newimage -s 5m md0 /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0 4718 4 4338 0% /mnt If you use the option without unit number, &man.mdmfs.8; will use &man.md.4; auto-unit feature to automatically select an unused device. For more details about &man.mdmfs.8;, please refer to the manual page. Memory-Based File System disks memory file system For a memory-based file system the swap backing should normally be used. Using swap backing does not mean that the memory disk will be swapped out to disk by default, but merely that the memory disk will be allocated from a memory pool which can be swapped out to disk if needed. It is also possible to create memory-based disk which are &man.malloc.9; backed, but using malloc backed memory disks, especially large ones, can result in a system panic if the kernel runs out of memory. Creating a New Memory-Based Disk with <command>mdconfig</command> &prompt.root; mdconfig -a -t swap -s 5m -u 1 &prompt.root; newfs -U md1 /dev/md1: 5.0MB (10240 sectors) block size 16384, fragment size 2048 using 4 cylinder groups of 1.27MB, 81 blks, 192 inodes. with soft updates super-block backups (for fsck -b #) at: 160, 2752, 5344, 7936 &prompt.root; mount /dev/md1 /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md1 4718 4 4338 0% /mnt Creating a New Memory-Based Disk with <command>mdmfs</command> &prompt.root; mdmfs -s 5m md2 /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md2 4846 2 4458 0% /mnt Detaching a Memory Disk from the System disks detaching a memory disk When a memory-based or file-based file system is not used, you should release all resources to the system. The first thing to do is to unmount the file system, then use &man.mdconfig.8; to detach the disk from the system and release the resources. For example to detach and free all resources used by /dev/md4: &prompt.root; mdconfig -d -u 4 It is possible to list information about configured &man.md.4; devices in using the command mdconfig -l. Tom Rhodes Contributed by File System Snapshots file systems snapshots FreeBSD offers a feature in conjunction with Soft Updates: File system snapshots. Snapshots allow a user to create images of specified file systems, and treat them as a file. Snapshot files must be created in the file system that the action is performed on, and a user may create no more than 20 snapshots per file system. Active snapshots are recorded in the superblock so they are persistent across unmount and remount operations along with system reboots. When a snapshot is no longer required, it can be removed with the standard &man.rm.1; command. Snapshots may be removed in any order, however all the used space may not be acquired because another snapshot will possibly claim some of the released blocks. The un-alterable file flag is set by &man.mksnap.ffs.8; after initial creation of a snapshot file. The &man.unlink.1; command makes an exception for snapshot files since it allows them to be removed. Snapshots are created with the &man.mount.8; command. To place a snapshot of /var in the file /var/snapshot/snap use the following command: &prompt.root; mount -u -o snapshot /var/snapshot/snap /var Alternatively, you can use &man.mksnap.ffs.8; to create a snapshot: &prompt.root; mksnap_ffs /var /var/snapshot/snap One can find snapshot files on a file system (e.g. /var) by using the &man.find.1; command: &prompt.root; find /var -flags snapshot Once a snapshot has been created, it has several uses: Some administrators will use a snapshot file for backup purposes, because the snapshot can be transfered to CDs or tape. The file system integrity checker, &man.fsck.8;, may be run on the snapshot. Assuming that the file system was clean when it was mounted, you should always get a clean (and unchanging) result. This is essentially what the background &man.fsck.8; process does. Run the &man.dump.8; utility on the snapshot. A dump will be returned that is consistent with the file system and the timestamp of the snapshot. &man.dump.8; can also take a snapshot, create a dump image and then remove the snapshot in one command using the flag. &man.mount.8; the snapshot as a frozen image of the file system. To &man.mount.8; the snapshot /var/snapshot/snap run: &prompt.root; mdconfig -a -t vnode -f /var/snapshot/snap -u 4 &prompt.root; mount -r /dev/md4 /mnt You can now walk the hierarchy of your frozen /var file system mounted at /mnt. Everything will initially be in the same state it was during the snapshot creation time. The only exception is that any earlier snapshots will appear as zero length files. When the use of a snapshot has delimited, it can be unmounted with: &prompt.root; umount /mnt &prompt.root; mdconfig -d -u 4 For more information about and file system snapshots, including technical papers, you can visit Marshall Kirk McKusick's website at . File System Quotas accounting disk space disk quotas Quotas are an optional feature of the operating system that allow you to limit the amount of disk space and/or the number of files a user or members of a group may allocate on a per-file system basis. This is used most often on timesharing systems where it is desirable to limit the amount of resources any one user or group of users may allocate. This will prevent one user or group of users from consuming all of the available disk space. Configuring Your System to Enable Disk Quotas Before attempting to use disk quotas, it is necessary to make sure that quotas are configured in your kernel. This is done by adding the following line to your kernel configuration file: options QUOTA The stock GENERIC kernel does not have this enabled by default, so you will have to configure, build and install a custom kernel in order to use disk quotas. Please refer to for more information on kernel configuration. Next you will need to enable disk quotas in /etc/rc.conf. This is done by adding the line: enable_quotas="YES" disk quotas checking For finer control over your quota startup, there is an additional configuration variable available. Normally on bootup, the quota integrity of each file system is checked by the &man.quotacheck.8; program. The &man.quotacheck.8; facility insures that the data in the quota database properly reflects the data on the file system. This is a very time consuming process that will significantly affect the time your system takes to boot. If you would like to skip this step, a variable in /etc/rc.conf is made available for the purpose: check_quotas="NO" Finally you will need to edit /etc/fstab to enable disk quotas on a per-file system basis. This is where you can either enable user or group quotas or both for all of your file systems. To enable per-user quotas on a file system, add the option to the options field in the /etc/fstab entry for the file system you want to enable quotas on. For example: /dev/da1s2g /home ufs rw,userquota 1 2 Similarly, to enable group quotas, use the option instead of . To enable both user and group quotas, change the entry as follows: /dev/da1s2g /home ufs rw,userquota,groupquota 1 2 By default, the quota files are stored in the root directory of the file system with the names quota.user and quota.group for user and group quotas respectively. See &man.fstab.5; for more information. Even though the &man.fstab.5; manual page says that you can specify an alternate location for the quota files, this is not recommended because the various quota utilities do not seem to handle this properly. At this point you should reboot your system with your new kernel. /etc/rc will automatically run the appropriate commands to create the initial quota files for all of the quotas you enabled in /etc/fstab, so there is no need to manually create any zero length quota files. In the normal course of operations you should not be required to run the &man.quotacheck.8;, &man.quotaon.8;, or &man.quotaoff.8; commands manually. However, you may want to read their manual pages just to be familiar with their operation. Setting Quota Limits disk quotas limits Once you have configured your system to enable quotas, verify that they really are enabled. An easy way to do this is to run: &prompt.root; quota -v You should see a one line summary of disk usage and current quota limits for each file system that quotas are enabled on. You are now ready to start assigning quota limits with the &man.edquota.8; command. You have several options on how to enforce limits on the amount of disk space a user or group may allocate, and how many files they may create. You may limit allocations based on disk space (block quotas) or number of files (inode quotas) or a combination of both. Each of these limits are further broken down into two categories: hard and soft limits. hard limit A hard limit may not be exceeded. Once a user reaches his hard limit he may not make any further allocations on the file system in question. For example, if the user has a hard limit of 500 kbytes on a file system and is currently using 490 kbytes, the user can only allocate an additional 10 kbytes. Attempting to allocate an additional 11 kbytes will fail. soft limit Soft limits, on the other hand, can be exceeded for a limited amount of time. This period of time is known as the grace period, which is one week by default. If a user stays over his or her soft limit longer than the grace period, the soft limit will turn into a hard limit and no further allocations will be allowed. When the user drops back below the soft limit, the grace period will be reset. The following is an example of what you might see when you run the &man.edquota.8; command. When the &man.edquota.8; command is invoked, you are placed into the editor specified by the EDITOR environment variable, or in the vi editor if the EDITOR variable is not set, to allow you to edit the quota limits. &prompt.root; edquota -u test Quotas for user test: /usr: kbytes in use: 65, limits (soft = 50, hard = 75) inodes in use: 7, limits (soft = 50, hard = 60) /usr/var: kbytes in use: 0, limits (soft = 50, hard = 75) inodes in use: 0, limits (soft = 50, hard = 60) You will normally see two lines for each file system that has quotas enabled. One line for the block limits, and one line for inode limits. Simply change the value you want updated to modify the quota limit. For example, to raise this user's block limit from a soft limit of 50 and a hard limit of 75 to a soft limit of 500 and a hard limit of 600, change: /usr: kbytes in use: 65, limits (soft = 50, hard = 75) to: /usr: kbytes in use: 65, limits (soft = 500, hard = 600) The new quota limits will be in place when you exit the editor. Sometimes it is desirable to set quota limits on a range of UIDs. This can be done by use of the option on the &man.edquota.8; command. First, assign the desired quota limit to a user, and then run edquota -p protouser startuid-enduid. For example, if user test has the desired quota limits, the following command can be used to duplicate those quota limits for UIDs 10,000 through 19,999: &prompt.root; edquota -p test 10000-19999 For more information see &man.edquota.8; manual page. Checking Quota Limits and Disk Usage disk quotas checking You can use either the &man.quota.1; or the &man.repquota.8; commands to check quota limits and disk usage. The &man.quota.1; command can be used to check individual user or group quotas and disk usage. A user may only examine his own quota, and the quota of a group he is a member of. Only the super-user may view all user and group quotas. The &man.repquota.8; command can be used to get a summary of all quotas and disk usage for file systems with quotas enabled. The following is some sample output from the quota -v command for a user that has quota limits on two file systems. Disk quotas for user test (uid 1002): Filesystem usage quota limit grace files quota limit grace /usr 65* 50 75 5days 7 50 60 /usr/var 0 50 75 0 50 60 grace period On the /usr file system in the above example, this user is currently 15 kbytes over the soft limit of 50 kbytes and has 5 days of the grace period left. Note the asterisk * which indicates that the user is currently over his quota limit. Normally file systems that the user is not using any disk space on will not show up in the output from the &man.quota.1; command, even if he has a quota limit assigned for that file system. The option will display those file systems, such as the /usr/var file system in the above example. Quotas over NFS NFS Quotas are enforced by the quota subsystem on the NFS server. The &man.rpc.rquotad.8; daemon makes quota information available to the &man.quota.1; command on NFS clients, allowing users on those machines to see their quota statistics. Enable rpc.rquotad in /etc/inetd.conf like so: rquotad/1 dgram rpc/udp wait root /usr/libexec/rpc.rquotad rpc.rquotad Now restart inetd: &prompt.root; /etc/rc.d/inetd restart Lucky Green Contributed by
shamrock@cypherpunks.to
 Encrypting Disk Partitions disks encrypting FreeBSD offers excellent online protections against unauthorized data access. File permissions and Mandatory Access Control (MAC) (see ) help prevent unauthorized third-parties from accessing data while the operating system is active and the computer is powered up. However, the permissions enforced by the operating system are irrelevant if an attacker has physical access to a computer and can simply move the computer's hard drive to another system to copy and analyze the sensitive data. Regardless of how an attacker may have come into possession of a hard drive or powered-down computer, both GEOM Based Disk Encryption (gbde) and geli cryptographic subsystems in &os; are able to protect the data on the computer's file systems against even highly-motivated attackers with significant resources. Unlike cumbersome encryption methods that encrypt only individual files, gbde and geli transparently encrypt entire file systems. No cleartext ever touches the hard drive's platter. Disk Encryption with <application>gbde</application> Become <username>root</username> Configuring gbde requires super-user privileges. &prompt.user; su - Password: Add &man.gbde.4; Support to the Kernel Configuration File Add the following line to the kernel configuration file: options GEOM_BDE Rebuild the kernel as described in . Reboot into the new kernel. An alternative to recompiling the kernel is to use kldload to load &man.gbde.4;: &prompt.root; kldload geom_bde Preparing the Encrypted Hard Drive The following example assumes that you are adding a new hard drive to your system that will hold a single encrypted partition. This partition will be mounted as /private. gbde can also be used to encrypt /home and /var/mail, but this requires more complex instructions which exceed the scope of this introduction. Add the New Hard Drive Install the new drive to the system as explained in . For the purposes of this example, a new hard drive partition has been added as /dev/ad4s1c. The /dev/ad0s1* devices represent existing standard FreeBSD partitions on the example system. &prompt.root; ls /dev/ad* /dev/ad0 /dev/ad0s1b /dev/ad0s1e /dev/ad4s1 /dev/ad0s1 /dev/ad0s1c /dev/ad0s1f /dev/ad4s1c /dev/ad0s1a /dev/ad0s1d /dev/ad4 Create a Directory to Hold gbde Lock Files &prompt.root; mkdir /etc/gbde The gbde lock file contains information that gbde requires to access encrypted partitions. Without access to the lock file, gbde will not be able to decrypt the data contained in the encrypted partition without significant manual intervention which is not supported by the software. Each encrypted partition uses a separate lock file. Initialize the gbde Partition A gbde partition must be initialized before it can be used. This initialization needs to be performed only once: &prompt.root; gbde init /dev/ad4s1c -i -L /etc/gbde/ad4s1c.lock &man.gbde.8; will open your editor, permitting you to set various configuration options in a template. For use with UFS1 or UFS2, set the sector_size to 2048: $FreeBSD: src/sbin/gbde/template.txt,v 1.1 2002/10/20 11:16:13 phk Exp $ # # Sector size is the smallest unit of data which can be read or written. # Making it too small decreases performance and decreases available space. # Making it too large may prevent filesystems from working. 512 is the # minimum and always safe. For UFS, use the fragment size # sector_size = 2048 [...] &man.gbde.8; will ask you twice to type the passphrase that should be used to secure the data. The passphrase must be the same both times. gbde's ability to protect your data depends entirely on the quality of the passphrase that you choose. For tips on how to select a secure passphrase that is easy to remember, see the Diceware Passphrase website. The gbde init command creates a lock file for your gbde partition that in this example is stored as /etc/gbde/ad4s1c.lock. gbde lock files must end in .lock in order to be correctly detected by the /etc/rc.d/gbde start up script. gbde lock files must be backed up together with the contents of any encrypted partitions. While deleting a lock file alone cannot prevent a determined attacker from decrypting a gbde partition, without the lock file, the legitimate owner will be unable to access the data on the encrypted partition without a significant amount of work that is totally unsupported by &man.gbde.8; and its designer. Attach the Encrypted Partition to the Kernel &prompt.root; gbde attach /dev/ad4s1c -l /etc/gbde/ad4s1c.lock You will be asked to provide the passphrase that you selected during the initialization of the encrypted partition. The new encrypted device will show up in /dev as /dev/device_name.bde: &prompt.root; ls /dev/ad* /dev/ad0 /dev/ad0s1b /dev/ad0s1e /dev/ad4s1 /dev/ad0s1 /dev/ad0s1c /dev/ad0s1f /dev/ad4s1c /dev/ad0s1a /dev/ad0s1d /dev/ad4 /dev/ad4s1c.bde Create a File System on the Encrypted Device Once the encrypted device has been attached to the kernel, you can create a file system on the device. To create a file system on the encrypted device, use &man.newfs.8;. Since it is much faster to initialize a new UFS2 file system than it is to initialize the old UFS1 file system, using &man.newfs.8; with the option is recommended. &prompt.root; newfs -U -O2 /dev/ad4s1c.bde The &man.newfs.8; command must be performed on an attached gbde partition which is identified by a *.bde extension to the device name. Mount the Encrypted Partition Create a mount point for the encrypted file system. &prompt.root; mkdir /private Mount the encrypted file system. &prompt.root; mount /dev/ad4s1c.bde /private Verify That the Encrypted File System is Available The encrypted file system should now be visible to &man.df.1; and be available for use. &prompt.user; df -H Filesystem Size Used Avail Capacity Mounted on /dev/ad0s1a 1037M 72M 883M 8% / /devfs 1.0K 1.0K 0B 100% /dev /dev/ad0s1f 8.1G 55K 7.5G 0% /home /dev/ad0s1e 1037M 1.1M 953M 0% /tmp /dev/ad0s1d 6.1G 1.9G 3.7G 35% /usr /dev/ad4s1c.bde 150G 4.1K 138G 0% /private Mounting Existing Encrypted File Systems After each boot, any encrypted file systems must be re-attached to the kernel, checked for errors, and mounted, before the file systems can be used. The required commands must be executed as user root. Attach the gbde Partition to the Kernel &prompt.root; gbde attach /dev/ad4s1c -l /etc/gbde/ad4s1c.lock You will be asked to provide the passphrase that you selected during initialization of the encrypted gbde partition. Check the File System for Errors Since encrypted file systems cannot yet be listed in /etc/fstab for automatic mounting, the file systems must be checked for errors by running &man.fsck.8; manually before mounting. &prompt.root; fsck -p -t ffs /dev/ad4s1c.bde Mount the Encrypted File System &prompt.root; mount /dev/ad4s1c.bde /private The encrypted file system is now available for use. Automatically Mounting Encrypted Partitions It is possible to create a script to automatically attach, check, and mount an encrypted partition, but for security reasons the script should not contain the &man.gbde.8; password. Instead, it is recommended that such scripts be run manually while providing the password via the console or &man.ssh.1;. As an alternative, an rc.d script is provided. Arguments for this script can be passed via &man.rc.conf.5;, for example: gbde_autoattach_all="YES" gbde_devices="ad4s1c" gbde_lockdir="/etc/gbde" This will require that the gbde passphrase be entered at boot time. After typing the correct passphrase, the gbde encrypted partition will be mounted automatically. This can be very useful when using gbde on notebooks. Cryptographic Protections Employed by gbde &man.gbde.8; encrypts the sector payload using 128-bit AES in CBC mode. Each sector on the disk is encrypted with a different AES key. For more information on gbde's cryptographic design, including how the sector keys are derived from the user-supplied passphrase, see &man.gbde.4;. Compatibility Issues &man.sysinstall.8; is incompatible with gbde-encrypted devices. All *.bde devices must be detached from the kernel before starting &man.sysinstall.8; or it will crash during its initial probing for devices. To detach the encrypted device used in our example, use the following command: &prompt.root; gbde detach /dev/ad4s1c Also note that, as &man.vinum.4; does not use the &man.geom.4; subsystem, you cannot use gbde with vinum volumes. Daniel Gerzo Contributed by Disk Encryption with <command>geli</command> A new cryptographic GEOM class is available as of &os; 6.0 - geli. It is currently being developed by &a.pjd;. The geli utility is different to gbde; it offers different features and uses a different scheme for doing cryptographic work. The most important features of &man.geli.8; are: Utilizes the &man.crypto.9; framework — when cryptographic hardware is available, geli will use it automatically. Supports multiple cryptographic algorithms (currently AES, Blowfish, and 3DES). Allows the root partition to be encrypted. The passphrase used to access the encrypted root partition will be requested during the system boot. Allows the use of two independent keys (e.g. a key and a company key). geli is fast - performs simple sector-to-sector encryption. Allows backup and restore of Master Keys. When a user has to destroy his keys, it will be possible to get access to the data again by restoring keys from the backup. Allows to attach a disk with a random, one-time key — useful for swap partitions and temporary file systems. More geli features can be found in the &man.geli.8; manual page. The next steps will describe how to enable support for geli in the &os; kernel and will explain how to create and use a geli encryption provider. In order to use geli, you must be running &os; 6.0-RELEASE or later. Super-user privileges will be required since modifications to the kernel are necessary. Adding <command>geli</command> Support to the Kernel Add the following lines to the kernel configuration file: options GEOM_ELI device crypto Rebuild the kernel as described in . Alternatively, the geli module can be loaded at boot time. Add the following line to the /boot/loader.conf: geom_eli_load="YES" &man.geli.8; should now be supported by the kernel. Generating the Master Key The following example will describe how to generate a key file, which will be used as part of the Master Key for the encrypted provider mounted under - /private. The key + /private. The key file will provide some random data used to encrypt the Master Key. The Master Key will be protected by a passphrase as well. Provider's sector size will be 4kB big. Furthermore, the discussion will describe how to attach the geli provider, create a file system on it, how to mount it, how to work with it, and finally how to detach it. It is recommended to use a bigger sector size (like 4kB) for better performance. The Master Key will be protected with a passphrase and the data source for key file will be /dev/random. The sector size of /dev/da2.eli, which we call provider, will be 4kB. &prompt.root; dd if=/dev/random of=/root/da2.key bs=64 count=1 &prompt.root; geli init -s 4096 -K /root/da2.key /dev/da2 Enter new passphrase: Reenter new passphrase: It is not mandatory that both a passphrase and a key file are used; either method of securing the Master Key can be used in isolation. If key file is given as -, standard input will be used. This example shows how more than one key file can be used. &prompt.root; cat keyfile1 keyfile2 keyfile3 | geli init -K - /dev/da2 Attaching the Provider with the generated Key &prompt.root; geli attach -k /root/da2.key /dev/da2 Enter passphrase: The new plaintext device will be named /dev/da2.eli. &prompt.root; ls /dev/da2* /dev/da2 /dev/da2.eli Creating the new File System &prompt.root; dd if=/dev/random of=/dev/da2.eli bs=1m &prompt.root; newfs /dev/da2.eli &prompt.root; mount /dev/da2.eli /private The encrypted file system should be visible to &man.df.1; and be available for use now: &prompt.root; df -H Filesystem Size Used Avail Capacity Mounted on /dev/ad0s1a 248M 89M 139M 38% / /devfs 1.0K 1.0K 0B 100% /dev /dev/ad0s1f 7.7G 2.3G 4.9G 32% /usr /dev/ad0s1d 989M 1.5M 909M 0% /tmp /dev/ad0s1e 3.9G 1.3G 2.3G 35% /var /dev/da2.eli 150G 4.1K 138G 0% /private Unmounting and Detaching the Provider Once the work on the encrypted partition is done, and - the /private partition + the /private partition is no longer needed, it is prudent to consider unmounting and detaching the geli encrypted partition from the kernel. &prompt.root; umount /private &prompt.root; geli detach da2.eli More information about the use of &man.geli.8; can be found in the manual page. Using the <filename>geli</filename> <filename>rc.d</filename> Script geli comes with a rc.d script which can be used to simplify the usage of geli. An example of configuring geli through &man.rc.conf.5; follows: geli_devices="da2" geli_da2_flags="-p -k /root/da2.key" This will configure /dev/da2 as a geli provider of which the Master Key file is located in /root/da2.key, and geli will not use a passphrase when attaching the provider (note that this can only be used if was given during the geli init phase). The system will detach the geli provider from the kernel before the system shuts down. More information about configuring rc.d is provided in the rc.d section of the Handbook. Christian Brüffer Written by Encrypting Swap Space swap encrypting Swap encryption in &os; is easy to configure and has been available since &os; 5.3-RELEASE. Depending on which version of &os; is being used, different options are available and configuration can vary slightly. From &os; 6.0-RELEASE onwards, the &man.gbde.8; or &man.geli.8; encryption systems can be used for swap encryption. With earlier versions, only &man.gbde.8; is available. Both systems use the encswap rc.d script. The previous section, Encrypting Disk Partitions, includes a short discussion on the different encryption systems. Why should Swap be Encrypted? Like the encryption of disk partitions, encryption of swap space is done to protect sensitive information. Imagine an application that e.g. deals with passwords. As long as these passwords stay in physical memory, all is well. However, if the operating system starts swapping out memory pages to free space for other applications, the passwords may be written to the disk platters unencrypted and easy to retrieve for an adversary. Encrypting swap space can be a solution for this scenario. Preparation For the remainder of this section, ad0s1b will be the swap partition. Up to this point the swap has been unencrypted. It is possible that there are already passwords or other sensitive data on the disk platters in cleartext. To rectify this, the data on the swap partition should be overwritten with random garbage: &prompt.root; dd if=/dev/random of=/dev/ad0s1b bs=1m Swap Encryption with &man.gbde.8; If &os; 6.0-RELEASE or newer is being used, the .bde suffix should be added to the device in the respective /etc/fstab swap line: # Device Mountpoint FStype Options Dump Pass# /dev/ad0s1b.bde none swap sw 0 0 For systems prior to &os; 6.0-RELEASE, the following line in /etc/rc.conf is also needed: gbde_swap_enable="YES" Swap Encryption with &man.geli.8; Alternatively, the procedure for using &man.geli.8; for swap encryption is similar to that of using &man.gbde.8;. The .eli suffix should be added to the device in the respective /etc/fstab swap line: # Device Mountpoint FStype Options Dump Pass# /dev/ad0s1b.eli none swap sw 0 0 &man.geli.8; uses the AES algorithm with a key length of 256 bit by default. Optionally, these defaults can be altered using the geli_swap_flags option in /etc/rc.conf. The following line tells the encswap rc.d script to create &man.geli.8; swap partitions using the Blowfish algorithm with a key length of 128 bit, a sectorsize of 4 kilobytes and the detach on last close option set: geli_swap_flags="-e blowfish -l 128 -s 4096 -d" For systems prior to &os; 6.2-RELEASE, use the following line: geli_swap_flags="-a blowfish -l 128 -s 4096 -d" Please refer to the description of the onetime command in the &man.geli.8; manual page for a list of possible options. Verifying that it Works Once the system has been rebooted, proper operation of the encrypted swap can be verified using the swapinfo command. If &man.gbde.8; is being used: &prompt.user; swapinfo Device 1K-blocks Used Avail Capacity /dev/ad0s1b.bde 542720 0 542720 0% If &man.geli.8; is being used: &prompt.user; swapinfo Device 1K-blocks Used Avail Capacity /dev/ad0s1b.eli 542720 0 542720 0% diff --git a/en_US.ISO8859-1/books/handbook/firewalls/chapter.sgml b/en_US.ISO8859-1/books/handbook/firewalls/chapter.sgml index 79fb73fc81..af77944824 100644 --- a/en_US.ISO8859-1/books/handbook/firewalls/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/firewalls/chapter.sgml @@ -1,3452 +1,3452 @@ Joseph J. Barbish Contributed by Brad Davis Converted to SGML and updated by Firewalls firewall security firewalls Introduction Firewalls make it possible to filter incoming and outgoing traffic that flows through your system. A firewall can use one or more sets of rules to inspect the network packets as they come in or go out of your network connections and either allows the traffic through or blocks it. The rules of a firewall can inspect one or more characteristics of the packets, including but not limited to the protocol type, the source or destination host address, and the source or destination port. Firewalls can greatly enhance the security of a host or a network. They can be used to do one or more of the following things: To protect and insulate the applications, services and machines of your internal network from unwanted traffic coming in from the public Internet. To limit or disable access from hosts of the internal network to services of the public Internet. To support network address translation (NAT), which allows your internal network to use private IP addresses and share a single connection to the public Internet (either with a single IP address or by a shared pool of automatically assigned public addresses). After reading this chapter, you will know: How to properly define packet filtering rules. The differences between the firewalls built into &os;. How to use and configure the OpenBSD PF firewall. How to use and configure IPFILTER. How to use and configure IPFW. Before reading this chapter, you should: Understand basic &os; and Internet concepts. Firewall Concepts firewall rulesets There are two basic ways to create firewall rulesets: inclusive or exclusive. An exclusive firewall allows all traffic through except for the traffic matching the ruleset. An inclusive firewall does the reverse. It only allows traffic matching the rules through and blocks everything else. Inclusive firewalls are generally safer than exclusive firewalls because they significantly reduce the risk of allowing unwanted traffic to pass through the firewall. Security can be tightened further using a stateful firewall. With a stateful firewall the firewall keeps track of which connections are opened through the firewall and will only allow traffic through which either matches an existing connection or opens a new one. The disadvantage of a stateful firewall is that it can be vulnerable to Denial of Service (DoS) attacks if a lot of new connections are opened very fast. With most firewalls it is possible to use a combination of stateful and non-stateful behavior to make an optimal firewall for the site. Firewall Packages &os; has three different firewall packages built into the base system. They are: IPFILTER (also known as IPF), IPFIREWALL (also known as IPFW), and OpenBSD's PacketFilter (also known as PF). &os; also has two built in packages for traffic shaping (basically controlling bandwidth usage): &man.altq.4; and &man.dummynet.4;. Dummynet has traditionally been closely tied with IPFW, and ALTQ with PF. Traffic shaping for IPFILTER can currently be done with IPFILTER for NAT and filtering and IPFW with &man.dummynet.4; or by using PF with ALTQ. IPFW, and PF all use rules to control the access of packets to and from your system, although they go about it different ways and have different rule syntaxes. The reason that &os; has multiple built in firewall packages is that different people have different requirements and preferences. No single firewall package is the best. The author prefers IPFILTER because its stateful rules are much less complicated to use in a NAT environment and it has a built in ftp proxy that simplifies the rules to allow secure outbound FTP usage. Since all firewalls are based on inspecting the values of selected packet control fields, the creator of the firewall rulesets must have an understanding of how TCP/IP works, what the different values in the packet control fields are and how these values are used in a normal session conversation. For a good explanation go to: . John Ferrell Revised and updated by The OpenBSD Packet Filter (PF) and <acronym>ALTQ</acronym> firewall PF As of July 2003 the OpenBSD firewall software application known as PF was ported to &os; and made available in the &os; Ports Collection. Released in 2004, &os; 5.3 was the first release that contained PF as an integrated part of the base system. PF is a complete, full-featured firewall that has optional support for ALTQ (Alternate Queuing). ALTQ provides Quality of Service (QoS) functionality. The OpenBSD Project does an outstanding job of maintaining the PF FAQ. As such, this section of the Handbook will focus on PF as it pertains to &os; while providing some general information regarding usage. For detailed usage information please refer to the PF FAQ. More information about PF for &os; can be found at . Using the PF loadable kernel module Since the release of &os; 5.3, PF has been included in the basic install as a separate run time loadable module. The system will dynamically load the PF kernel module when the &man.rc.conf.5; statement pf_enable="YES" is present. However, the PF module will not load if the system cannot find a PF ruleset configuration file. The default location is /etc/pf.conf. If your PF ruleset is located somewhere else put pf_rules="/path/pf.rules" to your /etc/rc.conf configuration file to specify the location. As of &os; 7.0 the sample pf.conf - that was in /etc/ has been + that was in /etc/ has been moved to /usr/share/examples/pf/. For &os; + class="directory">/usr/share/examples/pf/. For &os; versions prior to 7.0 there is an /etc/pf.conf by default. The PF module can also be loaded manually from the command line: &prompt.root; kldload pf.ko The loadable module was created with &man.pflog.4; enabled which provides support for logging. If you need other PF features you will need to compile PF support into the kernel. PF kernel options kernel options device pf kernel options device pflog kernel options device pfsync While it is not necessary that you compile PF support into the &os; kernel, you may want to do so to take advantage of one of PF's advanced features that is not included in the loadable module, namely &man.pfsync.4;, which is a pseudo-device that exposes certain changes to the state table used by PF. It can be paired with &man.carp.4; to create failover firewalls using PF. More information on CARP can be found in chapter 29 of the handbook. The PF kernel options can be found in /usr/src/sys/conf/NOTES and are reproduced below: device pf device pflog device pfsync The device pf option enables support for the Packet Filter firewall (&man.pf.4;). The device pflog option enables the optional &man.pflog.4; pseudo network device which can be used to log traffic to a &man.bpf.4; descriptor. The &man.pflogd.8; daemon can be used to store the logging information to disk. The device pfsync option enables the optional &man.pfsync.4; pseudo-network device that is used to monitor state changes. Available rc.conf Options The following &man.rc.conf.5; statements configure PF and &man.pflog.4; at boot: pf_enable="YES" # Enable PF (load module if required) pf_rules="/etc/pf.conf" # rules definition file for pf pf_flags="" # additional flags for pfctl startup pflog_enable="YES" # start pflogd(8) pflog_logfile="/var/log/pflog" # where pflogd should store the logfile pflog_flags="" # additional flags for pflogd startup If you have a LAN behind this firewall and have to forward packets for the computers on the LAN or want to do NAT, you will need the following option as well: gateway_enable="YES" # Enable as LAN gateway Creating Filtering Rules PF reads its configuration rules from &man.pf.conf.5; (/etc/pf.conf by default) and it modifies, drops, or passes packets according to the rules or definitions specified there. The &os; installation includes several sample files located in /usr/share/examples/pf/. Please refer to the PF FAQ for complete coverage of PF rulesets. When browsing the PF FAQ, please keep in mind that different versions of &os; contain different versions of PF: &os; 5.X — PF is at OpenBSD 3.5 &os; 6.X — PF is at OpenBSD 3.7 &os; 7.X — PF is at OpenBSD 4.1 The &a.pf; is a good place to ask questions about configuring and running the PF firewall. Do not forget to check the mailing list archives before asking questions! Working with PF Use &man.pfctl.8; to control PF. Below are some useful commands (be sure to review the &man.pfctl.8; man page for all available options): Command Purpose pfctl Enable PF pfctl Disable PF pfctl all /etc/pf.conf Flush all rules (nat, filter, state, table, etc.) and reload from the file /etc/pf.conf pfctl [ rules | nat | state ] Report on the filter rules, nat rules, or state table pfctl /etc/pf.conf Check /etc/pf.conf for errors, but do not load ruleset Enabling <acronym>ALTQ</acronym> ALTQ is only available by compiling support for it into the &os; kernel. ALTQ is not supported by all of the available network card drivers. Please see the &man.altq.4; manual page for a list of drivers that are supported in your release of &os;. The following kernel options will enable ALTQ and add additional functionality: options ALTQ options ALTQ_CBQ # Class Bases Queuing (CBQ) options ALTQ_RED # Random Early Detection (RED) options ALTQ_RIO # RED In/Out options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC) options ALTQ_PRIQ # Priority Queuing (PRIQ) options ALTQ_NOPCC # Required for SMP build options ALTQ enables the ALTQ framework. options ALTQ_CBQ enables Class Based Queuing (CBQ). CBQ allows you to divide a connection's bandwidth into different classes or queues to prioritize traffic based on filter rules. options ALTQ_RED enables Random Early Detection (RED). RED is used to avoid network congestion. RED does this by measuring the length of the queue and comparing it to the minimum and maximum thresholds for the queue. If the queue is over the maximum all new packets will be dropped. True to its name, RED drops packets from different connections randomly. options ALTQ_RIO enables Random Early Detection In and Out. options ALTQ_HFSC enables the Hierarchical Fair Service Curve Packet Scheduler. For more information about HFSC see: . options ALTQ_PRIQ enables Priority Queuing (PRIQ). PRIQ will always pass traffic that is in a higher queue first. options ALTQ_NOPCC enables SMP support for ALTQ. This option is required on SMP systems. The IPFILTER (IPF) Firewall firewall IPFILTER This section is work in progress. The contents might not be accurate at all times. The author of IPFILTER is Darren Reed. IPFILTER is not operating system dependent: it is an open source application and has been ported to &os;, NetBSD, OpenBSD, &sunos;, HP/UX, and &solaris; operating systems. IPFILTER is actively being supported and maintained, with updated versions being released regularly. IPFILTER is based on a kernel-side firewall and NAT mechanism that can be controlled and monitored by userland interface programs. The firewall rules can be set or deleted with the &man.ipf.8; utility. The NAT rules can be set or deleted with the &man.ipnat.1; utility. The &man.ipfstat.8; utility can print run-time statistics for the kernel parts of IPFILTER. The &man.ipmon.8; program can log IPFILTER actions to the system log files. IPF was originally written using a rule processing logic of the last matching rule wins and used only stateless type of rules. Over time IPF has been enhanced to include a quick option and a stateful keep state option which drastically modernized the rules processing logic. IPF's official documentation covers the legacy rule coding parameters and the legacy rule file processing logic. The modernized functions are only included as additional options, completely understating their benefits in producing a far superior secure firewall. The instructions contained in this section are based on using rules that contain the quick option and the stateful keep state option. This is the basic framework for coding an inclusive firewall rule set. An inclusive firewall only allows packets matching the rules to pass through. This way you can control what services can originate behind the firewall destined for the public Internet and also control the services which can originate from the public Internet accessing your private network. Everything else is blocked and logged by default design. Inclusive firewalls are much, much more secure than exclusive firewall rule sets and is the only rule set type covered herein. For detailed explanation of the legacy rules processing method see: and . The IPF FAQ is at . A searchable archive of the open-source IPFilter mailing list is available at . Enabling IPF IPFILTER enabling IPF is included in the basic &os; install as a separate run time loadable module. The system will dynamically load the IPF kernel loadable module when the rc.conf statement ipfilter_enable="YES" is used. The loadable module was created with logging enabled and the default pass all options. You do not need to compile IPF into the &os; kernel just to change the default to block all, you can do that by just coding a block all rule at the end of your rule set. Kernel options kernel options IPFILTER kernel options IPFILTER_LOG kernel options IPFILTER_DEFAULT_BLOCK IPFILTER kernel options It is not a mandatory requirement that you enable IPF by compiling the following options into the &os; kernel. It is only presented here as background information. Compiling IPF into the kernel causes the loadable module to never be used. Sample kernel config IPF option statements are in the /usr/src/sys/conf/NOTES kernel source and are reproduced here: options IPFILTER options IPFILTER_LOG options IPFILTER_DEFAULT_BLOCK options IPFILTER enables support for the IPFILTER firewall. options IPFILTER_LOG enables the option to have IPF log traffic by writing to the ipl packet logging pseudo—device for every rule that has the log keyword. options IPFILTER_DEFAULT_BLOCK changes the default behavior so any packet not matching a firewall pass rule gets blocked. These settings will take effect only after you have built and installed a kernel with them set. Available rc.conf Options You need the following statements in /etc/rc.conf to activate IPF at boot time: ipfilter_enable="YES" # Start ipf firewall ipfilter_rules="/etc/ipf.rules" # loads rules definition text file ipmon_enable="YES" # Start IP monitor log ipmon_flags="-Ds" # D = start as daemon # s = log to syslog # v = log tcp window, ack, seq # n = map IP & port to names If you have a LAN behind this firewall that uses the reserved private IP address ranges, then you need to add the following to enable NAT functionality: gateway_enable="YES" # Enable as LAN gateway ipnat_enable="YES" # Start ipnat function ipnat_rules="/etc/ipnat.rules" # rules definition file for ipnat IPF ipf The ipf command is used to load your rules file. Normally you create a file containing your custom rules and use this command to replace in mass the currently running firewall internal rules: &prompt.root; ipf -Fa -f /etc/ipf.rules means flush all internal rules tables. means this is the file to read for the rules to load. This gives you the ability to make changes to your custom rules file, run the above IPF command, and thus update the running firewall with a fresh copy of all the rules without having to reboot the system. This method is very convenient for testing new rules as the procedure can be executed as many times as needed. See the &man.ipf.8; manual page for details on the other flags available with this command. The &man.ipf.8; command expects the rules file to be a standard text file. It will not accept a rules file written as a script with symbolic substitution. There is a way to build IPF rules that utilizes the power of script symbolic substitution. For more information, see . IPFSTAT ipfstat IPFILTER statistics The default behavior of &man.ipfstat.8; is to retrieve and display the totals of the accumulated statistics gathered as a result of applying the user coded rules against packets going in and out of the firewall since it was last started, or since the last time the accumulators were reset to zero by the ipf -Z command. See the &man.ipfstat.8; manual page for details. The default &man.ipfstat.8; command output will look something like this: input packets: blocked 99286 passed 1255609 nomatch 14686 counted 0 output packets: blocked 4200 passed 1284345 nomatch 14687 counted 0 input packets logged: blocked 99286 passed 0 output packets logged: blocked 0 passed 0 packets logged: input 0 output 0 log failures: input 3898 output 0 fragment state(in): kept 0 lost 0 fragment state(out): kept 0 lost 0 packet state(in): kept 169364 lost 0 packet state(out): kept 431395 lost 0 ICMP replies: 0 TCP RSTs sent: 0 Result cache hits(in): 1215208 (out): 1098963 IN Pullups succeeded: 2 failed: 0 OUT Pullups succeeded: 0 failed: 0 Fastroute successes: 0 failures: 0 TCP cksum fails(in): 0 (out): 0 Packet log flags set: (0) When supplied with either for inbound or for outbound, it will retrieve and display the appropriate list of filter rules currently installed and in use by the kernel. ipfstat -in displays the inbound internal rules table with rule number. ipfstat -on displays the outbound internal rules table with the rule number. The output will look something like this: @1 pass out on xl0 from any to any @2 block out on dc0 from any to any @3 pass out quick on dc0 proto tcp/udp from any to any keep state ipfstat -ih displays the inbound internal rules table, prefixing each rule with a count of how many times the rule was matched. ipfstat -oh displays the outbound internal rules table, prefixing each rule with a count of how many times the rule was matched. The output will look something like this: 2451423 pass out on xl0 from any to any 354727 block out on dc0 from any to any 430918 pass out quick on dc0 proto tcp/udp from any to any keep state One of the most important functions of the ipfstat command is the flag which displays the state table in a way similar to the way &man.top.1; shows the &os; running process table. When your firewall is under attack this function gives you the ability to identify, drill down to, and see the attacking packets. The optional sub-flags give the ability to select the destination or source IP, port, or protocol that you want to monitor in real time. See the &man.ipfstat.8; manual page for details. IPMON ipmon IPFILTER logging In order for ipmon to work properly, the kernel option IPFILTER_LOG must be turned on. This command has two different modes that it can be used in. Native mode is the default mode when you type the command on the command line without the flag. Daemon mode is for when you want to have a continuous system log file available so that you can review logging of past events. This is how &os; and IPFILTER are configured to work together. &os; has a built in facility to automatically rotate system logs. That is why outputting the log information to syslogd is better than the default of outputting to a regular file. In the default rc.conf file you see the ipmon_flags statement uses the flags: ipmon_flags="-Ds" # D = start as daemon # s = log to syslog # v = log tcp window, ack, seq # n = map IP & port to names The benefits of logging are obvious. It provides the ability to review, after the fact, information such as which packets had been dropped, what addresses they came from and where they were going. These all give you a significant edge in tracking down attackers. Even with the logging facility enabled, IPF will not generate any rule logging on its own. The firewall administrator decides what rules in the rule set he wants to log and adds the log keyword to those rules. Normally only deny rules are logged. It is very customary to include a default deny everything rule with the log keyword included as your last rule in the rule set. This way you get to see all the packets that did not match any of the rules in the rule set. IPMON Logging Syslogd uses its own special method for segregation of log data. It uses special groupings called facility and level. IPMON in mode uses security as the facility name. All IPMON logged data goes to security The following levels can be used to further segregate the logged data if desired: LOG_INFO - packets logged using the "log" keyword as the action rather than pass or block. LOG_NOTICE - packets logged which are also passed LOG_WARNING - packets logged which are also blocked LOG_ERR - packets which have been logged and which can be considered short To setup IPFILTER to log all data to /var/log/ipfilter.log, you will need to create the file. The following command will do that: &prompt.root; touch /var/log/ipfilter.log The syslog function is controlled by definition statements in the /etc/syslog.conf file. The syslog.conf file offers considerable flexibility in how syslog will deal with system messages issued by software applications like IPF. Add the following statement to /etc/syslog.conf: security.* /var/log/ipfilter.log The security.* means to write all the logged messages to the coded file location. To activate the changes to /etc/syslog.conf you can reboot or bump the syslog task into re-reading /etc/syslog.conf by running /etc/rc.d/syslogd reload Do not forget to change /etc/newsyslog.conf to rotate the new log you just created above. The Format of Logged Messages Messages generated by ipmon consist of data fields separated by white space. Fields common to all messages are: The date of packet receipt. The time of packet receipt. This is in the form HH:MM:SS.F, for hours, minutes, seconds, and fractions of a second (which can be several digits long). The name of the interface the packet was processed on, e.g. dc0. The group and rule number of the rule, e.g. @0:17. These can be viewed with ipfstat -in. The action: p for passed, b for blocked, S for a short packet, n did not match any rules, L for a log rule. The order of precedence in showing flags is: S, p, b, n, L. A capital P or B means that the packet has been logged due to a global logging setting, not a particular rule. The addresses. This is actually three fields: the source address and port (separated by a comma), the -> symbol, and the destination address and port. 209.53.17.22,80 -> 198.73.220.17,1722. PR followed by the protocol name or number, e.g. PR tcp. len followed by the header length and total length of the packet, e.g. len 20 40. If the packet is a TCP packet, there will be an additional field starting with a hyphen followed by letters corresponding to any flags that were set. See the &man.ipmon.8; manual page for a list of letters and their flags. If the packet is an ICMP packet, there will be two fields at the end, the first always being ICMP, and the next being the ICMP message and sub-message type, separated by a slash, e.g. ICMP 3/3 for a port unreachable message. Building the Rule Script with Symbolic Substitution Some experienced IPF users create a file containing the rules and code them in a manner compatible with running them as a script with symbolic substitution. The major benefit of doing this is that you only have to change the value associated with the symbolic name and when the script is run all the rules containing the symbolic name will have the value substituted in the rules. Being a script, you can use symbolic substitution to code frequently used values and substitute them in multiple rules. You will see this in the following example. The script syntax used here is compatible with the sh, csh, and tcsh shells. Symbolic substitution fields are prefixed with a dollar sign: $. Symbolic fields do not have the $ prefix. The value to populate the symbolic field must be enclosed with double quotes ("). Start your rule file with something like this: ############# Start of IPF rules script ######################## oif="dc0" # name of the outbound interface odns="192.0.2.11" # ISP's DNS server IP address myip="192.0.2.7" # my static IP address from ISP ks="keep state" fks="flags S keep state" # You can choose between building /etc/ipf.rules file # from this script or running this script "as is". # # Uncomment only one line and comment out another. # # 1) This can be used for building /etc/ipf.rules: #cat > /etc/ipf.rules << EOF # # 2) This can be used to run script "as is": /sbin/ipf -Fa -f - << EOF # Allow out access to my ISP's Domain name server. pass out quick on $oif proto tcp from any to $odns port = 53 $fks pass out quick on $oif proto udp from any to $odns port = 53 $ks # Allow out non-secure standard www function pass out quick on $oif proto tcp from $myip to any port = 80 $fks # Allow out secure www function https over TLS SSL pass out quick on $oif proto tcp from $myip to any port = 443 $fks EOF ################## End of IPF rules script ######################## That is all there is to it. The rules are not important in this example; how the symbolic substitution fields are populated and used are. If the above example was in a file named /etc/ipf.rules.script, you could reload these rules by entering the following command: &prompt.root; sh /etc/ipf.rules.script There is one problem with using a rules file with embedded symbolics: IPF does not understand symbolic substitution, and cannot read such scripts directly. This script can be used in one of two ways: Uncomment the line that begins with cat, and comment out the line that begins with /sbin/ipf. Place ipfilter_enable="YES" into /etc/rc.conf as usual, and run script once after each modification to create or update /etc/ipf.rules. Disable IPFILTER in system startup scripts by adding ipfilter_enable="NO" (this is default value) into /etc/rc.conf file. Add a script like the following to your /usr/local/etc/rc.d/ startup directory. The script should have an obvious name like ipf.loadrules.sh. The .sh extension is mandatory. #!/bin/sh sh /etc/ipf.rules.script The permissions on this script file must be read, write, execute for owner root. &prompt.root; chmod 700 /usr/local/etc/rc.d/ipf.loadrules.sh Now, when your system boots, your IPF rules will be loaded. IPF Rule Sets A rule set is a group of ipf rules coded to pass or block packets based on the values contained in the packet. The bi-directional exchange of packets between hosts comprises a session conversation. The firewall rule set processes the packet two times, once on its arrival from the public Internet host and again as it leaves for its return trip back to the public Internet host. Each TCP/IP service (i.e. telnet, www, mail, etc.) is predefined by its protocol, source and destination IP address, or the source and destination port number. This is the basic selection criteria used to create rules which will pass or block services. IPFILTER rule processing order IPF was originally written using a rules processing logic of the last matching rule wins and used only stateless rules. Over time IPF has been enhanced to include a quick option and a stateful keep state option which drastically modernized the rule processing logic. The instructions contained in this section are based on using rules that contain the quick option and the stateful keep state option. This is the basic framework for coding an inclusive firewall rule set. An inclusive firewall only allows services matching the rules through. This way you can control what services can originate behind the firewall destined for the public Internet and also control the services which can originate from the public Internet accessing your private network. Everything else is blocked and logged by default design. Inclusive firewalls are much, much securer than exclusive firewall rule sets and is the only rule set type covered herein. When working with the firewall rules, be very careful. Some configurations will lock you out of the server. To be on the safe side, you may wish to consider performing the initial firewall configuration from the local console rather than doing it remotely e.g. via ssh. Rule Syntax IPFILTER rule syntax The rule syntax presented here has been simplified to only address the modern stateful rule context and first matching rule wins logic. For the complete legacy rule syntax description see the &man.ipf.8; manual page. A # character is used to mark the start of a comment and may appear at the end of a rule line or on its own line. Blank lines are ignored. Rules contain keywords. These keywords have to be coded in a specific order from left to right on the line. Keywords are identified in bold type. Some keywords have sub-options which may be keywords themselves and also include more sub-options. Each of the headings in the below syntax has a bold section header which expands on the content. ACTION IN-OUT OPTIONS SELECTION STATEFUL PROTO SRC_ADDR,DST_ADDR OBJECT PORT_NUM TCP_FLAG STATEFUL ACTION = block | pass IN-OUT = in | out OPTIONS = log | quick | on interface-name SELECTION = proto value | source/destination IP | port = number | flags flag-value PROTO = tcp/udp | udp | tcp | icmp SRC_ADD,DST_ADDR = all | from object to object OBJECT = IP address | any PORT_NUM = port number TCP_FLAG = S STATEFUL = keep state ACTION The action indicates what to do with the packet if it matches the rest of the filter rule. Each rule must have a action. The following actions are recognized: block indicates that the packet should be dropped if the selection parameters match the packet. pass indicates that the packet should exit the firewall if the selection parameters match the packet. IN-OUT A mandatory requirement is that each filter rule explicitly state which side of the I/O it is to be used on. The next keyword must be either in or out and one or the other has to be coded or the rule will not pass syntax checks. in means this rule is being applied against an inbound packet which has just been received on the interface facing the public Internet. out means this rule is being applied against an outbound packet destined for the interface facing the public Internet. OPTIONS These options must be used in the order shown here. log indicates that the packet header will be written to the ipl log (as described in the LOGGING section below) if the selection parameters match the packet. quick indicates that if the selection parameters match the packet, this rule will be the last rule checked, allowing a short-circuit path to avoid processing any following rules for this packet. This option is a mandatory requirement for the modernized rules processing logic. on indicates the interface name to be incorporated into the selection parameters. Interface names are as displayed by &man.ifconfig.8;. Using this option, the rule will only match if the packet is going through that interface in the specified direction (in/out). This option is a mandatory requirement for the modernized rules processing logic. When a packet is logged, the headers of the packet are written to the IPL packet logging pseudo-device. Immediately following the log keyword, the following qualifiers may be used (in this order): body indicates that the first 128 bytes of the packet contents will be logged after the headers. first If the log keyword is being used in conjunction with a keep state option, it is recommended that this option is also applied so that only the triggering packet is logged and not every packet which thereafter matches the keep state information. SELECTION The keywords described in this section are used to describe attributes of the packet to be interrogated when determining whether rules match or not. There is a keyword subject, and it has sub-option keywords, one of which has to be selected. The following general-purpose attributes are provided for matching, and must be used in this order: PROTO proto is the subject keyword and must be coded along with one of its corresponding keyword sub-option values. The value allows a specific protocol to be matched against. This option is a mandatory requirement for the modernized rules processing logic. tcp/udp | udp | tcp | icmp or any protocol names found in /etc/protocols are recognized and may be used. The special protocol keyword tcp/udp may be used to match either a TCP or a UDP packet, and has been added as a convenience to save duplication of otherwise identical rules. SRC_ADDR/DST_ADDR The all keyword is essentially a synonym for from any to any with no other match parameters. from src to dst: the from and to keywords are used to match against IP addresses. Rules must specify BOTH source and destination parameters. any is a special keyword that matches any IP address. Examples of use: from any to any or from 0.0.0.0/0 to any or from any to 0.0.0.0/0 or from 0.0.0.0 to any or from any to 0.0.0.0. IP addresses may be specified as a dotted IP address numeric form/mask-length, or as single dotted IP address numeric form. There is no way to match ranges of IP addresses which do not express themselves easily as mask-length. See this web page for help on writing mask-length: . PORT If a port match is included, for either or both of source and destination, then it is only applied to TCP and UDP packets. When composing port comparisons, either the service name from /etc/services or an integer port number may be used. When the port appears as part of the from object, it matches the source port number; when it appears as part of the to object, it matches the destination port number. The use of the port option with the to object is a mandatory requirement for the modernized rules processing logic. Example of use: from any to any port = 80 Port comparisons may be done in a number of forms, with a number of comparison operators, or port ranges may be specified. port "=" | "!=" | "<" | ">" | "<=" | ">=" | "eq" | "ne" | "lt" | "gt" | "le" | "ge". To specify port ranges, port "<>" | "><" Following the source and destination matching parameters, the following two parameters are mandatory requirements for the modernized rules processing logic. <acronym>TCP</acronym>_FLAG Flags are only effective for TCP filtering. The letters represents one of the possible flags that can be interrogated in the TCP packet header. The modernized rules processing logic uses the flags S parameter to identify the tcp session start request. STATEFUL keep state indicates that on a pass rule, any packets that match the rules selection parameters should activate the stateful filtering facility. This option is a mandatory requirement for the modernized rules processing logic. Stateful Filtering IPFILTER stateful filtering Stateful filtering treats traffic as a bi-directional exchange of packets comprising a session conversation. When activated, keep-state dynamically generates internal rules for each anticipated packet being exchanged during the bi-directional session conversation. It has the interrogation abilities to determine if the session conversation between the originating sender and the destination are following the valid procedure of bi-directional packet exchange. Any packets that do not properly fit the session conversation template are automatically rejected as impostors. Keep state will also allow ICMP packets related to a TCP or UDP session through. So if you get ICMP type 3 code 4 in response to some web surfing allowed out by a keep state rule, they will be automatically allowed in. Any packet that IPF can be certain is part of an active session, even if it is a different protocol, will be let in. What happens is: Packets destined to go out the interface connected to the public Internet are first checked against the dynamic state table, if the packet matches the next expected packet comprising in a active session conversation, then it exits the firewall and the state of the session conversation flow is updated in the dynamic state table, the remaining packets get checked against the outbound rule set. Packets coming in to the interface connected to the public Internet are first checked against the dynamic state table, if the packet matches the next expected packet comprising a active session conversation, then it exits the firewall and the state of the session conversation flow is updated in the dynamic state table, the remaining packets get checked against the inbound rule set. When the conversation completes it is removed from the dynamic state table. Stateful filtering allows you to focus on blocking/passing new sessions. If the new session is passed, all its subsequent packets will be allowed through automatically and any impostors automatically rejected. If a new session is blocked, none of its subsequent packets will be allowed through. Stateful filtering has technically advanced interrogation abilities capable of defending against the flood of different attack methods currently employed by attackers. Inclusive Rule Set Example The following rule set is an example of how to code a very secure inclusive type of firewall. An inclusive firewall only allows services matching pass rules through and blocks all other by default. All firewalls have at the minimum two interfaces which have to have rules to allow the firewall to function. All &unix; flavored systems including &os; are designed to use interface lo0 and IP address 127.0.0.1 for internal communication within the operating system. The firewall rules must contain rules to allow free unmolested movement of these special internally used packets. The interface which faces the public Internet is the one where you place your rules to authorize and control access out to the public Internet and access requests arriving from the public Internet. This can be your user PPP tun0 interface or your NIC that is connected to your DSL or cable modem. In cases where one or more NICs are cabled to private LANs behind the firewall, those interfaces must have a rule coded to allow free unmolested movement of packets originating from those LAN interfaces. The rules should be first organized into three major sections: all the free unmolested interfaces, the public interface outbound, and the public interface inbound. The rules in each of the public interface sections should have the most frequently matched rules placed before less commonly matched rules, with the last rule in the section blocking and logging all packets on that interface and direction. The Outbound section in the following rule set only contains 'pass' rules which contain selection values that uniquely identify the service that is authorized for public Internet access. All the rules have the 'quick', 'on', 'proto', 'port', and 'keep state' option coded. The 'proto tcp' rules have the 'flag' option included to identify the session start request as the triggering packet to activate the stateful facility. The Inbound section has all the blocking of undesirable packets first, for two different reasons. The first is that these things being blocked may be part of an otherwise valid packet which may be allowed in by the later authorized service rules. The second reason is that by having a rule that explicitly blocks selected packets that I receive on an infrequent basis and that I do not want to see in the log, they will not be caught by the last rule in the section which blocks and logs all packets which have fallen through the rules. The last rule in the section which blocks and logs all packets is how you create the legal evidence needed to prosecute the people who are attacking your system. Another thing you should take note of, is there is no response returned for any of the undesirable stuff, their packets just get dropped and vanish. This way the attacker has no knowledge if his packets have reached your system. The less the attackers can learn about your system, the more time they must invest before actually doing something bad. The inbound 'nmap OS fingerprint' attempts rule I log the first occurrence because this is something a attacker would do. Any time you see log messages on a rule with 'log first'. You should do an ipfstat -hio command to see the number of times the rule has been matched so you know if you are being flooded, i.e. under attack. When you log packets with port numbers you do not recognize, look it up in /etc/services or go to and do a port number lookup to find what the purpose of that port number is. Check out this link for port numbers used by Trojans . The following rule set is a complete very secure 'inclusive' type of firewall rule set that I have used on my system. You can not go wrong using this rule set for your own. Just comment out any pass rules for services that you do not want to authorize. If you see messages in your log that you want to stop seeing just add a block rule in the inbound section. You have to change the dc0 interface name in every rule to the interface name of the Nic card that connects your system to the public Internet. For user PPP it would be tun0. Add the following statements to /etc/ipf.rules: ################################################################# # No restrictions on Inside LAN Interface for private network # Not needed unless you have LAN ################################################################# #pass out quick on xl0 all #pass in quick on xl0 all ################################################################# # No restrictions on Loopback Interface ################################################################# pass in quick on lo0 all pass out quick on lo0 all ################################################################# # Interface facing Public Internet (Outbound Section) # Interrogate session start requests originating from behind the # firewall on the private network # or from this gateway server destine for the public Internet. ################################################################# # Allow out access to my ISP's Domain name server. # xxx must be the IP address of your ISP's DNS. # Dup these lines if your ISP has more than one DNS server # Get the IP addresses from /etc/resolv.conf file pass out quick on dc0 proto tcp from any to xxx port = 53 flags S keep state pass out quick on dc0 proto udp from any to xxx port = 53 keep state # Allow out access to my ISP's DHCP server for cable or DSL networks. # This rule is not needed for 'user ppp' type connection to the # public Internet, so you can delete this whole group. # Use the following rule and check log for IP address. # Then put IP address in commented out rule & delete first rule pass out log quick on dc0 proto udp from any to any port = 67 keep state #pass out quick on dc0 proto udp from any to z.z.z.z port = 67 keep state # Allow out non-secure standard www function pass out quick on dc0 proto tcp from any to any port = 80 flags S keep state # Allow out secure www function https over TLS SSL pass out quick on dc0 proto tcp from any to any port = 443 flags S keep state # Allow out send & get email function pass out quick on dc0 proto tcp from any to any port = 110 flags S keep state pass out quick on dc0 proto tcp from any to any port = 25 flags S keep state # Allow out Time pass out quick on dc0 proto tcp from any to any port = 37 flags S keep state # Allow out nntp news pass out quick on dc0 proto tcp from any to any port = 119 flags S keep state # Allow out gateway & LAN users non-secure FTP ( both passive & active modes) # This function uses the IPNAT built in FTP proxy function coded in # the nat rules file to make this single rule function correctly. # If you want to use the pkg_add command to install application packages # on your gateway system you need this rule. pass out quick on dc0 proto tcp from any to any port = 21 flags S keep state # Allow out secure FTP, Telnet, and SCP # This function is using SSH (secure shell) pass out quick on dc0 proto tcp from any to any port = 22 flags S keep state # Allow out non-secure Telnet pass out quick on dc0 proto tcp from any to any port = 23 flags S keep state # Allow out FBSD CVSUP function pass out quick on dc0 proto tcp from any to any port = 5999 flags S keep state # Allow out ping to public Internet pass out quick on dc0 proto icmp from any to any icmp-type 8 keep state # Allow out whois for LAN PC to public Internet pass out quick on dc0 proto tcp from any to any port = 43 flags S keep state # Block and log only the first occurrence of everything # else that's trying to get out. # This rule enforces the block all by default logic. block out log first quick on dc0 all ################################################################# # Interface facing Public Internet (Inbound Section) # Interrogate packets originating from the public Internet # destine for this gateway server or the private network. ################################################################# # Block all inbound traffic from non-routable or reserved address spaces block in quick on dc0 from 192.168.0.0/16 to any #RFC 1918 private IP block in quick on dc0 from 172.16.0.0/12 to any #RFC 1918 private IP block in quick on dc0 from 10.0.0.0/8 to any #RFC 1918 private IP block in quick on dc0 from 127.0.0.0/8 to any #loopback block in quick on dc0 from 0.0.0.0/8 to any #loopback block in quick on dc0 from 169.254.0.0/16 to any #DHCP auto-config block in quick on dc0 from 192.0.2.0/24 to any #reserved for docs block in quick on dc0 from 204.152.64.0/23 to any #Sun cluster interconnect block in quick on dc0 from 224.0.0.0/3 to any #Class D & E multicast ##### Block a bunch of different nasty things. ############ # That I do not want to see in the log # Block frags block in quick on dc0 all with frags # Block short tcp packets block in quick on dc0 proto tcp all with short # block source routed packets block in quick on dc0 all with opt lsrr block in quick on dc0 all with opt ssrr # Block nmap OS fingerprint attempts # Log first occurrence of these so I can get their IP address block in log first quick on dc0 proto tcp from any to any flags FUP # Block anything with special options block in quick on dc0 all with ipopts # Block public pings block in quick on dc0 proto icmp all icmp-type 8 # Block ident block in quick on dc0 proto tcp from any to any port = 113 # Block all Netbios service. 137=name, 138=datagram, 139=session # Netbios is MS/Windows sharing services. # Block MS/Windows hosts2 name server requests 81 block in log first quick on dc0 proto tcp/udp from any to any port = 137 block in log first quick on dc0 proto tcp/udp from any to any port = 138 block in log first quick on dc0 proto tcp/udp from any to any port = 139 block in log first quick on dc0 proto tcp/udp from any to any port = 81 # Allow traffic in from ISP's DHCP server. This rule must contain # the IP address of your ISP's DHCP server as it's the only # authorized source to send this packet type. Only necessary for # cable or DSL configurations. This rule is not needed for # 'user ppp' type connection to the public Internet. # This is the same IP address you captured and # used in the outbound section. pass in quick on dc0 proto udp from z.z.z.z to any port = 68 keep state # Allow in standard www function because I have apache server pass in quick on dc0 proto tcp from any to any port = 80 flags S keep state # Allow in non-secure Telnet session from public Internet # labeled non-secure because ID/PW passed over public Internet as clear text. # Delete this sample group if you do not have telnet server enabled. #pass in quick on dc0 proto tcp from any to any port = 23 flags S keep state # Allow in secure FTP, Telnet, and SCP from public Internet # This function is using SSH (secure shell) pass in quick on dc0 proto tcp from any to any port = 22 flags S keep state # Block and log only first occurrence of all remaining traffic # coming into the firewall. The logging of only the first # occurrence stops a .denial of service. attack targeted # at filling up your log file space. # This rule enforces the block all by default logic. block in log first quick on dc0 all ################### End of rules file ##################################### <acronym>NAT</acronym> NAT IP masquerading NAT network address translation NAT NAT stands for Network Address Translation. To those familiar with &linux;, this concept is called IP Masquerading; NAT and IP Masquerading are the same thing. One of the many things the IPF NAT function enables is the ability to have a private Local Area Network (LAN) behind the firewall sharing a single ISP assigned IP address on the public Internet. You may ask why would someone want to do this. ISPs normally assign a dynamic IP address to their non-commercial users. Dynamic means that the IP address can be different each time you dial in and log on to your ISP, or for cable and DSL modem users when you power off and then power on your modems you can get assigned a different IP address. This IP address is how you are known to the public Internet. Now lets say you have five PCs at home and each one needs Internet access. You would have to pay your ISP for an individual Internet account for each PC and have five phone lines. With NAT you only need a single account with your ISP, then cable your other four PCs to a switch and the switch to the NIC in your &os; system which is going to service your LAN as a gateway. NAT will automatically translate the private LAN IP address for each separate PC on the LAN to the single public IP address as it exits the firewall bound for the public Internet. It also does the reverse translation for returning packets. NAT is most often accomplished without the approval, or knowledge, of your ISP and in most cases is grounds for your ISP terminating your account if found out. Commercial users pay a lot more for their Internet connection and usually get assigned a block of static IP address which never change. The ISP also expects and consents to their Commercial customers using NAT for their internal private LANs. There is a special range of IP addresses reserved for NATed private LAN IP address. According to RFC 1918, you can use the following IP ranges for private nets which will never be routed directly to the public Internet: Start IP 10.0.0.0 - Ending IP 10.255.255.255 Start IP 172.16.0.0 - Ending IP 172.31.255.255 Start IP 192.168.0.0 - Ending IP 192.168.255.255 IP<acronym>NAT</acronym> NAT and IPFILTER ipnat NAT rules are loaded by using the ipnat command. Typically the NAT rules are stored in /etc/ipnat.rules. See &man.ipnat.1; for details. When changing the NAT rules after NAT has been started, make your changes to the file containing the NAT rules, then run ipnat command with the flags to delete the internal in use NAT rules and flush the contents of the translation table of all active entries. To reload the NAT rules issue a command like this: &prompt.root; ipnat -CF -f /etc/ipnat.rules To display some statistics about your NAT, use this command: &prompt.root; ipnat -s To list the NAT table's current mappings, use this command: &prompt.root; ipnat -l To turn verbose mode on, and display information relating to rule processing and active rules/table entries: &prompt.root; ipnat -v IP<acronym>NAT</acronym> Rules NAT rules are very flexible and can accomplish many different things to fit the needs of commercial and home users. The rule syntax presented here has been simplified to what is most commonly used in a non-commercial environment. For a complete rule syntax description see the &man.ipnat.5; manual page. The syntax for a NAT rule looks something like this: map IF LAN_IP_RANGE -> PUBLIC_ADDRESS The keyword map starts the rule. Replace IF with the external interface. The LAN_IP_RANGE is what your internal clients use for IP Addressing, usually this is something like 192.168.1.0/24. The PUBLIC_ADDRESS can either be the external IP address or the special keyword 0/32, which means to use the IP address assigned to IF. How <acronym>NAT</acronym> works A packet arrives at the firewall from the LAN with a public destination. It passes through the outbound filter rules, NAT gets his turn at the packet and applies its rules top down, first matching rule wins. NAT tests each of its rules against the packets interface name and source IP address. When a packets interface name matches a NAT rule then the [source IP address, i.e. private LAN IP address] of the packet is checked to see if it falls within the IP address range specified to the left of the arrow symbol on the NAT rule. On a match the packet has its source IP address rewritten with the public IP address obtained by the 0/32 keyword. NAT posts a entry in its internal NAT table so when the packet returns from the public Internet it can be mapped back to its original private IP address and then passed to the filter rules for processing. Enabling IP<acronym>NAT</acronym> To enable IPNAT add these statements to /etc/rc.conf. To enable your machine to route traffic between interfaces: gateway_enable="YES" To start IPNAT automatically each time: ipnat_enable="YES" To specify where to load the IPNAT rules from: ipnat_rules="/etc/ipnat.rules" <acronym>NAT</acronym> for a very large LAN For networks that have large numbers of PC's on the LAN or networks with more than a single LAN, the process of funneling all those private IP addresses into a single public IP address becomes a resource problem that may cause problems with the same port numbers being used many times across many NATed LAN PC's, causing collisions. There are two ways to relieve this resource problem. Assigning Ports to Use A normal NAT rule would look like: map dc0 192.168.1.0/24 -> 0/32 In the above rule the packet's source port is unchanged as the packet passes through IPNAT. By adding the portmap keyword you can tell IPNAT to only use source ports in a range. For example the following rule will tell IPNAT to modify the source port to be within that range: map dc0 192.168.1.0/24 -> 0/32 portmap tcp/udp 20000:60000 Additionally we can make things even easier by using the auto keyword to tell IPNAT to determine by itself which ports are available to use: map dc0 192.168.1.0/24 -> 0/32 portmap tcp/udp auto Using a pool of public addresses In very large LANs there comes a point where there are just too many LAN addresses to fit into a single public address. If a block of public IP addresses is available, you can use these addresses as a pool, and let IPNAT pick one of the public IP addresses as packet-addresses are mapped on their way out. For example, instead of mapping all packets through a single public IP address, as in: map dc0 192.168.1.0/24 -> 204.134.75.1 A range of public IP addresses can be specified either with a netmask: map dc0 192.168.1.0/24 -> 204.134.75.0/255.255.255.0 or using CIDR notation: map dc0 192.168.1.0/24 -> 204.134.75.0/24 Port Redirection A very common practice is to have a web server, email server, database server and DNS server each segregated to a different PC on the LAN. In this case the traffic from these servers still have to be NATed, but there has to be some way to direct the inbound traffic to the correct LAN PCs. IPNAT has the redirection facilities of NAT to solve this problem. Lets say you have your web server on LAN address 10.0.10.25 and your single public IP address is 20.20.20.5 you would code the rule like this: rdr dc0 20.20.20.5/32 port 80 -> 10.0.10.25 port 80 or: rdr dc0 0.0.0.0/0 port 80 -> 10.0.10.25 port 80 or for a LAN DNS Server on LAN address of 10.0.10.33 that needs to receive public DNS requests: rdr dc0 20.20.20.5/32 port 53 -> 10.0.10.33 port 53 udp FTP and <acronym>NAT</acronym> FTP is a dinosaur left over from the time before the Internet as it is known today, when research universities were leased lined together and FTP was used to share files among research Scientists. This was a time when data security was not a consideration. Over the years the FTP protocol became buried into the backbone of the emerging Internet and its username and password being sent in clear text was never changed to address new security concerns. FTP has two flavors, it can run in active mode or passive mode. The difference is in how the data channel is acquired. Passive mode is more secure as the data channel is acquired be the ordinal ftp session requester. For a real good explanation of FTP and the different modes see . IP<acronym>NAT</acronym> Rules IPNAT has a special built in FTP proxy option which can be specified on the NAT map rule. It can monitor all outbound packet traffic for FTP active or passive start session requests and dynamically create temporary filter rules containing only the port number really in use for the data channel. This eliminates the security risk FTP normally exposes the firewall to from having large ranges of high order port numbers open. This rule will handle all the traffic for the internal LAN: map dc0 10.0.10.0/29 -> 0/32 proxy port 21 ftp/tcp This rule handles the FTP traffic from the gateway: map dc0 0.0.0.0/0 -> 0/32 proxy port 21 ftp/tcp This rule handles all non-FTP traffic from the internal LAN: map dc0 10.0.10.0/29 -> 0/32 The FTP map rule goes before our regular map rule. All packets are tested against the first rule from the top. Matches on interface name, then private LAN source IP address, and then is it a FTP packet. If all that matches then the special FTP proxy creates temp filter rules to let the FTP session packets pass in and out, in addition to also NATing the FTP packets. All LAN packets that are not FTP do not match the first rule and fall through to the third rule and are tested, matching on interface and source IP, then are NATed. IP<acronym>NAT</acronym> FTP Filter Rules Only one filter rule is needed for FTP if the NAT FTP proxy is used. Without the FTP Proxy you will need the following three rules: # Allow out LAN PC client FTP to public Internet # Active and passive modes pass out quick on rl0 proto tcp from any to any port = 21 flags S keep state # Allow out passive mode data channel high order port numbers pass out quick on rl0 proto tcp from any to any port > 1024 flags S keep state # Active mode let data channel in from FTP server pass in quick on rl0 proto tcp from any to any port = 20 flags S keep state IPFW firewall IPFW This section is work in progress. The contents might not be accurate at all times. The IPFIREWALL (IPFW) is a &os; sponsored firewall software application authored and maintained by &os; volunteer staff members. It uses the legacy stateless rules and a legacy rule coding technique to achieve what is referred to as Simple Stateful logic. The IPFW sample rule set (found in /etc/rc.firewall) in the standard &os; install is rather simple and it is not expected that it used directly without modifications. The example does not use stateful filtering, which is beneficial in most setups, so it will not be used as base for this section. The IPFW stateless rule syntax is empowered with technically sophisticated selection capabilities which far surpasses the knowledge level of the customary firewall installer. IPFW is targeted at the professional user or the advanced technical computer hobbyist who have advanced packet selection requirements. A high degree of detailed knowledge into how different protocols use and create their unique packet header information is necessary before the power of the IPFW rules can be unleashed. Providing that level of explanation is out of the scope of this section of the handbook. IPFW is composed of seven components, the primary component is the kernel firewall filter rule processor and its integrated packet accounting facility, the logging facility, the 'divert' rule which triggers the NAT facility, and the advanced special purpose facilities, the dummynet traffic shaper facilities, the 'fwd rule' forward facility, the bridge facility, and the ipstealth facility. Enabling IPFW IPFW enabling IPFW is included in the basic &os; install as a separate run time loadable module. The system will dynamically load the kernel module when the rc.conf statement firewall_enable="YES" is used. You do not need to compile IPFW into the &os; kernel unless you want NAT function enabled. After rebooting your system with firewall_enable="YES" in rc.conf the following white highlighted message is displayed on the screen as part of the boot process: ipfw2 initialized, divert disabled, rule-based forwarding disabled, default to deny, logging disabled The loadable module does have logging ability compiled in. To enable logging and set the verbose logging limit, there is a knob you can set in /etc/sysctl.conf by adding these statements, logging will be enabled on future reboots: net.inet.ip.fw.verbose=1 net.inet.ip.fw.verbose_limit=5 Kernel Options kernel options IPFIREWALL kernel options IPFIREWALL_VERBOSE kernel options IPFIREWALL_VERBOSE_LIMIT IPFW kernel options It is not a mandatory requirement that you enable IPFW by compiling the following options into the &os; kernel unless you need NAT function. It is presented here as background information. options IPFIREWALL This option enables IPFW as part of the kernel options IPFIREWALL_VERBOSE Enables logging of packets that pass through IPFW and have the 'log' keyword specified in the rule set. options IPFIREWALL_VERBOSE_LIMIT=5 Limits the number of packets logged through &man.syslogd.8; on a per entry basis. You may wish to use this option in hostile environments which you want to log firewall activity. This will close a possible denial of service attack via syslog flooding. kernel options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_DEFAULT_TO_ACCEPT This option will allow everything to pass through the firewall by default, which is a good idea when you are first setting up your firewall. options IPV6FIREWALL options IPV6FIREWALL_VERBOSE options IPV6FIREWALL_VERBOSE_LIMIT options IPV6FIREWALL_DEFAULT_TO_ACCEPT These options are exactly the same as the IPv4 options but they are for IPv6. If you do not use IPv6 you might want to use IPV6FIREWALL without any rules to block all IPv6 kernel options IPDIVERT options IPDIVERT This enables the use of NAT functionality. If you do not include IPFIREWALL_DEFAULT_TO_ACCEPT or set your rules to allow incoming packets you will block all packets going to and from this machine. <filename>/etc/rc.conf</filename> Options Enable the firewall: firewall_enable="YES" To select one of the default firewall types provided by &os;, select one by reading the /etc/rc.firewall file and place it in the following: firewall_type="open" Available values for this setting are: open — pass all traffic. client — will protect only this machine. simple — protect the whole network. closed — entirely disables IP traffic except for the loopback interface. UNKNOWN — disables the loading of firewall rules. filename — absolute path of file containing firewall rules. It is possible to use two different ways to load custom rules for ipfw firewall. One is by setting firewall_type variable to absolute path of file, which contains firewall rules without any command-line options for &man.ipfw.8; itself. A simple example of ruleset file can be following: add block in all add block out all On the other hand, it is possible to set firewall_script variable to absolute path of executable script that includes ipfw commands being executed at system boot time. A valid ruleset script that would be equivalent to the ruleset file shown above would be following: #!/bin/sh ipfw -q flush ipfw add block in all ipfw add block out all If firewall_type is set to either client or simple, the default rules found in /etc/rc.firewall should be reviewed to fit to the configuration of the given machine. Also note that the examples used in this chapter expect that the firewall_script is set to /etc/ipfw.rules. Enable logging: firewall_logging="YES" The only thing that the firewall_logging variable will do is setting the net.inet.ip.fw.verbose sysctl variable to the value of 1 (see ). There is no rc.conf variable to set log limitations, but it can be set via sysctl variable, manually or from the /etc/sysctl.conf file: net.inet.ip.fw.verbose_limit=5 If your machine is acting as a gateway, i.e. providing Network Address Translation (NAT) via &man.natd.8;, please refer to for information regarding the required /etc/rc.conf options. The IPFW Command ipfw The ipfw command is the normal vehicle for making manual single rule additions or deletions to the firewall active internal rules while it is running. The problem with using this method is once your system is shutdown or halted all the rules you added or changed or deleted are lost. Writing all your rules in a file and using that file to load the rules at boot time, or to replace in mass the currently running firewall rules with changes you made to the files content is the recommended method used here. The ipfw command is still a very useful to display the running firewall rules to the console screen. The IPFW accounting facility dynamically creates a counter for each rule that counts each packet that matches the rule. During the process of testing a rule, listing the rule with its counter is the one of the ways of determining if the rule is functioning. To list all the rules in sequence: &prompt.root; ipfw list To list all the rules with a time stamp of when the last time the rule was matched: &prompt.root; ipfw -t list To list the accounting information, packet count for matched rules along with the rules themselves. The first column is the rule number, followed by the number of outgoing matched packets, followed by the number of incoming matched packets, and then the rule itself. &prompt.root; ipfw -a list List the dynamic rules in addition to the static rules: &prompt.root; ipfw -d list Also show the expired dynamic rules: &prompt.root; ipfw -d -e list Zero the counters: &prompt.root; ipfw zero Zero the counters for just rule NUM: &prompt.root; ipfw zero NUM IPFW Rule Sets A rule set is a group of ipfw rules coded to allow or deny packets based on the values contained in the packet. The bi-directional exchange of packets between hosts comprises a session conversation. The firewall rule set processes the packet twice: once on its arrival from the public Internet host and again as it leaves for its return trip back to the public Internet host. Each tcp/ip service (i.e. telnet, www, mail, etc.) is predefined by its protocol, and port number. This is the basic selection criteria used to create rules which will allow or deny services. IPFW rule processing order When a packet enters the firewall it is compared against the first rule in the rule set and progress one rule at a time moving from top to bottom of the set in ascending rule number sequence order. When the packet matches a rule selection parameters, the rules action field value is executed and the search of the rule set terminates for that packet. This is referred to as the first match wins search method. If the packet does not match any of the rules, it gets caught by the mandatory ipfw default rule, number 65535 which denies all packets and discards them without any reply back to the originating destination. The search continues after count, skipto and tee rules. The instructions contained here are based on using rules that contain the stateful 'keep state', 'limit', 'in'/'out', and via options. This is the basic framework for coding an inclusive type firewall rule set. An inclusive firewall only allows services matching the rules through. This way you can control what services can originate behind the firewall destine for the public Internet and also control the services which can originate from the public Internet accessing your private network. Everything else is denied by default design. Inclusive firewalls are much, much more secure than exclusive firewall rule sets and is the only rule set type covered here in. When working with the firewall rules be careful, you can end up locking your self out. Rule Syntax IPFW rule syntax The rule syntax presented here has been simplified to what is necessary to create a standard inclusive type firewall rule set. For a complete rule syntax description see the &man.ipfw.8; manual page. Rules contain keywords: these keywords have to be coded in a specific order from left to right on the line. Keywords are identified in bold type. Some keywords have sub-options which may be keywords them selves and also include more sub-options. # is used to mark the start of a comment and may appear at the end of a rule line or on its own lines. Blank lines are ignored. CMD RULE_NUMBER ACTION LOGGING SELECTION STATEFUL CMD Each new rule has to be prefixed with add to add the rule to the internal table. RULE_NUMBER Each rule has to have a rule number to go with it. ACTION A rule can be associated with one of the following actions, which will be executed when the packet matches the selection criterion of the rule. allow | accept | pass | permit These all mean the same thing which is to allow packets that match the rule to exit the firewall rule processing. The search terminates at this rule. check-state Checks the packet against the dynamic rules table. If a match is found, execute the action associated with the rule which generated this dynamic rule, otherwise move to the next rule. The check-state rule does not have selection criterion. If no check-state rule is present in the rule set, the dynamic rules table is checked at the first keep-state or limit rule. deny | drop Both words mean the same thing which is to discard packets that match this rule. The search terminates. Logging log or logamount When a packet matches a rule with the log keyword, a message will be logged to syslogd with a facility name of SECURITY. The logging only occurs if the number of packets logged so far for that particular rule does not exceed the logamount parameter. If no logamount is specified, the limit is taken from the sysctl variable net.inet.ip.fw.verbose_limit. In both cases, a value of zero removes the logging limit. Once the limit is reached, logging can be re-enabled by clearing the logging counter or the packet counter for that rule, see the ipfw reset log command. Logging is done after all other packet matching conditions have been successfully verified, and before performing the final action (accept, deny) on the packet. It is up to you to decide which rules you want to enable logging on. Selection The keywords described in this section are used to describe attributes of the packet to be interrogated when determining whether rules match the packet or not. The following general-purpose attributes are provided for matching, and must be used in this order: udp | tcp | icmp or any protocol names found in /etc/protocols are recognized and may be used. The value specified is protocol to be matched against. This is a mandatory requirement. from src to dst The from and to keywords are used to match against IP addresses. Rules must specify BOTH source and destination parameters. any is a special keyword that matches any IP address. me is a special keyword that matches any IP address configured on an interface in your &os; system to represent the PC the firewall is running on (i.e. this box) as in 'from me to any' or 'from any to me' or 'from 0.0.0.0/0 to any' or 'from any to 0.0.0.0/0' or 'from 0.0.0.0 to any' or 'from any to 0.0.0.0' or 'from me to 0.0.0.0'. IP addresses are specified as a dotted IP address numeric form/mask-length, or as single dotted IP address numeric form. This is a mandatory requirement. See this link for help on writing mask-lengths. port number For protocols which support port numbers (such as TCP and UDP). It is mandatory that you code the port number of the service you want to match on. Service names (from /etc/services) may be used instead of numeric port values. in | out Matches incoming or outgoing packets, respectively. The in and out are keywords and it is mandatory that you code one or the other as part of your rule matching criterion. via IF Matches packets going through the interface specified by exact name. The via keyword causes the interface to always be checked as part of the match process. setup This is a mandatory keyword that identifies the session start request for TCP packets. keep-state This is a mandatory keyword. Upon a match, the firewall will create a dynamic rule, whose default behavior is to match bidirectional traffic between source and destination IP/port using the same protocol. limit {src-addr | src-port | dst-addr | dst-port} The firewall will only allow N connections with the same set of parameters as specified in the rule. One or more of source and destination addresses and ports can be specified. The 'limit' and 'keep-state' can not be used on same rule. Limit provides the same stateful function as 'keep-state' plus its own functions. Stateful Rule Option IPFW stateful filtering Stateful filtering treats traffic as a bi-directional exchange of packets comprising a session conversation. It has the interrogation abilities to determine if the session conversation between the originating sender and the destination are following the valid procedure of bi-directional packet exchange. Any packets that do not properly fit the session conversation template are automatically rejected as impostors. 'check-state' is used to identify where in the IPFW rules set the packet is to be tested against the dynamic rules facility. On a match the packet exits the firewall to continue on its way and a new rule is dynamic created for the next anticipated packet being exchanged during this bi-directional session conversation. On a no match the packet advances to the next rule in the rule set for testing. The dynamic rules facility is vulnerable to resource depletion from a SYN-flood attack which would open a huge number of dynamic rules. To counter this attack, &os; added another new option named limit. This option is used to limit the number of simultaneous session conversations by interrogating the rules source or destinations fields as directed by the limit option and using the packet's IP address found there, in a search of the open dynamic rules counting the number of times this rule and IP address combination occurred, if this count is greater that the value specified on the limit option, the packet is discarded. Logging Firewall Messages IPFW logging The benefits of logging are obvious: it provides the ability to review after the fact the rules you activated logging on which provides information like, what packets had been dropped, what addresses they came from, where they were going, giving you a significant edge in tracking down attackers. Even with the logging facility enabled, IPFW will not generate any rule logging on it's own. The firewall administrator decides what rules in the rule set he wants to log and adds the log verb to those rules. Normally only deny rules are logged, like the deny rule for incoming ICMP pings. It is very customary to duplicate the ipfw default deny everything rule with the log verb included as your last rule in the rule set. This way you get to see all the packets that did not match any of the rules in the rule set. Logging is a two edged sword, if you are not careful, you can lose yourself in the over abundance of log data and fill your disk up with growing log files. DoS attacks that fill up disk drives is one of the oldest attacks around. These log message are not only written to syslogd, but also are displayed on the root console screen and soon become very annoying. The IPFIREWALL_VERBOSE_LIMIT=5 kernel option limits the number of consecutive messages sent to the system logger syslogd, concerning the packet matching of a given rule. When this option is enabled in the kernel, the number of consecutive messages concerning a particular rule is capped at the number specified. There is nothing to be gained from 200 log messages saying the same identical thing. For instance, five consecutive messages concerning a particular rule would be logged to syslogd, the remainder identical consecutive messages would be counted and posted to the syslogd with a phrase like this: last message repeated 45 times All logged packets messages are written by default to /var/log/security file, which is defined in the /etc/syslog.conf file. Building a Rule Script Most experienced IPFW users create a file containing the rules and code them in a manner compatible with running them as a script. The major benefit of doing this is the firewall rules can be refreshed in mass without the need of rebooting the system to activate the new rules. This method is very convenient in testing new rules as the procedure can be executed as many times as needed. Being a script, you can use symbolic substitution to code frequent used values and substitution them in multiple rules. You will see this in the following example. The script syntax used here is compatible with the 'sh', 'csh', 'tcsh' shells. Symbolic substitution fields are prefixed with a dollar sign $. Symbolic fields do not have the $ prefix. The value to populate the Symbolic field must be enclosed to "double quotes". Start your rules file like this: ############### start of example ipfw rules script ############# # ipfw -q -f flush # Delete all rules # Set defaults oif="tun0" # out interface odns="192.0.2.11" # ISP's DNS server IP address cmd="ipfw -q add " # build rule prefix ks="keep-state" # just too lazy to key this each time $cmd 00500 check-state $cmd 00502 deny all from any to any frag $cmd 00501 deny tcp from any to any established $cmd 00600 allow tcp from any to any 80 out via $oif setup $ks $cmd 00610 allow tcp from any to $odns 53 out via $oif setup $ks $cmd 00611 allow udp from any to $odns 53 out via $oif $ks ################### End of example ipfw rules script ############ That is all there is to it. The rules are not important in this example, how the Symbolic substitution field are populated and used are. If the above example was in /etc/ipfw.rules file, you could reload these rules by entering on the command line. &prompt.root; sh /etc/ipfw.rules The /etc/ipfw.rules file could be located anywhere you want and the file could be named any thing you would like. The same thing could also be accomplished by running these commands by hand: &prompt.root; ipfw -q -f flush &prompt.root; ipfw -q add check-state &prompt.root; ipfw -q add deny all from any to any frag &prompt.root; ipfw -q add deny tcp from any to any established &prompt.root; ipfw -q add allow tcp from any to any 80 out via tun0 setup keep-state &prompt.root; ipfw -q add allow tcp from any to 192.0.2.11 53 out via tun0 setup keep-state &prompt.root; ipfw -q add 00611 allow udp from any to 192.0.2.11 53 out via tun0 keep-state Stateful Ruleset The following non-NATed rule set is an example of how to code a very secure 'inclusive' type of firewall. An inclusive firewall only allows services matching pass rules through and blocks all other by default. All firewalls have at the minimum two interfaces which have to have rules to allow the firewall to function. All &unix; flavored operating systems, &os; included, are designed to use interface lo0 and IP address 127.0.0.1 for internal communication with in the operating system. The firewall rules must contain rules to allow free unmolested movement of these special internally used packets. The interface which faces the public Internet, is the one which you code your rules to authorize and control access out to the public Internet and access requests arriving from the public Internet. This can be your ppp tun0 interface or your NIC that is connected to your DSL or cable modem. In cases where one or more than one NIC are connected to a private LANs behind the firewall, those interfaces must have rules coded to allow free unmolested movement of packets originating from those LAN interfaces. The rules should be first organized into three major sections, all the free unmolested interfaces, public interface outbound, and the public interface inbound. The order of the rules in each of the public interface sections should be in order of the most used rules being placed before less often used rules with the last rule in the section being a block log all packets on that interface and direction. The Outbound section in the following rule set only contains 'allow' rules which contain selection values that uniquely identify the service that is authorized for public Internet access. All the rules have the, proto, port, in/out, via and keep state option coded. The 'proto tcp' rules have the 'setup' option included to identify the start session request as the trigger packet to be posted to the keep state stateful table. The Inbound section has all the blocking of undesirable packets first for two different reasons. First is these things being blocked may be part of an otherwise valid packet which may be allowed in by the later authorized service rules. Second reason is that by having a rule that explicitly blocks selected packets that I receive on an infrequent bases and do not want to see in the log, this keeps them from being caught by the last rule in the section which blocks and logs all packets which have fallen through the rules. The last rule in the section which blocks and logs all packets is how you create the legal evidence needed to prosecute the people who are attacking your system. Another thing you should take note of, is there is no response returned for any of the undesirable stuff, their packets just get dropped and vanish. This way the attackers has no knowledge if his packets have reached your system. The less the attackers can learn about your system the more secure it is. When you log packets with port numbers you do not recognize, look the numbers up in /etc/services/ or go to and do a port number lookup to find what the purpose of that port number is. Check out this link for port numbers used by Trojans: . An Example Inclusive Ruleset The following non-NATed rule set is a complete inclusive type ruleset. You can not go wrong using this rule set for you own. Just comment out any pass rules for services you do not want. If you see messages in your log that you want to stop seeing just add a deny rule in the inbound section. You have to change the 'dc0' interface name in every rule to the interface name of the NIC that connects your system to the public Internet. For user ppp it would be 'tun0'. You will see a pattern in the usage of these rules. All statements that are a request to start a session to the public Internet use keep-state. All the authorized services that originate from the public Internet have the limit option to stop flooding. All rules use in or out to clarify direction. All rules use via interface name to specify the interface the packet is traveling over. The following rules go into /etc/ipfw.rules. ################ Start of IPFW rules file ############################### # Flush out the list before we begin. ipfw -q -f flush # Set rules command prefix cmd="ipfw -q add" pif="dc0" # public interface name of NIC # facing the public Internet ################################################################# # No restrictions on Inside LAN Interface for private network # Not needed unless you have LAN. # Change xl0 to your LAN NIC interface name ################################################################# #$cmd 00005 allow all from any to any via xl0 ################################################################# # No restrictions on Loopback Interface ################################################################# $cmd 00010 allow all from any to any via lo0 ################################################################# # Allow the packet through if it has previous been added to the # the "dynamic" rules table by a allow keep-state statement. ################################################################# $cmd 00015 check-state ################################################################# # Interface facing Public Internet (Outbound Section) # Interrogate session start requests originating from behind the # firewall on the private network or from this gateway server # destine for the public Internet. ################################################################# # Allow out access to my ISP's Domain name server. # x.x.x.x must be the IP address of your ISP.s DNS # Dup these lines if your ISP has more than one DNS server # Get the IP addresses from /etc/resolv.conf file $cmd 00110 allow tcp from any to x.x.x.x 53 out via $pif setup keep-state $cmd 00111 allow udp from any to x.x.x.x 53 out via $pif keep-state # Allow out access to my ISP's DHCP server for cable/DSL configurations. # This rule is not needed for .user ppp. connection to the public Internet. # so you can delete this whole group. # Use the following rule and check log for IP address. # Then put IP address in commented out rule & delete first rule $cmd 00120 allow log udp from any to any 67 out via $pif keep-state #$cmd 00120 allow udp from any to x.x.x.x 67 out via $pif keep-state # Allow out non-secure standard www function $cmd 00200 allow tcp from any to any 80 out via $pif setup keep-state # Allow out secure www function https over TLS SSL $cmd 00220 allow tcp from any to any 443 out via $pif setup keep-state # Allow out send & get email function $cmd 00230 allow tcp from any to any 25 out via $pif setup keep-state $cmd 00231 allow tcp from any to any 110 out via $pif setup keep-state # Allow out FBSD (make install & CVSUP) functions # Basically give user root "GOD" privileges. $cmd 00240 allow tcp from me to any out via $pif setup keep-state uid root # Allow out ping $cmd 00250 allow icmp from any to any out via $pif keep-state # Allow out Time $cmd 00260 allow tcp from any to any 37 out via $pif setup keep-state # Allow out nntp news (i.e. news groups) $cmd 00270 allow tcp from any to any 119 out via $pif setup keep-state # Allow out secure FTP, Telnet, and SCP # This function is using SSH (secure shell) $cmd 00280 allow tcp from any to any 22 out via $pif setup keep-state # Allow out whois $cmd 00290 allow tcp from any to any 43 out via $pif setup keep-state # deny and log everything else that.s trying to get out. # This rule enforces the block all by default logic. $cmd 00299 deny log all from any to any out via $pif ################################################################# # Interface facing Public Internet (Inbound Section) # Interrogate packets originating from the public Internet # destine for this gateway server or the private network. ################################################################# # Deny all inbound traffic from non-routable reserved address spaces $cmd 00300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP $cmd 00301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP $cmd 00302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP $cmd 00303 deny all from 127.0.0.0/8 to any in via $pif #loopback $cmd 00304 deny all from 0.0.0.0/8 to any in via $pif #loopback $cmd 00305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config $cmd 00306 deny all from 192.0.2.0/24 to any in via $pif #reserved for docs $cmd 00307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster interconnect $cmd 00308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast # Deny public pings $cmd 00310 deny icmp from any to any in via $pif # Deny ident $cmd 00315 deny tcp from any to any 113 in via $pif # Deny all Netbios service. 137=name, 138=datagram, 139=session # Netbios is MS/Windows sharing services. # Block MS/Windows hosts2 name server requests 81 $cmd 00320 deny tcp from any to any 137 in via $pif $cmd 00321 deny tcp from any to any 138 in via $pif $cmd 00322 deny tcp from any to any 139 in via $pif $cmd 00323 deny tcp from any to any 81 in via $pif # Deny any late arriving packets $cmd 00330 deny all from any to any frag in via $pif # Deny ACK packets that did not match the dynamic rule table $cmd 00332 deny tcp from any to any established in via $pif # Allow traffic in from ISP's DHCP server. This rule must contain # the IP address of your ISP.s DHCP server as it.s the only # authorized source to send this packet type. # Only necessary for cable or DSL configurations. # This rule is not needed for .user ppp. type connection to # the public Internet. This is the same IP address you captured # and used in the outbound section. #$cmd 00360 allow udp from any to x.x.x.x 67 in via $pif keep-state # Allow in standard www function because I have apache server $cmd 00400 allow tcp from any to me 80 in via $pif setup limit src-addr 2 # Allow in secure FTP, Telnet, and SCP from public Internet $cmd 00410 allow tcp from any to me 22 in via $pif setup limit src-addr 2 # Allow in non-secure Telnet session from public Internet # labeled non-secure because ID & PW are passed over public # Internet as clear text. # Delete this sample group if you do not have telnet server enabled. $cmd 00420 allow tcp from any to me 23 in via $pif setup limit src-addr 2 # Reject & Log all incoming connections from the outside $cmd 00499 deny log all from any to any in via $pif # Everything else is denied by default # deny and log all packets that fell through to see what they are $cmd 00999 deny log all from any to any ################ End of IPFW rules file ############################### An Example <acronym>NAT</acronym> and Stateful Ruleset NAT and IPFW There are some additional configuration statements that need to be enabled to activate the NAT function of IPFW. The kernel source needs 'option IPDIVERT' statement added to the other IPFIREWALL statements compiled into a custom kernel. In addition to the normal IPFW options in /etc/rc.conf, the following are needed. natd_enable="YES" # Enable NATD function natd_interface="rl0" # interface name of public Internet NIC natd_flags="-dynamic -m" # -m = preserve port numbers if possible Utilizing stateful rules with divert natd rule (Network Address Translation) greatly complicates the rule set coding logic. The positioning of the check-state, and 'divert natd' rules in the rule set becomes very critical. This is no longer a simple fall-through logic flow. A new action type is used, called 'skipto'. To use the skipto command it is mandatory that you number each rule so you know exactly where the skipto rule number is you are really jumping to. The following is an uncommented example of one coding method, selected here to explain the sequence of the packet flow through the rule sets. The processing flow starts with the first rule from the top of the rule file and progress one rule at a time deeper into the file until the end is reach or the packet being tested to the selection criteria matches and the packet is released out of the firewall. It is important to take notice of the location of rule numbers 100 101, 450, 500, and 510. These rules control the translation of the outbound and inbound packets so their entries in the keep-state dynamic table always register the private LAN IP address. Next notice that all the allow and deny rules specified the direction the packet is going (IE outbound or inbound) and the interface. Also notice that all the start outbound session requests all skipto rule 500 for the network address translation. Lets say a LAN user uses their web browser to get a web page. Web pages use port 80 to communicate over. So the packet enters the firewall, It does not match 100 because it is headed out not in. It passes rule 101 because this is the first packet so it has not been posted to the keep-state dynamic table yet. The packet finally comes to rule 125 a matches. It is outbound through the NIC facing the public Internet. The packet still has it's source IP address as a private LAN IP address. On the match to this rule, two actions take place. The keep-state option will post this rule into the keep-state dynamic rules table and the specified action is executed. The action is part of the info posted to the dynamic table. In this case it is "skipto rule 500". Rule 500 NATs the packet IP address and out it goes. Remember this, this is very important. This packet makes its way to the destination and returns and enters the top of the rule set. This time it does match rule 100 and has it destination IP address mapped back to its corresponding LAN IP address. It then is processed by the check-state rule, it's found in the table as an existing session conversation and released to the LAN. It goes to the LAN PC that sent it and a new packet is sent requesting another segment of the data from the remote server. This time it gets checked by the check-state rule and its outbound entry is found, the associated action, 'skipto 500', is executed. The packet jumps to rule 500 gets NATed and released on it's way out. On the inbound side, everything coming in that is part of an existing session conversation is being automatically handled by the check-state rule and the properly placed divert natd rules. All we have to address is denying all the bad packets and only allowing in the authorized services. Lets say there is a apache server running on the firewall box and we want people on the public Internet to be able to access the local web site. The new inbound start request packet matches rule 100 and its IP address is mapped to LAN IP for the firewall box. The packet is them matched against all the nasty things we want to check for and finally matches against rule 425. On a match two things occur. The packet rule is posted to the keep-state dynamic table but this time any new session requests originating from that source IP address is limited to 2. This defends against DoS attacks of service running on the specified port number. The action is allow so the packet is released to the LAN. On return the check-state rule recognizes the packet as belonging to an existing session conversation sends it to rule 500 for NATing and released to outbound interface. Example Ruleset #1: #!/bin/sh cmd="ipfw -q add" skip="skipto 500" pif=rl0 ks="keep-state" good_tcpo="22,25,37,43,53,80,443,110,119" ipfw -q -f flush $cmd 002 allow all from any to any via xl0 # exclude LAN traffic $cmd 003 allow all from any to any via lo0 # exclude loopback traffic $cmd 100 divert natd ip from any to any in via $pif $cmd 101 check-state # Authorized outbound packets $cmd 120 $skip udp from any to xx.168.240.2 53 out via $pif $ks $cmd 121 $skip udp from any to xx.168.240.5 53 out via $pif $ks $cmd 125 $skip tcp from any to any $good_tcpo out via $pif setup $ks $cmd 130 $skip icmp from any to any out via $pif $ks $cmd 135 $skip udp from any to any 123 out via $pif $ks # Deny all inbound traffic from non-routable reserved address spaces $cmd 300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP $cmd 301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP $cmd 302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP $cmd 303 deny all from 127.0.0.0/8 to any in via $pif #loopback $cmd 304 deny all from 0.0.0.0/8 to any in via $pif #loopback $cmd 305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config $cmd 306 deny all from 192.0.2.0/24 to any in via $pif #reserved for docs $cmd 307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster $cmd 308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast # Authorized inbound packets $cmd 400 allow udp from xx.70.207.54 to any 68 in $ks $cmd 420 allow tcp from any to me 80 in via $pif setup limit src-addr 1 $cmd 450 deny log ip from any to any # This is skipto location for outbound stateful rules $cmd 500 divert natd ip from any to any out via $pif $cmd 510 allow ip from any to any ######################## end of rules ################## The following is pretty much the same as above, but uses a self documenting coding style full of description comments to help the inexperienced IPFW rule writer to better understand what the rules are doing. Example Ruleset #2: #!/bin/sh ################ Start of IPFW rules file ############################### # Flush out the list before we begin. ipfw -q -f flush # Set rules command prefix cmd="ipfw -q add" skip="skipto 800" pif="rl0" # public interface name of NIC # facing the public Internet ################################################################# # No restrictions on Inside LAN Interface for private network # Change xl0 to your LAN NIC interface name ################################################################# $cmd 005 allow all from any to any via xl0 ################################################################# # No restrictions on Loopback Interface ################################################################# $cmd 010 allow all from any to any via lo0 ################################################################# # check if packet is inbound and nat address if it is ################################################################# $cmd 014 divert natd ip from any to any in via $pif ################################################################# # Allow the packet through if it has previous been added to the # the "dynamic" rules table by a allow keep-state statement. ################################################################# $cmd 015 check-state ################################################################# # Interface facing Public Internet (Outbound Section) # Interrogate session start requests originating from behind the # firewall on the private network or from this gateway server # destine for the public Internet. ################################################################# # Allow out access to my ISP's Domain name server. # x.x.x.x must be the IP address of your ISP's DNS # Dup these lines if your ISP has more than one DNS server # Get the IP addresses from /etc/resolv.conf file $cmd 020 $skip tcp from any to x.x.x.x 53 out via $pif setup keep-state # Allow out access to my ISP's DHCP server for cable/DSL configurations. $cmd 030 $skip udp from any to x.x.x.x 67 out via $pif keep-state # Allow out non-secure standard www function $cmd 040 $skip tcp from any to any 80 out via $pif setup keep-state # Allow out secure www function https over TLS SSL $cmd 050 $skip tcp from any to any 443 out via $pif setup keep-state # Allow out send & get email function $cmd 060 $skip tcp from any to any 25 out via $pif setup keep-state $cmd 061 $skip tcp from any to any 110 out via $pif setup keep-state # Allow out FreeBSD (make install & CVSUP) functions # Basically give user root "GOD" privileges. $cmd 070 $skip tcp from me to any out via $pif setup keep-state uid root # Allow out ping $cmd 080 $skip icmp from any to any out via $pif keep-state # Allow out Time $cmd 090 $skip tcp from any to any 37 out via $pif setup keep-state # Allow out nntp news (i.e. news groups) $cmd 100 $skip tcp from any to any 119 out via $pif setup keep-state # Allow out secure FTP, Telnet, and SCP # This function is using SSH (secure shell) $cmd 110 $skip tcp from any to any 22 out via $pif setup keep-state # Allow out whois $cmd 120 $skip tcp from any to any 43 out via $pif setup keep-state # Allow ntp time server $cmd 130 $skip udp from any to any 123 out via $pif keep-state ################################################################# # Interface facing Public Internet (Inbound Section) # Interrogate packets originating from the public Internet # destine for this gateway server or the private network. ################################################################# # Deny all inbound traffic from non-routable reserved address spaces $cmd 300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP $cmd 301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP $cmd 302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP $cmd 303 deny all from 127.0.0.0/8 to any in via $pif #loopback $cmd 304 deny all from 0.0.0.0/8 to any in via $pif #loopback $cmd 305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config $cmd 306 deny all from 192.0.2.0/24 to any in via $pif #reserved for docs $cmd 307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster $cmd 308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast # Deny ident $cmd 315 deny tcp from any to any 113 in via $pif # Deny all Netbios service. 137=name, 138=datagram, 139=session # Netbios is MS/Windows sharing services. # Block MS/Windows hosts2 name server requests 81 $cmd 320 deny tcp from any to any 137 in via $pif $cmd 321 deny tcp from any to any 138 in via $pif $cmd 322 deny tcp from any to any 139 in via $pif $cmd 323 deny tcp from any to any 81 in via $pif # Deny any late arriving packets $cmd 330 deny all from any to any frag in via $pif # Deny ACK packets that did not match the dynamic rule table $cmd 332 deny tcp from any to any established in via $pif # Allow traffic in from ISP's DHCP server. This rule must contain # the IP address of your ISP's DHCP server as it's the only # authorized source to send this packet type. # Only necessary for cable or DSL configurations. # This rule is not needed for 'user ppp' type connection to # the public Internet. This is the same IP address you captured # and used in the outbound section. $cmd 360 allow udp from x.x.x.x to any 68 in via $pif keep-state # Allow in standard www function because I have Apache server $cmd 370 allow tcp from any to me 80 in via $pif setup limit src-addr 2 # Allow in secure FTP, Telnet, and SCP from public Internet $cmd 380 allow tcp from any to me 22 in via $pif setup limit src-addr 2 # Allow in non-secure Telnet session from public Internet # labeled non-secure because ID & PW are passed over public # Internet as clear text. # Delete this sample group if you do not have telnet server enabled. $cmd 390 allow tcp from any to me 23 in via $pif setup limit src-addr 2 # Reject & Log all unauthorized incoming connections from the public Internet $cmd 400 deny log all from any to any in via $pif # Reject & Log all unauthorized out going connections to the public Internet $cmd 450 deny log all from any to any out via $pif # This is skipto location for outbound stateful rules $cmd 800 divert natd ip from any to any out via $pif $cmd 801 allow ip from any to any # Everything else is denied by default # deny and log all packets that fell through to see what they are $cmd 999 deny log all from any to any ################ End of IPFW rules file ############################### diff --git a/en_US.ISO8859-1/books/handbook/geom/chapter.sgml b/en_US.ISO8859-1/books/handbook/geom/chapter.sgml index d417e773b8..487c9eaf8b 100644 --- a/en_US.ISO8859-1/books/handbook/geom/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/geom/chapter.sgml @@ -1,669 +1,669 @@ Tom Rhodes Written by GEOM: Modular Disk Transformation Framework Synopsis GEOM GEOM Disk Framework GEOM This chapter covers the use of disks under the GEOM framework in &os;. This includes the major RAID control utilities which use the framework for configuration. This chapter will not go into in depth discussion on how GEOM handles or controls I/O, the underlying subsystem, or code. This information is provided through the &man.geom.4; manual page and its various SEE ALSO references. This chapter is also not a definitive guide to RAID configurations. Only GEOM-supported RAID classifications will be discussed. After reading this chapter, you will know: What type of RAID support is available through GEOM. How to use the base utilities to configure, maintain, and manipulate the various RAID levels. How to mirror, stripe, encrypt, and remotely connect disk devices through GEOM. How to troubleshoot disks attached to the GEOM framework. Before reading this chapter, you should: Understand how &os; treats disk devices (). Know how to configure and install a new &os; kernel (). GEOM Introduction GEOM permits access and control to classes — Master Boot Records, BSD labels, etc — through the use of providers, or the special files in - /dev. Supporting various + /dev. Supporting various software RAID configurations, GEOM will transparently provide access to the operating system and operating system utilities. Tom Rhodes Written by Murray Stokely RAID0 - Striping GEOM Striping Striping is a method used to combine several disk drives into a single volume. In many cases, this is done through the use of hardware controllers. The GEOM disk subsystem provides software support for RAID0, also known as disk striping. In a RAID0 system, data are split up in blocks that get written across all the drives in the array. Instead of having to wait on the system to write 256k to one disk, a RAID0 system can simultaneously write 64k to each of four different disks, offering superior I/O performance. This performance can be enhanced further by using multiple disk controllers. Each disk in a RAID0 stripe must be of the same size, since I/O requests are interleaved to read or write to multiple disks in parallel. Disk Striping Illustration Creating a stripe of unformatted ATA disks Load the geom_stripe module: &prompt.root; kldload geom_stripe Ensure that a suitable mount point exists. If this volume will become a root partition, then temporarily use another mount point such as /mnt: + class="directory">/mnt: &prompt.root; mkdir /mnt Determine the device names for the disks which will be striped, and create the new stripe device. For example, to stripe two unused and unpartitioned ATA disks, for example /dev/ad2 and /dev/ad3: &prompt.root; gstripe label -v st0 /dev/ad2 /dev/ad3 Write a standard label, also known as a partition table, on the new volume and install the default bootstrap code: &prompt.root; bsdlabel -wB /dev/stripe/st0 This process should have created two other devices - in the /dev/stripe + in the /dev/stripe directory in addition to the st0 device. Those include st0a and st0c. At this point a file system may be created on the st0a device with the newfs utility: &prompt.root; newfs -U /dev/stripe/st0a Many numbers will glide across the screen, and after a few seconds, the process will be complete. The volume has been created and is ready to be mounted. To manually mount the created disk stripe: &prompt.root; mount /dev/stripe/st0a /mnt To mount this striped file system automatically during the boot process, place the volume information in /etc/fstab file: &prompt.root; echo "/dev/stripe/st0a /mnt ufs rw 2 2" \ >> /etc/fstab The geom_stripe module must also be automatically loaded during system initialization, by adding a line to /boot/loader.conf: &prompt.root; echo 'geom_stripe_load="YES"' >> /boot/loader.conf RAID1 - Mirroring GEOM Disk Mirroring Mirroring is a technology used by many corporations and home users to back up data without interruption. When a mirror exists, it simply means that diskB replicates diskA. Or, perhaps diskC+D replicates diskA+B. Regardless of the disk configuration, the important aspect is that information on one disk or partition is being replicated. Later, that information could be more easily restored, backed up without causing service or access interruption, and even be physically stored in a data safe. To begin, ensure the system has two disk drives of equal size, this exercise assumes they are direct access (&man.da.4;) SCSI disks. Begin by installing &os; on the first disk with only two partitions. One should be a swap partition, double the RAM size and all remaining space devoted to - the root (/) file system. + the root (/) file system. It is possible to have separate partitions for other mount points; however, this will increase the difficulty level ten fold due to manual alteration of the &man.bsdlabel.8; and &man.fdisk.8; settings. Reboot and wait for the system to fully initialize. Once this process has completed, log in as the root user. Create the /dev/mirror/gm device and link it with /dev/da1: &prompt.root; gmirror label -vnb round-robin gm0 /dev/da1 The system should respond with: Metadata value stored on /dev/da1. Done. Initialize GEOM, this will load the /boot/kernel/geom_mirror.ko kernel module: &prompt.root; gmirror load This command should have created the gm0, device node under the - /dev/mirror + /dev/mirror directory. Install a generic fdisk label and boot code to new gm0 device: &prompt.root; fdisk -vBI /dev/mirror/gm0 Now install generic bsdlabel information: &prompt.root; bsdlabel -wB /dev/mirror/gm0s1 If multiple slices and partitions exist, the flags for the previous two commands will require alteration. They must match the slice and partition size of the other disk. Use the &man.newfs.8; utility to construct a default UFS file system on the gm0s1a device node: &prompt.root; newfs -U /dev/mirror/gm0s1a This should have caused the system to spit out some information and a bunch of numbers. This is good. Examine the screen for any error messages and mount the device to the - /mnt mount point: + /mnt mount point: &prompt.root; mount /dev/mirror/gm0s1a /mnt Now move all data from the boot disk over to this new file system. This example uses the &man.dump.8; and &man.restore.8; commands; however, &man.dd.1; would also work with this scenario. &prompt.root; dump -L -0 -f- / |(cd /mnt && restore -r -v -f-) This must be done for each file system. Simply place the appropriate file system in the correct location when running the aforementioned command. Now edit the replicated /mnt/etc/fstab file and remove or comment out the swap file It should be noted that commenting out the swap file entry in fstab will most likely require you to re-establish a different way of enabling swap space. Please refer to for more information. . Change the other file system information to use the new disk as shown in the following example: # Device Mountpoint FStype Options Dump Pass# #/dev/da0s2b none swap sw 0 0 /dev/mirror/gm0s1a / ufs rw 1 1 Ensure the geom_mirror.ko module will load on boot by running the following command: &prompt.root; echo 'geom_mirror_load="YES"' >> /mnt/boot/loader.conf &prompt.root; echo 'geom_mirror_load="YES"' >> /boot/loader.conf Reboot the system: &prompt.root; shutdown -r now At the boot screen, select option four (4) to gain access to single user mode. At the console, ensure that the system booted from the gm0s1a. This can be done by viewing the output from &man.df.1;. If all has gone well, the system should have booted from the gm0s1a device. From here, the primary disk may be cleared and inserted into the mirror using the following commands: &prompt.root; dd if=/dev/zero of=/dev/da0 bs=512 count=79 &prompt.root; gmirror configure -a gm0 &prompt.root; gmirror insert gm0 /dev/da0 The flag tells &man.gmirror.8; to use automatic synchronization; i.e., mirror the disk writes automatically. The manual page explains how to rebuild and replace disks, although it uses data in place of gm0. As the mirror is built the status may be checked using the following command: &prompt.root; gmirror status Troubleshooting System refuses to boot If the system boots up to a prompt similar to: ffs_mountroot: can't find rootvp Root mount failed: 6 mountroot> Reboot the machine using the power or reset button. At the boot menu, select option six (6). This will drop the system to a &man.loader.8; prompt. Load the kernel module manually: OK? load geom_mirror OK? boot If this works then for whatever reason the module was not being loaded properly. Place: options GEOM_MIRROR in the kernel configuration file, rebuild and reinstall. That should remedy this issue. GEOM Gate Network Devices GEOM supports the remote use of devices, such as disks, CD-ROMs, files, etc. through the use of the gate utilities. This is similar to NFS. To begin, an exports file must be created. This file specifies who is permitted to access the exported resources and what level of access they are offered. For example, to export the fourth slice on the first SCSI disk, the following /etc/gg.exports is more than adequate: 192.168.1.0/24 RW /dev/da0s4d It will allow all hosts inside the private network access the file system on the da0s4d partition. To export this device, ensure it is not currently mounted, and start the &man.ggated.8; server daemon: &prompt.root; ggated Now to mount the device on the client machine, issue the following commands: &prompt.root; ggatec create -o rw 192.168.1.1 /dev/da0s4d ggate0 &prompt.root; mount /dev/ggate0 /mnt From here on, the device may be accessed through the - /mnt mount point. + /mnt mount point. It should be pointed out that this will fail if the device is currently mounted on either the server machine or any other machine on the network. When the device is no longer needed, it may be safely unmounted with the &man.umount.8; command, similar to any other disk device. Labeling Disk Devices GEOM Disk Labels During system initialization, the &os; kernel will create device nodes as devices are found. This method of probing for devices raises some issues, for instance what if a new disk device is added via USB? It is very likely that a flash device may be handed the device name of da0 and the original da0 shifted to da1. This will cause issues mounting file systems if they are listed in /etc/fstab, effectively, this may also prevent the system from booting. One solution to this issue is to chain the SCSI devices in order so a new device added to the SCSI card will be issued unused device numbers. But what about USB devices which may replace the primary SCSI disk? This happens because USB devices are usually probed before the SCSI card. One solution is to only insert these devices after the system has been booted. Another method could be to use only a single ATA drive and never list the SCSI devices in /etc/fstab. A better solution is available. By using the glabel utility, an administrator or user may label their disk devices and use these labels in /etc/fstab. Because glabel stores the label in the last sector of a given provider, the label will remain persistent across reboots. By using this label as a device, the file system may always be mounted regardless of what device node it is accessed through. This goes without saying that a label be permanent. The glabel utility may be used to create both a transient and permanent label. Only the permanent label will remain consistent across reboots. See the &man.glabel.8; manual page for more information on the differences between labels. Label Types and Examples There are two types of labels, a generic label and a file system label. The difference between the labels is the auto detection associated with permanent labels, and the fact that this type of label will be persistent across reboots. These labels are given a special directory in /dev, which will be named based on their file system type. For example, UFS2 file system labels will be created in the /dev/ufs directory. A generic label will go away with the next reboot. These labels will be created in the /dev/label directory and are perfect for experimentation. Permanent labels may be placed on the file system using the tunefs or newfs utilities. To create a permanent label for a UFS2 file system without destroying any data, issue the following command: &prompt.root; tunefs -L home /dev/da3 If the file system is full, this may cause data corruption; however, if the file system is full then the main goal should be removing stale files and not adding labels. A label should now exist in /dev/ufs which may be added to /etc/fstab: /dev/ufs/home /home ufs rw 2 2 The file system must not be mounted while attempting to run tunefs. Now the file system may be mounted like normal: &prompt.root; mount /home From this point on, so long as the geom_label.ko kernel module is loaded at boot with /boot/loader.conf or the GEOM_LABEL kernel option is present, the device node may change without any ill effect on the system. File systems may also be created with a default label by using the flag with newfs. See the &man.newfs.8; manual page for more information. The following command can be used to destroy the label: &prompt.root; glabel destroy home UFS Journaling Through GEOM GEOM Journaling With the release of &os; 7.0, the long awaited feature of UFS journals has been implemented. The implementation itself is provided through the GEOM subsystem and is easily configured via the &man.gjournal.8; utility. What is journaling? Journaling capability stores a log of file system transactions, i.e.: changes that make up a complete disk write operation, before meta-data and file writes are committed to the disk proper. This transaction log can later be replayed to redo file system transactions, preventing file system inconsistencies. This method is yet another mechanism to protect against data loss and inconsistencies of the file system. Unlike Soft Updates which tracks and enforces meta-data updates and Snapshots which is an image of the file system, an actual log is stored at the end sector and, in some cases, may be stored on another disk entirely. Unlike other file system journaling implementations, the gjournal method is block based and not implemented as part of the file system - only as a GEOM extension. To enable support for gjournal, the &os; kernel must have the following option - which is the default on 7.X systems: options UFS_GJOURNAL Creating a journal on a free file system may now be done using the following steps, considering that the da4 is a new SCSI disk: &prompt.root; gjournal label /dev/da4 &prompt.root; gjournal load At this point, there should be a /dev/da4 device node and a /dev/da4.journal device node. A file system may now be created on this device: &prompt.root; newfs -O 2 -J /dev/da4.journal The previously issued command will create a UFS2 file system with journaling being made active. Effectively mount the device at the desired point with: &prompt.root; mount /dev/da4.journal /mnt In the case of several slices, a journal will be created for each individual slice. For instance, if ad4s1 and ad4s2 are both slices, then gjournal will create ad4s1.journal and ad4s2.journal. In the case of the command being run twice, the result will be journals. Under some circumstances, keeping the journal on another disk may be desired. For these cases, the journal provider or storage device should be listed after the device to enable journaling on. Journaling may also be enabled on current file systems by using tunefs; however, always make a backup before attempting to alter a file system. In most cases, the gjournal will fail if it is unable to create the actual journal but this does not protect against data loss incurred as a result of misusing tunefs. diff --git a/en_US.ISO8859-1/books/handbook/jails/chapter.sgml b/en_US.ISO8859-1/books/handbook/jails/chapter.sgml index 101e45ef3e..f28f4efb6d 100644 --- a/en_US.ISO8859-1/books/handbook/jails/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/jails/chapter.sgml @@ -1,961 +1,961 @@ Matteo Riondato Contributed by Jails jails Synopsis This chapter will provide an explanation of what &os; jails are and how to use them. Jails, sometimes referred to as an enhanced replacement of chroot environments, are a very powerful tool for system administrators, but their basic usage can also be useful for advanced users. After reading this chapter, you will know: What a jail is, and what purpose it may serve in &os; installations. How to build, start, and stop a jail. The basics of jail administration, both from inside and outside the jail. Other sources of useful information about jails are: The &man.jail.8; manual page. This is the full reference of the jail utility — the administrative tool which can be used in &os; to start, stop, and control &os; jails. The mailing lists and their archives. The archives of the &a.questions; and other mailing lists hosted by the &a.mailman.lists; already contain a wealth of material for jails. It should always be engaging to search the archives, or post a new question to the &a.questions.name; mailing list. Terms Related to Jails To facilitate better understanding of parts of the &os; system related to jails, their internals and the way they interact with the rest of &os;, the following terms are used further in this chapter: &man.chroot.2; (command) A system call of &os;, which changes the root directory of a process and all its descendants. &man.chroot.2; (environment) The environment of processes running in a chroot. This includes resources such as the part of the file system which is visible, user and group IDs which are available, network interfaces and other IPC mechanisms, etc. &man.jail.8; (command) The system administration utility which allows launching of processes within a jail environment. host (system, process, user, etc.) The controlling system of a jail environment. The host system has access to all the hardware resources available, and can control processes both outside of and inside a jail environment. One of the important differences of the host system from a jail is that the limitations which apply to superuser processes inside a jail are not enforced for processes of the host system. hosted (system, process, user, etc.) A process, user or other entity, whose access to resources is restricted by an &os; jail. Introduction Since system administration is a difficult and perplexing task, many powerful tools were developed to make life easier for the administrator. These tools mostly provide enhancements of some sort to the way systems are installed, configured and maintained. Part of the tasks which an administrator is expected to do is to properly configure the security of a system, so that it can continue serving its real purpose, without allowing security violations. One of the tools which can be used to enhance the security of a &os; system are jails. Jails were introduced in &os; 4.X by &a.phk;, but were greatly improved in &os; 5.X to make them a powerful and flexible subsystem. Their development still goes on, enhancing their usefulness, performance, reliability, and security. What is a Jail BSD-like operating systems have had &man.chroot.2; since the time of 4.2BSD. The &man.chroot.8; utility can be used to change the root directory of a set of processes, creating a safe environment, separate from the rest of the system. Processes created in the chrooted environment can not access files or resources outside of it. For that reason, compromising a service running in a chrooted environment should not allow the attacker to compromise the entire system. The &man.chroot.8; utility is good for easy tasks, which do not require a lot of flexibility or complex and advanced features. Since the inception of the chroot concept, however, many ways have been found to escape from a chrooted environment and, although they have been fixed in modern versions of the &os; kernel, it was clear that &man.chroot.2; was not the ideal solution for securing services. A new subsystem had to be implemented. This is one of the main reasons why jails were developed. Jails improve on the concept of the traditional &man.chroot.2; environment, in several ways. In a traditional &man.chroot.2; environment, processes are only limited in the part of the file system they can access. The rest of the system resources (like the set of system users, the running processes, or the networking subsystem) are shared by the chrooted processes and the processes of the host system. Jails expand this model by virtualizing not only access to the file system, but also the set of users, the networking subsystem of the &os; kernel and a few other things. A more complete set of fine-grained controls available for tuning the access of a jailed environment is described in . A jail is characterized by four elements: A directory subtree — the starting point from which a jail is entered. Once inside the jail, a process is not permitted to escape outside of this subtree. Traditional security issues which plagued the original &man.chroot.2; design will not affect &os; jails. A hostname — the hostname which will be used within the jail. Jails are mainly used for hosting network services, therefore having a descriptive hostname for each jail can really help the system administrator. An IP address — this will be assigned to the jail and cannot be changed in any way during the jail's life span. The IP address of a jail is usually an alias address for an existing network interface, but this is not strictly necessary. A command — the path name of an executable to run inside the jail. This is relative to the root directory of the jail environment, and may vary a lot, depending on the type of the specific jail environment. Apart from these, jails can have their own set of users and their own root user. Naturally, the powers of the root user are limited within the jail environment and, from the point of view of the host system, the jail root user is not an omnipotent user. In addition, the root user of a jail is not allowed to perform critical operations to the system outside of the associated &man.jail.8; environment. More information about capabilities and restrictions of the root user will be discussed in below. Creating and Controlling Jails Some administrators divide jails into the following two types: complete jails, which resemble a real &os; system, and service jails, dedicated to one application or service, possibly running with privileges. This is only a conceptual division and the process of building a jail is not affected by it. The &man.jail.8; manual page is quite clear about the procedure for building a jail: &prompt.root; setenv D /here/is/the/jail &prompt.root; mkdir -p $D &prompt.root; cd /usr/src &prompt.root; make world DESTDIR=$D &prompt.root; cd etc/ This step is not required on &os; 6.0 and later. &prompt.root; make distribution DESTDIR=$D &prompt.root; mount -t devfs devfs $D/dev Selecting a location for a jail is the best starting point. This is where the jail will physically reside within the file system of the jail's host. A good choice can be /usr/jail/jailname, + class="directory">/usr/jail/jailname, where jailname is the hostname identifying the jail. The /usr/ file system usually has + class="directory">/usr/ file system usually has enough space for the jail file system, which for complete jails is, essentially, a replication of every file present in a default installation of the &os; base system. This command will populate the directory subtree chosen as jail's physical location on the file system with the necessary binaries, libraries, manual pages and so on. Everything is done in the typical &os; style — first everything is built/compiled, then installed to the destination path. The distribution target for make installs every needed configuration file. In simple words, it installs every installable file of - /usr/src/etc/ to the - /etc directory of the jail + /usr/src/etc/ to the + /etc directory of the jail environment: - $D/etc/. + $D/etc/. Mounting the &man.devfs.8; file system inside a jail is not required. On the other hand, any, or almost any application requires access to at least one device, depending on the purpose of the given application. It is very important to control access to devices from inside a jail, as improper settings could permit an attacker to do nasty things in the jail. Control over &man.devfs.8; is managed through rulesets which are described in the &man.devfs.8; and &man.devfs.conf.5; manual pages. Once a jail is installed, it can be started by using the &man.jail.8; utility. The &man.jail.8; utility takes four mandatory arguments which are described in the . Other arguments may be specified too, e.g., to run the jailed process with the credentials of a specific user. The argument depends on the type of the jail; for a virtual system, /etc/rc is a good choice, since it will replicate the startup sequence of a real &os; system. For a service jail, it depends on the service or application that will run within the jail. Jails are often started at boot time and the &os; rc mechanism provides an easy way to do this. A list of the jails which are enabled to start at boot time should be added to the &man.rc.conf.5; file: jail_enable="YES" # Set to NO to disable starting of any jails jail_list="www" # Space separated list of names of jails For each jail listed in jail_list, a group of &man.rc.conf.5; settings, which describe the particular jail, should be added: jail_www_rootdir="/usr/jail/www" # jail's root directory jail_www_hostname="www.example.org" # jail's hostname jail_www_ip="192.168.0.10" # jail's IP address jail_www_devfs_enable="YES" # mount devfs in the jail jail_www_devfs_ruleset="www_ruleset" # devfs ruleset to apply to jail The default startup of jails configured in &man.rc.conf.5;, will run the /etc/rc script of the jail, which assumes the jail is a complete virtual system. For service jails, the default startup command of the jail should be changed, by setting the jail_jailname_exec_start option appropriately. For a full list of available options, please see the &man.rc.conf.5; manual page. The /etc/rc.d/jail script can be used to start or stop a jail by hand, if an entry for it exists in rc.conf: &prompt.root; /etc/rc.d/jail start www &prompt.root; /etc/rc.d/jail stop www A clean way to shut down a &man.jail.8; is not available at the moment. This is because commands normally used to accomplish a clean system shutdown cannot be used inside a jail. The best way to shut down a jail is to run the following command from within the jail itself or using the &man.jexec.8; utility from outside the jail: &prompt.root; sh /etc/rc.shutdown More information about this can be found in the &man.jail.8; manual page. Fine Tuning and Administration There are several options which can be set for any jail, and various ways of combining a host &os; system with jails, to produce higher level applications. This section presents: Some of the options available for tuning the behavior and security restrictions implemented by a jail installation. Some of the high-level applications for jail management, which are available through the &os; Ports Collection, and can be used to implement overall jail-based solutions. System tools for jail tuning in &os; Fine tuning of a jail's configuration is mostly done by setting &man.sysctl.8; variables. A special subtree of sysctl exists as a basis for organizing all the relevant options: the security.jail.* hierarchy of &os; kernel options. Here is a list of the main jail-related sysctls, complete with their default value. Names should be self-explanatory, but for more information about them, please refer to the &man.jail.8; and &man.sysctl.8; manual pages. security.jail.set_hostname_allowed: 1 security.jail.socket_unixiproute_only: 1 security.jail.sysvipc_allowed: 0 security.jail.enforce_statfs: 2 security.jail.allow_raw_sockets: 0 security.jail.chflags_allowed: 0 security.jail.jailed: 0 These variables can be used by the system administrator of the host system to add or remove some of the limitations imposed by default on the root user. Note that there are some limitations which cannot be removed. The root user is not allowed to mount or unmount file systems from within a &man.jail.8;. The root inside a jail may not load or unload &man.devfs.8; rulesets, set firewall rules, or do many other administrative tasks which require modifications of in-kernel data, such as setting the securelevel of the kernel. The base system of &os; contains a basic set of tools for viewing information about the active jails, and attaching to a jail to run administrative commands. The &man.jls.8; and &man.jexec.8; commands are part of the base &os; system, and can be used to perform the following simple tasks: Print a list of active jails and their corresponding jail identifier (JID), IP address, hostname and path. Attach to a running jail, from its host system, and run a command inside the jail or perform administrative tasks inside the jail itself. This is especially useful when the root user wants to cleanly shut down a jail. The &man.jexec.8; utility can also be used to start a shell in a jail to do administration in it; for example: &prompt.root; jexec 1 tcsh High-level administrative tools in &os; Ports Collection Among the many third-party utilities for jail administration, one of the most complete and useful is sysutils/jailutils. It is a set of small applications that contribute to &man.jail.8; management. Please refer to its web page for more information. Application of Jails Daniel Gerzo Contributed by Service Jails This section is based upon an idea originally presented by &a.simon; at , and an updated article written by Ken Tom locals@gmail.com. This section illustrates how to set up a &os; system that adds an additional layer of security, using the &man.jail.8; feature. It is also assumed that the given system is at least running RELENG_6_0 and the information provided earlier in this chapter has been well understood. Design One of the major problems with jails is the management of their upgrade process. This tends to be a problem because every jail has to be rebuilt from scratch whenever it is updated. This is usually not a problem for a single jail, since the update process is fairly simple, but can be quite time consuming and tedious if a lot of jails are created. This setup requires advanced experience with &os; and usage of its features. If the presented steps below look too complicated, it is advised to take a look at a simpler system such as sysutils/ezjail, which provides an easier method of administering &os; jails and is not as sophisticated as this setup. This idea has been presented to resolve such issues by sharing as much as is possible between jails, in a safe way — using read-only &man.mount.nullfs.8; mounts, so that updating will be be simpler, and putting single services into individual jails will become more attractive. Additionally, it provides a simple way to add or remove jails as well as a way to upgrade them. Examples of services in this context are: an HTTP server, a DNS server, a SMTP server, and so forth. The goals of the setup described in this section are: Create a simple and easy to understand jail structure. This implies not having to run a full installworld on each and every jail. Make it easy to add new jails or remove existing ones. Make it easy to update or upgrade existing jails. Make it possible to run a customized &os; branch. Be paranoid about security, reducing as much as possible the possibility of compromise. Save space and inodes, as much as possible. As it has been already mentioned, this design relies heavily on having a single master template which is read-only (known as nullfs) mounted into each jail and one read-write device per jail. A device can be a separate physical disc, a partition, or a vnode backed &man.md.4; device. In this example, we will use read-write nullfs mounts. The file system layout is described in the following list: Each jail will be mounted under the /home/j directory. + class="directory">/home/j directory. - /home/j/mroot is + /home/j/mroot is the template for each jail and the read-only partition for all of the jails. A blank directory will be created for each jail under - the /home/j + the /home/j directory. Each jail will have a /s directory, that will be + class="directory">/s directory, that will be linked to the read-write portion of the system. Each jail shall have its own read-write system that is based upon /home/j/skel. + class="directory">/home/j/skel. Each jailspace (read-write portion of each jail) shall be created in /home/js. + class="directory">/home/js. This assumes that the jails are based under the - /home partition. This + /home partition. This can, of course, be changed to anything else, but this change will have to be reflected in each of the examples below. Creating the Template This section will describe the steps needed to create the master template that will be the read-only portion for the jails to use. It is always a good idea to update the &os; system to the latest -RELEASE branch. Check the corresponding Handbook Chapter to accomplish this task. In the case the update is not feasible, the buildworld will be required in order to be able to proceed. Additionally, the sysutils/cpdup package will be required. We will use the &man.portsnap.8; utility to download the &os; Ports Collection. The Handbook Portsnap Chapter is always good reading for newcomers. First, create a directory structure for the read-only file system which will contain the &os; binaries for our jails, then change directory to the &os; source tree and install the read-only file system to the jail template: &prompt.root; mkdir /home/j /home/j/mroot &prompt.root; cd /usr/src &prompt.root; make installworld DESTDIR=/home/j/mroot Next, prepare a &os; Ports Collection for the jails as well as a &os; source tree, which is required for mergemaster: &prompt.root; cd /home/j/mroot &prompt.root; mkdir usr/ports &prompt.root; portsnap -p /home/j/mroot/usr/ports fetch extract &prompt.root; cpdup /usr/src /home/j/mroot/usr/src Create a skeleton for the read-write portion of the system: &prompt.root; mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles &prompt.root; mv etc /home/j/skel &prompt.root; mv usr/local /home/j/skel/usr-local &prompt.root; mv tmp /home/j/skel &prompt.root; mv var /home/j/skel &prompt.root; mv root /home/j/skel Use mergemaster to install missing configuration files. Then get rid of the extra directories that mergemaster creates: &prompt.root; mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i &prompt.root; cd /home/j/skel &prompt.root; rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev Now, symlink the read-write file system to the read-only file system. Please make sure that the symlinks are created in the correct s/ locations. Real + class="directory">s/ locations. Real directories or the creation of directories in the wrong locations will cause the installation to fail. &prompt.root; cd /home/j/mroot &prompt.root; mkdir s &prompt.root; ln -s s/etc etc &prompt.root; ln -s s/home home &prompt.root; ln -s s/root root &prompt.root; ln -s ../s/usr-local usr/local &prompt.root; ln -s ../s/usr-X11R6 usr/X11R6 &prompt.root; ln -s ../../s/distfiles usr/ports/distfiles &prompt.root; ln -s s/tmp tmp &prompt.root; ln -s s/var var As a last step, create a generic /home/j/skel/etc/make.conf with its contents as shown below: WRKDIRPREFIX?= /s/portbuild Having WRKDIRPREFIX set up this way will make it possible to compile &os; ports inside each jail. Remember that the ports directory is part of the read-only system. The custom path for WRKDIRPREFIX allows builds to be done in the read-write portion of every jail. Creating Jails Now that we have a complete &os; jail template, we can setup and configure the jails in /etc/rc.conf. This example demonstrates the creation of 3 jails: NS, MAIL and WWW. Put the following lines into the /etc/fstab file, so that the read-only template for the jails and the read-write space will be available in the respective jails: /home/j/mroot /home/j/ns nullfs ro 0 0 /home/j/mroot /home/j/mail nullfs ro 0 0 /home/j/mroot /home/j/www nullfs ro 0 0 /home/js/ns /home/j/ns/s nullfs rw 0 0 /home/js/mail /home/j/mail/s nullfs rw 0 0 /home/js/www /home/j/www/s nullfs rw 0 0 Partitions marked with a 0 pass number are not checked by &man.fsck.8; during boot, and partitions marked with a 0 dump number are not backed up by &man.dump.8;. We do not want fsck to check nullfs mounts or dump to back up the read-only nullfs mounts of the jails. This is why they are marked with 0 0 in the last two columns of each fstab entry above. Configure the jails in /etc/rc.conf: jail_enable="YES" jail_set_hostname_allow="NO" jail_list="ns mail www" jail_ns_hostname="ns.example.org" jail_ns_ip="192.168.3.17" jail_ns_rootdir="/usr/home/j/ns" jail_ns_devfs_enable="YES" jail_mail_hostname="mail.example.org" jail_mail_ip="192.168.3.18" jail_mail_rootdir="/usr/home/j/mail" jail_mail_devfs_enable="YES" jail_www_hostname="www.example.org" jail_www_ip="62.123.43.14" jail_www_rootdir="/usr/home/j/www" jail_www_devfs_enable="YES" The reason why the jail_name_rootdir variable is set to /usr/home instead of - /home is that the + class="directory">/usr/home instead of + /home is that the physical path of the /home directory on a + class="directory">/home directory on a default &os; installation is /usr/home. The + class="directory">/usr/home. The jail_name_rootdir variable must not be set to a path which includes a symbolic link, otherwise the jails will refuse to start. Use the &man.realpath.1; utility to determine a value which should be set to this variable. Please see the &os;-SA-07:01.jail Security Advisory for more information. Create the required mount points for the read-only file system of each jail: &prompt.root; mkdir /home/j/ns /home/j/mail /home/j/www Install the read-write template into each jail. Note the use of sysutils/cpdup, which helps to ensure that a correct copy is done of each directory: &prompt.root; mkdir /home/js &prompt.root; cpdup /home/j/skel /home/js/ns &prompt.root; cpdup /home/j/skel /home/js/mail &prompt.root; cpdup /home/j/skel /home/js/www In this phase, the jails are built and prepared to run. First, mount the required file systems for each jail, and then start them using the /etc/rc.d/jail script: &prompt.root; mount -a &prompt.root; /etc/rc.d/jail start The jails should be running now. To check if they have started correctly, use the &man.jls.8; command. Its output should be similar to the following: &prompt.root; jls JID IP Address Hostname Path 3 192.168.3.17 ns.example.org /home/j/ns 2 192.168.3.18 mail.example.org /home/j/mail 1 62.123.43.14 www.example.org /home/j/www At this point, it should be possible to log onto each jail, add new users or configure daemons. The JID column indicates the jail identification number of each running jail. Use the following command in order to perform administrative tasks in the jail whose JID is 3: &prompt.root; jexec 3 tcsh Upgrading In time, there will be a need to upgrade the system to a newer version of &os;, either because of a security issue, or because new features have been implemented which are useful for the existing jails. The design of this setup provides an easy way to upgrade existing jails. Additionally, it minimizes their downtime, as the jails will be brought down only in the very last minute. Also, it provides a way to roll back to the older versions should any problems occur. The first step is to upgrade the host system in the usual manner. Then create a new temporary read-only template in /home/j/mroot2. + class="directory">/home/j/mroot2. &prompt.root; mkdir /home/j/mroot2 &prompt.root; cd /usr/src &prompt.root; make installworld DESTDIR=/home/j/mroot2 &prompt.root; cd /home/j/mroot2 &prompt.root; cpdup /usr/src usr/src &prompt.root; mkdir s The installworld run creates a few unnecessary directories, which should be removed: &prompt.root; chflags -R 0 var &prompt.root; rm -R etc var root usr/local tmp Recreate the read-write symlinks for the master file system: &prompt.root; ln -s s/etc etc &prompt.root; ln -s s/root root &prompt.root; ln -s s/home home &prompt.root; ln -s ../s/usr-local usr/local &prompt.root; ln -s ../s/usr-X11R6 usr/X11R6 &prompt.root; ln -s s/tmp tmp &prompt.root; ln -s s/var var The right time to stop the jails is now: &prompt.root; /etc/rc.d/jail stop Unmount the original file systems: &prompt.root; umount /home/j/ns/s &prompt.root; umount /home/j/ns &prompt.root; umount /home/j/mail/s &prompt.root; umount /home/j/mail &prompt.root; umount /home/j/www/s &prompt.root; umount /home/j/www The read-write systems are attached to the read-only - system (/s) and + system (/s) and must be unmounted first. Move the old read-only file system and replace it with the new one. This will serve as a backup and archive of the old read-only file system should something go wrong. The naming convention used here corresponds to when a new read-only file system has been created. Move the original &os; Ports Collection over to the new file system to save some space and inodes: &prompt.root; cd /home/j &prompt.root; mv mroot mroot.20060601 &prompt.root; mv mroot2 mroot &prompt.root; mv mroot.20060601/usr/ports mroot/usr At this point the new read-only template is ready, so the only remaining task is to remount the file systems and start the jails: &prompt.root; mount -a &prompt.root; /etc/rc.d/jail start Use &man.jls.8; to check if the jails started correctly. Do not forget to run mergemaster in each jail. The configuration files will need to be updated as well as the rc.d scripts. diff --git a/en_US.ISO8859-1/books/handbook/kernelconfig/chapter.sgml b/en_US.ISO8859-1/books/handbook/kernelconfig/chapter.sgml index ff9848423b..c98db31594 100644 --- a/en_US.ISO8859-1/books/handbook/kernelconfig/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/kernelconfig/chapter.sgml @@ -1,1531 +1,1531 @@ Jim Mock Updated and restructured by Jake Hamby Originally contributed by Configuring the FreeBSD Kernel Synopsis kernel building a custom kernel The kernel is the core of the &os; operating system. It is responsible for managing memory, enforcing security controls, networking, disk access, and much more. While more and more of &os; becomes dynamically configurable it is still occasionally necessary to reconfigure and recompile your kernel. After reading this chapter, you will know: Why you might need to build a custom kernel. How to write a kernel configuration file, or alter an existing configuration file. How to use the kernel configuration file to create and build a new kernel. How to install the new kernel. How to troubleshoot if things go wrong. All of the commands listed within this chapter by way of example should be executed as root in order to succeed. Why Build a Custom Kernel? Traditionally, &os; has had what is called a monolithic kernel. This means that the kernel was one large program, supported a fixed list of devices, and if you wanted to change the kernel's behavior then you had to compile a new kernel, and then reboot your computer with the new kernel. Today, &os; is rapidly moving to a model where much of the kernel's functionality is contained in modules which can be dynamically loaded and unloaded from the kernel as necessary. This allows the kernel to adapt to new hardware suddenly becoming available (such as PCMCIA cards in a laptop), or for new functionality to be brought into the kernel that was not necessary when the kernel was originally compiled. This is known as a modular kernel. Despite this, it is still necessary to carry out some static kernel configuration. In some cases this is because the functionality is so tied to the kernel that it can not be made dynamically loadable. In others it may simply be because no one has yet taken the time to write a dynamic loadable kernel module for that functionality. Building a custom kernel is one of the most important rites of passage nearly every BSD user must endure. This process, while time consuming, will provide many benefits to your &os; system. Unlike the GENERIC kernel, which must support a wide range of hardware, a custom kernel only contains support for your PC's hardware. This has a number of benefits, such as: Faster boot time. Since the kernel will only probe the hardware you have on your system, the time it takes your system to boot can decrease dramatically. Lower memory usage. A custom kernel often uses less memory than the GENERIC kernel, which is important because the kernel must always be present in real memory. For this reason, a custom kernel is especially useful on a system with a small amount of RAM. Additional hardware support. A custom kernel allows you to add in support for devices which are not present in the GENERIC kernel, such as sound cards. Tom Rhodes Written by Finding the System Hardware Before venturing into kernel configuration, it would be wise to get an inventory of the machine's hardware. In cases where &os; is not the primary operating system, the inventory list may easily be created by viewing the current operating system configuration. For example, µsoft;'s Device Manager normally contains important information about installed devices. The Device Manager is located in the control panel. Some versions of µsoft.windows; have a System icon which will display a screen where Device Manager may be accessed. If another operating system does not exist on the machine, the administrator must find this information out manually. One method is using the &man.dmesg.8; utility and the &man.man.1; commands. Most device drivers on &os; have a manual page, listing supported hardware, and during the boot probe, found hardware will be listed. For example, the following lines indicate that the psm driver found a mouse: psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 This driver will need to be included in the custom kernel configuration file or loaded using &man.loader.conf.5;. On occasion, the data from dmesg will only show system messages instead of the boot probe output. In these situations, the output may be obtained by viewing the /var/run/dmesg.boot file. Another method of finding hardware is by using the &man.pciconf.8; utility which provides more verbose output. For example: ath0@pci0:3:0:0: class=0x020000 card=0x058a1014 chip=0x1014168c rev=0x01 hdr=0x00 vendor = 'Atheros Communications Inc.' device = 'AR5212 Atheros AR5212 802.11abg wireless' class = network subclass = ethernet This bit of output, obtained using pciconf shows that the ath driver located a wireless Ethernet device. Using man ath will return the &man.ath.4; manual page. The flag, when passed to &man.man.1; can also be used to provide useful information. From the above, one can issue: &prompt.root; man -k Atheros To get a list of manual pages which contain that particular word: ath(4) - Atheros IEEE 802.11 wireless network driver ath_hal(4) - Atheros Hardware Access Layer (HAL) Armed with a hardware inventory list, the process of building a custom kernel should appear less daunting. Kernel Drivers, Subsystems, and Modules kernel drivers / modules / subsystems Before building a custom kernel, consider the reasons for doing so. If there is a need for specific hardware support, it may already exist as a module. Kernel modules exist in the - /boot/kernel directory + /boot/kernel directory and may be dynamically loaded into the running kernel using &man.kldload.8;. Most, if not all kernel drivers have a specific module and manual page. For example, the last section noted the ath wireless Ethernet driver. This device has the following information in its manual page: Alternatively, to load the driver as a module at boot time, place the following line in &man.loader.conf.5: if_ath_load="YES" As instructed, adding the if_ath_load="YES" line to the /boot/loader.conf file will enable loading this module dynamically at boot time. In some cases; however, there is no associated module. This is mostly true for certain subsystems and very important drivers, for instance, the fast file system (FFS) is a required option in the kernel. As is network support (INET). Unfortunately the only way to tell if a driver is required is to check for the module itself. It is considerably easy to remove built in support for a device or option and have a broken kernel. For example, if the &man.ata.4; driver is pulled from the kernel configuration file, a system using ATA disk drivers may not boot without the line added to loader.conf. When in doubt, check for the module and then just leave support in the kernel. Building and Installing a Custom Kernel kernel building / installing First, let us take a quick tour of the kernel build directory. All directories mentioned will be relative to the main /usr/src/sys directory, which is also accessible through the path name /sys. There are a number of subdirectories here representing different parts of the kernel, but the most important for our purposes are arch/conf, where you will edit your custom kernel configuration, and compile, which is the staging area where your kernel will be built. arch represents one of i386, alpha, amd64, ia64, powerpc, sparc64, or pc98 (an alternative development branch of PC hardware, popular in Japan). Everything inside a particular architecture's directory deals with that architecture only; the rest of the code is machine independent code common to all platforms to which &os; could potentially be ported. Notice the logical organization of the directory structure, with each supported device, file system, and option in its own subdirectory. This chapter assumes that you are using the i386 architecture in the examples. If this is not the case for your situation, make appropriate adjustments to the path names for your system's architecture. If there is not a /usr/src/sys directory on your system, then the kernel source has not been installed. The easiest way to do this is by running sysinstall as root, choosing Configure, then Distributions, then src, then base and sys. If you have an aversion to sysinstall and you have access to an official &os; CDROM, then you can also install the source from the command line: &prompt.root; mount /cdrom &prompt.root; mkdir -p /usr/src/sys &prompt.root; ln -s /usr/src/sys /sys &prompt.root; cat /cdrom/src/ssys.[a-d]* | tar -xzvf - &prompt.root; cat /cdrom/src/sbase.[a-d]* | tar -xzvf - Next, move to the arch/conf directory and copy the GENERIC configuration file to the name you want to give your kernel. For example: &prompt.root; cd /usr/src/sys/i386/conf &prompt.root; cp GENERIC MYKERNEL Traditionally, this name is in all capital letters and, if you are maintaining multiple &os; machines with different hardware, it is a good idea to name it after your machine's hostname. We will call it MYKERNEL for the purpose of this example. Storing your kernel configuration file directly under /usr/src can be a bad idea. If you are experiencing problems it can be tempting to just delete /usr/src and start again. After doing this, it usually only takes a few seconds for you to realize that you have deleted your custom kernel configuration file. Also, do not edit GENERIC directly, as it may get overwritten the next time you update your source tree, and your kernel modifications will be lost. You might want to keep your kernel configuration file elsewhere, and then create a symbolic link to the file in the i386 directory. For example: &prompt.root; cd /usr/src/sys/i386/conf &prompt.root; mkdir /root/kernels &prompt.root; cp GENERIC /root/kernels/MYKERNEL &prompt.root; ln -s /root/kernels/MYKERNEL Now, edit MYKERNEL with your favorite text editor. If you are just starting out, the only editor available will probably be vi, which is too complex to explain here, but is covered well in many books in the bibliography. However, &os; does offer an easier editor called ee which, if you are a beginner, should be your editor of choice. Feel free to change the comment lines at the top to reflect your configuration or the changes you have made to differentiate it from GENERIC. SunOS If you have built a kernel under &sunos; or some other BSD operating system, much of this file will be very familiar to you. If you are coming from some other operating system such as DOS, on the other hand, the GENERIC configuration file might seem overwhelming to you, so follow the descriptions in the Configuration File section slowly and carefully. If you sync your source tree with the latest sources of the &os; project, be sure to always check the file /usr/src/UPDATING before you perform any update steps. This file describes any important issues or areas requiring special attention within the updated source code. /usr/src/UPDATING always matches your version of the &os; source, and is therefore more up to date with new information than this handbook. You must now compile the source code for the kernel. Building a Kernel Change to the /usr/src directory: + class="directory">/usr/src directory: &prompt.root; cd /usr/src Compile the kernel: &prompt.root; make buildkernel KERNCONF=MYKERNEL Install the new kernel: &prompt.root; make installkernel KERNCONF=MYKERNEL It is required to have full &os; source tree to build the kernel. By default, when you build a custom kernel, all kernel modules will be rebuilt as well. If you want to update a kernel faster or to build only custom modules, you should edit /etc/make.conf before starting to build the kernel: MODULES_OVERRIDE = linux acpi sound/sound sound/driver/ds1 ntfs This variable sets up a list of modules to build instead of all of them. WITHOUT_MODULES = linux acpi sound/sound sound/driver/ds1 ntfs This variable sets up a list of modules to exclude from the build process. For other variables which you may find useful in the process of building kernel, refer to &man.make.conf.5; manual page. /boot/kernel.old The new kernel will be copied to the /boot/kernel directory as /boot/kernel/kernel and the old kernel will be moved to /boot/kernel.old/kernel. Now, shutdown the system and reboot to use your new kernel. If something goes wrong, there are some troubleshooting instructions at the end of this chapter that you may find useful. Be sure to read the section which explains how to recover in case your new kernel does not boot. Other files relating to the boot process, such as the boot &man.loader.8; and configuration are stored in /boot. Third party or custom modules can be placed in /boot/kernel, although users should be aware that keeping modules in sync with the compiled kernel is very important. Modules not intended to run with the compiled kernel may result in instability or incorrectness. Joel Dahl Updated for &os; 6.X by The Configuration File kernel NOTES NOTES kernel configuration file The general format of a configuration file is quite simple. Each line contains a keyword and one or more arguments. For simplicity, most lines only contain one argument. Anything following a # is considered a comment and ignored. The following sections describe each keyword, in the order they are listed in GENERIC. For an exhaustive list of architecture dependent options and devices, see the NOTES file in the same directory as the GENERIC file. For architecture independent options, see /usr/src/sys/conf/NOTES. To build a file which contains all available options, as normally done for testing purposes, run the following command as root: &prompt.root; cd /usr/src/sys/i386/conf && make LINT kernel configuration file The following is an example of the GENERIC kernel configuration file with various additional comments where needed for clarity. This example should match your copy in /usr/src/sys/i386/conf/GENERIC fairly closely. kernel options machine machine i386 This is the machine architecture. It must be either alpha, amd64, i386, ia64, pc98, powerpc, or sparc64. kernel options cpu cpu I486_CPU cpu I586_CPU cpu I686_CPU The above option specifies the type of CPU you have in your system. You may have multiple instances of the CPU line (if, for example, you are not sure whether you should use I586_CPU or I686_CPU), but for a custom kernel it is best to specify only the CPU you have. If you are unsure of your CPU type, you can check the /var/run/dmesg.boot file to view your boot messages. kernel options ident ident GENERIC This is the identification of the kernel. You should change this to whatever you named your kernel, i.e. MYKERNEL if you have followed the instructions of the previous examples. The value you put in the ident string will print when you boot up the kernel, so it is useful to give the new kernel a different name if you want to keep it separate from your usual kernel (e.g., you want to build an experimental kernel). #To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. The &man.device.hints.5; is used to configure options of the device drivers. The default location that &man.loader.8; will check at boot time is /boot/device.hints. Using the hints option you can compile these hints statically into your kernel. Then there is no need to create a device.hints file in /boot. makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols The normal build process of &os; includes debugging information when building the kernel with the the option, which enables debugging information when passed to &man.gcc.1;. options SCHED_4BSD # 4BSD scheduler The traditional and default system scheduler for &os;. Keep this. options PREEMPTION # Enable kernel thread preemption Allows threads that are in the kernel to be preempted by higher priority threads. It helps with interactivity and allows interrupt threads to run sooner rather than waiting. options INET # InterNETworking Networking support. Leave this in, even if you do not plan to be connected to a network. Most programs require at least loopback networking (i.e., making network connections within your PC), so this is essentially mandatory. options INET6 # IPv6 communications protocols This enables the IPv6 communication protocols. options FFS # Berkeley Fast Filesystem This is the basic hard drive file system. Leave it in if you boot from the hard disk. options SOFTUPDATES # Enable FFS Soft Updates support This option enables Soft Updates in the kernel, this will help speed up write access on the disks. Even when this functionality is provided by the kernel, it must be turned on for specific disks. Review the output from &man.mount.8; to see if Soft Updates is enabled for your system disks. If you do not see the soft-updates option then you will need to activate it using the &man.tunefs.8; (for existing file systems) or &man.newfs.8; (for new file systems) commands. options UFS_ACL # Support for access control lists This option enables kernel support for access control lists. This relies on the use of extended attributes and UFS2, and the feature is described in detail in . ACLs are enabled by default and should not be disabled in the kernel if they have been used previously on a file system, as this will remove the access control lists, changing the way files are protected in unpredictable ways. options UFS_DIRHASH # Improve performance on big directories This option includes functionality to speed up disk operations on large directories, at the expense of using additional memory. You would normally keep this for a large server, or interactive workstation, and remove it if you are using &os; on a smaller system where memory is at a premium and disk access speed is less important, such as a firewall. options MD_ROOT # MD is a potential root device This option enables support for a memory backed virtual disk used as a root device. kernel options NFS kernel options NFS_ROOT options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options NFS_ROOT # NFS usable as /, requires NFSCLIENT The network file system. Unless you plan to mount partitions from a &unix; file server over TCP/IP, you can comment these out. kernel options MSDOSFS options MSDOSFS # MSDOS Filesystem The &ms-dos; file system. Unless you plan to mount a DOS formatted hard drive partition at boot time, you can safely comment this out. It will be automatically loaded the first time you mount a DOS partition, as described above. Also, the excellent emulators/mtools software allows you to access DOS floppies without having to mount and unmount them (and does not require MSDOSFS at all). options CD9660 # ISO 9660 Filesystem The ISO 9660 file system for CDROMs. Comment it out if you do not have a CDROM drive or only mount data CDs occasionally (since it will be dynamically loaded the first time you mount a data CD). Audio CDs do not need this file system. options PROCFS # Process filesystem (requires PSEUDOFS) The process file system. This is a pretend file system mounted on /proc which allows programs like &man.ps.1; to give you more information on what processes are running. Use of PROCFS is not required under most circumstances, as most debugging and monitoring tools have been adapted to run without PROCFS: installs will not mount this file system by default. options PSEUDOFS # Pseudo-filesystem framework 6.X kernels making use of PROCFS must also include support for PSEUDOFS. options GEOM_GPT # GUID Partition Tables. This option brings the ability to have a large number of partitions on a single disk. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] Compatibility with 4.3BSD. Leave this in; some programs will act strangely if you comment this out. options COMPAT_FREEBSD4 # Compatible with &os;4 This option is required on &os; 5.X &i386; and Alpha systems to support applications compiled on older versions of &os; that use older system call interfaces. It is recommended that this option be used on all &i386; and Alpha systems that may run older applications; platforms that gained support only in 5.X, such as ia64 and &sparc64;, do not require this option. options COMPAT_FREEBSD5 # Compatible with &os;5 This option is required on &os; 6.X and above to support applications compiled on &os; 5.X versions that use &os; 5.X system call interfaces. options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI This causes the kernel to pause for 5 seconds before probing each SCSI device in your system. If you only have IDE hard drives, you can ignore this, otherwise you can try to lower this number, to speed up booting. Of course, if you do this and &os; has trouble recognizing your SCSI devices, you will have to raise it again. options KTRACE # ktrace(1) support This enables kernel process tracing, which is useful in debugging. options SYSVSHM # SYSV-style shared memory This option provides for System V shared memory. The most common use of this is the XSHM extension in X, which many graphics-intensive programs will automatically take advantage of for extra speed. If you use X, you will definitely want to include this. options SYSVMSG # SYSV-style message queues Support for System V messages. This option only adds a few hundred bytes to the kernel. options SYSVSEM # SYSV-style semaphores Support for System V semaphores. Less commonly used but only adds a few hundred bytes to the kernel. The option of the &man.ipcs.1; command will list any processes using each of these System V facilities. options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions Real-time extensions added in the 1993 &posix;. Certain applications in the Ports Collection use these (such as &staroffice;). options KBD_INSTALL_CDEV # install a CDEV entry in /dev This option is required to allow the creation of keyboard device nodes in /dev. options ADAPTIVE_GIANT # Giant mutex is adaptive. Giant is the name of a mutual exclusion mechanism (a sleep mutex) that protects a large set of kernel resources. Today, this is an unacceptable performance bottleneck which is actively being replaced with locks that protect individual resources. The ADAPTIVE_GIANT option causes Giant to be included in the set of mutexes adaptively spun on. That is, when a thread wants to lock the Giant mutex, but it is already locked by a thread on another CPU, the first thread will keep running and wait for the lock to be released. Normally, the thread would instead go back to sleep and wait for its next chance to run. If you are not sure, leave this in. Note that on &os; 8.0-CURRENT and later versions, all mutexes are adaptive by default, unless explicitly set to non-adaptive by compiling with the NO_ADAPTIVE_MUTEXES option. As a result, Giant is adaptive by default now, and the ADAPTIVE_GIANT option has been removed from the kernel configuration. kernel options SMP device apic # I/O APIC The apic device enables the use of the I/O APIC for interrupt delivery. The apic device can be used in both UP and SMP kernels, but is required for SMP kernels. Add options SMP to include support for multiple processors. The apic device exists only on the i386 architecture, this configuration line should not be used on other architectures. device eisa Include this if you have an EISA motherboard. This enables auto-detection and configuration support for all devices on the EISA bus. device pci Include this if you have a PCI motherboard. This enables auto-detection of PCI cards and gatewaying from the PCI to ISA bus. # Floppy drives device fdc This is the floppy drive controller. # ATA and ATAPI devices device ata This driver supports all ATA and ATAPI devices. You only need one device ata line for the kernel to detect all PCI ATA/ATAPI devices on modern machines. device atadisk # ATA disk drives This is needed along with device ata for ATA disk drives. device ataraid # ATA RAID drives This is needed along with device ata for ATA RAID drives. device atapicd # ATAPI CDROM drives This is needed along with device ata for ATAPI CDROM drives. device atapifd # ATAPI floppy drives This is needed along with device ata for ATAPI floppy drives. device atapist # ATAPI tape drives This is needed along with device ata for ATAPI tape drives. options ATA_STATIC_ID # Static device numbering This makes the controller number static; without this, the device numbers are dynamically allocated. # SCSI Controllers device ahb # EISA AHA1742 family device ahc # AHA2940 and onboard AIC7xxx devices options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. device ahd # AHA39320/29320 and onboard AIC79xx devices options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. device amd # AMD 53C974 (Teckram DC-390(T)) device isp # Qlogic family #device ispfw # Firmware for QLogic HBAs- normally a module device mpt # LSI-Logic MPT-Fusion #device ncr # NCR/Symbios Logic device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device trm # Tekram DC395U/UW/F DC315U adapters device adv # Advansys SCSI adapters device adw # Advansys wide SCSI adapters device aha # Adaptec 154x SCSI adapters device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. device bt # Buslogic/Mylex MultiMaster SCSI adapters device ncv # NCR 53C500 device nsp # Workbit Ninja SCSI-3 device stg # TMC 18C30/18C50 SCSI controllers. Comment out any you do not have in your system. If you have an IDE only system, you can remove these altogether. The *_REG_PRETTY_PRINT lines are debugging options for their respective drivers. # SCSI peripherals device scbus # SCSI bus (required for SCSI) device ch # SCSI media changers device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) device ses # SCSI Environmental Services (and SAF-TE) SCSI peripherals. Again, comment out any you do not have, or if you have only IDE hardware, you can remove them completely. The USB &man.umass.4; driver and a few other drivers use the SCSI subsystem even though they are not real SCSI devices. Therefore make sure not to remove SCSI support, if any such drivers are included in the kernel configuration. # RAID controllers interfaced to the SCSI subsystem device amr # AMI MegaRAID device arcmsr # Areca SATA II RAID device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID device ciss # Compaq Smart RAID 5* device dpt # DPT Smartcache III, IV - See NOTES for options device hptmv # Highpoint RocketRAID 182x device rr232x # Highpoint RocketRAID 232x device iir # Intel Integrated RAID device ips # IBM (Adaptec) ServeRAID device mly # Mylex AcceleRAID/eXtremeRAID device twa # 3ware 9000 series PATA/SATA RAID # RAID controllers device aac # Adaptec FSA RAID device aacp # SCSI passthrough for aac (requires CAM) device ida # Compaq Smart RAID device mfi # LSI MegaRAID SAS device mlx # Mylex DAC960 family device pst # Promise Supertrak SX6000 device twe # 3ware ATA RAID Supported RAID controllers. If you do not have any of these, you can comment them out or remove them. # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller The keyboard controller (atkbdc) provides I/O services for the AT keyboard and PS/2 style pointing devices. This controller is required by the keyboard driver (atkbd) and the PS/2 pointing device driver (psm). device atkbd # AT keyboard The atkbd driver, together with atkbdc controller, provides access to the AT 84 keyboard or the AT enhanced keyboard which is connected to the AT keyboard controller. device psm # PS/2 mouse Use this device if your mouse plugs into the PS/2 mouse port. device kbdmux # keyboard multiplexer Basic support for keyboard multiplexing. If you do not plan to use more than one keyboard on the system, you can safely remove that line. device vga # VGA video card driver The video card driver. device splash # Splash screen and screen saver support Splash screen at start up! Screen savers require this too. # syscons is the default console driver, resembling an SCO console device sc sc is the default console driver and resembles a SCO console. Since most full-screen programs access the console through a terminal database library like termcap, it should not matter whether you use this or vt, the VT220 compatible console driver. When you log in, set your TERM variable to scoansi if full-screen programs have trouble running under this console. # Enable this for the pcvt (VT220 compatible) console driver #device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor This is a VT220-compatible console driver, backward compatible to VT100/102. It works well on some laptops which have hardware incompatibilities with sc. Also set your TERM variable to vt100 or vt220 when you log in. This driver might also prove useful when connecting to a large number of different machines over the network, where termcap or terminfo entries for the sc device are often not available — vt100 should be available on virtually any platform. device agp Include this if you have an AGP card in the system. This will enable support for AGP, and AGP GART for boards which have these features. APM # Power management support (see NOTES for more options) #device apm Advanced Power Management support. Useful for laptops, although this is disabled in GENERIC by default. # Add suspend/resume support for the i8254. device pmtimer Timer device driver for power management events, such as APM and ACPI. # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support device cbb # cardbus (yenta) bridge device pccard # PC Card (16-bit) bus device cardbus # CardBus (32-bit) bus PCMCIA support. You want this if you are using a laptop. # Serial (COM) ports device sio # 8250, 16[45]50 based serial ports These are the serial ports referred to as COM ports in the &ms-dos;/&windows; world. If you have an internal modem on COM4 and a serial port at COM2, you will have to change the IRQ of the modem to 2 (for obscure technical reasons, IRQ2 = IRQ 9) in order to access it from &os;. If you have a multiport serial card, check the manual page for &man.sio.4; for more information on the proper values to add to your /boot/device.hints. Some video cards (notably those based on S3 chips) use IO addresses in the form of 0x*2e8, and since many cheap serial cards do not fully decode the 16-bit IO address space, they clash with these cards making the COM4 port practically unavailable. Each serial port is required to have a unique IRQ (unless you are using one of the multiport cards where shared interrupts are supported), so the default IRQs for COM3 and COM4 cannot be used. # Parallel port device ppc This is the ISA-bus parallel port interface. device ppbus # Parallel port bus (required) Provides support for the parallel port bus. device lpt # Printer Support for parallel port printers. All three of the above are required to enable parallel printer support. device plip # TCP/IP over parallel This is the driver for the parallel network interface. device ppi # Parallel port interface device The general-purpose I/O (geek port) + IEEE1284 I/O. #device vpo # Requires scbus and da zip drive This is for an Iomega Zip drive. It requires scbus and da support. Best performance is achieved with ports in EPP 1.9 mode. #device puc Uncomment this device if you have a dumb serial or parallel PCI card that is supported by the &man.puc.4; glue driver. # PCI Ethernet NICs. device de # DEC/Intel DC21x4x (Tulip) device em # Intel PRO/1000 adapter Gigabit Ethernet Card device ixgb # Intel PRO/10GbE Ethernet Card device txp # 3Com 3cR990 (Typhoon) device vx # 3Com 3c590, 3c595 (Vortex) Various PCI network card drivers. Comment out or remove any of these not present in your system. # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support MII bus support is required for some PCI 10/100 Ethernet NICs, namely those which use MII-compliant transceivers or implement transceiver control interfaces that operate like an MII. Adding device miibus to the kernel config pulls in support for the generic miibus API and all of the PHY drivers, including a generic one for PHYs that are not specifically handled by an individual driver. device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet device bfe # Broadcom BCM440x 10/100 Ethernet device bge # Broadcom BCM570xx Gigabit Ethernet device dc # DEC/Intel 21143 and various workalikes device fxp # Intel EtherExpress PRO/100B (82557, 82558) device lge # Level 1 LXT1001 gigabit ethernet device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet device nge # NatSemi DP83820 gigabit ethernet device nve # nVidia nForce MCP on-board Ethernet Networking device pcn # AMD Am79C97x PCI 10/100 (precedence over 'lnc') device re # RealTek 8139C+/8169/8169S/8110S device rl # RealTek 8129/8139 device sf # Adaptec AIC-6915 (Starfire) device sis # Silicon Integrated Systems SiS 900/SiS 7016 device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet device ste # Sundance ST201 (D-Link DFE-550TX) device stge # Sundance/Tamarack TC9021 gigabit Ethernet device ti # Alteon Networks Tigon I/II gigabit Ethernet device tl # Texas Instruments ThunderLAN device tx # SMC EtherPower II (83c170 EPIC) device vge # VIA VT612x gigabit ethernet device vr # VIA Rhine, Rhine II device wb # Winbond W89C840F device xl # 3Com 3c90x (Boomerang, Cyclone) Drivers that use the MII bus controller code. # ISA Ethernet NICs. pccard NICs included. device cs # Crystal Semiconductor CS89x0 NIC # 'device ed' requires 'device miibus' device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards device ex # Intel EtherExpress Pro/10 and Pro/10+ device ep # Etherlink III based cards device fe # Fujitsu MB8696x based cards device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. device lnc # NE2100, NE32-VL Lance Ethernet cards device sn # SMC's 9000 series of Ethernet chips device xe # Xircom pccard Ethernet # ISA devices that use the old ISA shims #device le ISA Ethernet drivers. See /usr/src/sys/i386/conf/NOTES for details of which cards are supported by which driver. # Wireless NIC cards device wlan # 802.11 support Generic 802.11 support. This line is required for wireless networking. device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support device wlan_tkip # 802.11 TKIP support Crypto support for 802.11 devices. These lines are needed if you intend to use encryption and 802.11i security protocols. device an # Aironet 4500/4800 802.11 wireless NICs. device ath # Atheros pci/cardbus NIC's device ath_hal # Atheros HAL (Hardware Access Layer) device ath_rate_sample # SampleRate tx rate control for ath device awi # BayStack 660 and others device ral # Ralink Technology RT2500 wireless NICs. device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. #device wl # Older non 802.11 Wavelan wireless NIC. Support for various wireless cards. # Pseudo devices device loop # Network loopback This is the generic loopback device for TCP/IP. If you telnet or FTP to localhost (a.k.a. 127.0.0.1) it will come back at you through this device. This is mandatory. device random # Entropy device Cryptographically secure random number generator. device ether # Ethernet support ether is only needed if you have an Ethernet card. It includes generic Ethernet protocol code. device sl # Kernel SLIP sl is for SLIP support. This has been almost entirely supplanted by PPP, which is easier to set up, better suited for modem-to-modem connection, and more powerful. device ppp # Kernel PPP This is for kernel PPP support for dial-up connections. There is also a version of PPP implemented as a userland application that uses tun and offers more flexibility and features such as demand dialing. device tun # Packet tunnel. This is used by the userland PPP software. See the PPP section of this book for more information. device pty # Pseudo-ttys (telnet etc) This is a pseudo-terminal or simulated login port. It is used by incoming telnet and rlogin sessions, xterm, and some other applications such as Emacs. device md # Memory disks Memory disk pseudo-devices. device gif # IPv6 and IPv4 tunneling This implements IPv6 over IPv4 tunneling, IPv4 over IPv6 tunneling, IPv4 over IPv4 tunneling, and IPv6 over IPv6 tunneling. The gif device is auto-cloning, and will create device nodes as needed. device faith # IPv6-to-IPv4 relaying (translation) This pseudo-device captures packets that are sent to it and diverts them to the IPv4/IPv6 translation daemon. # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter This is the Berkeley Packet Filter. This pseudo-device allows network interfaces to be placed in promiscuous mode, capturing every packet on a broadcast network (e.g., an Ethernet). These packets can be captured to disk and or examined with the &man.tcpdump.1; program. The &man.bpf.4; device is also used by &man.dhclient.8; to obtain the IP address of the default router (gateway) and so on. If you use DHCP, leave this uncommented. # USB support device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device ehci # EHCI PCI->USB interface (USB 2.0) device usb # USB Bus (required) #device udbp # USB Double Bulk Pipe devices device ugen # Generic device uhid # Human Interface Devices device ukbd # Keyboard device ulpt # Printer device umass # Disks/Mass storage - Requires scbus and da device ums # Mouse device ural # Ralink Technology RT2500USB wireless NICs device urio # Diamond Rio 500 MP3 player device uscanner # Scanners # USB Ethernet, requires mii device aue # ADMtek USB Ethernet device axe # ASIX Electronics USB Ethernet device cdce # Generic USB over Ethernet device cue # CATC USB Ethernet device kue # Kawasaki LSI USB Ethernet device rue # RealTek RTL8150 USB Ethernet Support for various USB devices. # FireWire support device firewire # FireWire bus code device sbp # SCSI over FireWire (Requires scbus and da) device fwe # Ethernet over FireWire (non-standard!) Support for various Firewire devices. For more information and additional devices supported by &os;, see /usr/src/sys/i386/conf/NOTES. Large Memory Configurations (<acronym>PAE</acronym>) Physical Address Extensions (PAE) large memory Large memory configuration machines require access to more than the 4 gigabyte limit on User+Kernel Virtual Address (KVA) space. Due to this limitation, Intel added support for 36-bit physical address space access in the &pentium; Pro and later line of CPUs. The Physical Address Extension (PAE) capability of the &intel; &pentium; Pro and later CPUs allows memory configurations of up to 64 gigabytes. &os; provides support for this capability via the kernel configuration option, available in all current release versions of &os;. Due to the limitations of the Intel memory architecture, no distinction is made for memory above or below 4 gigabytes. Memory allocated above 4 gigabytes is simply added to the pool of available memory. To enable PAE support in the kernel, simply add the following line to your kernel configuration file: options PAE The PAE support in &os; is only available for &intel; IA-32 processors. It should also be noted, that the PAE support in &os; has not received wide testing, and should be considered beta quality compared to other stable features of &os;. PAE support in &os; has a few limitations: A process is not able to access more than 4 gigabytes of VM space. KLD modules cannot be loaded into a PAE enabled kernel, due to the differences in the build framework of a module and the kernel. Device drivers that do not use the &man.bus.dma.9; interface will cause data corruption in a PAE enabled kernel and are not recommended for use. For this reason, a PAE kernel configuration file is provided in &os; which excludes all drivers not known to work in a PAE enabled kernel. Some system tunables determine memory resource usage by the amount of available physical memory. Such tunables can unnecessarily over-allocate due to the large memory nature of a PAE system. One such example is the sysctl, which controls the maximum number of vnodes allowed in the kernel. It is advised to adjust this and other such tunables to a reasonable value. It might be necessary to increase the kernel virtual address (KVA) space or to reduce the amount of specific kernel resource that is heavily used (see above) in order to avoid KVA exhaustion. The kernel option can be used for increasing the KVA space. For performance and stability concerns, it is advised to consult the &man.tuning.7; manual page. The &man.pae.4; manual page contains up-to-date information on &os;'s PAE support. If Something Goes Wrong There are four categories of trouble that can occur when building a custom kernel. They are: config fails: If the &man.config.8; command fails when you give it your kernel description, you have probably made a simple error somewhere. Fortunately, &man.config.8; will print the line number that it had trouble with, so that you can quickly locate the line containing the error. For example, if you see: config: line 17: syntax error Make sure the keyword is typed correctly by comparing it to the GENERIC kernel or another reference. make fails: If the make command fails, it usually signals an error in your kernel description which is not severe enough for &man.config.8; to catch. Again, look over your configuration, and if you still cannot resolve the problem, send mail to the &a.questions; with your kernel configuration, and it should be diagnosed quickly. The kernel does not boot: If your new kernel does not boot, or fails to recognize your devices, do not panic! Fortunately, &os; has an excellent mechanism for recovering from incompatible kernels. Simply choose the kernel you want to boot from at the &os; boot loader. You can access this when the system boot menu appears. Select the Escape to a loader prompt option, number six. At the prompt, type unload kernel and then type boot /boot/kernel.old/kernel, or the filename of any other kernel that will boot properly. When reconfiguring a kernel, it is always a good idea to keep a kernel that is known to work on hand. After booting with a good kernel you can check over your configuration file and try to build it again. One helpful resource is the /var/log/messages file which records, among other things, all of the kernel messages from every successful boot. Also, the &man.dmesg.8; command will print the kernel messages from the current boot. If you are having trouble building a kernel, make sure to keep a GENERIC, or some other kernel that is known to work on hand as a different name that will not get erased on the next build. You cannot rely on kernel.old because when installing a new kernel, kernel.old is overwritten with the last installed kernel which may be non-functional. Also, as soon as possible, move the working kernel to the proper /boot/kernel location or commands such as &man.ps.1; may not work properly. To do this, simply rename the directory containing the good kernel: &prompt.root; mv /boot/kernel /boot/kernel.bad &prompt.root; mv /boot/kernel.good /boot/kernel The kernel works, but &man.ps.1; does not work any more: If you have installed a different version of the kernel from the one that the system utilities have been built with, for example, a -CURRENT kernel on a -RELEASE, many system-status commands like &man.ps.1; and &man.vmstat.8; will not work any more. You should recompile and install a world built with the same version of the source tree as your kernel. This is one reason it is not normally a good idea to use a different version of the kernel from the rest of the operating system. diff --git a/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml b/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml index 5efbf37d9f..cc0117b04b 100644 --- a/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml @@ -1,3359 +1,3359 @@ Jim Mock Restructured and parts updated by Brian N. Handy Originally contributed by Rich Murphey Linux Binary Compatibility Synopsis Linux binary compatibility binary compatibility Linux FreeBSD provides binary compatibility with several other &unix; like operating systems, including Linux. At this point, you may be asking yourself why exactly, does FreeBSD need to be able to run Linux binaries? The answer to that question is quite simple. Many companies and developers develop only for Linux, since it is the latest hot thing in the computing world. That leaves the rest of us FreeBSD users bugging these same companies and developers to put out native FreeBSD versions of their applications. The problem is, that most of these companies do not really realize how many people would use their product if there were FreeBSD versions too, and most continue to only develop for Linux. So what is a FreeBSD user to do? This is where the Linux binary compatibility of FreeBSD comes into play. In a nutshell, the compatibility allows FreeBSD users to run about 90% of all Linux applications without modification. This includes applications such as &staroffice;, the Linux version of &netscape;, &adobe; &acrobat;, &realplayer;, VMware, &oracle;, &wordperfect;, Doom, Quake, and more. It is also reported that in some situations, Linux binaries perform better on FreeBSD than they do under Linux. There are, however, some Linux-specific operating system features that are not supported under FreeBSD. Linux binaries will not work on FreeBSD if they overly use &i386; specific calls, such as enabling virtual 8086 mode. After reading this chapter, you will know: How to enable Linux binary compatibility on your system. How to install additional Linux shared libraries. How to install Linux applications on your FreeBSD system. The implementation details of Linux compatibility in FreeBSD. Before reading this chapter, you should: Know how to install additional third-party software (). Installation KLD (kernel loadable object) Linux binary compatibility is not turned on by default. The easiest way to enable this functionality is to load the linux KLD object (Kernel LoaDable object). You can load this module by typing the following as root: &prompt.root; kldload linux If you would like Linux compatibility to always be enabled, then you should add the following line to /etc/rc.conf: linux_enable="YES" The &man.kldstat.8; command can be used to verify that the KLD is loaded: &prompt.user; kldstat Id Refs Address Size Name 1 2 0xc0100000 16bdb8 kernel 7 1 0xc24db000 d000 linux.ko kernel options COMPAT_LINUX If for some reason you do not want to or cannot load the KLD, then you may statically link Linux binary compatibility into the kernel by adding options COMPAT_LINUX to your kernel configuration file. Then install your new kernel as described in . Installing Linux Runtime Libraries Linux installing Linux libraries This can be done one of two ways, either by using the linux_base port, or by installing them manually. Installing Using the linux_base Port Ports Collection This is by far the easiest method to use when installing the runtime libraries. It is just like installing any other port from the Ports Collection. Simply do the following: &prompt.root; cd /usr/ports/emulators/linux_base-fc4 &prompt.root; make install distclean You should now have working Linux binary compatibility. Some programs may complain about incorrect minor versions of the system libraries. In general, however, this does not seem to be a problem. There may be multiple versions of the emulators/linux_base port available, corresponding to different versions of various Linux distributions. You should install the port most closely resembling the requirements of the Linux applications you would like to install. Installing Libraries Manually If you do not have the ports collection installed, you can install the libraries by hand instead. You will need the Linux shared libraries that the program depends on and the runtime linker. Also, you will need to create a shadow root directory, /compat/linux, for Linux libraries on your FreeBSD system. Any shared libraries opened by Linux programs run under FreeBSD will look in this tree first. So, if a Linux program loads, for example, /lib/libc.so, FreeBSD will first try to open /compat/linux/lib/libc.so, and if that does not exist, it will then try /lib/libc.so. Shared libraries should be installed in the shadow tree /compat/linux/lib rather than the paths that the Linux ld.so reports. Generally, you will need to look for the shared libraries that Linux binaries depend on only the first few times that you install a Linux program on your FreeBSD system. After a while, you will have a sufficient set of Linux shared libraries on your system to be able to run newly imported Linux binaries without any extra work. How to Install Additional Shared Libraries shared libraries What if you install the linux_base port and your application still complains about missing shared libraries? How do you know which shared libraries Linux binaries need, and where to get them? Basically, there are 2 possibilities (when following these instructions you will need to be root on your FreeBSD system). If you have access to a Linux system, see what shared libraries the application needs, and copy them to your FreeBSD system. Look at the following example: Let us assume you used FTP to get the Linux binary of Doom, and put it on a Linux system you have access to. You then can check which shared libraries it needs by running ldd linuxdoom, like so: &prompt.user; ldd linuxdoom libXt.so.3 (DLL Jump 3.1) => /usr/X11/lib/libXt.so.3.1.0 libX11.so.3 (DLL Jump 3.1) => /usr/X11/lib/libX11.so.3.1.0 libc.so.4 (DLL Jump 4.5pl26) => /lib/libc.so.4.6.29 symbolic links You would need to get all the files from the last column, and put them under /compat/linux, with the names in the first column as symbolic links pointing to them. This means you eventually have these files on your FreeBSD system: /compat/linux/usr/X11/lib/libXt.so.3.1.0 /compat/linux/usr/X11/lib/libXt.so.3 -> libXt.so.3.1.0 /compat/linux/usr/X11/lib/libX11.so.3.1.0 /compat/linux/usr/X11/lib/libX11.so.3 -> libX11.so.3.1.0 /compat/linux/lib/libc.so.4.6.29 /compat/linux/lib/libc.so.4 -> libc.so.4.6.29
Note that if you already have a Linux shared library with a matching major revision number to the first column of the ldd output, you will not need to copy the file named in the last column to your system, the one you already have should work. It is advisable to copy the shared library anyway if it is a newer version, though. You can remove the old one, as long as you make the symbolic link point to the new one. So, if you have these libraries on your system: /compat/linux/lib/libc.so.4.6.27 /compat/linux/lib/libc.so.4 -> libc.so.4.6.27 and you find a new binary that claims to require a later version according to the output of ldd: libc.so.4 (DLL Jump 4.5pl26) -> libc.so.4.6.29 If it is only one or two versions out of date in the trailing digit then do not worry about copying /lib/libc.so.4.6.29 too, because the program should work fine with the slightly older version. However, if you like, you can decide to replace the libc.so anyway, and that should leave you with: /compat/linux/lib/libc.so.4.6.29 /compat/linux/lib/libc.so.4 -> libc.so.4.6.29
The symbolic link mechanism is only needed for Linux binaries. The FreeBSD runtime linker takes care of looking for matching major revision numbers itself and you do not need to worry about it.
Installing Linux ELF Binaries Linux ELF binaries ELF binaries sometimes require an extra step of branding. If you attempt to run an unbranded ELF binary, you will get an error message like the following: &prompt.user; ./my-linux-elf-binary ELF binary type not known Abort To help the FreeBSD kernel distinguish between a FreeBSD ELF binary from a Linux binary, use the &man.brandelf.1; utility. &prompt.user; brandelf -t Linux my-linux-elf-binary GNU toolchain The GNU toolchain now places the appropriate branding information into ELF binaries automatically, so this step should become increasingly unnecessary in the future. Configuring the Hostname Resolver If DNS does not work or you get this message: resolv+: "bind" is an invalid keyword resolv+: "hosts" is an invalid keyword You will need to configure a /compat/linux/etc/host.conf file containing: order hosts, bind multi on The order here specifies that /etc/hosts is searched first and DNS is searched second. When /compat/linux/etc/host.conf is not installed, Linux applications find FreeBSD's /etc/host.conf and complain about the incompatible FreeBSD syntax. You should remove bind if you have not configured a name server using the /etc/resolv.conf file. Boris Hollas Updated for Mathematica 5.X by Installing &mathematica; applications Mathematica This document describes the process of installing the Linux version of &mathematica; 5.X onto a FreeBSD system. The Linux version of &mathematica; or &mathematica; for Students can be ordered directly from Wolfram at . Running the &mathematica; Installer First, you have to tell &os; that &mathematica;'s Linux binaries use the Linux ABI. The easiest way to do so is to set the default ELF brand to Linux for all unbranded binaries with the command: &prompt.root; sysctl kern.fallback_elf_brand=3 This will make &os; assume that unbranded ELF binaries use the Linux ABI and so you should be able to run the installer straight from the CDROM. Now, copy the file MathInstaller to your hard drive: &prompt.root; mount /cdrom &prompt.root; cp /cdrom/Unix/Installers/Linux/MathInstaller /localdir/ and in this file, replace /bin/sh in the first line by /compat/linux/bin/sh. This makes sure that the installer is executed by the Linux version of &man.sh.1;. Next, replace all occurrences of Linux) by FreeBSD) with a text editor or the script below in the next section. This tells the &mathematica; installer, who calls uname -s to determine the operating system, to treat &os; as a Linux-like operating system. Invoking MathInstaller will now install &mathematica;. Modifying the &mathematica; Executables The shell scripts that &mathematica; created during installation have to be modified before you can use them. If - you chose /usr/local/bin + you chose /usr/local/bin as the directory to place the &mathematica; executables in, you will find symlinks in this directory to files called math, mathematica, Mathematica, and MathKernel. In each of these, replace Linux) by FreeBSD) with a text editor or the following shell script: #!/bin/sh cd /usr/local/bin for i in math mathematica Mathematica MathKernel do sed 's/Linux)/FreeBSD)/g' $i > $i.tmp sed 's/\/bin\/sh/\/compat\/linux\/bin\/sh/g' $i.tmp > $i rm $i.tmp chmod a+x $i done Obtaining Your &mathematica; Password Ethernet MAC address When you start &mathematica; for the first time, you will be asked for a password. If you have not yet obtained a password from Wolfram, run the program mathinfo in the installation directory to obtain your machine ID. This machine ID is based solely on the MAC address of your first Ethernet card, so you cannot run your copy of &mathematica; on different machines. When you register with Wolfram, either by email, phone or fax, you will give them the machine ID and they will respond with a corresponding password consisting of groups of numbers. Running the &mathematica; Frontend over a Network &mathematica; uses some special fonts to display characters not present in any of the standard font sets (integrals, sums, Greek letters, etc.). The X protocol requires these fonts to be install locally. This means you will have to copy these fonts from the CDROM or from a host with &mathematica; installed to your local machine. These fonts are normally stored in /cdrom/Unix/Files/SystemFiles/Fonts on the CDROM, or /usr/local/mathematica/SystemFiles/Fonts on your hard drive. The actual fonts are in the subdirectories Type1 and X. There are several ways to use them, as described below. The first way is to copy them into one of the existing font directories in /usr/X11R6/lib/X11/fonts. This will require editing the fonts.dir file, adding the font names to it, and changing the number of fonts on the first line. Alternatively, you should also just be able to run &man.mkfontdir.1; in the directory you have copied them to. The second way to do this is to copy the directories to /usr/X11R6/lib/X11/fonts: &prompt.root; cd /usr/X11R6/lib/X11/fonts &prompt.root; mkdir X &prompt.root; mkdir MathType1 &prompt.root; cd /cdrom/Unix/Files/SystemFiles/Fonts &prompt.root; cp X/* /usr/X11R6/lib/X11/fonts/X &prompt.root; cp Type1/* /usr/X11R6/lib/X11/fonts/MathType1 &prompt.root; cd /usr/X11R6/lib/X11/fonts/X &prompt.root; mkfontdir &prompt.root; cd ../MathType1 &prompt.root; mkfontdir Now add the new font directories to your font path: &prompt.root; xset fp+ /usr/X11R6/lib/X11/fonts/X &prompt.root; xset fp+ /usr/X11R6/lib/X11/fonts/MathType1 &prompt.root; xset fp rehash If you are using the &xorg; server, you can have these font directories loaded automatically by adding them to your xorg.conf file. For &xfree86; servers, the configuration file is XF86Config. fonts If you do not already have a directory called /usr/X11R6/lib/X11/fonts/Type1, you can change the name of the MathType1 directory in the example above to Type1. Aaron Kaplan Contributed by Robert Getschmann Thanks to Installing &maple; applications Maple &maple; is a commercial mathematics program similar to &mathematica;. You must purchase this software from and then register there for a license file. To install this software on FreeBSD, please follow these simple steps. Execute the INSTALL shell script from the product distribution. Choose the RedHat option when prompted by the installation program. A typical installation directory might be /usr/local/maple. If you have not done so, order a license for &maple; from Maple Waterloo Software () and copy it to /usr/local/maple/license/license.dat. Install the FLEXlm license manager by running the INSTALL_LIC install shell script that comes with &maple;. Specify the primary hostname for your machine for the license server. Patch the /usr/local/maple/bin/maple.system.type file with the following: ----- snip ------------------ *** maple.system.type.orig Sun Jul 8 16:35:33 2001 --- maple.system.type Sun Jul 8 16:35:51 2001 *************** *** 72,77 **** --- 72,78 ---- # the IBM RS/6000 AIX case MAPLE_BIN="bin.IBM_RISC_UNIX" ;; + "FreeBSD"|\ "Linux") # the Linux/x86 case # We have two Linux implementations, one for Red Hat and ----- snip end of patch ----- Please note that after the "FreeBSD"|\ no other whitespace should be present. This patch instructs &maple; to recognize FreeBSD as a type of Linux system. The bin/maple shell script calls the bin/maple.system.type shell script which in turn calls uname -a to find out the operating system name. Depending on the OS name it will find out which binaries to use. Start the license server. The following script, installed as /usr/local/etc/rc.d/lmgrd.sh is a convenient way to start up lmgrd: ----- snip ------------ #! /bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin PATH=${PATH}:/usr/local/maple/bin:/usr/local/maple/FLEXlm/UNIX/LINUX export PATH LICENSE_FILE=/usr/local/maple/license/license.dat LOG=/var/log/lmgrd.log case "$1" in start) lmgrd -c ${LICENSE_FILE} 2>> ${LOG} 1>&2 echo -n " lmgrd" ;; stop) lmgrd -c ${LICENSE_FILE} -x lmdown 2>> ${LOG} 1>&2 ;; *) echo "Usage: `basename $0` {start|stop}" 1>&2 exit 64 ;; esac exit 0 ----- snip ------------ Test-start &maple;: &prompt.user; cd /usr/local/maple/bin &prompt.user; ./xmaple You should be up and running. Make sure to write Maplesoft to let them know you would like a native FreeBSD version! Common Pitfalls The FLEXlm license manager can be a difficult tool to work with. Additional documentation on the subject can be found at . lmgrd is known to be very picky about the license file and to core dump if there are any problems. A correct license file should look like this: # ======================================================= # License File for UNIX Installations ("Pointer File") # ======================================================= SERVER chillig ANY #USE_SERVER VENDOR maplelmg FEATURE Maple maplelmg 2000.0831 permanent 1 XXXXXXXXXXXX \ PLATFORMS=i86_r ISSUER="Waterloo Maple Inc." \ ISSUED=11-may-2000 NOTICE=" Technische Universitat Wien" \ SN=XXXXXXXXX Serial number and key 'X''ed out. chillig is a hostname. Editing the license file works as long as you do not touch the FEATURE line (which is protected by the license key). Dan Pelleg Contributed by Installing &matlab; applications MATLAB This document describes the process of installing the Linux version of &matlab; version 6.5 onto a &os; system. It works quite well, with the exception of the &java.virtual.machine; (see ). The Linux version of &matlab; can be ordered directly from The MathWorks at . Make sure you also get the license file or instructions how to create it. While you are there, let them know you would like a native &os; version of their software. Installing &matlab; To install &matlab;, do the following: Insert the installation CD and mount it. Become root, as recommended by the installation script. To start the installation script type: &prompt.root; /compat/linux/bin/sh /cdrom/install The installer is graphical. If you get errors about not being able to open a display, type setenv HOME ~USER, where USER is the user you did a &man.su.1; as. When asked for the &matlab; root directory, type: /compat/linux/usr/local/matlab. For easier typing on the rest of the installation process, type this at your shell prompt: set MATLAB=/compat/linux/usr/local/matlab Edit the license file as instructed when obtaining the &matlab; license. You can prepare this file in advance using your favorite editor, and copy it to $MATLAB/license.dat before the installer asks you to edit it. Complete the installation process. At this point your &matlab; installation is complete. The following steps apply glue to connect it to your &os; system. License Manager Startup Create symlinks for the license manager scripts: &prompt.root; ln -s $MATLAB/etc/lmboot /usr/local/etc/lmboot_TMW &prompt.root; ln -s $MATLAB/etc/lmdown /usr/local/etc/lmdown_TMW Create a startup file at /usr/local/etc/rc.d/flexlm.sh. The example below is a modified version of the distributed $MATLAB/etc/rc.lm.glnx86. The changes are file locations, and startup of the license manager under Linux emulation. #!/bin/sh case "$1" in start) if [ -f /usr/local/etc/lmboot_TMW ]; then /compat/linux/bin/sh /usr/local/etc/lmboot_TMW -u username && echo 'MATLAB_lmgrd' fi ;; stop) if [ -f /usr/local/etc/lmdown_TMW ]; then /compat/linux/bin/sh /usr/local/etc/lmdown_TMW > /dev/null 2>&1 fi ;; *) echo "Usage: $0 {start|stop}" exit 1 ;; esac exit 0 The file must be made executable: &prompt.root; chmod +x /usr/local/etc/rc.d/flexlm.sh You must also replace username above with the name of a valid user on your system (and not root). Start the license manager with the command: &prompt.root; /usr/local/etc/rc.d/flexlm.sh start Linking the &java; Runtime Environment Change the &java; Runtime Environment (JRE) link to one working under &os;: &prompt.root; cd $MATLAB/sys/java/jre/glnx86/ &prompt.root; unlink jre; ln -s ./jre1.1.8 ./jre Creating a &matlab; Startup Script Place the following startup script in /usr/local/bin/matlab: #!/bin/sh /compat/linux/bin/sh /compat/linux/usr/local/matlab/bin/matlab "$@" Then type the command chmod +x /usr/local/bin/matlab. Depending on your version of emulators/linux_base, you may run into errors when running this script. To avoid that, edit the file /compat/linux/usr/local/matlab/bin/matlab, and change the line that says: if [ `expr "$lscmd" : '.*->.*'` -ne 0 ]; then (in version 13.0.1 it is on line 410) to this line: if test -L $newbase; then Creating a &matlab; Shutdown Script The following is needed to solve a problem with &matlab; not exiting correctly. Create a file $MATLAB/toolbox/local/finish.m, and in it put the single line: ! $MATLAB/bin/finish.sh The $MATLAB is literal. In the same directory, you will find the files finishsav.m and finishdlg.m, which let you save your workspace before quitting. If you use either of them, insert the line above immediately after the save command. Create a file $MATLAB/bin/finish.sh, which will contain the following: #!/usr/compat/linux/bin/sh (sleep 5; killall -1 matlab_helper) & exit 0 Make the file executable: &prompt.root; chmod +x $MATLAB/bin/finish.sh Using &matlab; At this point you are ready to type matlab and start using it. Marcel Moolenaar Contributed by Installing &oracle; applications Oracle Preface This document describes the process of installing &oracle; 8.0.5 and &oracle; 8.0.5.1 Enterprise Edition for Linux onto a FreeBSD machine. Installing the Linux Environment Make sure you have both emulators/linux_base and devel/linux_devtools from the Ports Collection installed. If you run into difficulties with these ports, you may have to use the packages or older versions available in the Ports Collection. If you want to run the intelligent agent, you will also need to install the Red Hat Tcl package: tcl-8.0.3-20.i386.rpm. The general command for installing packages with the official RPM port (archivers/rpm) is: &prompt.root; rpm -i --ignoreos --root /compat/linux --dbpath /var/lib/rpm package Installation of the package should not generate any errors. Creating the &oracle; Environment Before you can install &oracle;, you need to set up a proper environment. This document only describes what to do specially to run &oracle; for Linux on FreeBSD, not what has been described in the &oracle; installation guide. Kernel Tuning kernel tuning As described in the &oracle; installation guide, you need to set the maximum size of shared memory. Do not use SHMMAX under FreeBSD. SHMMAX is merely calculated out of SHMMAXPGS and PGSIZE. Therefore define SHMMAXPGS. All other options can be used as described in the guide. For example: options SHMMAXPGS=10000 options SHMMNI=100 options SHMSEG=10 options SEMMNS=200 options SEMMNI=70 options SEMMSL=61 Set these options to suit your intended use of &oracle;. Also, make sure you have the following options in your kernel configuration file: options SYSVSHM #SysV shared memory options SYSVSEM #SysV semaphores options SYSVMSG #SysV interprocess communication &oracle; Account Create an oracle account just as you would create any other account. The oracle account is special only that you need to give it a Linux shell. Add /compat/linux/bin/bash to /etc/shells and set the shell for the oracle account to /compat/linux/bin/bash. Environment Besides the normal &oracle; variables, such as ORACLE_HOME and ORACLE_SID you must set the following environment variables: Variable Value LD_LIBRARY_PATH $ORACLE_HOME/lib CLASSPATH $ORACLE_HOME/jdbc/lib/classes111.zip PATH /compat/linux/bin /compat/linux/sbin /compat/linux/usr/bin /compat/linux/usr/sbin /bin /sbin /usr/bin /usr/sbin /usr/local/bin $ORACLE_HOME/bin It is advised to set all the environment variables in .profile. A complete example is: ORACLE_BASE=/oracle; export ORACLE_BASE ORACLE_HOME=/oracle; export ORACLE_HOME LD_LIBRARY_PATH=$ORACLE_HOME/lib export LD_LIBRARY_PATH ORACLE_SID=ORCL; export ORACLE_SID ORACLE_TERM=386x; export ORACLE_TERM CLASSPATH=$ORACLE_HOME/jdbc/lib/classes111.zip export CLASSPATH PATH=/compat/linux/bin:/compat/linux/sbin:/compat/linux/usr/bin PATH=$PATH:/compat/linux/usr/sbin:/bin:/sbin:/usr/bin:/usr/sbin PATH=$PATH:/usr/local/bin:$ORACLE_HOME/bin export PATH Installing &oracle; Due to a slight inconsistency in the Linux emulator, you need to create a directory named .oracle in /var/tmp before you start the installer. Let it be owned by the oracle user. You should be able to install &oracle; without any problems. If you have problems, check your &oracle; distribution and/or configuration first! After you have installed &oracle;, apply the patches described in the next two subsections. A frequent problem is that the TCP protocol adapter is not installed right. As a consequence, you cannot start any TCP listeners. The following actions help solve this problem: &prompt.root; cd $ORACLE_HOME/network/lib &prompt.root; make -f ins_network.mk ntcontab.o &prompt.root; cd $ORACLE_HOME/lib &prompt.root; ar r libnetwork.a ntcontab.o &prompt.root; cd $ORACLE_HOME/network/lib &prompt.root; make -f ins_network.mk install Do not forget to run root.sh again! Patching root.sh When installing &oracle;, some actions, which need to be performed as root, are recorded in a shell script called root.sh. This script is written in the orainst directory. Apply the following patch to root.sh, to have it use to proper location of chown or alternatively run the script under a Linux native shell. *** orainst/root.sh.orig Tue Oct 6 21:57:33 1998 --- orainst/root.sh Mon Dec 28 15:58:53 1998 *************** *** 31,37 **** # This is the default value for CHOWN # It will redefined later in this script for those ports # which have it conditionally defined in ss_install.h ! CHOWN=/bin/chown # # Define variables to be used in this script --- 31,37 ---- # This is the default value for CHOWN # It will redefined later in this script for those ports # which have it conditionally defined in ss_install.h ! CHOWN=/usr/sbin/chown # # Define variables to be used in this script When you do not install &oracle; from CD, you can patch the source for root.sh. It is called rthd.sh and is located in the orainst directory in the source tree. Patching genclntsh The script genclntsh is used to create a single shared client library. It is used when building the demos. Apply the following patch to comment out the definition of PATH: *** bin/genclntsh.orig Wed Sep 30 07:37:19 1998 --- bin/genclntsh Tue Dec 22 15:36:49 1998 *************** *** 32,38 **** # # Explicit path to ensure that we're using the correct commands #PATH=/usr/bin:/usr/ccs/bin export PATH ! PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin export PATH # # each product MUST provide a $PRODUCT/admin/shrept.lst --- 32,38 ---- # # Explicit path to ensure that we're using the correct commands #PATH=/usr/bin:/usr/ccs/bin export PATH ! #PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin export PATH # # each product MUST provide a $PRODUCT/admin/shrept.lst Running &oracle; When you have followed the instructions, you should be able to run &oracle; as if it was run on Linux itself. Holger Kipp Contributed by Valentino Vaschetto Original version converted to SGML by Installing &sap.r3; applications SAP R/3 Installations of &sap; Systems using FreeBSD will not be supported by the &sap; support team — they only offer support for certified platforms. Preface This document describes a possible way of installing a &sap.r3; System with &oracle; Database for Linux onto a FreeBSD machine, including the installation of FreeBSD and &oracle;. Two different configurations will be described: &sap.r3; 4.6B (IDES) with &oracle; 8.0.5 on FreeBSD 4.3-STABLE &sap.r3; 4.6C with &oracle; 8.1.7 on FreeBSD 4.5-STABLE Even though this document tries to describe all important steps in a greater detail, it is not intended as a replacement for the &oracle; and &sap.r3; installation guides. Please see the documentation that comes with the &sap.r3; Linux edition for &sap; and &oracle; specific questions, as well as resources from &oracle; and &sap; OSS. Software The following CD-ROMs have been used for &sap; installations: &sap.r3; 4.6B, &oracle; 8.0.5 Name Number Description KERNEL 51009113 SAP Kernel Oracle / Installation / AIX, Linux, Solaris RDBMS 51007558 Oracle / RDBMS 8.0.5.X / Linux EXPORT1 51010208 IDES / DB-Export / Disc 1 of 6 EXPORT2 51010209 IDES / DB-Export / Disc 2 of 6 EXPORT3 51010210 IDES / DB-Export / Disc 3 of 6 EXPORT4 51010211 IDES / DB-Export / Disc 4 of 6 EXPORT5 51010212 IDES / DB-Export / Disc 5 of 6 EXPORT6 51010213 IDES / DB-Export / Disc 6 of 6 Additionally, we used the &oracle; 8 Server (Pre-production version 8.0.5 for Linux, Kernel Version 2.0.33) CD which is not really necessary, and FreeBSD 4.3-STABLE (it was only a few days past 4.3 RELEASE). &sap.r3; 4.6C SR2, &oracle; 8.1.7 Name Number Description KERNEL 51014004 SAP Kernel Oracle / SAP Kernel Version 4.6D / DEC, Linux RDBMS 51012930 Oracle 8.1.7/ RDBMS / Linux EXPORT1 51013953 Release 4.6C SR2 / Export / Disc 1 of 4 EXPORT1 51013953 Release 4.6C SR2 / Export / Disc 2 of 4 EXPORT1 51013953 Release 4.6C SR2 / Export / Disc 3 of 4 EXPORT1 51013953 Release 4.6C SR2 / Export / Disc 4 of 4 LANG1 51013954 Release 4.6C SR2 / Language / DE, EN, FR / Disc 1 of 3 Depending on the languages you would like to install, additional language CDs might be necessary. Here we are just using DE and EN, so the first language CD is the only one needed. As a little note, the numbers for all four EXPORT CDs are identical. All three language CDs also have the same number (this is different from the 4.6B IDES release CD numbering). At the time of writing this installation is running on FreeBSD 4.5-STABLE (20.03.2002). &sap; Notes The following notes should be read before installing &sap.r3; and proved to be useful during installation: &sap.r3; 4.6B, &oracle; 8.0.5 Number Title 0171356 SAP Software on Linux: Essential Comments 0201147 INST: 4.6C R/3 Inst. on UNIX - Oracle 0373203 Update / Migration Oracle 8.0.5 --> 8.0.6/8.1.6 LINUX 0072984 Release of Digital UNIX 4.0B for Oracle 0130581 R3SETUP step DIPGNTAB terminates 0144978 Your system has not been installed correctly 0162266 Questions and tips for R3SETUP on Windows NT / W2K &sap.r3; 4.6C, &oracle; 8.1.7 Number Title 0015023 Initializing table TCPDB (RSXP0004) (EBCDIC) 0045619 R/3 with several languages or typefaces 0171356 SAP Software on Linux: Essential Comments 0195603 RedHat 6.1 Enterprise version: Known problems 0212876 The new archiving tool SAPCAR 0300900 Linux: Released DELL Hardware 0377187 RedHat 6.2: important remarks 0387074 INST: R/3 4.6C SR2 Installation on UNIX 0387077 INST: R/3 4.6C SR2 Inst. on UNIX - Oracle 0387078 SAP Software on UNIX: OS Dependencies 4.6C SR2 Hardware Requirements The following equipment is sufficient for the installation of a &sap.r3; System. For production use, a more exact sizing is of course needed: Component 4.6B 4.6C Processor 2 x 800MHz &pentium; III 2 x 800MHz &pentium; III Memory 1GB ECC 2GB ECC Hard Disk Space 50-60GB (IDES) 50-60GB (IDES) For use in production, &xeon; Processors with large cache, high-speed disk access (SCSI, RAID hardware controller), USV and ECC-RAM is recommended. The large amount of hard disk space is due to the preconfigured IDES System, which creates 27 GB of database files during installation. This space is also sufficient for initial production systems and application data. &sap.r3; 4.6B, &oracle; 8.0.5 The following off-the-shelf hardware was used: a dual processor board with 2 800 MHz &pentium; III processors, &adaptec; 29160 Ultra160 SCSI adapter (for accessing a 40/80 GB DLT tape drive and CDROM), &mylex; &acceleraid; (2 channels, firmware 6.00-1-00 with 32 MB RAM). To the &mylex; RAID controller are attached two 17 GB hard disks (mirrored) and four 36 GB hard disks (RAID level 5). &sap.r3; 4.6C, &oracle; 8.1.7 For this installation a &dell; &poweredge; 2500 was used: a dual processor board with two 1000 MHz &pentium; III processors (256 kB Cache), 2 GB PC133 ECC SDRAM, PERC/3 DC PCI RAID Controller with 128 MB, and an EIDE DVD-ROM drive. To the RAID controller are attached two 18 GB hard disks (mirrored) and four 36 GB hard disks (RAID level 5). Installation of FreeBSD First you have to install FreeBSD. There are several ways to do this, for more information read the . Disk Layout To keep it simple, the same disk layout both for the &sap.r3; 46B and &sap.r3; 46C SR2 installation was used. Only the device names changed, as the installations were on different hardware (/dev/da and /dev/amr respectively, so if using an AMI &megaraid;, one will see /dev/amr0s1a instead of /dev/da0s1a): File system Size (1k-blocks) Size (GB) Mounted on /dev/da0s1a 1.016.303 1 / /dev/da0s1b 6 swap /dev/da0s1e 2.032.623 2 /var /dev/da0s1f 8.205.339 8 /usr /dev/da1s1e 45.734.361 45 /compat/linux/oracle /dev/da1s1f 2.032.623 2 /compat/linux/sapmnt /dev/da1s1g 2.032.623 2 /compat/linux/usr/sap Configure and initialize the two logical drives with the &mylex; or PERC/3 RAID software beforehand. The software can be started during the BIOS boot phase. Please note that this disk layout differs slightly from the &sap; recommendations, as &sap; suggests mounting the &oracle; subdirectories (and some others) separately — we decided to just create them as real subdirectories for simplicity. <command>make world</command> and a New Kernel Download the latest -STABLE sources. Rebuild world and your custom kernel after configuring your kernel configuration file. Here you should also include the kernel parameters which are required for both &sap.r3; and &oracle;. Installing the Linux Environment Installing the Linux Base System First the linux_base port needs to be installed (as root): &prompt.root; cd /usr/ports/emulators/linux_base &prompt.root; make install distclean Installing Linux Development Environment The Linux development environment is needed, if you want to install &oracle; on FreeBSD according to the : &prompt.root; cd /usr/ports/devel/linux_devtools &prompt.root; make install distclean The Linux development environment has only been installed for the &sap.r3; 46B IDES installation. It is not needed, if the &oracle; DB is not relinked on the FreeBSD system. This is the case if you are using the &oracle; tarball from a Linux system. Installing the Necessary RPMs RPMs To start the R3SETUP program, PAM support is needed. During the first &sap; Installation on FreeBSD 4.3-STABLE we tried to install PAM with all the required packages and finally forced the installation of the PAM package, which worked. For &sap.r3; 4.6C SR2 we directly forced the installation of the PAM RPM, which also works, so it seems the dependent packages are not needed: &prompt.root; rpm -i --ignoreos --nodeps --root /compat/linux --dbpath /var/lib/rpm \ pam-0.68-7.i386.rpm For &oracle; 8.0.5 to run the intelligent agent, we also had to install the RedHat Tcl package tcl-8.0.5-30.i386.rpm (otherwise the relinking during &oracle; installation will not work). There are some other issues regarding relinking of &oracle;, but that is a &oracle; Linux issue, not FreeBSD specific. Some Additional Hints It might also be a good idea to add linprocfs to /etc/fstab, for more information, see the &man.linprocfs.5; manual page. Another parameter to set is kern.fallback_elf_brand=3 which is done in the file /etc/sysctl.conf. Creating the &sap.r3; Environment Creating the Necessary File Systems and Mountpoints For a simple installation, it is sufficient to create the following file systems: mount point size in GB /compat/linux/oracle 45 GB /compat/linux/sapmnt 2 GB /compat/linux/usr/sap 2 GB It is also necessary to created some links. Otherwise the &sap; Installer will complain, as it is checking the created links: &prompt.root; ln -s /compat/linux/oracle /oracle &prompt.root; ln -s /compat/linux/sapmnt /sapmnt &prompt.root; ln -s /compat/linux/usr/sap /usr/sap Possible error message during installation (here with System PRD and the &sap.r3; 4.6C SR2 installation): INFO 2002-03-19 16:45:36 R3LINKS_IND_IND SyLinkCreate:200 Checking existence of symbolic link /usr/sap/PRD/SYS/exe/dbg to /sapmnt/PRD/exe. Creating if it does not exist... WARNING 2002-03-19 16:45:36 R3LINKS_IND_IND SyLinkCreate:400 Link /usr/sap/PRD/SYS/exe/dbg exists but it points to file /compat/linux/sapmnt/PRD/exe instead of /sapmnt/PRD/exe. The program cannot go on as long as this link exists at this location. Move the link to another location. ERROR 2002-03-19 16:45:36 R3LINKS_IND_IND Ins_SetupLinks:0 can not setup link '/usr/sap/PRD/SYS/exe/dbg' with content '/sapmnt/PRD/exe' Creating Users and Directories &sap.r3; needs two users and three groups. The user names depend on the &sap; system ID (SID) which consists of three letters. Some of these SIDs are reserved by &sap; (for example SAP and NIX. For a complete list please see the &sap; documentation). For the IDES installation we used IDS, for the 4.6C SR2 installation PRD, as that system is intended for production use. We have therefore the following groups (group IDs might differ, these are just the values we used with our installation): group ID group name description 100 dba Data Base Administrator 101 sapsys &sap; System 102 oper Data Base Operator For a default &oracle; installation, only group dba is used. As oper group, one also uses group dba (see &oracle; and &sap; documentation for further information). We also need the following users: user ID user name generic name group additional groups description 1000 idsadm/prdadm sidadm sapsys oper &sap; Administrator 1002 oraids/oraprd orasid dba oper &oracle; Administrator Adding the users with &man.adduser.8; requires the following (please note shell and home directory) entries for &sap; Administrator: Name: sidadm Password: ****** Fullname: SAP Administrator SID Uid: 1000 Gid: 101 (sapsys) Class: Groups: sapsys dba HOME: /home/sidadm Shell: bash (/compat/linux/bin/bash) and for &oracle; Administrator: Name: orasid Password: ****** Fullname: Oracle Administrator SID Uid: 1002 Gid: 100 (dba) Class: Groups: dba HOME: /oracle/sid Shell: bash (/compat/linux/bin/bash) This should also include group oper in case you are using both groups dba and oper. Creating Directories These directories are usually created as separate file systems. This depends entirely on your requirements. We choose to create them as simple directories, as they are all located on the same RAID 5 anyway: First we will set owners and rights of some directories (as user root): &prompt.root; chmod 775 /oracle &prompt.root; chmod 777 /sapmnt &prompt.root; chown root:dba /oracle &prompt.root; chown sidadm:sapsys /compat/linux/usr/sap &prompt.root; chmod 775 /compat/linux/usr/sap Second we will create directories as user orasid. These will all be subdirectories of /oracle/SID: &prompt.root; su - orasid &prompt.root; cd /oracle/SID &prompt.root; mkdir mirrlogA mirrlogB origlogA origlogB &prompt.root; mkdir sapdata1 sapdata2 sapdata3 sapdata4 sapdata5 sapdata6 &prompt.root; mkdir saparch sapreorg &prompt.root; exit For the &oracle; 8.1.7 installation some additional directories are needed: &prompt.root; su - orasid &prompt.root; cd /oracle &prompt.root; mkdir 805_32 &prompt.root; mkdir client stage &prompt.root; mkdir client/80x_32 &prompt.root; mkdir stage/817_32 &prompt.root; cd /oracle/SID &prompt.root; mkdir 817_32 The directory client/80x_32 is used with exactly this name. Do not replace the x with some number or anything. In the third step we create directories as user sidadm: &prompt.root; su - sidadm &prompt.root; cd /usr/sap &prompt.root; mkdir SID &prompt.root; mkdir trans &prompt.root; exit Entries in <filename>/etc/services</filename> &sap.r3; requires some entries in file /etc/services, which will not be set correctly during installation under FreeBSD. Please add the following entries (you need at least those entries corresponding to the instance number — in this case, 00. It will do no harm adding all entries from 00 to 99 for dp, gw, sp and ms). If you are going to use a SAProuter or need to access &sap; OSS, you also need 99, as port 3299 is usually used for the SAProuter process on the target system: sapdp00 3200/tcp # SAP Dispatcher. 3200 + Instance-Number sapgw00 3300/tcp # SAP Gateway. 3300 + Instance-Number sapsp00 3400/tcp # 3400 + Instance-Number sapms00 3500/tcp # 3500 + Instance-Number sapmsSID 3600/tcp # SAP Message Server. 3600 + Instance-Number sapgw00s 4800/tcp # SAP Secure Gateway 4800 + Instance-Number Necessary Locales locale &sap; requires at least two locales that are not part of the default RedHat installation. &sap; offers the required RPMs as download from their FTP server (which is only accessible if you are a customer with OSS access). See note 0171356 for a list of RPMs you need. It is also possible to just create appropriate links (for example from de_DE and en_US ), but we would not recommend this for a production system (so far it worked with the IDES system without any problems, though). The following locales are needed: de_DE.ISO-8859-1 en_US.ISO-8859-1 Create the links like this: &prompt.root; cd /compat/linux/usr/share/locale &prompt.root; ln -s de_DE de_DE.ISO-8859-1 &prompt.root; ln -s en_US en_US.ISO-8859-1 If they are not present, there will be some problems during the installation. If these are then subsequently ignored (by setting the STATUS of the offending steps to OK in file CENTRDB.R3S), it will be impossible to log onto the &sap; system without some additional effort. Kernel Tuning kernel tuning &sap.r3; systems need a lot of resources. We therefore added the following parameters to the kernel configuration file: # Set these for memory pigs (SAP and Oracle): options MAXDSIZ="(1024*1024*1024)" options DFLDSIZ="(1024*1024*1024)" # System V options needed. options SYSVSHM #SYSV-style shared memory options SHMMAXPGS=262144 #max amount of shared mem. pages #options SHMMAXPGS=393216 #use this for the 46C inst.parameters options SHMMNI=256 #max number of shared memory ident if. options SHMSEG=100 #max shared mem.segs per process options SYSVMSG #SYSV-style message queues options MSGSEG=32767 #max num. of mes.segments in system options MSGSSZ=32 #size of msg-seg. MUST be power of 2 options MSGMNB=65535 #max char. per message queue options MSGTQL=2046 #max amount of msgs in system options SYSVSEM #SYSV-style semaphores options SEMMNU=256 #number of semaphore UNDO structures options SEMMNS=1024 #number of semaphores in system options SEMMNI=520 #number of semaphore identifiers options SEMUME=100 #number of UNDO keys The minimum values are specified in the documentation that comes from &sap;. As there is no description for Linux, see the HP-UX section (32-bit) for further information. As the system for the 4.6C SR2 installation has more main memory, the shared segments can be larger both for &sap; and &oracle;, therefore choose a larger number of shared memory pages. With the default installation of FreeBSD on &i386;, leave MAXDSIZ and DFLDSIZ at 1 GB maximum. Otherwise, strange errors like ORA-27102: out of memory and Linux Error: 12: Cannot allocate memory might happen. Installing &sap.r3; Preparing &sap; CDROMs There are many CDROMs to mount and unmount during the installation. Assuming you have enough CDROM drives, you can just mount them all. We decided to copy the CDROMs contents to corresponding directories: /oracle/SID/sapreorg/cd-name where cd-name was one of KERNEL, RDBMS, EXPORT1, EXPORT2, EXPORT3, EXPORT4, EXPORT5 and EXPORT6 for the 4.6B/IDES installation, and KERNEL, RDBMS, DISK1, DISK2, DISK3, DISK4 and LANG for the 4.6C SR2 installation. All the filenames on the mounted CDs should be in capital letters, otherwise use the option for mounting. So use the following commands: &prompt.root; mount_cd9660 -g /dev/cd0a /mnt &prompt.root; cp -R /mnt/* /oracle/SID/sapreorg/cd-name &prompt.root; umount /mnt Running the Installation Script First you have to prepare an install directory: &prompt.root; cd /oracle/SID/sapreorg &prompt.root; mkdir install &prompt.root; cd install Then the installation script is started, which will copy nearly all the relevant files into the install directory: &prompt.root; /oracle/SID/sapreorg/KERNEL/UNIX/INSTTOOL.SH The IDES installation (4.6B) comes with a fully customized &sap.r3; demonstration system, so there are six instead of just three EXPORT CDs. At this point the installation template CENTRDB.R3S is for installing a standard central instance (&r3; and database), not the IDES central instance, so one needs to copy the corresponding CENTRDB.R3S from the EXPORT1 directory, otherwise R3SETUP will only ask for three EXPORT CDs. The newer &sap; 4.6C SR2 release comes with four EXPORT CDs. The parameter file that controls the installation steps is CENTRAL.R3S. Contrary to earlier releases there are no separate installation templates for a central instance with or without database. &sap; is using a separate template for database installation. To restart the installation later it is however sufficient to restart with the original file. During and after installation, &sap; requires hostname to return the computer name only, not the fully qualified domain name. So either set the hostname accordingly, or set an alias with alias hostname='hostname -s' for both orasid and sidadm (and for root at least during installation steps performed as root). It is also possible to adjust the installed .profile and .login files of both users that are installed during &sap; installation. Start <command>R3SETUP</command> 4.6B Make sure LD_LIBRARY_PATH is set correctly: &prompt.root; export LD_LIBRARY_PATH=/oracle/IDS/lib:/sapmnt/IDS/exe:/oracle/805_32/lib Start R3SETUP as root from installation directory: &prompt.root; cd /oracle/IDS/sapreorg/install &prompt.root; ./R3SETUP -f CENTRDB.R3S The script then asks some questions (defaults in brackets, followed by actual input): Question Default Input Enter SAP System ID [C11] IDSEnter Enter SAP Instance Number [00] Enter Enter SAPMOUNT Directory [/sapmnt] Enter Enter name of SAP central host [troubadix.domain.de] Enter Enter name of SAP db host [troubadix] Enter Select character set [1] (WE8DEC) Enter Enter Oracle server version (1) Oracle 8.0.5, (2) Oracle 8.0.6, (3) Oracle 8.1.5, (4) Oracle 8.1.6 1Enter Extract Oracle Client archive [1] (Yes, extract) Enter Enter path to KERNEL CD [/sapcd] /oracle/IDS/sapreorg/KERNEL Enter path to RDBMS CD [/sapcd] /oracle/IDS/sapreorg/RDBMS Enter path to EXPORT1 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT1 Directory to copy EXPORT1 CD [/oracle/IDS/sapreorg/CD4_DIR] Enter Enter path to EXPORT2 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT2 Directory to copy EXPORT2 CD [/oracle/IDS/sapreorg/CD5_DIR] Enter Enter path to EXPORT3 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT3 Directory to copy EXPORT3 CD [/oracle/IDS/sapreorg/CD6_DIR] Enter Enter path to EXPORT4 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT4 Directory to copy EXPORT4 CD [/oracle/IDS/sapreorg/CD7_DIR] Enter Enter path to EXPORT5 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT5 Directory to copy EXPORT5 CD [/oracle/IDS/sapreorg/CD8_DIR] Enter Enter path to EXPORT6 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT6 Directory to copy EXPORT6 CD [/oracle/IDS/sapreorg/CD9_DIR] Enter Enter amount of RAM for SAP + DB 850Enter (in Megabytes) Service Entry Message Server [3600] Enter Enter Group-ID of sapsys [101] Enter Enter Group-ID of oper [102] Enter Enter Group-ID of dba [100] Enter Enter User-ID of sidadm [1000] Enter Enter User-ID of orasid [1002] Enter Number of parallel procs [2] Enter If you had not copied the CDs to the different locations, then the &sap; installer cannot find the CD needed (identified by the LABEL.ASC file on the CD) and would then ask you to insert and mount the CD and confirm or enter the mount path. The CENTRDB.R3S might not be error free. In our case, it requested EXPORT4 CD again but indicated the correct key (6_LOCATION, then 7_LOCATION etc.), so one can just continue with entering the correct values. Apart from some problems mentioned below, everything should go straight through up to the point where the &oracle; database software needs to be installed. Start <command>R3SETUP</command> 4.6C SR2 Make sure LD_LIBRARY_PATH is set correctly. This is a different value from the 4.6B installation with &oracle; 8.0.5: &prompt.root; export LD_LIBRARY_PATH=/sapmnt/PRD/exe:/oracle/PRD/817_32/lib Start R3SETUP as user root from installation directory: &prompt.root; cd /oracle/PRD/sapreorg/install &prompt.root; ./R3SETUP -f CENTRAL.R3S The script then asks some questions (defaults in brackets, followed by actual input): Question Default Input Enter SAP System ID [C11] PRDEnter Enter SAP Instance Number [00] Enter Enter SAPMOUNT Directory [/sapmnt] Enter Enter name of SAP central host [majestix] Enter Enter Database System ID [PRD] PRDEnter Enter name of SAP db host [majestix] Enter Select character set [1] (WE8DEC) Enter Enter Oracle server version (2) Oracle 8.1.7 2Enter Extract Oracle Client archive [1] (Yes, extract) Enter Enter path to KERNEL CD [/sapcd] /oracle/PRD/sapreorg/KERNEL Enter amount of RAM for SAP + DB 2044 1800Enter (in Megabytes) Service Entry Message Server [3600] Enter Enter Group-ID of sapsys [100] Enter Enter Group-ID of oper [101] Enter Enter Group-ID of dba [102] Enter Enter User-ID of oraprd [1002] Enter Enter User-ID of prdadm [1000] Enter LDAP support 3Enter (no support) Installation step completed [1] (continue) Enter Choose installation service [1] (DB inst,file) Enter So far, creation of users gives an error during installation in phases OSUSERDBSID_IND_ORA (for creating user orasid) and OSUSERSIDADM_IND_ORA (creating user sidadm). Apart from some problems mentioned below, everything should go straight through up to the point where the &oracle; database software needs to be installed. Installing &oracle; 8.0.5 Please see the corresponding &sap; Notes and &oracle; Readmes regarding Linux and &oracle; DB for possible problems. Most if not all problems stem from incompatible libraries. For more information on installing &oracle;, refer to the Installing &oracle; chapter. Installing the &oracle; 8.0.5 with <command>orainst</command> If &oracle; 8.0.5 is to be used, some additional libraries are needed for successfully relinking, as &oracle; 8.0.5 was linked with an old glibc (RedHat 6.0), but RedHat 6.1 already uses a new glibc. So you have to install the following additional packages to ensure that linking will work: compat-libs-5.2-2.i386.rpm compat-glibc-5.2-2.0.7.2.i386.rpm compat-egcs-5.2-1.0.3a.1.i386.rpm compat-egcs-c++-5.2-1.0.3a.1.i386.rpm compat-binutils-5.2-2.9.1.0.23.1.i386.rpm See the corresponding &sap; Notes or &oracle; Readmes for further information. If this is no option (at the time of installation we did not have enough time to check this), one could use the original binaries, or use the relinked binaries from an original RedHat system. For compiling the intelligent agent, the RedHat Tcl package must be installed. If you cannot get tcl-8.0.3-20.i386.rpm, a newer one like tcl-8.0.5-30.i386.rpm for RedHat 6.1 should also do. Apart from relinking, the installation is straightforward: &prompt.root; su - oraids &prompt.root; export TERM=xterm &prompt.root; export ORACLE_TERM=xterm &prompt.root; export ORACLE_HOME=/oracle/IDS &prompt.root; cd $ORACLE_HOME/orainst_sap &prompt.root; ./orainst Confirm all screens with Enter until the software is installed, except that one has to deselect the &oracle; On-Line Text Viewer, as this is not currently available for Linux. &oracle; then wants to relink with i386-glibc20-linux-gcc instead of the available gcc, egcs or i386-redhat-linux-gcc . Due to time constrains we decided to use the binaries from an &oracle; 8.0.5 PreProduction release, after the first attempt at getting the version from the RDBMS CD working, failed, and finding and accessing the correct RPMs was a nightmare at that time. Installing the &oracle; 8.0.5 Pre-production Release for Linux (Kernel 2.0.33) This installation is quite easy. Mount the CD, start the installer. It will then ask for the location of the &oracle; home directory, and copy all binaries there. We did not delete the remains of our previous RDBMS installation tries, though. Afterwards, &oracle; Database could be started with no problems. Installing the &oracle; 8.1.7 Linux Tarball Take the tarball oracle81732.tgz you produced from the installation directory on a Linux system and untar it to /oracle/SID/817_32/. Continue with &sap.r3; Installation First check the environment settings of users idsamd (sidadm) and oraids (orasid). They should now both have the files .profile, .login and .cshrc which are all using hostname. In case the system's hostname is the fully qualified name, you need to change hostname to hostname -s within all three files. Database Load Afterwards, R3SETUP can either be restarted or continued (depending on whether exit was chosen or not). R3SETUP then creates the tablespaces and loads the data (for 46B IDES, from EXPORT1 to EXPORT6, for 46C from DISK1 to DISK4) with R3load into the database. When the database load is finished (might take a few hours), some passwords are requested. For test installations, one can use the well known default passwords (use different ones if security is an issue!): Question Input Enter Password for sapr3 sapEnter Confirum Password for sapr3 sapEnter Enter Password for sys change_on_installEnter Confirm Password for sys change_on_installEnter Enter Password for system managerEnter Confirm Password for system managerEnter At this point We had a few problems with dipgntab during the 4.6B installation. Listener Start the &oracle; Listener as user orasid as follows: &prompt.user; umask 0; lsnrctl start Otherwise you might get the error ORA-12546 as the sockets will not have the correct permissions. See &sap; Note 072984. Updating MNLS Tables If you plan to import non-Latin-1 languages into the &sap; system, you have to update the Multi National Language Support tables. This is described in the &sap; OSS Notes 15023 and 45619. Otherwise, you can skip this question during &sap; installation. If you do not need MNLS, it is still necessary to check the table TCPDB and initializing it if this has not been done. See &sap; note 0015023 and 0045619 for further information. Post-installation Steps Request &sap.r3; License Key You have to request your &sap.r3; License Key. This is needed, as the temporary license that was installed during installation is only valid for four weeks. First get the hardware key. Log on as user idsadm and call saplicense: &prompt.root; /sapmnt/IDS/exe/saplicense -get Calling saplicense without parameters gives a list of options. Upon receiving the license key, it can be installed using: &prompt.root; /sapmnt/IDS/exe/saplicense -install You are then required to enter the following values: SAP SYSTEM ID = SID, 3 chars CUSTOMER KEY = hardware key, 11 chars INSTALLATION NO = installation, 10 digits EXPIRATION DATE = yyyymmdd, usually "99991231" LICENSE KEY = license key, 24 chars Creating Users Create a user within client 000 (for some tasks required to be done within client 000, but with a user different from users sap* and ddic). As a user name, We usually choose wartung (or service in English). Profiles required are sap_new and sap_all. For additional safety the passwords of default users within all clients should be changed (this includes users sap* and ddic). Configure Transport System, Profile, Operation Modes, Etc. Within client 000, user different from ddic and sap*, do at least the following: Task Transaction Configure Transport System, e.g. as Stand-Alone Transport Domain Entity STMS Create / Edit Profile for System RZ10 Maintain Operation Modes and Instances RZ04 These and all the other post-installation steps are thoroughly described in &sap; installation guides. Edit <filename>init<replaceable>sid</replaceable>.sap</filename> (<filename>initIDS.sap</filename>) The file /oracle/IDS/dbs/initIDS.sap contains the &sap; backup profile. Here the size of the tape to be used, type of compression and so on need to be defined. To get this running with sapdba / brbackup, we changed the following values: compress = hardware archive_function = copy_delete_save cpio_flags = "-ov --format=newc --block-size=128 --quiet" cpio_in_flags = "-iuv --block-size=128 --quiet" tape_size = 38000M tape_address = /dev/nsa0 tape_address_rew = /dev/sa0 Explanations: compress: The tape we use is a HP DLT1 which does hardware compression. archive_function: This defines the default behavior for saving &oracle; archive logs: new logfiles are saved to tape, already saved logfiles are saved again and are then deleted. This prevents lots of trouble if you need to recover the database, and one of the archive-tapes has gone bad. cpio_flags: Default is to use which sets block size to 5120 Bytes. For DLT Tapes, HP recommends at least 32 K block size, so we used for 64 K. is needed because we have inode numbers greater than 65535. The last option is needed as otherwise brbackup complains as soon as cpio outputs the numbers of blocks saved. cpio_in_flags: Flags needed for loading data back from tape. Format is recognized automatically. tape_size: This usually gives the raw storage capability of the tape. For security reason (we use hardware compression), the value is slightly lower than the actual value. tape_address: The non-rewindable device to be used with cpio. tape_address_rew: The rewindable device to be used with cpio. Configuration Issues after Installation The following &sap; parameters should be tuned after installation (examples for IDES 46B, 1 GB memory): Name Value ztta/roll_extension 250000000 abap/heap_area_dia 300000000 abap/heap_area_nondia 400000000 em/initial_size_MB 256 em/blocksize_kB 1024 ipc/shm_psize_40 70000000 &sap; Note 0013026: Name Value ztta/dynpro_area 2500000 &sap; Note 0157246: Name Value rdisp/ROLL_MAXFS 16000 rdisp/PG_MAXFS 30000 With the above parameters, on a system with 1 gigabyte of memory, one may find memory consumption similar to: Mem: 547M Active, 305M Inact, 109M Wired, 40M Cache, 112M Buf, 3492K Free Problems during Installation Restart <command>R3SETUP</command> after Fixing a Problem R3SETUP stops if it encounters an error. If you have looked at the corresponding logfiles and fixed the error, you have to start R3SETUP again, usually selecting REPEAT as option for the last step R3SETUP complained about. To restart R3SETUP, just start it with the corresponding R3S file: &prompt.root; ./R3SETUP -f CENTRDB.R3S for 4.6B, or with &prompt.root; ./R3SETUP -f CENTRAL.R3S for 4.6C, no matter whether the error occurred with CENTRAL.R3S or DATABASE.R3S. At some stages, R3SETUP assumes that both database and &sap; processes are up and running (as those were steps it already completed). Should errors occur and for example the database could not be started, you have to start both database and &sap; by hand after you fixed the errors and before starting R3SETUP again. Do not forget to also start the &oracle; listener again (as orasid with umask 0; lsnrctl start) if it was also stopped (for example due to a necessary reboot of the system). OSUSERSIDADM_IND_ORA during <command>R3SETUP</command> If R3SETUP complains at this stage, edit the template file R3SETUP used at that time (CENTRDB.R3S (4.6B) or either CENTRAL.R3S or DATABASE.R3S (4.6C)). Locate [OSUSERSIDADM_IND_ORA] or search for the only STATUS=ERROR entry and edit the following values: HOME=/home/sidadm (was empty) STATUS=OK (had status ERROR) Then you can restart R3SETUP again. OSUSERDBSID_IND_ORA during <command>R3SETUP</command> Possibly R3SETUP also complains at this stage. The error here is similar to the one in phase OSUSERSIDADM_IND_ORA. Just edit the template file R3SETUP used at that time (CENTRDB.R3S (4.6B) or either CENTRAL.R3S or DATABASE.R3S (4.6C)). Locate [OSUSERDBSID_IND_ORA] or search for the only STATUS=ERROR entry and edit the following value in that section: STATUS=OK Then restart R3SETUP. <errorname>oraview.vrf FILE NOT FOUND</errorname> during &oracle; Installation You have not deselected &oracle; On-Line Text Viewer before starting the installation. This is marked for installation even though this option is currently not available for Linux. Deselect this product inside the &oracle; installation menu and restart installation. <errorname>TEXTENV_INVALID</errorname> during <command>R3SETUP</command>, RFC or SAPgui Start If this error is encountered, the correct locale is missing. &sap; Note 0171356 lists the necessary RPMs that need be installed (e.g. saplocales-1.0-3, saposcheck-1.0-1 for RedHat 6.1). In case you ignored all the related errors and set the corresponding STATUS from ERROR to OK (in CENTRDB.R3S) every time R3SETUP complained and just restarted R3SETUP, the &sap; system will not be properly configured and you will then not be able to connect to the system with a SAPgui, even though the system can be started. Trying to connect with the old Linux SAPgui gave the following messages: Sat May 5 14:23:14 2001 *** ERROR => no valid userarea given [trgmsgo. 0401] Sat May 5 14:23:22 2001 *** ERROR => ERROR NR 24 occured [trgmsgi. 0410] *** ERROR => Error when generating text environment. [trgmsgi. 0435] *** ERROR => function failed [trgmsgi. 0447] *** ERROR => no socket operation allowed [trxio.c 3363] Speicherzugriffsfehler This behavior is due to &sap.r3; being unable to correctly assign a locale and also not being properly configured itself (missing entries in some database tables). To be able to connect to &sap;, add the following entries to file DEFAULT.PFL (see Note 0043288): abap/set_etct_env_at_new_mode = 0 install/collate/active = 0 rscp/TCP0B = TCP0B Restart the &sap; system. Now you can connect to the system, even though country-specific language settings might not work as expected. After correcting country settings (and providing the correct locales), these entries can be removed from DEFAULT.PFL and the &sap; system can be restarted. <errorcode>ORA-00001</errorcode> This error only happened with &oracle; 8.1.7 on FreeBSD. The reason was that the &oracle; database could not initialize itself properly and crashed, leaving semaphores and shared memory on the system. The next try to start the database then returned ORA-00001. Find them with ipcs -a and remove them with ipcrm. <errorcode>ORA-00445</errorcode> (Background Process PMON Did Not Start) This error happened with &oracle; 8.1.7. This error is reported if the database is started with the usual startsap script (for example startsap_majestix_00) as user prdadm. A possible workaround is to start the database as user oraprd instead with svrmgrl: &prompt.user; svrmgrl SVRMGR> connect internal; SVRMGR> startup; SVRMGR> exit <errorcode>ORA-12546</errorcode> (Start Listener with Correct Permissions) Start the &oracle; listener as user oraids with the following commands: &prompt.root; umask 0; lsnrctl start Otherwise you might get ORA-12546 as the sockets will not have the correct permissions. See &sap; Note 0072984. <errorcode>ORA-27102</errorcode> (Out of Memory) This error happened whilst trying to use values for MAXDSIZ and DFLDSIZ greater than 1 GB (1024x1024x1024). Additionally, we got Linux Error 12: Cannot allocate memory. [DIPGNTAB_IND_IND] during <command>R3SETUP</command> In general, see &sap; Note 0130581 (R3SETUP step DIPGNTAB terminates). During the IDES-specific installation, for some reason the installation process was not using the proper &sap; system name IDS, but the empty string "" instead. This leads to some minor problems with accessing directories, as the paths are generated dynamically using SID (in this case IDS). So instead of accessing: /usr/sap/IDS/SYS/... /usr/sap/IDS/DVMGS00 the following paths were used: /usr/sap//SYS/... /usr/sap/D00 To continue with the installation, we created a link and an additional directory: &prompt.root; pwd /compat/linux/usr/sap &prompt.root; ls -l total 4 drwxr-xr-x 3 idsadm sapsys 512 May 5 11:20 D00 drwxr-x--x 5 idsadm sapsys 512 May 5 11:35 IDS lrwxr-xr-x 1 root sapsys 7 May 5 11:35 SYS -> IDS/SYS drwxrwxr-x 2 idsadm sapsys 512 May 5 13:00 tmp drwxrwxr-x 11 idsadm sapsys 512 May 4 14:20 trans We also found &sap; Notes (0029227 and 0008401) describing this behavior. We did not encounter any of these problems with the &sap; 4.6C installation. [RFCRSWBOINI_IND_IND] during <command>R3SETUP</command> During installation of &sap; 4.6C, this error was just the result of another error happening earlier during installation. In this case, you have to look through the corresponding logfiles and correct the real problem. If after looking through the logfiles this error is indeed the correct one (check the &sap; Notes), you can set STATUS of the offending step from ERROR to OK (file CENTRDB.R3S) and restart R3SETUP. After installation, you have to execute the report RSWBOINS from transaction SE38. See &sap; Note 0162266 for additional information about phase RFCRSWBOINI and RFCRADDBDIF. [RFCRADDBDIF_IND_IND] during <command>R3SETUP</command> Here the same restrictions apply: make sure by looking through the logfiles, that this error is not caused by some previous problems. If you can confirm that &sap; Note 0162266 applies, just set STATUS of the offending step from ERROR to OK (file CENTRDB.R3S) and restart R3SETUP. After installation, you have to execute the report RADDBDIF from transaction SE38. <errorcode>sigaction sig31: File size limit exceeded</errorcode> This error occurred during start of &sap; processes disp+work. If starting &sap; with the startsap script, subprocesses are then started which detach and do the dirty work of starting all other &sap; processes. As a result, the script itself will not notice if something goes wrong. To check whether the &sap; processes did start properly, have a look at the process status with ps ax | grep SID, which will give you a list of all &oracle; and &sap; processes. If it looks like some processes are missing or if you cannot connect to the &sap; system, look at the corresponding logfiles which can be found at /usr/sap/SID/DVEBMGSnr/work/. The files to look at are dev_ms and dev_disp. Signal 31 happens here if the amount of shared memory used by &oracle; and &sap; exceed the one defined within the kernel configuration file and could be resolved by using a larger value: # larger value for 46C production systems: options SHMMAXPGS=393216 # smaller value sufficient for 46B: #options SHMMAXPGS=262144 Start of <command>saposcol</command> Failed There are some problems with the program saposcol (version 4.6D). The &sap; system is using saposcol to collect data about the system performance. This program is not needed to use the &sap; system, so this problem can be considered a minor one. The older versions (4.6B) does work, but does not collect all the data (many calls will just return 0, for example for CPU usage). Advanced Topics If you are curious as to how the Linux binary compatibility works, this is the section you want to read. Most of what follows is based heavily on an email written to &a.chat; by Terry Lambert tlambert@primenet.com (Message ID: <199906020108.SAA07001@usr09.primenet.com>). How Does It Work? execution class loader FreeBSD has an abstraction called an execution class loader. This is a wedge into the &man.execve.2; system call. What happens is that FreeBSD has a list of loaders, instead of a single loader with a fallback to the #! loader for running any shell interpreters or shell scripts. Historically, the only loader on the &unix; platform examined the magic number (generally the first 4 or 8 bytes of the file) to see if it was a binary known to the system, and if so, invoked the binary loader. If it was not the binary type for the system, the &man.execve.2; call returned a failure, and the shell attempted to start executing it as shell commands. The assumption was a default of whatever the current shell is. Later, a hack was made for &man.sh.1; to examine the first two characters, and if they were :\n, then it invoked the &man.csh.1; shell instead (we believe SCO first made this hack). What FreeBSD does now is go through a list of loaders, with a generic #! loader that knows about interpreters as the characters which follow to the next whitespace next to last, followed by a fallback to /bin/sh. ELF For the Linux ABI support, FreeBSD sees the magic number as an ELF binary (it makes no distinction between FreeBSD, &solaris;, Linux, or any other OS which has an ELF image type, at this point). Solaris The ELF loader looks for a specialized brand, which is a comment section in the ELF image, and which is not present on SVR4/&solaris; ELF binaries. For Linux binaries to function, they must be branded as type Linux from &man.brandelf.1;: &prompt.root; brandelf -t Linux file When this is done, the ELF loader will see the Linux brand on the file. ELF branding When the ELF loader sees the Linux brand, the loader replaces a pointer in the proc structure. All system calls are indexed through this pointer (in a traditional &unix; system, this would be the sysent[] structure array, containing the system calls). In addition, the process is flagged for special handling of the trap vector for the signal trampoline code, and several other (minor) fix-ups that are handled by the Linux kernel module. The Linux system call vector contains, among other things, a list of sysent[] entries whose addresses reside in the kernel module. When a system call is called by the Linux binary, the trap code dereferences the system call function pointer off the proc structure, and gets the Linux, not the FreeBSD, system call entry points. In addition, the Linux mode dynamically reroots lookups; this is, in effect, what the option to file system mounts (not the unionfs file system type!) does. First, an attempt is made to lookup the file in the /compat/linux/original-path directory, then only if that fails, the lookup is done in the /original-path directory. This makes sure that binaries that require other binaries can run (e.g., the Linux toolchain can all run under Linux ABI support). It also means that the Linux binaries can load and execute FreeBSD binaries, if there are no corresponding Linux binaries present, and that you could place a &man.uname.1; command in the /compat/linux directory tree to ensure that the Linux binaries could not tell they were not running on Linux. In effect, there is a Linux kernel in the FreeBSD kernel; the various underlying functions that implement all of the services provided by the kernel are identical to both the FreeBSD system call table entries, and the Linux system call table entries: file system operations, virtual memory operations, signal delivery, System V IPC, etc… The only difference is that FreeBSD binaries get the FreeBSD glue functions, and Linux binaries get the Linux glue functions (most older OS's only had their own glue functions: addresses of functions in a static global sysent[] structure array, instead of addresses of functions dereferenced off a dynamically initialized pointer in the proc structure of the process making the call). Which one is the native FreeBSD ABI? It does not matter. Basically the only difference is that (currently; this could easily be changed in a future release, and probably will be after this) the FreeBSD glue functions are statically linked into the kernel, and the Linux glue functions can be statically linked, or they can be accessed via a kernel module. Yeah, but is this really emulation? No. It is an ABI implementation, not an emulation. There is no emulator (or simulator, to cut off the next question) involved. So why is it sometimes called Linux emulation? To make it hard to sell FreeBSD! Really, it is because the historical implementation was done at a time when there was really no word other than that to describe what was going on; saying that FreeBSD ran Linux binaries was not true, if you did not compile the code in or load a module, and there needed to be a word to describe what was being loaded—hence the Linux emulator. diff --git a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml index 64ddb9f9d2..9acad52963 100644 --- a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml @@ -1,3341 +1,3341 @@ Obtaining FreeBSD CDROM and DVD Publishers Retail Boxed Products FreeBSD is available as a boxed product (FreeBSD CDs, additional software, and printed documentation) from several retailers:
CompUSA WWW:
Frys Electronics WWW:
CD and DVD Sets FreeBSD CD and DVD sets are available from many online retailers:
BSD Mall by Daemon News PO Box 161 Nauvoo, IL 62354 USA Phone: +1 866 273-6255 Fax: +1 217 453-9956 Email: sales@bsdmall.com WWW:
FreeBSD Mall, Inc. 3623 Sanford Street Concord, CA 94520-1405 USA Phone: +1 925 240-6652 Fax: +1 925 674-0821 Email: info@freebsdmall.com WWW:
Dr. Hinner EDV St. Augustinus-Str. 10 D-81825 München Germany Phone: (089) 428 419 WWW:
Ikarios 22-24 rue Voltaire 92000 Nanterre France WWW:
JMC Software Ireland Phone: 353 1 6291282 WWW:
The Linux Emporium Hilliard House, Lester Way Wallingford OX10 9TA United Kingdom Phone: +44 1491 837010 Fax: +44 1491 837016 WWW:
Linux+ DVD Magazine Lewartowskiego 6 Warsaw 00-190 Poland Phone: +48 22 860 18 18 Email: editors@lpmagazine.org WWW:
Linux System Labs Australia 21 Ray Drive Balwyn North VIC - 3104 Australia Phone: +61 3 9857 5918 Fax: +61 3 9857 8974 WWW:
LinuxCenter.Ru Galernaya Street, 55 Saint-Petersburg 190000 Russia Phone: +7-812-3125208 Email: info@linuxcenter.ru WWW:
Distributors If you are a reseller and want to carry FreeBSD CDROM products, please contact a distributor:
Cylogistics 809B Cuesta Dr., #2149 Mountain View, CA 94040 USA Phone: +1 650 694-4949 Fax: +1 650 694-4953 Email: sales@cylogistics.com WWW:
Ingram Micro 1600 E. St. Andrew Place Santa Ana, CA 92705-4926 USA Phone: 1 (800) 456-8000 WWW:
Kudzu, LLC 7375 Washington Ave. S. Edina, MN 55439 USA Phone: +1 952 947-0822 Fax: +1 952 947-0876 Email: sales@kudzuenterprises.com
LinuxCenter.Ru Galernaya Street, 55 Saint-Petersburg 190000 Russia Phone: +7-812-3125208 Email: info@linuxcenter.ru WWW:
Navarre Corp 7400 49th Ave South New Hope, MN 55428 USA Phone: +1 763 535-8333 Fax: +1 763 535-0341 WWW:
FTP Sites The official sources for FreeBSD are available via anonymous FTP from a worldwide set of mirror sites. The site is well connected and allows a large number of connections to it, but you are probably better off finding a closer mirror site (especially if you decide to set up some sort of mirror site). The FreeBSD mirror sites database is more accurate than the mirror listing in the Handbook, as it gets its information from the DNS rather than relying on static lists of hosts. Additionally, FreeBSD is available via anonymous FTP from the following mirror sites. If you choose to obtain FreeBSD via anonymous FTP, please try to use a site near you. The mirror sites listed as Primary Mirror Sites typically have the entire FreeBSD archive (all the currently available versions for each of the architectures) but you will probably have faster download times from a site that is in your country or region. The regional sites carry the most recent versions for the most popular architecture(s) but might not carry the entire FreeBSD archive. All sites provide access via anonymous FTP but some sites also provide access via other methods. The access methods available for each site are provided in parentheses after the hostname. &chap.mirrors.ftp.inc; Anonymous CVS <anchor id="anoncvs-intro">Introduction CVS anonymous Anonymous CVS (or, as it is otherwise known, anoncvs) is a feature provided by the CVS utilities bundled with FreeBSD for synchronizing with a remote CVS repository. Among other things, it allows users of FreeBSD to perform, with no special privileges, read-only CVS operations against one of the FreeBSD project's official anoncvs servers. To use it, one simply sets the CVSROOT environment variable to point at the appropriate anoncvs server, provides the well-known password anoncvs with the cvs login command, and then uses the &man.cvs.1; command to access it like any local repository. The cvs login command, stores the passwords that are used for authenticating to the CVS server in a file called .cvspass in your HOME directory. If this file does not exist, you might get an error when trying to use cvs login for the first time. Just make an empty .cvspass file, and retry to login. While it can also be said that the CVSup and anoncvs services both perform essentially the same function, there are various trade-offs which can influence the user's choice of synchronization methods. In a nutshell, CVSup is much more efficient in its usage of network resources and is by far the most technically sophisticated of the two, but at a price. To use CVSup, a special client must first be installed and configured before any bits can be grabbed, and then only in the fairly large chunks which CVSup calls collections. Anoncvs, by contrast, can be used to examine anything from an individual file to a specific program (like ls or grep) by referencing the CVS module name. Of course, anoncvs is also only good for read-only operations on the CVS repository, so if it is your intention to support local development in one repository shared with the FreeBSD project bits then CVSup is really your only option. <anchor id="anoncvs-usage">Using Anonymous CVS Configuring &man.cvs.1; to use an Anonymous CVS repository is a simple matter of setting the CVSROOT environment variable to point to one of the FreeBSD project's anoncvs servers. At the time of this writing, the following servers are available: France: :pserver:anoncvs@anoncvs.fr.FreeBSD.org:/home/ncvs (pserver (password anoncvs), ssh (no password)) Japan: :pserver:anoncvs@anoncvs.jp.FreeBSD.org:/home/ncvs (Use cvs login and enter the password anoncvs when prompted.) Taiwan: :pserver:anoncvs@anoncvs.tw.FreeBSD.org:/home/ncvs (pserver (use cvs login and enter any password when prompted), ssh (no password)) SSH2 HostKey: 1024 e8:3b:29:7b:ca:9f:ac:e9:45:cb:c8:17:ae:9b:eb:55 /etc/ssh/ssh_host_dsa_key.pub USA: freebsdanoncvs@anoncvs.FreeBSD.org:/home/ncvs (ssh only - no password) SSH HostKey: 1024 a1:e7:46:de:fb:56:ef:05:bc:73:aa:91:09:da:f7:f4 root@sanmateo.ecn.purdue.edu SSH2 HostKey: 1024 52:02:38:1a:2f:a8:71:d3:f5:83:93:8d:aa:00:6f:65 ssh_host_dsa_key.pub USA: anoncvs@anoncvs1.FreeBSD.org:/home/ncvs (ssh2 only - no password) SSH2 HostKey: 2048 53:1f:15:a3:72:5c:43:f6:44:0e:6a:e9:bb:f8:01:62 /etc/ssh/ssh_host_dsa_key.pub Since CVS allows one to check out virtually any version of the FreeBSD sources that ever existed (or, in some cases, will exist), you need to be familiar with the revision () flag to &man.cvs.1; and what some of the permissible values for it in the FreeBSD Project repository are. There are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific revision. Its meaning stays the same from day to day. A branch tag, on the other hand, refers to the latest revision on a given line of development, at any given time. Because a branch tag does not refer to a specific revision, it may mean something different tomorrow than it means today. contains revision tags that users might be interested in. Again, none of these are valid for the Ports Collection since the Ports Collection does not have multiple branches of development. When you specify a branch tag, you normally receive the latest versions of the files on that line of development. If you wish to receive some past version, you can do so by specifying a date with the flag. See the &man.cvs.1; manual page for more details. Examples While it really is recommended that you read the manual page for &man.cvs.1; thoroughly before doing anything, here are some quick examples which essentially show how to use Anonymous CVS: Checking Out Something from -CURRENT (&man.ls.1;): &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.tw.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter any word for password. &prompt.user; cvs co ls Using SSH to check out the <filename>src/</filename> tree: &prompt.user; cvs -d freebsdanoncvs@anoncvs.FreeBSD.org:/home/ncvs co src The authenticity of host 'anoncvs.freebsd.org (128.46.156.46)' can't be established. DSA key fingerprint is 52:02:38:1a:2f:a8:71:d3:f5:83:93:8d:aa:00:6f:65. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'anoncvs.freebsd.org' (DSA) to the list of known hosts. Checking Out the Version of &man.ls.1; in the 6-STABLE Branch: &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.tw.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter any word for password. &prompt.user; cvs co -rRELENG_6 ls Creating a List of Changes (as Unified Diffs) to &man.ls.1; &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.tw.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter any word for password. &prompt.user; cvs rdiff -u -rRELENG_5_3_0_RELEASE -rRELENG_5_4_0_RELEASE ls Finding Out What Other Module Names Can Be Used: &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.tw.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter any word for password. &prompt.user; cvs co modules &prompt.user; more modules/modules Other Resources The following additional resources may be helpful in learning CVS: CVS Tutorial from Cal Poly. CVS Home, the CVS development and support community. CVSweb is the FreeBSD Project web interface for CVS. Using CTM CTM CTM is a method for keeping a remote directory tree in sync with a central one. It has been developed for usage with FreeBSD's source trees, though other people may find it useful for other purposes as time goes by. Little, if any, documentation currently exists at this time on the process of creating deltas, so contact the &a.ctm-users.name; mailing list for more information and if you wish to use CTM for other things. Why Should I Use <application>CTM</application>? CTM will give you a local copy of the FreeBSD source trees. There are a number of flavors of the tree available. Whether you wish to track the entire CVS tree or just one of the branches, CTM can provide you the information. If you are an active developer on FreeBSD, but have lousy or non-existent TCP/IP connectivity, or simply wish to have the changes automatically sent to you, CTM was made for you. You will need to obtain up to three deltas per day for the most active branches. However, you should consider having them sent by automatic email. The sizes of the updates are always kept as small as possible. This is typically less than 5K, with an occasional (one in ten) being 10-50K and every now and then a large 100K+ or more coming around. You will also need to make yourself aware of the various caveats related to working directly from the development sources rather than a pre-packaged release. This is particularly true if you choose the current sources. It is recommended that you read Staying current with FreeBSD. What Do I Need to Use <application>CTM</application>? You will need two things: The CTM program, and the initial deltas to feed it (to get up to current levels). The CTM program has been part of FreeBSD ever since version 2.0 was released, and lives in /usr/src/usr.sbin/ctm if you have a copy of the source available. The deltas you feed CTM can be had two ways, FTP or email. If you have general FTP access to the Internet then the following FTP sites support access to CTM: or see section mirrors. FTP the relevant directory and fetch the README file, starting from there. If you wish to get your deltas via email: Subscribe to one of the CTM distribution lists. &a.ctm-cvs-cur.name; supports the entire CVS tree. &a.ctm-src-cur.name; supports the head of the development branch. &a.ctm-src-4.name; supports the 4.X release branch, etc.. (If you do not know how to subscribe yourself to a list, click on the list name above or go to &a.mailman.lists.link; and click on the list that you wish to subscribe to. The list page should contain all of the necessary subscription instructions.) When you begin receiving your CTM updates in the mail, you may use the ctm_rmail program to unpack and apply them. You can actually use the ctm_rmail program directly from a entry in /etc/aliases if you want to have the process run in a fully automated fashion. Check the ctm_rmail manual page for more details. No matter what method you use to get the CTM deltas, you should subscribe to the &a.ctm-announce.name; mailing list. In the future, this will be the only place where announcements concerning the operations of the CTM system will be posted. Click on the list name above and follow the instructions to subscribe to the list. Using <application>CTM</application> for the First Time Before you can start using CTM deltas, you will need to get to a starting point for the deltas produced subsequently to it. First you should determine what you already have. Everyone can start from an empty directory. You must use an initial Empty delta to start off your CTM supported tree. At some point it is intended that one of these started deltas be distributed on the CD for your convenience, however, this does not currently happen. Since the trees are many tens of megabytes, you should prefer to start from something already at hand. If you have a -RELEASE CD, you can copy or extract an initial source from it. This will save a significant transfer of data. You can recognize these starter deltas by the X appended to the number (src-cur.3210XEmpty.gz for instance). The designation following the X corresponds to the origin of your initial seed. Empty is an empty directory. As a rule a base transition from Empty is produced every 100 deltas. By the way, they are large! 70 to 80 Megabytes of gzip'd data is common for the XEmpty deltas. Once you have picked a base delta to start from, you will also need all deltas with higher numbers following it. Using <application>CTM</application> in Your Daily Life To apply the deltas, simply say: &prompt.root; cd /where/ever/you/want/the/stuff &prompt.root; ctm -v -v /where/you/store/your/deltas/src-xxx.* CTM understands deltas which have been put through gzip, so you do not need to gunzip them first, this saves disk space. Unless it feels very secure about the entire process, CTM will not touch your tree. To verify a delta you can also use the flag and CTM will not actually touch your tree; it will merely verify the integrity of the delta and see if it would apply cleanly to your current tree. There are other options to CTM as well, see the manual pages or look in the sources for more information. That is really all there is to it. Every time you get a new delta, just run it through CTM to keep your sources up to date. Do not remove the deltas if they are hard to download again. You just might want to keep them around in case something bad happens. Even if you only have floppy disks, consider using fdwrite to make a copy. Keeping Your Local Changes As a developer one would like to experiment with and change files in the source tree. CTM supports local modifications in a limited way: before checking for the presence of a file foo, it first looks for foo.ctm. If this file exists, CTM will operate on it instead of foo. This behavior gives us a simple way to maintain local changes: simply copy the files you plan to modify to the corresponding file names with a .ctm suffix. Then you can freely hack the code, while CTM keeps the .ctm file up-to-date. Other Interesting <application>CTM</application> Options Finding Out Exactly What Would Be Touched by an Update You can determine the list of changes that CTM will make on your source repository using the option to CTM. This is useful if you would like to keep logs of the changes, pre- or post- process the modified files in any manner, or just are feeling a tad paranoid. Making Backups Before Updating Sometimes you may want to backup all the files that would be changed by a CTM update. Specifying the option causes CTM to backup all files that would be touched by a given CTM delta to backup-file. Restricting the Files Touched by an Update Sometimes you would be interested in restricting the scope of a given CTM update, or may be interested in extracting just a few files from a sequence of deltas. You can control the list of files that CTM would operate on by specifying filtering regular expressions using the and options. For example, to extract an up-to-date copy of lib/libc/Makefile from your collection of saved CTM deltas, run the commands: &prompt.root; cd /where/ever/you/want/to/extract/it/ &prompt.root; ctm -e '^lib/libc/Makefile' ~ctm/src-xxx.* For every file specified in a CTM delta, the and options are applied in the order given on the command line. The file is processed by CTM only if it is marked as eligible after all the and options are applied to it. Future Plans for <application>CTM</application> Tons of them: Use some kind of authentication into the CTM system, so as to allow detection of spoofed CTM updates. Clean up the options to CTM, they became confusing and counter intuitive. Miscellaneous Stuff There is a sequence of deltas for the ports collection too, but interest has not been all that high yet. CTM Mirrors CTM/FreeBSD is available via anonymous FTP from the following mirror sites. If you choose to obtain CTM via anonymous FTP, please try to use a site near you. In case of problems, please contact the &a.ctm-users.name; mailing list. California, Bay Area, official source South Africa, backup server for old deltas Taiwan/R.O.C. If you did not find a mirror near to you or the mirror is incomplete, try to use a search engine such as alltheweb. Using CVSup Introduction CVSup is a software package for distributing and updating source trees from a master CVS repository on a remote server host. The FreeBSD sources are maintained in a CVS repository on a central development machine in California. With CVSup, FreeBSD users can easily keep their own source trees up to date. CVSup uses the so-called pull model of updating. Under the pull model, each client asks the server for updates, if and when they are wanted. The server waits passively for update requests from its clients. Thus all updates are instigated by the client. The server never sends unsolicited updates. Users must either run the CVSup client manually to get an update, or they must set up a cron job to run it automatically on a regular basis. The term CVSup, capitalized just so, refers to the entire software package. Its main components are the client cvsup which runs on each user's machine, and the server cvsupd which runs at each of the FreeBSD mirror sites. As you read the FreeBSD documentation and mailing lists, you may see references to sup. Sup was the predecessor of CVSup, and it served a similar purpose. CVSup is used much in the same way as sup and, in fact, uses configuration files which are backward-compatible with sup's. Sup is no longer used in the FreeBSD project, because CVSup is both faster and more flexible. The csup utility is a rewrite of the CVSup software in C. Its biggest advantage is, that it is faster and does not depend on the Modula-3 language, thus you do not need to install it as a requirement. Moreover, if you are using &os; 6.2 or later, you can use it out-of-the-box, since it is included in the base system. Older &os; versions do not have &man.csup.1; in their base system but you can easily install the net/csup port, or a precompiled package. The csup utility does not support CVS mode, though. If you want to mirror complete repositories, you will still need to use CVSup. If you decided to use csup, just skip the steps on the installation of CVSup and substitute the references of CVSup with csup while following the remainder of this article. Installation The easiest way to install CVSup is to use the precompiled net/cvsup package from the FreeBSD packages collection. If you prefer to build CVSup from source, you can use the net/cvsup port instead. But be forewarned: the net/cvsup port depends on the Modula-3 system, which takes a substantial amount of time and disk space to download and build. If you are going to be using CVSup on a machine which will not have &xfree86; or &xorg; installed, such as a server, be sure to use the port which does not include the CVSup GUI, net/cvsup-without-gui. If you want to install csup on FreeBSD 6.1 or earlier, you can use the precompiled net/csup package from the FreeBSD packages collection. If you prefer to build csup from source, you can use the net/csup port instead. CVSup Configuration CVSup's operation is controlled by a configuration file called the supfile. There are some sample supfiles in the directory /usr/share/examples/cvsup/. The information in a supfile answers the following questions for CVSup: Which files do you want to receive? Which versions of them do you want? Where do you want to get them from? Where do you want to put them on your own machine? Where do you want to put your status files? In the following sections, we will construct a typical supfile by answering each of these questions in turn. First, we describe the overall structure of a supfile. A supfile is a text file. Comments begin with # and extend to the end of the line. Lines that are blank and lines that contain only comments are ignored. Each remaining line describes a set of files that the user wishes to receive. The line begins with the name of a collection, a logical grouping of files defined by the server. The name of the collection tells the server which files you want. After the collection name come zero or more fields, separated by white space. These fields answer the questions listed above. There are two types of fields: flag fields and value fields. A flag field consists of a keyword standing alone, e.g., delete or compress. A value field also begins with a keyword, but the keyword is followed without intervening white space by = and a second word. For example, release=cvs is a value field. A supfile typically specifies more than one collection to receive. One way to structure a supfile is to specify all of the relevant fields explicitly for each collection. However, that tends to make the supfile lines quite long, and it is inconvenient because most fields are the same for all of the collections in a supfile. CVSup provides a defaulting mechanism to avoid these problems. Lines beginning with the special pseudo-collection name *default can be used to set flags and values which will be used as defaults for the subsequent collections in the supfile. A default value can be overridden for an individual collection, by specifying a different value with the collection itself. Defaults can also be changed or augmented in mid-supfile by additional *default lines. With this background, we will now proceed to construct a supfile for receiving and updating the main source tree of FreeBSD-CURRENT. Which files do you want to receive? The files available via CVSup are organized into named groups called collections. The collections that are available are described in the following section. In this example, we wish to receive the entire main source tree for the FreeBSD system. There is a single large collection src-all which will give us all of that. As a first step toward constructing our supfile, we simply list the collections, one per line (in this case, only one line): src-all Which version(s) of them do you want? With CVSup, you can receive virtually any version of the sources that ever existed. That is possible because the cvsupd server works directly from the CVS repository, which contains all of the versions. You specify which one of them you want using the tag= and value fields. Be very careful to specify any tag= fields correctly. Some tags are valid only for certain collections of files. If you specify an incorrect or misspelled tag, CVSup will delete files which you probably do not want deleted. In particular, use only tag=. for the ports-* collections. The tag= field names a symbolic tag in the repository. There are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific revision. Its meaning stays the same from day to day. A branch tag, on the other hand, refers to the latest revision on a given line of development, at any given time. Because a branch tag does not refer to a specific revision, it may mean something different tomorrow than it means today. contains branch tags that users might be interested in. When specifying a tag in CVSup's configuration file, it must be preceded with tag= (RELENG_4 will become tag=RELENG_4). Keep in mind that only the tag=. is relevant for the Ports Collection. Be very careful to type the tag name exactly as shown. CVSup cannot distinguish between valid and invalid tags. If you misspell the tag, CVSup will behave as though you had specified a valid tag which happens to refer to no files at all. It will delete your existing sources in that case. When you specify a branch tag, you normally receive the latest versions of the files on that line of development. If you wish to receive some past version, you can do so by specifying a date with the value field. The &man.cvsup.1; manual page explains how to do that. For our example, we wish to receive FreeBSD-CURRENT. We add this line at the beginning of our supfile: *default tag=. There is an important special case that comes into play if you specify neither a tag= field nor a date= field. In that case, you receive the actual RCS files directly from the server's CVS repository, rather than receiving a particular version. Developers generally prefer this mode of operation. By maintaining a copy of the repository itself on their systems, they gain the ability to browse the revision histories and examine past versions of files. This gain is achieved at a large cost in terms of disk space, however. Where do you want to get them from? We use the host= field to tell cvsup where to obtain its updates. Any of the CVSup mirror sites will do, though you should try to select one that is close to you in cyberspace. In this example we will use a fictional FreeBSD distribution site, cvsup99.FreeBSD.org: *default host=cvsup99.FreeBSD.org You will need to change the host to one that actually exists before running CVSup. On any particular run of cvsup, you can override the host setting on the command line, with . Where do you want to put them on your own machine? The prefix= field tells cvsup where to put the files it receives. In this example, we will put the source files directly into our main source tree, /usr/src. The src directory is already implicit in the collections we have chosen to receive, so this is the correct specification: *default prefix=/usr Where should cvsup maintain its status files? The CVSup client maintains certain status files in what is called the base directory. These files help CVSup to work more efficiently, by keeping track of which updates you have already received. We will use the standard base directory, /var/db: *default base=/var/db If your base directory does not already exist, now would be a good time to create it. The cvsup client will refuse to run if the base directory does not exist. Miscellaneous supfile settings: There is one more line of boiler plate that normally needs to be present in the supfile: *default release=cvs delete use-rel-suffix compress release=cvs indicates that the server should get its information out of the main FreeBSD CVS repository. This is virtually always the case, but there are other possibilities which are beyond the scope of this discussion. delete gives CVSup permission to delete files. You should always specify this, so that CVSup can keep your source tree fully up-to-date. CVSup is careful to delete only those files for which it is responsible. Any extra files you happen to have will be left strictly alone. use-rel-suffix is ... arcane. If you really want to know about it, see the &man.cvsup.1; manual page. Otherwise, just specify it and do not worry about it. compress enables the use of gzip-style compression on the communication channel. If your network link is T1 speed or faster, you probably should not use compression. Otherwise, it helps substantially. Putting it all together: Here is the entire supfile for our example: *default tag=. *default host=cvsup99.FreeBSD.org *default prefix=/usr *default base=/var/db *default release=cvs delete use-rel-suffix compress src-all The <filename>refuse</filename> File As mentioned above, CVSup uses a pull method. Basically, this means that you connect to the CVSup server, and it says, Here is what you can download from me..., and your client responds OK, I will take this, this, this, and this. In the default configuration, the CVSup client will take every file associated with the collection and tag you chose in the configuration file. However, this is not always what you want, especially if you are synching the doc, ports, or www trees — most people cannot read four or five languages, and therefore they do not need to download the language-specific files. If you are CVSuping the Ports Collection, you can get around this by specifying each collection individually (e.g., ports-astrology, ports-biology, etc instead of simply saying ports-all). However, since the doc and www trees do not have language-specific collections, you must use one of CVSup's many nifty features: the refuse file. The refuse file essentially tells CVSup that it should not take every single file from a collection; in other words, it tells the client to refuse certain files from the server. The refuse file can be found (or, if you do not yet have one, should be placed) in base/sup/. base is defined in your supfile; our defined base is /var/db, which means that by default the refuse file is /var/db/sup/refuse. The refuse file has a very simple format; it simply contains the names of files or directories that you do not wish to download. For example, if you cannot speak any languages other than English and some German, and you do not feel the need to read the German translation of documentation, you can put the following in your refuse file: doc/bn_* doc/da_* doc/de_* doc/el_* doc/es_* doc/fr_* doc/hu_* doc/it_* doc/ja_* doc/mn_* doc/nl_* doc/no_* doc/pl_* doc/pt_* doc/ru_* doc/sr_* doc/tr_* doc/zh_* and so forth for the other languages (you can find the full list by browsing the FreeBSD CVS repository). With this very useful feature, those users who are on slow links or pay by the minute for their Internet connection will be able to save valuable time as they will no longer need to download files that they will never use. For more information on refuse files and other neat features of CVSup, please view its manual page. Running <application>CVSup</application> You are now ready to try an update. The command line for doing this is quite simple: &prompt.root; cvsup supfile where supfile is of course the name of the supfile you have just created. Assuming you are running under X11, cvsup will display a GUI window with some buttons to do the usual things. Press the go button, and watch it run. Since you are updating your actual /usr/src tree in this example, you will need to run the program as root so that cvsup has the permissions it needs to update your files. Having just created your configuration file, and having never used this program before, that might understandably make you nervous. There is an easy way to do a trial run without touching your precious files. Just create an empty directory somewhere convenient, and name it as an extra argument on the command line: &prompt.root; mkdir /var/tmp/dest &prompt.root; cvsup supfile /var/tmp/dest The directory you specify will be used as the destination directory for all file updates. CVSup will examine your usual files in /usr/src, but it will not modify or delete any of them. Any file updates will instead land in /var/tmp/dest/usr/src. CVSup will also leave its base directory status files untouched when run this way. The new versions of those files will be written into the specified directory. As long as you have read access to /usr/src, you do not even need to be root to perform this kind of trial run. If you are not running X11 or if you just do not like GUIs, you should add a couple of options to the command line when you run cvsup: &prompt.root; cvsup -g -L 2 supfile The tells CVSup not to use its GUI. This is automatic if you are not running X11, but otherwise you have to specify it. The tells CVSup to print out the details of all the file updates it is doing. There are three levels of verbosity, from to . The default is 0, which means total silence except for error messages. There are plenty of other options available. For a brief list of them, type cvsup -H. For more detailed descriptions, see the manual page. Once you are satisfied with the way updates are working, you can arrange for regular runs of CVSup using &man.cron.8;. Obviously, you should not let CVSup use its GUI when running it from &man.cron.8;. <application>CVSup</application> File Collections The file collections available via CVSup are organized hierarchically. There are a few large collections, and they are divided into smaller sub-collections. Receiving a large collection is equivalent to receiving each of its sub-collections. The hierarchical relationships among collections are reflected by the use of indentation in the list below. The most commonly used collections are src-all, and ports-all. The other collections are used only by small groups of people for specialized purposes, and some mirror sites may not carry all of them. cvs-all release=cvs The main FreeBSD CVS repository, including the cryptography code. distrib release=cvs Files related to the distribution and mirroring of FreeBSD. doc-all release=cvs Sources for the FreeBSD Handbook and other documentation. This does not include files for the FreeBSD web site. ports-all release=cvs The FreeBSD Ports Collection. If you do not want to update the whole of ports-all (the whole ports tree), but use one of the subcollections listed below, make sure that you always update the ports-base subcollection! Whenever something changes in the ports build infrastructure represented by ports-base, it is virtually certain that those changes will be used by real ports real soon. Thus, if you only update the real ports and they use some of the new features, there is a very high chance that their build will fail with some mysterious error message. The very first thing to do in this case is to make sure that your ports-base subcollection is up to date. If you are going to be building your own local copy of ports/INDEX, you must accept ports-all (the whole ports tree). Building ports/INDEX with a partial tree is not supported. See the FAQ. ports-accessibility release=cvs Software to help disabled users. ports-arabic release=cvs Arabic language support. ports-archivers release=cvs Archiving tools. ports-astro release=cvs Astronomical ports. ports-audio release=cvs Sound support. ports-base release=cvs The Ports Collection build infrastructure - various files located in the Mk/ and Tools/ subdirectories of /usr/ports. Please see the important warning above: you should always update this subcollection, whenever you update any part of the FreeBSD Ports Collection! ports-benchmarks release=cvs Benchmarks. ports-biology release=cvs Biology. ports-cad release=cvs Computer aided design tools. ports-chinese release=cvs Chinese language support. ports-comms release=cvs Communication software. ports-converters release=cvs character code converters. ports-databases release=cvs Databases. ports-deskutils release=cvs Things that used to be on the desktop before computers were invented. ports-devel release=cvs Development utilities. ports-dns release=cvs DNS related software. ports-editors release=cvs Editors. ports-emulators release=cvs Emulators for other operating systems. ports-finance release=cvs Monetary, financial and related applications. ports-ftp release=cvs FTP client and server utilities. ports-games release=cvs Games. ports-german release=cvs German language support. ports-graphics release=cvs Graphics utilities. ports-hebrew release=cvs Hebrew language support. ports-hungarian release=cvs Hungarian language support. ports-irc release=cvs Internet Relay Chat utilities. ports-japanese release=cvs Japanese language support. ports-java release=cvs &java; utilities. ports-korean release=cvs Korean language support. ports-lang release=cvs Programming languages. ports-mail release=cvs Mail software. ports-math release=cvs Numerical computation software. ports-mbone release=cvs MBone applications. ports-misc release=cvs Miscellaneous utilities. ports-multimedia release=cvs Multimedia software. ports-net release=cvs Networking software. ports-net-im release=cvs Instant messaging software. ports-net-mgmt release=cvs Network management software. ports-net-p2p release=cvs Peer to peer networking. ports-news release=cvs USENET news software. ports-palm release=cvs Software support for Palm series. ports-polish release=cvs Polish language support. ports-ports-mgmt release=cvs Utilities to manage ports and packages. ports-portuguese release=cvs Portuguese language support. ports-print release=cvs Printing software. ports-russian release=cvs Russian language support. ports-science release=cvs Science. ports-security release=cvs Security utilities. ports-shells release=cvs Command line shells. ports-sysutils release=cvs System utilities. ports-textproc release=cvs text processing utilities (does not include desktop publishing). ports-ukrainian release=cvs Ukrainian language support. ports-vietnamese release=cvs Vietnamese language support. ports-www release=cvs Software related to the World Wide Web. ports-x11 release=cvs Ports to support the X window system. ports-x11-clocks release=cvs X11 clocks. ports-x11-drivers release=cvs X11 drivers. ports-x11-fm release=cvs X11 file managers. ports-x11-fonts release=cvs X11 fonts and font utilities. ports-x11-toolkits release=cvs X11 toolkits. ports-x11-servers release=cvs X11 servers. ports-x11-themes release=cvs X11 themes. ports-x11-wm release=cvs X11 window managers. projects-all release=cvs Sources for the FreeBSD projects repository. src-all release=cvs The main FreeBSD sources, including the cryptography code. src-base release=cvs Miscellaneous files at the top of /usr/src. src-bin release=cvs User utilities that may be needed in single-user mode (/usr/src/bin). src-cddl release=cvs Utilities and libraries covered by the CDDL license (/usr/src/cddl). src-contrib release=cvs Utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/contrib). src-crypto release=cvs Cryptography utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/crypto). src-eBones release=cvs Kerberos and DES (/usr/src/eBones). Not used in current releases of FreeBSD. src-etc release=cvs System configuration files (/usr/src/etc). src-games release=cvs Games (/usr/src/games). src-gnu release=cvs Utilities covered by the GNU Public License (/usr/src/gnu). src-include release=cvs Header files (/usr/src/include). src-kerberos5 release=cvs Kerberos5 security package (/usr/src/kerberos5). src-kerberosIV release=cvs KerberosIV security package (/usr/src/kerberosIV). src-lib release=cvs Libraries (/usr/src/lib). src-libexec release=cvs System programs normally executed by other programs (/usr/src/libexec). src-release release=cvs Files required to produce a FreeBSD release (/usr/src/release). src-rescue release=cvs Statically linked programs for emergency recovery; see &man.rescue.8; (/usr/src/rescue). src-sbin release=cvs System utilities for single-user mode (/usr/src/sbin). src-secure release=cvs Cryptographic libraries and commands (/usr/src/secure). src-share release=cvs Files that can be shared across multiple systems (/usr/src/share). src-sys release=cvs The kernel (/usr/src/sys). src-sys-crypto release=cvs Kernel cryptography code (/usr/src/sys/crypto). src-tools release=cvs Various tools for the maintenance of FreeBSD (/usr/src/tools). src-usrbin release=cvs User utilities (/usr/src/usr.bin). src-usrsbin release=cvs System utilities (/usr/src/usr.sbin). www release=cvs The sources for the FreeBSD WWW site. distrib release=self The CVSup server's own configuration files. Used by CVSup mirror sites. gnats release=current The GNATS bug-tracking database. mail-archive release=current FreeBSD mailing list archive. www release=current The pre-processed FreeBSD WWW site files (not the source files). Used by WWW mirror sites. For More Information For the CVSup FAQ and other information about CVSup, see The CVSup Home Page. Most FreeBSD-related discussion of CVSup takes place on the &a.hackers;. New versions of the software are announced there, as well as on the &a.announce;. For questions or bug reports about CVSup take a look at the CVSup FAQ. CVSup Sites CVSup servers for FreeBSD are running at the following sites: &chap.mirrors.cvsup.inc; Using Portsnap Introduction Portsnap is a system for securely distributing the &os; ports tree. Approximately once an hour, a snapshot of the ports tree is generated, repackaged, and cryptographically signed. The resulting files are then distributed via HTTP. Like CVSup, Portsnap uses a pull model of updating: The packaged and signed ports trees are placed on a web server which waits passively for clients to request files. Users must either run &man.portsnap.8; manually to download updates or set up a &man.cron.8; job to download updates automatically on a regular basis. For technical reasons, Portsnap does not update the live ports tree in /usr/ports/ directly; instead, it works via a compressed copy of the ports tree stored in /var/db/portsnap/ by default. This compressed copy is then used to update the live ports tree. If Portsnap is installed from the &os; Ports Collection, then the default location for its compressed snapshot will be /usr/local/portsnap/ instead of /var/db/portsnap/. Installation On &os; 6.0 and more recent versions, Portsnap is contained in the &os; base system. On older versions of &os;, it can be installed using the ports-mgmt/portsnap port. Portsnap Configuration Portsnap's operation is controlled by the /etc/portsnap.conf configuration file. For most users, the default configuration file will suffice; for more details, consult the &man.portsnap.conf.5; manual page. If Portsnap is installed from the &os; Ports Collection, it will use the configuration file /usr/local/etc/portsnap.conf instead of /etc/portsnap.conf. This configuration file is not created when the port is installed, but a sample configuration file is distributed; to copy it into place, run the following command: &prompt.root; cd /usr/local/etc && cp portsnap.conf.sample portsnap.conf Running <application>Portsnap</application> for the First Time The first time &man.portsnap.8; is run, it will need to download a compressed snapshot of the entire ports tree into /var/db/portsnap/ (or /usr/local/portsnap/ if Portsnap was installed from the Ports Collection). For the beginning of 2006 this is approximately a 41 MB download. &prompt.root; portsnap fetch Once the compressed snapshot has been downloaded, a live copy of the ports tree can be extracted into /usr/ports/. This is necessary even if a ports tree has already been created in that directory (e.g., by using CVSup), since it establishes a baseline from which portsnap can determine which parts of the ports tree need to be updated later. &prompt.root; portsnap extract In the default installation - /usr/ports is not + /usr/ports is not created. If you run &os; 6.0-RELEASE, it should be created before portsnap is used. On more recent versions of &os; or Portsnap, this operation will be done automatically at first use of the portsnap command. Updating the Ports Tree After an initial compressed snapshot of the ports tree has been downloaded and extracted into /usr/ports/, updating the ports tree consists of two steps: fetching updates to the compressed snapshot, and using them to update the live ports tree. These two steps can be specified to portsnap as a single command: &prompt.root; portsnap fetch update Some older versions of portsnap do not support this syntax; if it fails, try instead the following: &prompt.root; portsnap fetch &prompt.root; portsnap update Running Portsnap from cron In order to avoid problems with flash crowds accessing the Portsnap servers, portsnap fetch will not run from a &man.cron.8; job. Instead, a special portsnap cron command exists, which waits for a random duration up to 3600 seconds before fetching updates. In addition, it is strongly recommended that portsnap update not be run from a cron job, since it is liable to cause major problems if it happens to run at the same time as a port is being built or installed. However, it is safe to update the ports' INDEX files, and this can be done by passing the flag to portsnap. (Obviously, if portsnap -I update is run from cron, then it will be necessary to run portsnap update without the flag at a later time in order to update the rest of the tree.) Adding the following line to /etc/crontab will cause portsnap to update its compressed snapshot and the INDEX files in /usr/ports/, and will send an email if any installed ports are out of date: 0 3 * * * root portsnap -I cron update && pkg_version -vIL= If the system clock is not set to the local time zone, please replace 3 with a random value between 0 and 23, in order to spread the load on the Portsnap servers more evenly. Some older versions of portsnap do not support listing multiple commands (e.g., cron update) in the same invocation of portsnap. If the line above fails, try replacing portsnap -I cron update with portsnap cron && portsnap -I update. CVS Tags When obtaining or updating sources using cvs or CVSup, a revision tag must be specified. A revision tag refers to either a particular line of &os; development, or a specific point in time. The first type are called branch tags, and the second type are called release tags. Branch Tags All of these, with the exception of HEAD (which is always a valid tag), only apply to the src/ tree. The ports/, doc/, and www/ trees are not branched. HEAD Symbolic name for the main line, or FreeBSD-CURRENT. Also the default when no revision is specified. In CVSup, this tag is represented by a . (not punctuation, but a literal . character). In CVS, this is the default when no revision tag is specified. It is usually not a good idea to checkout or update to CURRENT sources on a STABLE machine, unless that is your intent. RELENG_7 The line of development for FreeBSD-7.X, also known as FreeBSD 7-STABLE RELENG_7_0 The release branch for FreeBSD-7.0, used only for security advisories and other critical fixes. RELENG_6 The line of development for FreeBSD-6.X, also known as FreeBSD 6-STABLE RELENG_6_3 The release branch for FreeBSD-6.3, used only for security advisories and other critical fixes. RELENG_6_2 The release branch for FreeBSD-6.2, used only for security advisories and other critical fixes. RELENG_6_1 The release branch for FreeBSD-6.1, used only for security advisories and other critical fixes. RELENG_6_0 The release branch for FreeBSD-6.0, used only for security advisories and other critical fixes. RELENG_5 The line of development for FreeBSD-5.X, also known as FreeBSD 5-STABLE. RELENG_5_5 The release branch for FreeBSD-5.5, used only for security advisories and other critical fixes. RELENG_5_4 The release branch for FreeBSD-5.4, used only for security advisories and other critical fixes. RELENG_5_3 The release branch for FreeBSD-5.3, used only for security advisories and other critical fixes. RELENG_5_2 The release branch for FreeBSD-5.2 and FreeBSD-5.2.1, used only for security advisories and other critical fixes. RELENG_5_1 The release branch for FreeBSD-5.1, used only for security advisories and other critical fixes. RELENG_5_0 The release branch for FreeBSD-5.0, used only for security advisories and other critical fixes. RELENG_4 The line of development for FreeBSD-4.X, also known as FreeBSD 4-STABLE. RELENG_4_11 The release branch for FreeBSD-4.11, used only for security advisories and other critical fixes. RELENG_4_10 The release branch for FreeBSD-4.10, used only for security advisories and other critical fixes. RELENG_4_9 The release branch for FreeBSD-4.9, used only for security advisories and other critical fixes. RELENG_4_8 The release branch for FreeBSD-4.8, used only for security advisories and other critical fixes. RELENG_4_7 The release branch for FreeBSD-4.7, used only for security advisories and other critical fixes. RELENG_4_6 The release branch for FreeBSD-4.6 and FreeBSD-4.6.2, used only for security advisories and other critical fixes. RELENG_4_5 The release branch for FreeBSD-4.5, used only for security advisories and other critical fixes. RELENG_4_4 The release branch for FreeBSD-4.4, used only for security advisories and other critical fixes. RELENG_4_3 The release branch for FreeBSD-4.3, used only for security advisories and other critical fixes. RELENG_3 The line of development for FreeBSD-3.X, also known as 3.X-STABLE. RELENG_2_2 The line of development for FreeBSD-2.2.X, also known as 2.2-STABLE. This branch is mostly obsolete. Release Tags These tags refer to a specific point in time when a particular version of &os; was released. The release engineering process is documented in more detail by the Release Engineering Information and Release Process documents. The src tree uses tag names that start with RELENG_ tags. The ports and doc trees use tags whose names begin with RELEASE tags. Finally, the www tree is not tagged with any special name for releases. RELENG_7_0_0_RELEASE FreeBSD 7.0 RELENG_6_3_0_RELEASE FreeBSD 6.3 RELENG_6_2_0_RELEASE FreeBSD 6.2 RELENG_6_1_0_RELEASE FreeBSD 6.1 RELENG_6_0_0_RELEASE FreeBSD 6.0 RELENG_5_5_0_RELEASE FreeBSD 5.5 RELENG_5_4_0_RELEASE FreeBSD 5.4 RELENG_4_11_0_RELEASE FreeBSD 4.11 RELENG_5_3_0_RELEASE FreeBSD 5.3 RELENG_4_10_0_RELEASE FreeBSD 4.10 RELENG_5_2_1_RELEASE FreeBSD 5.2.1 RELENG_5_2_0_RELEASE FreeBSD 5.2 RELENG_4_9_0_RELEASE FreeBSD 4.9 RELENG_5_1_0_RELEASE FreeBSD 5.1 RELENG_4_8_0_RELEASE FreeBSD 4.8 RELENG_5_0_0_RELEASE FreeBSD 5.0 RELENG_4_7_0_RELEASE FreeBSD 4.7 RELENG_4_6_2_RELEASE FreeBSD 4.6.2 RELENG_4_6_1_RELEASE FreeBSD 4.6.1 RELENG_4_6_0_RELEASE FreeBSD 4.6 RELENG_4_5_0_RELEASE FreeBSD 4.5 RELENG_4_4_0_RELEASE FreeBSD 4.4 RELENG_4_3_0_RELEASE FreeBSD 4.3 RELENG_4_2_0_RELEASE FreeBSD 4.2 RELENG_4_1_1_RELEASE FreeBSD 4.1.1 RELENG_4_1_0_RELEASE FreeBSD 4.1 RELENG_4_0_0_RELEASE FreeBSD 4.0 RELENG_3_5_0_RELEASE FreeBSD-3.5 RELENG_3_4_0_RELEASE FreeBSD-3.4 RELENG_3_3_0_RELEASE FreeBSD-3.3 RELENG_3_2_0_RELEASE FreeBSD-3.2 RELENG_3_1_0_RELEASE FreeBSD-3.1 RELENG_3_0_0_RELEASE FreeBSD-3.0 RELENG_2_2_8_RELEASE FreeBSD-2.2.8 RELENG_2_2_7_RELEASE FreeBSD-2.2.7 RELENG_2_2_6_RELEASE FreeBSD-2.2.6 RELENG_2_2_5_RELEASE FreeBSD-2.2.5 RELENG_2_2_2_RELEASE FreeBSD-2.2.2 RELENG_2_2_1_RELEASE FreeBSD-2.2.1 RELENG_2_2_0_RELEASE FreeBSD-2.2.0 AFS Sites AFS servers for FreeBSD are running at the following sites: Sweden The path to the files are: /afs/stacken.kth.se/ftp/pub/FreeBSD/ stacken.kth.se # Stacken Computer Club, KTH, Sweden 130.237.234.43 #hot.stacken.kth.se 130.237.237.230 #fishburger.stacken.kth.se 130.237.234.3 #milko.stacken.kth.se Maintainer ftp@stacken.kth.se rsync Sites The following sites make FreeBSD available through the rsync protocol. The rsync utility works in much the same way as the &man.rcp.1; command, but has more options and uses the rsync remote-update protocol which transfers only the differences between two sets of files, thus greatly speeding up the synchronization over the network. This is most useful if you are a mirror site for the FreeBSD FTP server, or the CVS repository. The rsync suite is available for many operating systems, on FreeBSD, see the net/rsync port or use the package. Czech Republic rsync://ftp.cz.FreeBSD.org/ Available collections: ftp: A partial mirror of the FreeBSD FTP server. FreeBSD: A full mirror of the FreeBSD FTP server. Germany rsync://grappa.unix-ag.uni-kl.de/ Available collections: freebsd-cvs: The full FreeBSD CVS repository. This machine also mirrors the CVS repositories of the NetBSD and the OpenBSD projects, among others. Netherlands rsync://ftp.nl.FreeBSD.org/ Available collections: vol/4/freebsd-core: A full mirror of the FreeBSD FTP server. Russia rsync://cvsup4.ru.FreeBSD.org/ Available collections: FreeBSD-gnats: The GNATS bug-tracking database. Taiwan rsync://ftp.tw.FreeBSD.org/ rsync://ftp2.tw.FreeBSD.org/ rsync://ftp6.tw.FreeBSD.org/ Available collections: FreeBSD: A full mirror of the FreeBSD FTP server. United Kingdom rsync://rsync.mirror.ac.uk/ Available collections: ftp.FreeBSD.org: A full mirror of the FreeBSD FTP server. United States of America rsync://ftp-master.FreeBSD.org/ This server may only be used by FreeBSD primary mirror sites. Available collections: FreeBSD: The master archive of the FreeBSD FTP server. acl: The FreeBSD master ACL list. rsync://ftp13.FreeBSD.org/ Available collections: FreeBSD: A full mirror of the FreeBSD FTP server. diff --git a/en_US.ISO8859-1/books/handbook/network-servers/chapter.sgml b/en_US.ISO8859-1/books/handbook/network-servers/chapter.sgml index 0be2e0509a..27d2db17ae 100644 --- a/en_US.ISO8859-1/books/handbook/network-servers/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/network-servers/chapter.sgml @@ -1,4938 +1,4938 @@ Murray Stokely Reorganized by Network Servers Synopsis This chapter will cover some of the more frequently used network services on &unix; systems. We will cover how to install, configure, test, and maintain many different types of network services. Example configuration files are included throughout this chapter for you to benefit from. After reading this chapter, you will know: How to manage the inetd daemon. How to set up a network file system. How to set up a network information server for sharing user accounts. How to set up automatic network settings using DHCP. How to set up a domain name server. How to set up the Apache HTTP Server. How to set up a File Transfer Protocol (FTP) Server. How to set up a file and print server for &windows; clients using Samba. How to synchronize the time and date, and set up a time server, with the NTP protocol. Before reading this chapter, you should: Understand the basics of the /etc/rc scripts. Be familiar with basic network terminology. Know how to install additional third-party software (). Chern Lee Contributed by Updated for &os; 6.1-RELEASE by The &os; Documentation Project The <application>inetd</application> <quote>Super-Server</quote> Overview &man.inetd.8; is sometimes referred to as the Internet Super-Server because it manages connections for several services. When a connection is received by inetd, it determines which program the connection is destined for, spawns the particular process and delegates the socket to it (the program is invoked with the service socket as its standard input, output and error descriptors). Running inetd for servers that are not heavily used can reduce the overall system load, when compared to running each daemon individually in stand-alone mode. Primarily, inetd is used to spawn other daemons, but several trivial protocols are handled directly, such as chargen, auth, and daytime. This section will cover the basics in configuring inetd through its command-line options and its configuration file, /etc/inetd.conf. Settings inetd is initialized through the &man.rc.8; system. The inetd_enable option is set to NO by default, but may be turned on by sysinstall during installation, depending on the configuration chosen by the user. Placing: inetd_enable="YES" or inetd_enable="NO" into /etc/rc.conf will enable or disable inetd starting at boot time. The command: &prompt.root; /etc/rc.d/inetd rcvar can be run to display the current effective setting. Additionally, different command-line options can be passed to inetd via the inetd_flags option. Command-Line Options Like most server daemons, inetd has a number of options that it can be passed in order to modify its behaviour. The full list of options reads: inetd Options can be passed to inetd using the inetd_flags option in /etc/rc.conf. By default, inetd_flags is set to -wW -C 60, which turns on TCP wrapping for inetd's services, and prevents any single IP address from requesting any service more than 60 times in any given minute. Novice users may be pleased to note that these parameters usually do not need to be modified, although we mention the rate-limiting options below as they be useful should you find that you are receiving an excessive amount of connections. A full list of options can be found in the &man.inetd.8; manual. -c maximum Specify the default maximum number of simultaneous invocations of each service; the default is unlimited. May be overridden on a per-service basis with the parameter. -C rate Specify the default maximum number of times a service can be invoked from a single IP address in one minute; the default is unlimited. May be overridden on a per-service basis with the parameter. -R rate Specify the maximum number of times a service can be invoked in one minute; the default is 256. A rate of 0 allows an unlimited number of invocations. -s maximum Specify the maximum number of times a service can be invoked from a single IP address at any one time; the default is unlimited. May be overridden on a per-service basis with the parameter. <filename>inetd.conf</filename> Configuration of inetd is done via the file /etc/inetd.conf. When a modification is made to /etc/inetd.conf, inetd can be forced to re-read its configuration file by running the command: Reloading the <application>inetd</application> configuration file &prompt.root; /etc/rc.d/inetd reload Each line of the configuration file specifies an individual daemon. Comments in the file are preceded by a #. The format of each entry in /etc/inetd.conf is as follows: service-name socket-type protocol {wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]] user[:group][/login-class] server-program server-program-arguments An example entry for the &man.ftpd.8; daemon using IPv4 might read: ftp stream tcp nowait root /usr/libexec/ftpd ftpd -l service-name This is the service name of the particular daemon. It must correspond to a service listed in /etc/services. This determines which port inetd must listen to. If a new service is being created, it must be placed in /etc/services first. socket-type Either stream, dgram, raw, or seqpacket. stream must be used for connection-based, TCP daemons, while dgram is used for daemons utilizing the UDP transport protocol. protocol One of the following: Protocol Explanation tcp, tcp4 TCP IPv4 udp, udp4 UDP IPv4 tcp6 TCP IPv6 udp6 UDP IPv6 tcp46 Both TCP IPv4 and v6 udp46 Both UDP IPv4 and v6 {wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]] indicates whether the daemon invoked from inetd is able to handle its own socket or not. socket types must use the option, while stream socket daemons, which are usually multi-threaded, should use . usually hands off multiple sockets to a single daemon, while spawns a child daemon for each new socket. The maximum number of child daemons inetd may spawn can be set using the option. If a limit of ten instances of a particular daemon is needed, a /10 would be placed after . Specifying /0 allows an unlimited number of children In addition to , two other options which limit the maximum connections from a single place to a particular daemon can be enabled. limits the number of connections from any particular IP address per minutes, e.g. a value of ten would limit any particular IP address connecting to a particular service to ten attempts per minute. limits the number of children that can be started on behalf on any single IP address at any moment. These options are useful to prevent intentional or unintentional excessive resource consumption and Denial of Service (DoS) attacks to a machine. In this field, either of or is mandatory. , and are optional. A stream-type multi-threaded daemon without any , or limits would simply be: nowait. The same daemon with a maximum limit of ten daemons would read: nowait/10. The same setup with a limit of twenty connections per IP address per minute and a maximum total limit of ten child daemons would read: nowait/10/20. These options are utilized by the default settings of the &man.fingerd.8; daemon, as seen here: finger stream tcp nowait/3/10 nobody /usr/libexec/fingerd fingerd -s Finally, an example of this field with a maximum of 100 children in total, with a maximum of 5 for any one IP address would read: nowait/100/0/5. user This is the username that the particular daemon should run as. Most commonly, daemons run as the root user. For security purposes, it is common to find some servers running as the daemon user, or the least privileged nobody user. server-program The full path of the daemon to be executed when a connection is received. If the daemon is a service provided by inetd internally, then should be used. server-program-arguments This works in conjunction with by specifying the arguments, starting with argv[0], passed to the daemon on invocation. If mydaemon -d is the command line, mydaemon -d would be the value of . Again, if the daemon is an internal service, use here. Security Depending on the choices made at install time, many of inetd's services may be enabled by default. If there is no apparent need for a particular daemon, consider disabling it. Place a # in front of the daemon in question in /etc/inetd.conf, and then reload the inetd configuration. Some daemons, such as fingerd, may not be desired at all because they provide information that may be useful to an attacker. Some daemons are not security-conscious and have long, or non-existent, timeouts for connection attempts. This allows an attacker to slowly send connections to a particular daemon, thus saturating available resources. It may be a good idea to place , or limitations on certain daemons if you find that you have too many connections. By default, TCP wrapping is turned on. Consult the &man.hosts.access.5; manual page for more information on placing TCP restrictions on various inetd invoked daemons. Miscellaneous daytime, time, echo, discard, chargen, and auth are all internally provided services of inetd. The auth service provides identity network services, and is configurable to a certain degree, whilst the others are simply on or off. Consult the &man.inetd.8; manual page for more in-depth information. Tom Rhodes Reorganized and enhanced by Bill Swingle Written by Network File System (NFS) NFS Among the many different file systems that FreeBSD supports is the Network File System, also known as NFS. NFS allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files. Some of the most notable benefits that NFS can provide are: Local workstations use less disk space because commonly used data can be stored on a single machine and still remain accessible to others over the network. There is no need for users to have separate home directories on every network machine. Home directories could be set up on the NFS server and made available throughout the network. Storage devices such as floppy disks, CDROM drives, and &iomegazip; drives can be used by other machines on the network. This may reduce the number of removable media drives throughout the network. How <acronym>NFS</acronym> Works NFS consists of at least two main parts: a server and one or more clients. The client remotely accesses the data that is stored on the server machine. In order for this to function properly a few processes have to be configured and running. The server has to be running the following daemons: NFS server file server UNIX clients rpcbind mountd nfsd Daemon Description nfsd The NFS daemon which services requests from the NFS clients. mountd The NFS mount daemon which carries out the requests that &man.nfsd.8; passes on to it. rpcbind This daemon allows NFS clients to discover which port the NFS server is using. The client can also run a daemon, known as nfsiod. The nfsiod daemon services the requests from the NFS server. This is optional, and improves performance, but is not required for normal and correct operation. See the &man.nfsiod.8; manual page for more information. Configuring <acronym>NFS</acronym> NFS configuration NFS configuration is a relatively straightforward process. The processes that need to be running can all start at boot time with a few modifications to your /etc/rc.conf file. On the NFS server, make sure that the following options are configured in the /etc/rc.conf file: rpcbind_enable="YES" nfs_server_enable="YES" mountd_flags="-r" mountd runs automatically whenever the NFS server is enabled. On the client, make sure this option is present in /etc/rc.conf: nfs_client_enable="YES" The /etc/exports file specifies which file systems NFS should export (sometimes referred to as share). Each line in /etc/exports specifies a file system to be exported and which machines have access to that file system. Along with what machines have access to that file system, access options may also be specified. There are many such options that can be used in this file but only a few will be mentioned here. You can easily discover other options by reading over the &man.exports.5; manual page. Here are a few example /etc/exports entries: NFS export examples The following examples give an idea of how to export file systems, although the settings may be different depending on your environment and network configuration. For instance, to export the /cdrom directory to three example machines that have the same domain name as the server (hence the lack of a domain name for each) or have entries in your /etc/hosts file. The flag makes the exported file system read-only. With this flag, the remote system will not be able to write any changes to the exported file system. /cdrom -ro host1 host2 host3 The following line exports /home to three hosts by IP address. This is a useful setup if you have a private network without a DNS server configured. Optionally the /etc/hosts file could be configured for internal hostnames; please review &man.hosts.5; for more information. The flag allows the subdirectories to be mount points. In other words, it will not mount the subdirectories but permit the client to mount only the directories that are required or needed. /home -alldirs 10.0.0.2 10.0.0.3 10.0.0.4 The following line exports /a so that two clients from different domains may access the file system. The flag allows the root user on the remote system to write data on the exported file system as root. If the -maproot=root flag is not specified, then even if a user has root access on the remote system, he will not be able to modify files on the exported file system. /a -maproot=root host.example.com box.example.org In order for a client to access an exported file system, the client must have permission to do so. Make sure the client is listed in your /etc/exports file. In /etc/exports, each line represents the export information for one file system to one host. A remote host can only be specified once per file system, and may only have one default entry. For example, assume that /usr is a single file system. The following /etc/exports would be invalid: # Invalid when /usr is one file system /usr/src client /usr/ports client One file system, /usr, has two lines specifying exports to the same host, client. The correct format for this situation is: /usr/src /usr/ports client The properties of one file system exported to a given host must all occur on one line. Lines without a client specified are treated as a single host. This limits how you can export file systems, but for most people this is not an issue. The following is an example of a valid export list, where /usr and /exports are local file systems: # Export src and ports to client01 and client02, but only # client01 has root privileges on it /usr/src /usr/ports -maproot=root client01 /usr/src /usr/ports client02 # The client machines have root and can mount anywhere # on /exports. Anyone in the world can mount /exports/obj read-only /exports -alldirs -maproot=root client01 client02 /exports/obj -ro The mountd daemon must be forced to recheck the /etc/exports file whenever it has been modified, so the changes can take effect. This can be accomplished either by sending a HUP signal to the running daemon: &prompt.root; kill -HUP `cat /var/run/mountd.pid` or by invoking the mountd &man.rc.8; script with the appropriate parameter: &prompt.root; /etc/rc.d/mountd onereload Please refer to for more information about using rc scripts. Alternatively, a reboot will make FreeBSD set everything up properly. A reboot is not necessary though. Executing the following commands as root should start everything up. On the NFS server: &prompt.root; rpcbind &prompt.root; nfsd -u -t -n 4 &prompt.root; mountd -r On the NFS client: &prompt.root; nfsiod -n 4 Now everything should be ready to actually mount a remote file system. In these examples the server's name will be server and the client's name will be client. If you only want to temporarily mount a remote file system or would rather test the configuration, just execute a command like this as root on the client: NFS mounting &prompt.root; mount server:/home /mnt This will mount the /home directory on the server at /mnt on the client. If everything is set up correctly you should be able to enter /mnt on the client and see all the files that are on the server. If you want to automatically mount a remote file system each time the computer boots, add the file system to the /etc/fstab file. Here is an example: server:/home /mnt nfs rw 0 0 The &man.fstab.5; manual page lists all the available options. Locking Some applications (e.g. mutt) require file locking to operate correctly. In the case of NFS, rpc.lockd can be used for file locking. To enable it, add the following to the /etc/rc.conf file on both client and server (it is assumed that the NFS client and server are configured already): rpc_lockd_enable="YES" rpc_statd_enable="YES" Start the application by using: &prompt.root; /etc/rc.d/lockd start &prompt.root; /etc/rc.d/statd start If real locking between the NFS clients and NFS server is not required, it is possible to let the NFS client do locking locally by passing to &man.mount.nfs.8;. Refer to the &man.mount.nfs.8; manual page for further details. Practical Uses NFS has many practical uses. Some of the more common ones are listed below: NFS uses Set several machines to share a CDROM or other media among them. This is cheaper and often a more convenient method to install software on multiple machines. On large networks, it might be more convenient to configure a central NFS server in which to store all the user home directories. These home directories can then be exported to the network so that users would always have the same home directory, regardless of which workstation they log in to. Several machines could have a common /usr/ports/distfiles directory. That way, when you need to install a port on several machines, you can quickly access the source without downloading it on each machine. Wylie Stilwell Contributed by Chern Lee Rewritten by Automatic Mounts with <application>amd</application> amd automatic mounter daemon &man.amd.8; (the automatic mounter daemon) automatically mounts a remote file system whenever a file or directory within that file system is accessed. Filesystems that are inactive for a period of time will also be automatically unmounted by amd. Using amd provides a simple alternative to permanent mounts, as permanent mounts are usually listed in /etc/fstab. amd operates by attaching itself as an NFS server to the /host and /net directories. When a file is accessed within one of these directories, amd looks up the corresponding remote mount and automatically mounts it. /net is used to mount an exported file system from an IP address, while /host is used to mount an export from a remote hostname. An access to a file within /host/foobar/usr would tell amd to attempt to mount the /usr export on the host foobar. Mounting an Export with <application>amd</application> You can view the available mounts of a remote host with the showmount command. For example, to view the mounts of a host named foobar, you can use: &prompt.user; showmount -e foobar Exports list on foobar: /usr 10.10.10.0 /a 10.10.10.0 &prompt.user; cd /host/foobar/usr As seen in the example, the showmount shows /usr as an export. When changing directories to /host/foobar/usr, amd attempts to resolve the hostname foobar and automatically mount the desired export. amd can be started by the startup scripts by placing the following lines in /etc/rc.conf: amd_enable="YES" Additionally, custom flags can be passed to amd from the amd_flags option. By default, amd_flags is set to: amd_flags="-a /.amd_mnt -l syslog /host /etc/amd.map /net /etc/amd.map" The /etc/amd.map file defines the default options that exports are mounted with. The /etc/amd.conf file defines some of the more advanced features of amd. Consult the &man.amd.8; and &man.amd.conf.5; manual pages for more information. John Lind Contributed by Problems Integrating with Other Systems Certain Ethernet adapters for ISA PC systems have limitations which can lead to serious network problems, particularly with NFS. This difficulty is not specific to FreeBSD, but FreeBSD systems are affected by it. The problem nearly always occurs when (FreeBSD) PC systems are networked with high-performance workstations, such as those made by Silicon Graphics, Inc., and Sun Microsystems, Inc. The NFS mount will work fine, and some operations may succeed, but suddenly the server will seem to become unresponsive to the client, even though requests to and from other systems continue to be processed. This happens to the client system, whether the client is the FreeBSD system or the workstation. On many systems, there is no way to shut down the client gracefully once this problem has manifested itself. The only solution is often to reset the client, because the NFS situation cannot be resolved. Though the correct solution is to get a higher performance and capacity Ethernet adapter for the FreeBSD system, there is a simple workaround that will allow satisfactory operation. If the FreeBSD system is the server, include the option on the mount from the client. If the FreeBSD system is the client, then mount the NFS file system with the option . These options may be specified using the fourth field of the fstab entry on the client for automatic mounts, or by using the parameter of the &man.mount.8; command for manual mounts. It should be noted that there is a different problem, sometimes mistaken for this one, when the NFS servers and clients are on different networks. If that is the case, make certain that your routers are routing the necessary UDP information, or you will not get anywhere, no matter what else you are doing. In the following examples, fastws is the host (interface) name of a high-performance workstation, and freebox is the host (interface) name of a FreeBSD system with a lower-performance Ethernet adapter. Also, /sharedfs will be the exported NFS file system (see &man.exports.5;), and /project will be the mount point on the client for the exported file system. In all cases, note that additional options, such as or and may be desirable in your application. Examples for the FreeBSD system (freebox) as the client in /etc/fstab on freebox: fastws:/sharedfs /project nfs rw,-r=1024 0 0 As a manual mount command on freebox: &prompt.root; mount -t nfs -o -r=1024 fastws:/sharedfs /project Examples for the FreeBSD system as the server in /etc/fstab on fastws: freebox:/sharedfs /project nfs rw,-w=1024 0 0 As a manual mount command on fastws: &prompt.root; mount -t nfs -o -w=1024 freebox:/sharedfs /project Nearly any 16-bit Ethernet adapter will allow operation without the above restrictions on the read or write size. For anyone who cares, here is what happens when the failure occurs, which also explains why it is unrecoverable. NFS typically works with a block size of 8 K (though it may do fragments of smaller sizes). Since the maximum Ethernet packet is around 1500 bytes, the NFS block gets split into multiple Ethernet packets, even though it is still a single unit to the upper-level code, and must be received, assembled, and acknowledged as a unit. The high-performance workstations can pump out the packets which comprise the NFS unit one right after the other, just as close together as the standard allows. On the smaller, lower capacity cards, the later packets overrun the earlier packets of the same unit before they can be transferred to the host and the unit as a whole cannot be reconstructed or acknowledged. As a result, the workstation will time out and try again, but it will try again with the entire 8 K unit, and the process will be repeated, ad infinitum. By keeping the unit size below the Ethernet packet size limitation, we ensure that any complete Ethernet packet received can be acknowledged individually, avoiding the deadlock situation. Overruns may still occur when a high-performance workstations is slamming data out to a PC system, but with the better cards, such overruns are not guaranteed on NFS units. When an overrun occurs, the units affected will be retransmitted, and there will be a fair chance that they will be received, assembled, and acknowledged. Bill Swingle Written by Eric Ogren Enhanced by Udo Erdelhoff Network Information System (NIS/YP) What Is It? NIS Solaris HP-UX AIX Linux NetBSD OpenBSD NIS, which stands for Network Information Services, was developed by Sun Microsystems to centralize administration of &unix; (originally &sunos;) systems. It has now essentially become an industry standard; all major &unix; like systems (&solaris;, HP-UX, &aix;, Linux, NetBSD, OpenBSD, FreeBSD, etc) support NIS. yellow pagesNIS NIS was formerly known as Yellow Pages, but because of trademark issues, Sun changed the name. The old term (and yp) is still often seen and used. NIS domains It is a RPC-based client/server system that allows a group of machines within an NIS domain to share a common set of configuration files. This permits a system administrator to set up NIS client systems with only minimal configuration data and add, remove or modify configuration data from a single location. Windows NT It is similar to the &windowsnt; domain system; although the internal implementation of the two are not at all similar, the basic functionality can be compared. Terms/Processes You Should Know There are several terms and several important user processes that you will come across when attempting to implement NIS on FreeBSD, whether you are trying to create an NIS server or act as an NIS client: rpcbind portmap Term Description NIS domainname An NIS master server and all of its clients (including its slave servers) have a NIS domainname. Similar to an &windowsnt; domain name, the NIS domainname does not have anything to do with DNS. rpcbind Must be running in order to enable RPC (Remote Procedure Call, a network protocol used by NIS). If rpcbind is not running, it will be impossible to run an NIS server, or to act as an NIS client. ypbind Binds an NIS client to its NIS server. It will take the NIS domainname from the system, and using RPC, connect to the server. ypbind is the core of client-server communication in an NIS environment; if ypbind dies on a client machine, it will not be able to access the NIS server. ypserv Should only be running on NIS servers; this is the NIS server process itself. If &man.ypserv.8; dies, then the server will no longer be able to respond to NIS requests (hopefully, there is a slave server to take over for it). There are some implementations of NIS (but not the FreeBSD one), that do not try to reconnect to another server if the server it used before dies. Often, the only thing that helps in this case is to restart the server process (or even the whole server) or the ypbind process on the client. rpc.yppasswdd Another process that should only be running on NIS master servers; this is a daemon that will allow NIS clients to change their NIS passwords. If this daemon is not running, users will have to login to the NIS master server and change their passwords there. How Does It Work? There are three types of hosts in an NIS environment: master servers, slave servers, and clients. Servers act as a central repository for host configuration information. Master servers hold the authoritative copy of this information, while slave servers mirror this information for redundancy. Clients rely on the servers to provide this information to them. Information in many files can be shared in this manner. The master.passwd, group, and hosts files are commonly shared via NIS. Whenever a process on a client needs information that would normally be found in these files locally, it makes a query to the NIS server that it is bound to instead. Machine Types NIS master server A NIS master server. This server, analogous to a &windowsnt; primary domain controller, maintains the files used by all of the NIS clients. The passwd, group, and other various files used by the NIS clients live on the master server. It is possible for one machine to be an NIS master server for more than one NIS domain. However, this will not be covered in this introduction, which assumes a relatively small-scale NIS environment. NIS slave server NIS slave servers. Similar to the &windowsnt; backup domain controllers, NIS slave servers maintain copies of the NIS master's data files. NIS slave servers provide the redundancy, which is needed in important environments. They also help to balance the load of the master server: NIS Clients always attach to the NIS server whose response they get first, and this includes slave-server-replies. NIS client NIS clients. NIS clients, like most &windowsnt; workstations, authenticate against the NIS server (or the &windowsnt; domain controller in the &windowsnt; workstations case) to log on. Using NIS/YP This section will deal with setting up a sample NIS environment. Planning Let us assume that you are the administrator of a small university lab. This lab, which consists of 15 FreeBSD machines, currently has no centralized point of administration; each machine has its own /etc/passwd and /etc/master.passwd. These files are kept in sync with each other only through manual intervention; currently, when you add a user to the lab, you must run adduser on all 15 machines. Clearly, this has to change, so you have decided to convert the lab to use NIS, using two of the machines as servers. Therefore, the configuration of the lab now looks something like: Machine name IP address Machine role ellington 10.0.0.2 NIS master coltrane 10.0.0.3 NIS slave basie 10.0.0.4 Faculty workstation bird 10.0.0.5 Client machine cli[1-11] 10.0.0.[6-17] Other client machines If you are setting up a NIS scheme for the first time, it is a good idea to think through how you want to go about it. No matter what the size of your network, there are a few decisions that need to be made. Choosing a NIS Domain Name NIS domainname This might not be the domainname that you are used to. It is more accurately called the NIS domainname. When a client broadcasts its requests for info, it includes the name of the NIS domain that it is part of. This is how multiple servers on one network can tell which server should answer which request. Think of the NIS domainname as the name for a group of hosts that are related in some way. Some organizations choose to use their Internet domainname for their NIS domainname. This is not recommended as it can cause confusion when trying to debug network problems. The NIS domainname should be unique within your network and it is helpful if it describes the group of machines it represents. For example, the Art department at Acme Inc. might be in the acme-art NIS domain. For this example, assume you have chosen the name test-domain. SunOS However, some operating systems (notably &sunos;) use their NIS domain name as their Internet domain name. If one or more machines on your network have this restriction, you must use the Internet domain name as your NIS domain name. Physical Server Requirements There are several things to keep in mind when choosing a machine to use as a NIS server. One of the unfortunate things about NIS is the level of dependency the clients have on the server. If a client cannot contact the server for its NIS domain, very often the machine becomes unusable. The lack of user and group information causes most systems to temporarily freeze up. With this in mind you should make sure to choose a machine that will not be prone to being rebooted regularly, or one that might be used for development. The NIS server should ideally be a stand alone machine whose sole purpose in life is to be an NIS server. If you have a network that is not very heavily used, it is acceptable to put the NIS server on a machine running other services, just keep in mind that if the NIS server becomes unavailable, it will affect all of your NIS clients adversely. NIS Servers The canonical copies of all NIS information are stored on a single machine called the NIS master server. The databases used to store the information are called NIS maps. In FreeBSD, these maps are stored in /var/yp/[domainname] where [domainname] is the name of the NIS domain being served. A single NIS server can support several domains at once, therefore it is possible to have several such directories, one for each supported domain. Each domain will have its own independent set of maps. NIS master and slave servers handle all NIS requests with the ypserv daemon. ypserv is responsible for receiving incoming requests from NIS clients, translating the requested domain and map name to a path to the corresponding database file and transmitting data from the database back to the client. Setting Up a NIS Master Server NIS server configuration Setting up a master NIS server can be relatively straight forward, depending on your needs. FreeBSD comes with support for NIS out-of-the-box. All you need is to add the following lines to /etc/rc.conf, and FreeBSD will do the rest for you. nisdomainname="test-domain" This line will set the NIS domainname to test-domain upon network setup (e.g. after reboot). nis_server_enable="YES" This will tell FreeBSD to start up the NIS server processes when the networking is next brought up. nis_yppasswdd_enable="YES" This will enable the rpc.yppasswdd daemon which, as mentioned above, will allow users to change their NIS password from a client machine. Depending on your NIS setup, you may need to add further entries. See the section about NIS servers that are also NIS clients, below, for details. Now, all you have to do is to run the command /etc/netstart as superuser. It will set up everything for you, using the values you defined in /etc/rc.conf. Initializing the NIS Maps NIS maps The NIS maps are database files, that are kept in the /var/yp directory. They are generated from configuration files in the /etc directory of the NIS master, with one exception: the /etc/master.passwd file. This is for a good reason, you do not want to propagate passwords to your root and other administrative accounts to all the servers in the NIS domain. Therefore, before we initialize the NIS maps, you should: &prompt.root; cp /etc/master.passwd /var/yp/master.passwd &prompt.root; cd /var/yp &prompt.root; vi master.passwd You should remove all entries regarding system accounts (bin, tty, kmem, games, etc), as well as any accounts that you do not want to be propagated to the NIS clients (for example root and any other UID 0 (superuser) accounts). Make sure the /var/yp/master.passwd is neither group nor world readable (mode 600)! Use the chmod command, if appropriate. Tru64 UNIX When you have finished, it is time to initialize the NIS maps! FreeBSD includes a script named ypinit to do this for you (see its manual page for more information). Note that this script is available on most &unix; Operating Systems, but not on all. On Digital UNIX/Compaq Tru64 UNIX it is called ypsetup. Because we are generating maps for an NIS master, we are going to pass the option to ypinit. To generate the NIS maps, assuming you already performed the steps above, run: ellington&prompt.root; ypinit -m test-domain Server Type: MASTER Domain: test-domain Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If you don't, something might not work. At this point, we have to construct a list of this domains YP servers. rod.darktech.org is already known as master server. Please continue to add any slave servers, one per line. When you are done with the list, type a <control D>. master server : ellington next host to add: coltrane next host to add: ^D The current list of NIS servers looks like this: ellington coltrane Is this correct? [y/n: y] y [..output from map generation..] NIS Map update completed. ellington has been setup as an YP master server without any errors. ypinit should have created /var/yp/Makefile from /var/yp/Makefile.dist. When created, this file assumes that you are operating in a single server NIS environment with only FreeBSD machines. Since test-domain has a slave server as well, you must edit /var/yp/Makefile: ellington&prompt.root; vi /var/yp/Makefile You should comment out the line that says NOPUSH = "True" (if it is not commented out already). Setting up a NIS Slave Server NIS slave server Setting up an NIS slave server is even more simple than setting up the master. Log on to the slave server and edit the file /etc/rc.conf as you did before. The only difference is that we now must use the option when running ypinit. The option requires the name of the NIS master be passed to it as well, so our command line looks like: coltrane&prompt.root; ypinit -s ellington test-domain Server Type: SLAVE Domain: test-domain Master: ellington Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If you don't, something might not work. There will be no further questions. The remainder of the procedure should take a few minutes, to copy the databases from ellington. Transferring netgroup... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byuser... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byhost... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring group.bygid... ypxfr: Exiting: Map successfully transferred Transferring group.byname... ypxfr: Exiting: Map successfully transferred Transferring services.byname... ypxfr: Exiting: Map successfully transferred Transferring rpc.bynumber... ypxfr: Exiting: Map successfully transferred Transferring rpc.byname... ypxfr: Exiting: Map successfully transferred Transferring protocols.byname... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byaddr... ypxfr: Exiting: Map successfully transferred Transferring netid.byname... ypxfr: Exiting: Map successfully transferred Transferring hosts.byaddr... ypxfr: Exiting: Map successfully transferred Transferring protocols.bynumber... ypxfr: Exiting: Map successfully transferred Transferring ypservers... ypxfr: Exiting: Map successfully transferred Transferring hosts.byname... ypxfr: Exiting: Map successfully transferred coltrane has been setup as an YP slave server without any errors. Don't forget to update map ypservers on ellington. You should now have a directory called /var/yp/test-domain. Copies of the NIS master server's maps should be in this directory. You will need to make sure that these stay updated. The following /etc/crontab entries on your slave servers should do the job: 20 * * * * root /usr/libexec/ypxfr passwd.byname 21 * * * * root /usr/libexec/ypxfr passwd.byuid These two lines force the slave to sync its maps with the maps on the master server. Although these entries are not mandatory, since the master server attempts to ensure any changes to its NIS maps are communicated to its slaves and because password information is vital to systems depending on the server, it is a good idea to force the updates. This is more important on busy networks where map updates might not always complete. Now, run the command /etc/netstart on the slave server as well, which again starts the NIS server. NIS Clients An NIS client establishes what is called a binding to a particular NIS server using the ypbind daemon. ypbind checks the system's default domain (as set by the domainname command), and begins broadcasting RPC requests on the local network. These requests specify the name of the domain for which ypbind is attempting to establish a binding. If a server that has been configured to serve the requested domain receives one of the broadcasts, it will respond to ypbind, which will record the server's address. If there are several servers available (a master and several slaves, for example), ypbind will use the address of the first one to respond. From that point on, the client system will direct all of its NIS requests to that server. ypbind will occasionally ping the server to make sure it is still up and running. If it fails to receive a reply to one of its pings within a reasonable amount of time, ypbind will mark the domain as unbound and begin broadcasting again in the hopes of locating another server. Setting Up a NIS Client NIS client configuration Setting up a FreeBSD machine to be a NIS client is fairly straightforward. Edit the file /etc/rc.conf and add the following lines in order to set the NIS domainname and start ypbind upon network startup: nisdomainname="test-domain" nis_client_enable="YES" To import all possible password entries from the NIS server, remove all user accounts from your /etc/master.passwd file and use vipw to add the following line to the end of the file: +::::::::: This line will afford anyone with a valid account in the NIS server's password maps an account. There are many ways to configure your NIS client by changing this line. See the netgroups section below for more information. For more detailed reading see O'Reilly's book on Managing NFS and NIS. You should keep at least one local account (i.e. not imported via NIS) in your /etc/master.passwd and this account should also be a member of the group wheel. If there is something wrong with NIS, this account can be used to log in remotely, become root, and fix things. To import all possible group entries from the NIS server, add this line to your /etc/group file: +:*:: After completing these steps, you should be able to run ypcat passwd and see the NIS server's passwd map. NIS Security In general, any remote user can issue an RPC to &man.ypserv.8; and retrieve the contents of your NIS maps, provided the remote user knows your domainname. To prevent such unauthorized transactions, &man.ypserv.8; supports a feature called securenets which can be used to restrict access to a given set of hosts. At startup, &man.ypserv.8; will attempt to load the securenets information from a file called /var/yp/securenets. This path varies depending on the path specified with the option. This file contains entries that consist of a network specification and a network mask separated by white space. Lines starting with # are considered to be comments. A sample securenets file might look like this: # allow connections from local host -- mandatory 127.0.0.1 255.255.255.255 # allow connections from any host # on the 192.168.128.0 network 192.168.128.0 255.255.255.0 # allow connections from any host # between 10.0.0.0 to 10.0.15.255 # this includes the machines in the testlab 10.0.0.0 255.255.240.0 If &man.ypserv.8; receives a request from an address that matches one of these rules, it will process the request normally. If the address fails to match a rule, the request will be ignored and a warning message will be logged. If the /var/yp/securenets file does not exist, ypserv will allow connections from any host. The ypserv program also has support for Wietse Venema's TCP Wrapper package. This allows the administrator to use the TCP Wrapper configuration files for access control instead of /var/yp/securenets. While both of these access control mechanisms provide some security, they, like the privileged port test, are vulnerable to IP spoofing attacks. All NIS-related traffic should be blocked at your firewall. Servers using /var/yp/securenets may fail to serve legitimate NIS clients with archaic TCP/IP implementations. Some of these implementations set all host bits to zero when doing broadcasts and/or fail to observe the subnet mask when calculating the broadcast address. While some of these problems can be fixed by changing the client configuration, other problems may force the retirement of the client systems in question or the abandonment of /var/yp/securenets. Using /var/yp/securenets on a server with such an archaic implementation of TCP/IP is a really bad idea and will lead to loss of NIS functionality for large parts of your network. TCP Wrappers The use of the TCP Wrapper package increases the latency of your NIS server. The additional delay may be long enough to cause timeouts in client programs, especially in busy networks or with slow NIS servers. If one or more of your client systems suffers from these symptoms, you should convert the client systems in question into NIS slave servers and force them to bind to themselves. Barring Some Users from Logging On In our lab, there is a machine basie that is supposed to be a faculty only workstation. We do not want to take this machine out of the NIS domain, yet the passwd file on the master NIS server contains accounts for both faculty and students. What can we do? There is a way to bar specific users from logging on to a machine, even if they are present in the NIS database. To do this, all you must do is add -username to the end of the /etc/master.passwd file on the client machine, where username is the username of the user you wish to bar from logging in. This should preferably be done using vipw, since vipw will sanity check your changes to /etc/master.passwd, as well as automatically rebuild the password database when you finish editing. For example, if we wanted to bar user bill from logging on to basie we would: basie&prompt.root; vipw [add -bill to the end, exit] vipw: rebuilding the database... vipw: done basie&prompt.root; cat /etc/master.passwd root:[password]:0:0::0:0:The super-user:/root:/bin/csh toor:[password]:0:0::0:0:The other super-user:/root:/bin/sh daemon:*:1:1::0:0:Owner of many system processes:/root:/sbin/nologin operator:*:2:5::0:0:System &:/:/sbin/nologin bin:*:3:7::0:0:Binaries Commands and Source,,,:/:/sbin/nologin tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin games:*:7:13::0:0:Games pseudo-user:/usr/games:/sbin/nologin news:*:8:8::0:0:News Subsystem:/:/sbin/nologin man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico xten:*:67:67::0:0:X-10 daemon:/usr/local/xten:/sbin/nologin pop:*:68:6::0:0:Post Office Owner:/nonexistent:/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/sbin/nologin +::::::::: -bill basie&prompt.root; Udo Erdelhoff Contributed by Using Netgroups netgroups The method shown in the previous section works reasonably well if you need special rules for a very small number of users and/or machines. On larger networks, you will forget to bar some users from logging onto sensitive machines, or you may even have to modify each machine separately, thus losing the main benefit of NIS: centralized administration. The NIS developers' solution for this problem is called netgroups. Their purpose and semantics can be compared to the normal groups used by &unix; file systems. The main differences are the lack of a numeric ID and the ability to define a netgroup by including both user accounts and other netgroups. Netgroups were developed to handle large, complex networks with hundreds of users and machines. On one hand, this is a Good Thing if you are forced to deal with such a situation. On the other hand, this complexity makes it almost impossible to explain netgroups with really simple examples. The example used in the remainder of this section demonstrates this problem. Let us assume that your successful introduction of NIS in your laboratory caught your superiors' interest. Your next job is to extend your NIS domain to cover some of the other machines on campus. The two tables contain the names of the new users and new machines as well as brief descriptions of them. User Name(s) Description alpha, beta Normal employees of the IT department charlie, delta The new apprentices of the IT department echo, foxtrott, golf, ... Ordinary employees able, baker, ... The current interns Machine Name(s) Description war, death, famine, pollution Your most important servers. Only the IT employees are allowed to log onto these machines. pride, greed, envy, wrath, lust, sloth Less important servers. All members of the IT department are allowed to login onto these machines. one, two, three, four, ... Ordinary workstations. Only the real employees are allowed to use these machines. trashcan A very old machine without any critical data. Even the intern is allowed to use this box. If you tried to implement these restrictions by separately blocking each user, you would have to add one -user line to each system's passwd for each user who is not allowed to login onto that system. If you forget just one entry, you could be in trouble. It may be feasible to do this correctly during the initial setup, however you will eventually forget to add the lines for new users during day-to-day operations. After all, Murphy was an optimist. Handling this situation with netgroups offers several advantages. Each user need not be handled separately; you assign a user to one or more netgroups and allow or forbid logins for all members of the netgroup. If you add a new machine, you will only have to define login restrictions for netgroups. If a new user is added, you will only have to add the user to one or more netgroups. Those changes are independent of each other: no more for each combination of user and machine do... If your NIS setup is planned carefully, you will only have to modify exactly one central configuration file to grant or deny access to machines. The first step is the initialization of the NIS map netgroup. FreeBSD's &man.ypinit.8; does not create this map by default, but its NIS implementation will support it once it has been created. To create an empty map, simply type ellington&prompt.root; vi /var/yp/netgroup and start adding content. For our example, we need at least four netgroups: IT employees, IT apprentices, normal employees and interns. IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) USERS (,echo,test-domain) (,foxtrott,test-domain) \ (,golf,test-domain) INTERNS (,able,test-domain) (,baker,test-domain) IT_EMP, IT_APP etc. are the names of the netgroups. Each bracketed group adds one or more user accounts to it. The three fields inside a group are: The name of the host(s) where the following items are valid. If you do not specify a hostname, the entry is valid on all hosts. If you do specify a hostname, you will enter a realm of darkness, horror and utter confusion. The name of the account that belongs to this netgroup. The NIS domain for the account. You can import accounts from other NIS domains into your netgroup if you are one of the unlucky fellows with more than one NIS domain. Each of these fields can contain wildcards. See &man.netgroup.5; for details. netgroups Netgroup names longer than 8 characters should not be used, especially if you have machines running other operating systems within your NIS domain. The names are case sensitive; using capital letters for your netgroup names is an easy way to distinguish between user, machine and netgroup names. Some NIS clients (other than FreeBSD) cannot handle netgroups with a large number of entries. For example, some older versions of &sunos; start to cause trouble if a netgroup contains more than 15 entries. You can circumvent this limit by creating several sub-netgroups with 15 users or less and a real netgroup that consists of the sub-netgroups: BIGGRP1 (,joe1,domain) (,joe2,domain) (,joe3,domain) [...] BIGGRP2 (,joe16,domain) (,joe17,domain) [...] BIGGRP3 (,joe31,domain) (,joe32,domain) BIGGROUP BIGGRP1 BIGGRP2 BIGGRP3 You can repeat this process if you need more than 225 users within a single netgroup. Activating and distributing your new NIS map is easy: ellington&prompt.root; cd /var/yp ellington&prompt.root; make This will generate the three NIS maps netgroup, netgroup.byhost and netgroup.byuser. Use &man.ypcat.1; to check if your new NIS maps are available: ellington&prompt.user; ypcat -k netgroup ellington&prompt.user; ypcat -k netgroup.byhost ellington&prompt.user; ypcat -k netgroup.byuser The output of the first command should resemble the contents of /var/yp/netgroup. The second command will not produce output if you have not specified host-specific netgroups. The third command can be used to get the list of netgroups for a user. The client setup is quite simple. To configure the server war, you only have to start &man.vipw.8; and replace the line +::::::::: with +@IT_EMP::::::::: Now, only the data for the users defined in the netgroup IT_EMP is imported into war's password database and only these users are allowed to login. Unfortunately, this limitation also applies to the ~ function of the shell and all routines converting between user names and numerical user IDs. In other words, cd ~user will not work, ls -l will show the numerical ID instead of the username and find . -user joe -print will fail with No such user. To fix this, you will have to import all user entries without allowing them to login onto your servers. This can be achieved by adding another line to /etc/master.passwd. This line should contain: +:::::::::/sbin/nologin, meaning Import all entries but replace the shell with /sbin/nologin in the imported entries. You can replace any field in the passwd entry by placing a default value in your /etc/master.passwd. Make sure that the line +:::::::::/sbin/nologin is placed after +@IT_EMP:::::::::. Otherwise, all user accounts imported from NIS will have /sbin/nologin as their login shell. After this change, you will only have to change one NIS map if a new employee joins the IT department. You could use a similar approach for the less important servers by replacing the old +::::::::: in their local version of /etc/master.passwd with something like this: +@IT_EMP::::::::: +@IT_APP::::::::: +:::::::::/sbin/nologin The corresponding lines for the normal workstations could be: +@IT_EMP::::::::: +@USERS::::::::: +:::::::::/sbin/nologin And everything would be fine until there is a policy change a few weeks later: The IT department starts hiring interns. The IT interns are allowed to use the normal workstations and the less important servers; and the IT apprentices are allowed to login onto the main servers. You add a new netgroup IT_INTERN, add the new IT interns to this netgroup and start to change the configuration on each and every machine... As the old saying goes: Errors in centralized planning lead to global mess. NIS' ability to create netgroups from other netgroups can be used to prevent situations like these. One possibility is the creation of role-based netgroups. For example, you could create a netgroup called BIGSRV to define the login restrictions for the important servers, another netgroup called SMALLSRV for the less important servers and a third netgroup called USERBOX for the normal workstations. Each of these netgroups contains the netgroups that are allowed to login onto these machines. The new entries for your NIS map netgroup should look like this: BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS This method of defining login restrictions works reasonably well if you can define groups of machines with identical restrictions. Unfortunately, this is the exception and not the rule. Most of the time, you will need the ability to define login restrictions on a per-machine basis. Machine-specific netgroup definitions are the other possibility to deal with the policy change outlined above. In this scenario, the /etc/master.passwd of each box contains two lines starting with +. The first of them adds a netgroup with the accounts allowed to login onto this machine, the second one adds all other accounts with /sbin/nologin as shell. It is a good idea to use the ALL-CAPS version of the machine name as the name of the netgroup. In other words, the lines should look like this: +@BOXNAME::::::::: +:::::::::/sbin/nologin Once you have completed this task for all your machines, you will not have to modify the local versions of /etc/master.passwd ever again. All further changes can be handled by modifying the NIS map. Here is an example of a possible netgroup map for this scenario with some additional goodies: # Define groups of users first IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) DEPT1 (,echo,test-domain) (,foxtrott,test-domain) DEPT2 (,golf,test-domain) (,hotel,test-domain) DEPT3 (,india,test-domain) (,juliet,test-domain) ITINTERN (,kilo,test-domain) (,lima,test-domain) D_INTERNS (,able,test-domain) (,baker,test-domain) # # Now, define some groups based on roles USERS DEPT1 DEPT2 DEPT3 BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS # # And a groups for a special tasks # Allow echo and golf to access our anti-virus-machine SECURITY IT_EMP (,echo,test-domain) (,golf,test-domain) # # machine-based netgroups # Our main servers WAR BIGSRV FAMINE BIGSRV # User india needs access to this server POLLUTION BIGSRV (,india,test-domain) # # This one is really important and needs more access restrictions DEATH IT_EMP # # The anti-virus-machine mentioned above ONE SECURITY # # Restrict a machine to a single user TWO (,hotel,test-domain) # [...more groups to follow] If you are using some kind of database to manage your user accounts, you should be able to create the first part of the map with your database's report tools. This way, new users will automatically have access to the boxes. One last word of caution: It may not always be advisable to use machine-based netgroups. If you are deploying a couple of dozen or even hundreds of identical machines for student labs, you should use role-based netgroups instead of machine-based netgroups to keep the size of the NIS map within reasonable limits. Important Things to Remember There are still a couple of things that you will need to do differently now that you are in an NIS environment. Every time you wish to add a user to the lab, you must add it to the master NIS server only, and you must remember to rebuild the NIS maps. If you forget to do this, the new user will not be able to login anywhere except on the NIS master. For example, if we needed to add a new user jsmith to the lab, we would: &prompt.root; pw useradd jsmith &prompt.root; cd /var/yp &prompt.root; make test-domain You could also run adduser jsmith instead of pw useradd jsmith. Keep the administration accounts out of the NIS maps. You do not want to be propagating administrative accounts and passwords to machines that will have users that should not have access to those accounts. Keep the NIS master and slave secure, and minimize their downtime. If somebody either hacks or simply turns off these machines, they have effectively rendered many people without the ability to login to the lab. This is the chief weakness of any centralized administration system. If you do not protect your NIS servers, you will have a lot of angry users! NIS v1 Compatibility FreeBSD's ypserv has some support for serving NIS v1 clients. FreeBSD's NIS implementation only uses the NIS v2 protocol, however other implementations include support for the v1 protocol for backwards compatibility with older systems. The ypbind daemons supplied with these systems will try to establish a binding to an NIS v1 server even though they may never actually need it (and they may persist in broadcasting in search of one even after they receive a response from a v2 server). Note that while support for normal client calls is provided, this version of ypserv does not handle v1 map transfer requests; consequently, it cannot be used as a master or slave in conjunction with older NIS servers that only support the v1 protocol. Fortunately, there probably are not any such servers still in use today. NIS Servers That Are Also NIS Clients Care must be taken when running ypserv in a multi-server domain where the server machines are also NIS clients. It is generally a good idea to force the servers to bind to themselves rather than allowing them to broadcast bind requests and possibly become bound to each other. Strange failure modes can result if one server goes down and others are dependent upon it. Eventually all the clients will time out and attempt to bind to other servers, but the delay involved can be considerable and the failure mode is still present since the servers might bind to each other all over again. You can force a host to bind to a particular server by running ypbind with the flag. If you do not want to do this manually each time you reboot your NIS server, you can add the following lines to your /etc/rc.conf: nis_client_enable="YES" # run client stuff as well nis_client_flags="-S NIS domain,server" See &man.ypbind.8; for further information. Password Formats NIS password formats One of the most common issues that people run into when trying to implement NIS is password format compatibility. If your NIS server is using DES encrypted passwords, it will only support clients that are also using DES. For example, if you have &solaris; NIS clients in your network, then you will almost certainly need to use DES encrypted passwords. To check which format your servers and clients are using, look at /etc/login.conf. If the host is configured to use DES encrypted passwords, then the default class will contain an entry like this: default:\ :passwd_format=des:\ :copyright=/etc/COPYRIGHT:\ [Further entries elided] Other possible values for the passwd_format capability include blf and md5 (for Blowfish and MD5 encrypted passwords, respectively). If you have made changes to /etc/login.conf, you will also need to rebuild the login capability database, which is achieved by running the following command as root: &prompt.root; cap_mkdb /etc/login.conf The format of passwords already in /etc/master.passwd will not be updated until a user changes his password for the first time after the login capability database is rebuilt. Next, in order to ensure that passwords are encrypted with the format that you have chosen, you should also check that the crypt_default in /etc/auth.conf gives precedence to your chosen password format. To do this, place the format that you have chosen first in the list. For example, when using DES encrypted passwords, the entry would be: crypt_default = des blf md5 Having followed the above steps on each of the &os; based NIS servers and clients, you can be sure that they all agree on which password format is used within your network. If you have trouble authenticating on an NIS client, this is a pretty good place to start looking for possible problems. Remember: if you want to deploy an NIS server for a heterogenous network, you will probably have to use DES on all systems because it is the lowest common standard. Greg Sutter Written by Automatic Network Configuration (DHCP) What Is DHCP? Dynamic Host Configuration Protocol DHCP Internet Software Consortium (ISC) DHCP, the Dynamic Host Configuration Protocol, describes the means by which a system can connect to a network and obtain the necessary information for communication upon that network. FreeBSD versions prior to 6.0 use the ISC (Internet Software Consortium) DHCP client (&man.dhclient.8;) implementation. Later versions use the OpenBSD dhclient taken from OpenBSD 3.7. All information here regarding dhclient is for use with either of the ISC or OpenBSD DHCP clients. The DHCP server is the one included in the ISC distribution. What This Section Covers This section describes both the client-side components of the ISC and OpenBSD DHCP client and server-side components of the ISC DHCP system. The client-side program, dhclient, comes integrated within FreeBSD, and the server-side portion is available from the net/isc-dhcp3-server port. The &man.dhclient.8;, &man.dhcp-options.5;, and &man.dhclient.conf.5; manual pages, in addition to the references below, are useful resources. How It Works UDP When dhclient, the DHCP client, is executed on the client machine, it begins broadcasting requests for configuration information. By default, these requests are on UDP port 68. The server replies on UDP 67, giving the client an IP address and other relevant network information such as netmask, router, and DNS servers. All of this information comes in the form of a DHCP lease and is only valid for a certain time (configured by the DHCP server maintainer). In this manner, stale IP addresses for clients no longer connected to the network can be automatically reclaimed. DHCP clients can obtain a great deal of information from the server. An exhaustive list may be found in &man.dhcp-options.5;. FreeBSD Integration &os; fully integrates the ISC or OpenBSD DHCP client, dhclient (according to the &os; version you run). DHCP client support is provided within both the installer and the base system, obviating the need for detailed knowledge of network configurations on any network that runs a DHCP server. dhclient has been included in all FreeBSD distributions since 3.2. sysinstall DHCP is supported by sysinstall. When configuring a network interface within sysinstall, the second question asked is: Do you want to try DHCP configuration of the interface?. Answering affirmatively will execute dhclient, and if successful, will fill in the network configuration information automatically. There are two things you must do to have your system use DHCP upon startup: DHCP requirements Make sure that the bpf device is compiled into your kernel. To do this, add device bpf to your kernel configuration file, and rebuild the kernel. For more information about building kernels, see . The bpf device is already part of the GENERIC kernel that is supplied with FreeBSD, so if you do not have a custom kernel, you should not need to create one in order to get DHCP working. For those who are particularly security conscious, you should be warned that bpf is also the device that allows packet sniffers to work correctly (although they still have to be run as root). bpf is required to use DHCP, but if you are very sensitive about security, you probably should not add bpf to your kernel in the expectation that at some point in the future you will be using DHCP. Edit your /etc/rc.conf to include the following: ifconfig_fxp0="DHCP" Be sure to replace fxp0 with the designation for the interface that you wish to dynamically configure, as described in . If you are using a different location for dhclient, or if you wish to pass additional flags to dhclient, also include the following (editing as necessary): dhcp_program="/sbin/dhclient" dhcp_flags="" DHCP server The DHCP server, dhcpd, is included as part of the net/isc-dhcp3-server port in the ports collection. This port contains the ISC DHCP server and documentation. Files DHCP configuration files /etc/dhclient.conf dhclient requires a configuration file, /etc/dhclient.conf. Typically the file contains only comments, the defaults being reasonably sane. This configuration file is described by the &man.dhclient.conf.5; manual page. /sbin/dhclient dhclient is statically linked and resides in /sbin. The &man.dhclient.8; manual page gives more information about dhclient. /sbin/dhclient-script dhclient-script is the FreeBSD-specific DHCP client configuration script. It is described in &man.dhclient-script.8;, but should not need any user modification to function properly. /var/db/dhclient.leases The DHCP client keeps a database of valid leases in this file, which is written as a log. &man.dhclient.leases.5; gives a slightly longer description. Further Reading The DHCP protocol is fully described in RFC 2131. An informational resource has also been set up at . Installing and Configuring a DHCP Server What This Section Covers This section provides information on how to configure a FreeBSD system to act as a DHCP server using the ISC (Internet Software Consortium) implementation of the DHCP server. The server is not provided as part of FreeBSD, and so you will need to install the net/isc-dhcp3-server port to provide this service. See for more information on using the Ports Collection. DHCP Server Installation DHCP installation In order to configure your FreeBSD system as a DHCP server, you will need to ensure that the &man.bpf.4; device is compiled into your kernel. To do this, add device bpf to your kernel configuration file, and rebuild the kernel. For more information about building kernels, see . The bpf device is already part of the GENERIC kernel that is supplied with FreeBSD, so you do not need to create a custom kernel in order to get DHCP working. Those who are particularly security conscious should note that bpf is also the device that allows packet sniffers to work correctly (although such programs still need privileged access). bpf is required to use DHCP, but if you are very sensitive about security, you probably should not include bpf in your kernel purely because you expect to use DHCP at some point in the future. The next thing that you will need to do is edit the sample dhcpd.conf which was installed by the net/isc-dhcp3-server port. By default, this will be /usr/local/etc/dhcpd.conf.sample, and you should copy this to /usr/local/etc/dhcpd.conf before proceeding to make changes. Configuring the DHCP Server DHCP dhcpd.conf dhcpd.conf is comprised of declarations regarding subnets and hosts, and is perhaps most easily explained using an example : option domain-name "example.com"; option domain-name-servers 192.168.4.100; option subnet-mask 255.255.255.0; default-lease-time 3600; max-lease-time 86400; ddns-update-style none; subnet 192.168.4.0 netmask 255.255.255.0 { range 192.168.4.129 192.168.4.254; option routers 192.168.4.1; } host mailhost { hardware ethernet 02:03:04:05:06:07; fixed-address mailhost.example.com; } This option specifies the domain that will be provided to clients as the default search domain. See &man.resolv.conf.5; for more information on what this means. This option specifies a comma separated list of DNS servers that the client should use. The netmask that will be provided to clients. A client may request a specific length of time that a lease will be valid. Otherwise the server will assign a lease with this expiry value (in seconds). This is the maximum length of time that the server will lease for. Should a client request a longer lease, a lease will be issued, although it will only be valid for max-lease-time seconds. This option specifies whether the DHCP server should attempt to update DNS when a lease is accepted or released. In the ISC implementation, this option is required. This denotes which IP addresses should be used in the pool reserved for allocating to clients. IP addresses between, and including, the ones stated are handed out to clients. Declares the default gateway that will be provided to clients. The hardware MAC address of a host (so that the DHCP server can recognize a host when it makes a request). Specifies that the host should always be given the same IP address. Note that using a hostname is correct here, since the DHCP server will resolve the hostname itself before returning the lease information. Once you have finished writing your dhcpd.conf, you should enable the DHCP server in /etc/rc.conf, i.e. by adding: dhcpd_enable="YES" dhcpd_ifaces="dc0" Replace the dc0 interface name with the interface (or interfaces, separated by whitespace) that your DHCP server should listen on for DHCP client requests. Then, you can proceed to start the server by issuing the following command: &prompt.root; /usr/local/etc/rc.d/isc-dhcpd.sh start Should you need to make changes to the configuration of your server in the future, it is important to note that sending a SIGHUP signal to dhcpd does not result in the configuration being reloaded, as it does with most daemons. You will need to send a SIGTERM signal to stop the process, and then restart it using the command above. Files DHCP configuration files /usr/local/sbin/dhcpd dhcpd is statically linked and resides in /usr/local/sbin. The &man.dhcpd.8; manual page installed with the port gives more information about dhcpd. /usr/local/etc/dhcpd.conf dhcpd requires a configuration file, /usr/local/etc/dhcpd.conf before it will start providing service to clients. This file needs to contain all the information that should be provided to clients that are being serviced, along with information regarding the operation of the server. This configuration file is described by the &man.dhcpd.conf.5; manual page installed by the port. /var/db/dhcpd.leases The DHCP server keeps a database of leases it has issued in this file, which is written as a log. The manual page &man.dhcpd.leases.5;, installed by the port gives a slightly longer description. /usr/local/sbin/dhcrelay dhcrelay is used in advanced environments where one DHCP server forwards a request from a client to another DHCP server on a separate network. If you require this functionality, then install the net/isc-dhcp3-relay port. The &man.dhcrelay.8; manual page provided with the port contains more detail. Chern Lee Contributed by Tom Rhodes Daniel Gerzo Domain Name System (<acronym>DNS</acronym>) Overview BIND &os; utilizes, by default, a version of BIND (Berkeley Internet Name Domain), which is the most common implementation of the DNS protocol. DNS is the protocol through which names are mapped to IP addresses, and vice versa. For example, a query for www.FreeBSD.org will receive a reply with the IP address of The &os; Project's web server, whereas, a query for ftp.FreeBSD.org will return the IP address of the corresponding FTP machine. Likewise, the opposite can happen. A query for an IP address can resolve its hostname. It is not necessary to run a name server to perform DNS lookups on a system. &os; currently comes with BIND9 DNS server software by default. Our installation provides enhanced security features, a new file system layout and automated &man.chroot.8; configuration. DNS DNS is coordinated across the Internet through a somewhat complex system of authoritative root, Top Level Domain (TLD), and other smaller-scale name servers which host and cache individual domain information. Currently, BIND is maintained by the Internet Software Consortium . Terminology To understand this document, some terms related to DNS must be understood. resolver reverse DNS root zone Term Definition Forward DNS Mapping of hostnames to IP addresses. Origin Refers to the domain covered in a particular zone file. named, BIND, name server Common names for the BIND name server package within &os;. Resolver A system process through which a machine queries a name server for zone information. Reverse DNS The opposite of forward DNS; mapping of IP addresses to hostnames. Root zone The beginning of the Internet zone hierarchy. All zones fall under the root zone, similar to how all files in a file system fall under the root directory. Zone An individual domain, subdomain, or portion of the DNS administered by the same authority. zones examples Examples of zones: . is the root zone. org. is a Top Level Domain (TLD) under the root zone. example.org. is a zone under the org. TLD. 1.168.192.in-addr.arpa is a zone referencing all IP addresses which fall under the 192.168.1.* IP space. As one can see, the more specific part of a hostname appears to its left. For example, example.org. is more specific than org., as org. is more specific than the root zone. The layout of each part of a hostname is much like a file system: the - /dev directory falls + /dev directory falls within the root, and so on. Reasons to Run a Name Server Name servers usually come in two forms: an authoritative name server, and a caching name server. An authoritative name server is needed when: One wants to serve DNS information to the world, replying authoritatively to queries. A domain, such as example.org, is registered and IP addresses need to be assigned to hostnames under it. An IP address block requires reverse DNS entries (IP to hostname). A backup or second name server, called a slave, will reply to queries. A caching name server is needed when: A local DNS server may cache and respond more quickly than querying an outside name server. When one queries for www.FreeBSD.org, the resolver usually queries the uplink ISP's name server, and retrieves the reply. With a local, caching DNS server, the query only has to be made once to the outside world by the caching DNS server. Every additional query will not have to look to the outside of the local network, since the information is cached locally. How It Works In &os;, the BIND daemon is called named for obvious reasons. File Description &man.named.8; The BIND daemon. &man.rndc.8; Name server control utility. - /etc/namedb + /etc/namedb Directory where BIND zone information resides. /etc/namedb/named.conf Configuration file of the daemon. Depending on how a given zone is configured on the server, the files related to that zone can be found in the master, slave, or dynamic subdirectories of the - /etc/namedb directory. + class="directory">master, slave, or dynamic subdirectories of the + /etc/namedb directory. These files contain the DNS information that will be given out by the name server in response to queries. Starting BIND BIND starting Since BIND is installed by default, configuring it all is relatively simple. The default named configuration is that of a basic resolving name server, ran in a &man.chroot.8; environment. To start the server one time with this configuration, use the following command: &prompt.root; /etc/rc.d/named forcestart To ensure the named daemon is started at boot each time, put the following line into the /etc/rc.conf: named_enable="YES" There are obviously many configuration options for /etc/namedb/named.conf that are beyond the scope of this document. However, if you are interested in the startup options for named on &os;, take a look at the named_* flags in /etc/defaults/rc.conf and consult the &man.rc.conf.5; manual page. The section is also a good read. Configuration Files BIND configuration files Configuration files for named currently reside in - /etc/namedb directory and + /etc/namedb directory and will need modification before use, unless all that is needed is a simple resolver. This is where most of the configuration will be performed. Using <command>make-localhost</command> To configure a master zone for the localhost visit the - /etc/namedb directory + /etc/namedb directory and run the following command: &prompt.root; sh make-localhost If all went well, a new file should exist in the master subdirectory. The filenames should be localhost.rev for the local domain name and localhost-v6.rev for IPv6 configurations. As the default configuration file, required information will be present in the named.conf file. <filename>/etc/namedb/named.conf</filename> // $FreeBSD$ // // Refer to the named.conf(5) and named(8) man pages, and the documentation // in /usr/share/doc/bind9 for more details. // // If you are going to set up an authoritative server, make sure you // understand the hairy details of how DNS works. Even with // simple mistakes, you can break connectivity for affected parties, // or cause huge amounts of useless Internet traffic. options { directory "/etc/namedb"; pid-file "/var/run/named/pid"; dump-file "/var/dump/named_dump.db"; statistics-file "/var/stats/named.stats"; // If named is being used only as a local resolver, this is a safe default. // For named to be accessible to the network, comment this option, specify // the proper IP address, or delete this option. listen-on { 127.0.0.1; }; // If you have IPv6 enabled on this system, uncomment this option for // use as a local resolver. To give access to the network, specify // an IPv6 address, or the keyword "any". // listen-on-v6 { ::1; }; // In addition to the "forwarders" clause, you can force your name // server to never initiate queries of its own, but always ask its // forwarders only, by enabling the following line: // // forward only; // If you've got a DNS server around at your upstream provider, enter // its IP address here, and enable the line below. This will make you // benefit from its cache, thus reduce overall DNS traffic in the Internet. /* forwarders { 127.0.0.1; }; */ Just as the comment says, to benefit from an uplink's cache, forwarders can be enabled here. Under normal circumstances, a name server will recursively query the Internet looking at certain name servers until it finds the answer it is looking for. Having this enabled will have it query the uplink's name server (or name server provided) first, taking advantage of its cache. If the uplink name server in question is a heavily trafficked, fast name server, enabling this may be worthwhile. 127.0.0.1 will not work here. Change this IP address to a name server at your uplink. /* * If there is a firewall between you and nameservers you want * to talk to, you might need to uncomment the query-source * directive below. Previous versions of BIND always asked * questions using port 53, but BIND versions 8 and later * use a pseudo-random unprivileged UDP port by default. */ // query-source address * port 53; }; // If you enable a local name server, don't forget to enter 127.0.0.1 // first in your /etc/resolv.conf so this server will be queried. // Also, make sure to enable it in /etc/rc.conf. zone "." { type hint; file "named.root"; }; zone "0.0.127.IN-ADDR.ARPA" { type master; file "master/localhost.rev"; }; // RFC 3152 zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.ARPA" { type master; file "master/localhost-v6.rev"; }; // NB: Do not use the IP addresses below, they are faked, and only // serve demonstration/documentation purposes! // // Example slave zone config entries. It can be convenient to become // a slave at least for the zone your own domain is in. Ask // your network administrator for the IP address of the responsible // primary. // // Never forget to include the reverse lookup (IN-ADDR.ARPA) zone! // (This is named after the first bytes of the IP address, in reverse // order, with ".IN-ADDR.ARPA" appended.) // // Before starting to set up a primary zone, make sure you fully // understand how DNS and BIND works. There are sometimes // non-obvious pitfalls. Setting up a slave zone is simpler. // // NB: Don't blindly enable the examples below. :-) Use actual names // and addresses instead. /* An example master zone zone "example.net" { type master; file "master/example.net"; }; */ /* An example dynamic zone key "exampleorgkey" { algorithm hmac-md5; secret "sf87HJqjkqh8ac87a02lla=="; }; zone "example.org" { type master; allow-update { key "exampleorgkey"; }; file "dynamic/example.org"; }; */ /* Examples of forward and reverse slave zones zone "example.com" { type slave; file "slave/example.com"; masters { 192.168.1.1; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "slave/1.168.192.in-addr.arpa"; masters { 192.168.1.1; }; }; */ In named.conf, these are examples of slave entries for a forward and reverse zone. For each new zone served, a new zone entry must be added to named.conf. For example, the simplest zone entry for example.org can look like: zone "example.org" { type master; file "master/example.org"; }; The zone is a master, as indicated by the statement, holding its zone information in /etc/namedb/master/example.org indicated by the statement. zone "example.org" { type slave; file "slave/example.org"; }; In the slave case, the zone information is transferred from the master name server for the particular zone, and saved in the file specified. If and when the master server dies or is unreachable, the slave name server will have the transferred zone information and will be able to serve it. Zone Files BIND zone files An example master zone file for example.org (existing within /etc/namedb/master/example.org) is as follows: $TTL 3600 ; 1 hour example.org. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 86400 ; Minimum TTL ) ; DNS Servers IN NS ns1.example.org. IN NS ns2.example.org. ; MX Records IN MX 10 mx.example.org. IN MX 20 mail.example.org. IN A 192.168.1.1 ; Machine Names localhost IN A 127.0.0.1 ns1 IN A 192.168.1.2 ns2 IN A 192.168.1.3 mx IN A 192.168.1.4 mail IN A 192.168.1.5 ; Aliases www IN CNAME @ Note that every hostname ending in a . is an exact hostname, whereas everything without a trailing . is referenced to the origin. For example, www is translated into www.origin. In our fictitious zone file, our origin is example.org., so www would translate to www.example.org. The format of a zone file follows: recordname IN recordtype value DNS records The most commonly used DNS records: SOA start of zone authority NS an authoritative name server A a host address CNAME the canonical name for an alias MX mail exchanger PTR a domain name pointer (used in reverse DNS) example.org. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh after 3 hours 3600 ; Retry after 1 hour 604800 ; Expire after 1 week 86400 ) ; Minimum TTL of 1 day example.org. the domain name, also the origin for this zone file. ns1.example.org. the primary/authoritative name server for this zone. admin.example.org. the responsible person for this zone, email address with @ replaced. (admin@example.org becomes admin.example.org) 2006051501 the serial number of the file. This must be incremented each time the zone file is modified. Nowadays, many admins prefer a yyyymmddrr format for the serial number. 2006051501 would mean last modified 05/15/2006, the latter 01 being the first time the zone file has been modified this day. The serial number is important as it alerts slave name servers for a zone when it is updated. IN NS ns1.example.org. This is an NS entry. Every name server that is going to reply authoritatively for the zone must have one of these entries. localhost IN A 127.0.0.1 ns1 IN A 192.168.1.2 ns2 IN A 192.168.1.3 mx IN A 192.168.1.4 mail IN A 192.168.1.5 The A record indicates machine names. As seen above, ns1.example.org would resolve to 192.168.1.2. IN A 192.168.1.1 This line assigns IP address 192.168.1.1 to the current origin, in this case example.org. www IN CNAME @ The canonical name record is usually used for giving aliases to a machine. In the example, www is aliased to the master machine which name equals to domain name example.org (192.168.1.1). CNAMEs can be used to provide alias hostnames, or round robin one hostname among multiple machines. MX record IN MX 10 mail.example.org. The MX record indicates which mail servers are responsible for handling incoming mail for the zone. mail.example.org is the hostname of the mail server, and 10 being the priority of that mail server. One can have several mail servers, with priorities of 10, 20 and so on. A mail server attempting to deliver to example.org would first try the highest priority MX (the record with the lowest priority number), then the second highest, etc, until the mail can be properly delivered. For in-addr.arpa zone files (reverse DNS), the same format is used, except with PTR entries instead of A or CNAME. $TTL 3600 1.168.192.in-addr.arpa. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 3600 ) ; Minimum IN NS ns1.example.org. IN NS ns2.example.org. 1 IN PTR example.org. 2 IN PTR ns1.example.org. 3 IN PTR ns2.example.org. 4 IN PTR mx.example.org. 5 IN PTR mail.example.org. This file gives the proper IP address to hostname mappings of our above fictitious domain. Caching Name Server BIND caching name server A caching name server is a name server that is not authoritative for any zones. It simply asks queries of its own, and remembers them for later use. To set one up, just configure the name server as usual, omitting any inclusions of zones. Security Although BIND is the most common implementation of DNS, there is always the issue of security. Possible and exploitable security holes are sometimes found. While &os; automatically drops named into a &man.chroot.8; environment; there are several other security mechanisms in place which could help to lure off possible DNS service attacks. It is always good idea to read CERT's security advisories and to subscribe to the &a.security-notifications; to stay up to date with the current Internet and &os; security issues. If a problem arises, keeping sources up to date and having a fresh build of named would not hurt. Further Reading BIND/named manual pages: &man.rndc.8; &man.named.8; &man.named.conf.5; Official ISC BIND Page Official ISC BIND Forum BIND FAQ O'Reilly DNS and BIND 5th Edition RFC1034 - Domain Names - Concepts and Facilities RFC1035 - Domain Names - Implementation and Specification Murray Stokely Contributed by Apache HTTP Server web servers setting up Apache Overview &os; is used to run some of the busiest web sites in the world. The majority of web servers on the Internet are using the Apache HTTP Server. Apache software packages should be included on your FreeBSD installation media. If you did not install Apache when you first installed FreeBSD, then you can install it from the www/apache13 or www/apache22 port. Once Apache has been installed successfully, it must be configured. This section covers version 1.3.X of the Apache HTTP Server as that is the most widely used version for &os;. Apache 2.X introduces many new technologies but they are not discussed here. For more information about Apache 2.X, please see . Configuration Apache configuration file The main Apache HTTP Server configuration file is installed as /usr/local/etc/apache/httpd.conf on &os;. This file is a typical &unix; text configuration file with comment lines beginning with the # character. A comprehensive description of all possible configuration options is outside the scope of this book, so only the most frequently modified directives will be described here. ServerRoot "/usr/local" This specifies the default directory hierarchy for the Apache installation. Binaries are stored in the bin and sbin subdirectories of the server root, and configuration files are stored in etc/apache. ServerAdmin you@your.address The address to which problems with the server should be emailed. This address appears on some server-generated pages, such as error documents. ServerName www.example.com ServerName allows you to set a host name which is sent back to clients for your server if it is different to the one that the host is configured with (i.e., use www instead of the host's real name). DocumentRoot "/usr/local/www/data" DocumentRoot: The directory out of which you will serve your documents. By default, all requests are taken from this directory, but symbolic links and aliases may be used to point to other locations. It is always a good idea to make backup copies of your Apache configuration file before making changes. Once you are satisfied with your initial configuration you are ready to start running Apache. Running <application>Apache</application> Apache starting or stopping Apache does not run from the inetd super server as many other network servers do. It is configured to run standalone for better performance for incoming HTTP requests from client web browsers. A shell script wrapper is included to make starting, stopping, and restarting the server as simple as possible. To start up Apache for the first time, just run: &prompt.root; /usr/local/sbin/apachectl start You can stop the server at any time by typing: &prompt.root; /usr/local/sbin/apachectl stop After making changes to the configuration file for any reason, you will need to restart the server: &prompt.root; /usr/local/sbin/apachectl restart To restart Apache without aborting current connections, run: &prompt.root; /usr/local/sbin/apachectl graceful Additional information available at &man.apachectl.8; manual page. To launch Apache at system startup, add the following line to /etc/rc.conf: apache_enable="YES" or for Apache 2.2: apache22_enable="YES" If you would like to supply additional command line options for the Apache httpd program started at system boot, you may specify them with an additional line in rc.conf: apache_flags="" Now that the web server is running, you can view your web site by pointing a web browser to http://localhost/. The default web page that is displayed is /usr/local/www/data/index.html. Virtual Hosting Apache supports two different types of Virtual Hosting. The first method is Name-based Virtual Hosting. Name-based virtual hosting uses the clients HTTP/1.1 headers to figure out the hostname. This allows many different domains to share the same IP address. To setup Apache to use Name-based Virtual Hosting add an entry like the following to your httpd.conf: NameVirtualHost * If your webserver was named www.domain.tld and you wanted to setup a virtual domain for www.someotherdomain.tld then you would add the following entries to httpd.conf: <VirtualHost *> ServerName www.domain.tld DocumentRoot /www/domain.tld </VirtualHost> <VirtualHost *> ServerName www.someotherdomain.tld DocumentRoot /www/someotherdomain.tld </VirtualHost> Replace the addresses with the addresses you want to use and the path to the documents with what you are using. For more information about setting up virtual hosts, please consult the official Apache documentation at: . Apache Modules Apache modules There are many different Apache modules available to add functionality to the basic server. The FreeBSD Ports Collection provides an easy way to install Apache together with some of the more popular add-on modules. mod_ssl web servers secure SSL cryptography The mod_ssl module uses the OpenSSL library to provide strong cryptography via the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols. This module provides everything necessary to request a signed certificate from a trusted certificate signing authority so that you can run a secure web server on &os;. If you have not yet installed Apache, then a version of Apache 1.3.X that includes mod_ssl may be installed with the www/apache13-modssl port. SSL support is also available for Apache 2.X in the www/apache22 port, where it is enabled by default. Language Bindings There are Apache modules for most major scripting languages. These modules typically make it possible to write Apache modules entirely in a scripting language. They are also often used as a persistent interpreter embedded into the server that avoids the overhead of starting an external interpreter and the startup-time penalty for dynamic websites, as described in the next section. Dynamic Websites web servers dynamic In the last decade, more businesses have turned to the Internet in order to enhance their revenue and increase exposure. This has also increased the need for interactive web content. While some companies, such as µsoft;, have introduced solutions into their proprietary products, the open source community answered the call. Modern options for dynamic web content include Django, Ruby on Rails, mod_perl, and mod_php. Django Python Django Django is a BSD licensed framework designed to allow developers to write high performance, elegant web applications quickly. It provides an object-relational mapper so that data types are developed as Python objects, and a rich dynamic database-access API is provided for those objects without the developer ever having to write SQL. It also provides an extensible template system so that the logic of the application is separated from the HTML presentation. Django depends on mod_python, Apache, and an SQL database engine of your choice. The FreeBSD Port will install all of these pre-requisites for you with the appropriate flags. Installing Django with Apache2, mod_python3, and PostgreSQL &prompt.root; cd /usr/ports/www/py-django; make all install clean -DWITH_MOD_PYTHON3 -DWITH_POSTGRESQL Once Django and these pre-requisites are installed, you will need to create a Django project directory and then configure Apache to use the embedded Python interpreter to call your application for specific URLs on your site. Apache Configuration for Django/mod_python You will need to add a line to the apache httpd.conf file to configure Apache to pass requests for certain URLs to your web application: <Location "/"> SetHandler python-program PythonPath "['/dir/to/your/django/packages/'] + sys.path" PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE mysite.settings PythonAutoReload On PythonDebug On </Location> Ruby on Rails Ruby on Rails Ruby on Rails is another open source web framework that provides a full development stack and is optimized to make web developers more productive and capable of writing powerful applications quickly. It can be installed easily from the ports system. &prompt.root; cd /usr/ports/www/rubygem-rails; make all install clean mod_perl mod_perl Perl The Apache/Perl integration project brings together the full power of the Perl programming language and the Apache HTTP Server. With the mod_perl module it is possible to write Apache modules entirely in Perl. In addition, the persistent interpreter embedded in the server avoids the overhead of starting an external interpreter and the penalty of Perl start-up time. mod_perl is available a few different ways. To use mod_perl remember that mod_perl 1.0 only works with Apache 1.3 and mod_perl 2.0 only works with Apache 2.X. mod_perl 1.0 is available in www/mod_perl and a statically compiled version is available in www/apache13-modperl. mod_perl 2.0 is avaliable in www/mod_perl2. Tom Rhodes Written by mod_php mod_php PHP PHP, also known as PHP: Hypertext Preprocessor is a general-purpose scripting language that is especially suited for Web development. Capable of being embedded into HTML its syntax draws upon C, &java;, and Perl with the intention of allowing web developers to write dynamically generated webpages quickly. To gain support for PHP5 for the Apache web server, begin by installing the lang/php5 port. If the lang/php5 port is being installed for the first time, available OPTIONS will be displayed automatically. If a menu is not displayed, i.e. because the lang/php5 port has been installed some time in the past, it is always possible to bring the options dialog up again by running: &prompt.root; make config in the port directory. In the options dialog, check the APACHE option to build mod_php5 as a loadable module for the Apache web server. A lot of sites are still using PHP4 for various reasons (i.e. compatibility issues or already deployed web applications). If the mod_php4 is needed instead of mod_php5, then please use the lang/php4 port. The lang/php4 port supports many of the configuration and build-time options of the lang/php5 port. This will install and configure the modules required to support dynamic PHP applications. Check to ensure the following sections have been added to /usr/local/etc/apache/httpd.conf: LoadModule php5_module libexec/apache/libphp5.so AddModule mod_php5.c <IfModule mod_php5.c> DirectoryIndex index.php index.html </IfModule> <IfModule mod_php5.c> AddType application/x-httpd-php .php AddType application/x-httpd-php-source .phps </IfModule> Once completed, a simple call to the apachectl command for a graceful restart is needed to load the PHP module: &prompt.root; apachectl graceful For future upgrades of PHP, the make config command will not be required; the selected OPTIONS are saved automatically by the &os; Ports framework. The PHP support in &os; is extremely modular so the base install is very limited. It is very easy to add support using the lang/php5-extensions port. This port provides a menu driven interface to PHP extension installation. Alternatively, individual extensions can be installed using the appropriate port. For instance, to add support for the MySQL database server to PHP5, simply install the databases/php5-mysql port. After installing an extension, the Apache server must be reloaded to pick up the new configuration changes: &prompt.root; apachectl graceful Murray Stokely Contributed by File Transfer Protocol (FTP) FTP servers Overview The File Transfer Protocol (FTP) provides users with a simple way to transfer files to and from an FTP server. &os; includes FTP server software, ftpd, in the base system. This makes setting up and administering an FTP server on FreeBSD very straightforward. Configuration The most important configuration step is deciding which accounts will be allowed access to the FTP server. A normal FreeBSD system has a number of system accounts used for various daemons, but unknown users should not be allowed to log in with these accounts. The /etc/ftpusers file is a list of users disallowed any FTP access. By default, it includes the aforementioned system accounts, but it is possible to add specific users here that should not be allowed access to FTP. You may want to restrict the access of some users without preventing them completely from using FTP. This can be accomplished with the /etc/ftpchroot file. This file lists users and groups subject to FTP access restrictions. The &man.ftpchroot.5; manual page has all of the details so it will not be described in detail here. FTP anonymous If you would like to enable anonymous FTP access to your server, then you must create a user named ftp on your &os; system. Users will then be able to log on to your FTP server with a username of ftp or anonymous and with any password (by convention an email address for the user should be used as the password). The FTP server will call &man.chroot.2; when an anonymous user logs in, to restrict access to only the home directory of the ftp user. There are two text files that specify welcome messages to be displayed to FTP clients. The contents of the file /etc/ftpwelcome will be displayed to users before they reach the login prompt. After a successful login, the contents of the file /etc/ftpmotd will be displayed. Note that the path to this file is relative to the login environment, so the file ~ftp/etc/ftpmotd would be displayed for anonymous users. Once the FTP server has been configured properly, it must be enabled in /etc/inetd.conf. All that is required here is to remove the comment symbol # from in front of the existing ftpd line : ftp stream tcp nowait root /usr/libexec/ftpd ftpd -l As explained in , the inetd configuration must be reloaded after this configuration file is changed. You can now log on to your FTP server by typing: &prompt.user; ftp localhost Maintaining syslog log files FTP The ftpd daemon uses &man.syslog.3; to log messages. By default, the system log daemon will put messages related to FTP in the /var/log/xferlog file. The location of the FTP log can be modified by changing the following line in /etc/syslog.conf: ftp.info /var/log/xferlog FTP anonymous Be aware of the potential problems involved with running an anonymous FTP server. In particular, you should think twice about allowing anonymous users to upload files. You may find that your FTP site becomes a forum for the trade of unlicensed commercial software or worse. If you do need to allow anonymous FTP uploads, then you should set up the permissions so that these files can not be read by other anonymous users until they have been reviewed. Murray Stokely Contributed by File and Print Services for µsoft.windows; clients (Samba) Samba server Microsoft Windows file server Windows clients print server Windows clients Overview Samba is a popular open source software package that provides file and print services for µsoft.windows; clients. Such clients can connect to and use FreeBSD filespace as if it was a local disk drive, or FreeBSD printers as if they were local printers. Samba software packages should be included on your FreeBSD installation media. If you did not install Samba when you first installed FreeBSD, then you can install it from the net/samba3 port or package. Configuration A default Samba configuration file is installed as /usr/local/share/examples/samba/smb.conf.default. This file must be copied to /usr/local/etc/smb.conf and customized before Samba can be used. The smb.conf file contains runtime configuration information for Samba, such as definitions of the printers and file system shares that you would like to share with &windows; clients. The Samba package includes a web based tool called swat which provides a simple way of configuring the smb.conf file. Using the Samba Web Administration Tool (SWAT) The Samba Web Administration Tool (SWAT) runs as a daemon from inetd. Therefore, the following line in /etc/inetd.conf should be uncommented before swat can be used to configure Samba: swat stream tcp nowait/400 root /usr/local/sbin/swat swat As explained in , the inetd configuration must be reloaded after this configuration file is changed. Once swat has been enabled in inetd.conf, you can use a browser to connect to . You will first have to log on with the system root account. Once you have successfully logged on to the main Samba configuration page, you can browse the system documentation, or begin by clicking on the Globals tab. The Globals section corresponds to the variables that are set in the [global] section of /usr/local/etc/smb.conf. Global Settings Whether you are using swat or editing /usr/local/etc/smb.conf directly, the first directives you are likely to encounter when configuring Samba are: workgroup NT Domain-Name or Workgroup-Name for the computers that will be accessing this server. netbios name NetBIOS This sets the NetBIOS name by which a Samba server is known. By default it is the same as the first component of the host's DNS name. server string This sets the string that will be displayed with the net view command and some other networking tools that seek to display descriptive text about the server. Security Settings Two of the most important settings in /usr/local/etc/smb.conf are the security model chosen, and the backend password format for client users. The following directives control these options: security The two most common options here are security = share and security = user. If your clients use usernames that are the same as their usernames on your &os; machine then you will want to use user level security. This is the default security policy and it requires clients to first log on before they can access shared resources. In share level security, client do not need to log onto the server with a valid username and password before attempting to connect to a shared resource. This was the default security model for older versions of Samba. passdb backend NIS+ LDAP SQL database Samba has several different backend authentication models. You can authenticate clients with LDAP, NIS+, a SQL database, or a modified password file. The default authentication method is smbpasswd, and that is all that will be covered here. Assuming that the default smbpasswd backend is used, the /usr/local/private/smbpasswd file must be created to allow Samba to authenticate clients. If you would like to give your &unix; user accounts access from &windows; clients, use the following command: &prompt.root; smbpasswd -a username Please see the Official Samba HOWTO for additional information about configuration options. With the basics outlined here, you should have everything you need to start running Samba. Starting <application>Samba</application> The net/samba3 port adds a new startup script, which can be used to control Samba. To enable this script, so that it can be used for example to start, stop or restart Samba, add the following line to the /etc/rc.conf file: samba_enable="YES" Or, for fine grain control: nmbd_enable="YES" smbd_enable="YES" This will also configure Samba to automatically start at system boot time. It is possible then to start Samba at any time by typing: &prompt.root; /usr/local/etc/rc.d/samba start Starting SAMBA: removing stale tdbs : Starting nmbd. Starting smbd. Please refer to for more information about using rc scripts. Samba actually consists of three separate daemons. You should see that both the nmbd and smbd daemons are started by the samba script. If you enabled winbind name resolution services in smb.conf, then you will also see that the winbindd daemon is started. You can stop Samba at any time by typing : &prompt.root; /usr/local/etc/rc.d/samba stop Samba is a complex software suite with functionality that allows broad integration with µsoft.windows; networks. For more information about functionality beyond the basic installation described here, please see . Tom Hukins Contributed by Clock Synchronization with NTP NTP Overview Over time, a computer's clock is prone to drift. The Network Time Protocol (NTP) is one way to ensure your clock stays accurate. Many Internet services rely on, or greatly benefit from, computers' clocks being accurate. For example, a web server may receive requests to send a file if it has been modified since a certain time. In a local area network environment, it is essential that computers sharing files from the same file server have synchronized clocks so that file timestamps stay consistent. Services such as &man.cron.8; also rely on an accurate system clock to run commands at the specified times. NTP ntpd FreeBSD ships with the &man.ntpd.8; NTP server which can be used to query other NTP servers to set the clock on your machine or provide time services to others. Choosing Appropriate NTP Servers NTP choosing servers In order to synchronize your clock, you will need to find one or more NTP servers to use. Your network administrator or ISP may have set up an NTP server for this purpose—check their documentation to see if this is the case. There is an online list of publicly accessible NTP servers which you can use to find an NTP server near to you. Make sure you are aware of the policy for any servers you choose, and ask for permission if required. Choosing several unconnected NTP servers is a good idea in case one of the servers you are using becomes unreachable or its clock is unreliable. &man.ntpd.8; uses the responses it receives from other servers intelligently—it will favor unreliable servers less than reliable ones. Configuring Your Machine NTP configuration Basic Configuration ntpdate If you only wish to synchronize your clock when the machine boots up, you can use &man.ntpdate.8;. This may be appropriate for some desktop machines which are frequently rebooted and only require infrequent synchronization, but most machines should run &man.ntpd.8;. Using &man.ntpdate.8; at boot time is also a good idea for machines that run &man.ntpd.8;. The &man.ntpd.8; program changes the clock gradually, whereas &man.ntpdate.8; sets the clock, no matter how great the difference between a machine's current clock setting and the correct time. To enable &man.ntpdate.8; at boot time, add ntpdate_enable="YES" to /etc/rc.conf. You will also need to specify all servers you wish to synchronize with and any flags to be passed to &man.ntpdate.8; in ntpdate_flags. NTP ntp.conf General Configuration NTP is configured by the /etc/ntp.conf file in the format described in &man.ntp.conf.5;. Here is a simple example: server ntplocal.example.com prefer server timeserver.example.org server ntp2a.example.net driftfile /var/db/ntp.drift The server option specifies which servers are to be used, with one server listed on each line. If a server is specified with the prefer argument, as with ntplocal.example.com, that server is preferred over other servers. A response from a preferred server will be discarded if it differs significantly from other servers' responses, otherwise it will be used without any consideration to other responses. The prefer argument is normally used for NTP servers that are known to be highly accurate, such as those with special time monitoring hardware. The driftfile option specifies which file is used to store the system clock's frequency offset. The &man.ntpd.8; program uses this to automatically compensate for the clock's natural drift, allowing it to maintain a reasonably correct setting even if it is cut off from all external time sources for a period of time. The driftfile option specifies which file is used to store information about previous responses from the NTP servers you are using. This file contains internal information for NTP. It should not be modified by any other process. Controlling Access to Your Server By default, your NTP server will be accessible to all hosts on the Internet. The restrict option in /etc/ntp.conf allows you to control which machines can access your server. If you want to deny all machines from accessing your NTP server, add the following line to /etc/ntp.conf: restrict default ignore This will also prevent access from your server to any servers listed in your local configuration. If you need to synchronise your NTP server with an external NTP server you should allow the specific server. See the &man.ntp.conf.5; manual for more information. If you only want to allow machines within your own network to synchronize their clocks with your server, but ensure they are not allowed to configure the server or used as peers to synchronize against, add restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap instead, where 192.168.1.0 is an IP address on your network and 255.255.255.0 is your network's netmask. /etc/ntp.conf can contain multiple restrict options. For more details, see the Access Control Support subsection of &man.ntp.conf.5;. Running the NTP Server To ensure the NTP server is started at boot time, add the line ntpd_enable="YES" to /etc/rc.conf. If you wish to pass additional flags to &man.ntpd.8;, edit the ntpd_flags parameter in /etc/rc.conf. To start the server without rebooting your machine, run ntpd being sure to specify any additional parameters from ntpd_flags in /etc/rc.conf. For example: &prompt.root; ntpd -p /var/run/ntpd.pid Using ntpd with a Temporary Internet Connection The &man.ntpd.8; program does not need a permanent connection to the Internet to function properly. However, if you have a temporary connection that is configured to dial out on demand, it is a good idea to prevent NTP traffic from triggering a dial out or keeping the connection alive. If you are using user PPP, you can use filter directives in /etc/ppp/ppp.conf. For example: set filter dial 0 deny udp src eq 123 # Prevent NTP traffic from initiating dial out set filter dial 1 permit 0 0 set filter alive 0 deny udp src eq 123 # Prevent incoming NTP traffic from keeping the connection open set filter alive 1 deny udp dst eq 123 # Prevent outgoing NTP traffic from keeping the connection open set filter alive 2 permit 0/0 0/0 For more details see the PACKET FILTERING section in &man.ppp.8; and the examples in /usr/share/examples/ppp/. Some Internet access providers block low-numbered ports, preventing NTP from functioning since replies never reach your machine. Further Information Documentation for the NTP server can be found in /usr/share/doc/ntp/ in HTML format. diff --git a/en_US.ISO8859-1/books/handbook/ports/chapter.sgml b/en_US.ISO8859-1/books/handbook/ports/chapter.sgml index 3b58aa85c4..3b29b5fe94 100644 --- a/en_US.ISO8859-1/books/handbook/ports/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/ports/chapter.sgml @@ -1,1579 +1,1579 @@ Installing Applications: Packages and Ports Synopsis ports packages FreeBSD is bundled with a rich collection of system tools as part of the base system. However, there is only so much one can do before needing to install an additional third-party application to get real work done. FreeBSD provides two complementary technologies for installing third-party software on your system: the FreeBSD Ports Collection (for installing from source), and packages (for installing from pre-built binaries). Either method may be used to install the newest version of your favorite applications from local media or straight off the network. After reading this chapter, you will know: How to install third-party binary software packages. How to build third-party software from source by using the ports collection. How to remove previously installed packages or ports. How to override the default values that the ports collection uses. How to find the appropriate software package. How to upgrade your applications. Overview of Software Installation If you have used a &unix; system before you will know that the typical procedure for installing third-party software goes something like this: Download the software, which might be distributed in source code format, or as a binary. Unpack the software from its distribution format (typically a tarball compressed with &man.compress.1;, &man.gzip.1;, or &man.bzip2.1;). Locate the documentation (perhaps an INSTALL or README file, or some files in a doc/ subdirectory) and read up on how to install the software. If the software was distributed in source format, compile it. This may involve editing a Makefile, or running a configure script, and other work. Test and install the software. And that is only if everything goes well. If you are installing a software package that was not deliberately ported to FreeBSD you may even have to go in and edit the code to make it work properly. Should you want to, you can continue to install software the traditional way with FreeBSD. However, FreeBSD provides two technologies which can save you a lot of effort: packages and ports. At the time of writing, over &os.numports; third-party applications have been made available in this way. For any given application, the FreeBSD package for that application is a single file which you must download. The package contains pre-compiled copies of all the commands for the application, as well as any configuration files or documentation. A downloaded package file can be manipulated with FreeBSD package management commands, such as &man.pkg.add.1;, &man.pkg.delete.1;, &man.pkg.info.1;, and so on. Installing a new application can be carried out with a single command. A FreeBSD port for an application is a collection of files designed to automate the process of compiling an application from source code. Remember that there are a number of steps you would normally carry out if you compiled a program yourself (downloading, unpacking, patching, compiling, installing). The files that make up a port contain all the necessary information to allow the system to do this for you. You run a handful of simple commands and the source code for the application is automatically downloaded, extracted, patched, compiled, and installed for you. In fact, the ports system can also be used to generate packages which can later be manipulated with pkg_add and the other package management commands that will be introduced shortly. Both packages and ports understand dependencies. Suppose you want to install an application that depends on a specific library being installed. Both the application and the library have been made available as FreeBSD ports and packages. If you use the pkg_add command or the ports system to add the application, both will notice that the library has not been installed, and automatically install the library first. Given that the two technologies are quite similar, you might be wondering why FreeBSD bothers with both. Packages and ports both have their own strengths, and which one you use will depend on your own preference. Package Benefits A compressed package tarball is typically smaller than the compressed tarball containing the source code for the application. Packages do not require any additional compilation. For large applications, such as Mozilla, KDE, or GNOME this can be important, particularly if you are on a slow system. Packages do not require any understanding of the process involved in compiling software on FreeBSD. Ports Benefits Packages are normally compiled with conservative options, because they have to run on the maximum number of systems. By installing from the port, you can tweak the compilation options to (for example) generate code that is specific to a Pentium 4 or Athlon processor. Some applications have compile-time options relating to what they can and cannot do. For example, Apache can be configured with a wide variety of different built-in options. By building from the port you do not have to accept the default options, and can set them yourself. In some cases, multiple packages will exist for the same application to specify certain settings. For example, Ghostscript is available as a ghostscript package and a ghostscript-nox11 package, depending on whether or not you have installed an X11 server. This sort of rough tweaking is possible with packages, but rapidly becomes impossible if an application has more than one or two different compile-time options. The licensing conditions of some software distributions forbid binary distribution. They must be distributed as source code. Some people do not trust binary distributions. At least with source code, you can (in theory) read through it and look for potential problems yourself. If you have local patches, you will need the source in order to apply them. Some people like having code around, so they can read it if they get bored, hack it, borrow from it (license permitting, of course), and so on. To keep track of updated ports, subscribe to the &a.ports; and the &a.ports-bugs;. Before installing any application, you should check for security issues related to your application. You can also install ports-mgmt/portaudit which will automatically check all installed applications for known vulnerabilities; a check will be also performed before any port build. Meanwhile, you can use the command portaudit -F -a after you have installed some packages. The remainder of this chapter will explain how to use packages and ports to install and manage third-party software on FreeBSD. Finding Your Application Before you can install any applications you need to know what you want, and what the application is called. FreeBSD's list of available applications is growing all the time. Fortunately, there are a number of ways to find what you want: The FreeBSD web site maintains an up-to-date searchable list of all the available applications, at http://www.FreeBSD.org/ports/. The ports are divided into categories, and you may either search for an application by name (if you know it), or see all the applications available in a category. FreshPorts Dan Langille maintains FreshPorts, at . FreshPorts tracks changes to the applications in the ports tree as they happen, allows you to watch one or more ports, and can send you email when they are updated. FreshMeat If you do not know the name of the application you want, try using a site like FreshMeat () to find an application, then check back at the FreeBSD site to see if the application has been ported yet. If you know the exact name of the port, but just need to find out which category it is in, you can use the &man.whereis.1; command. Simply type whereis file, where file is the program you want to install. If it is found on your system, you will be told where it is, as follows: &prompt.root; whereis lsof lsof: /usr/ports/sysutils/lsof This tells us that lsof (a system utility) can be found in the /usr/ports/sysutils/lsof directory. Additionally, you can use a simple &man.echo.1; statement to find where a port exists in the ports tree. For example: &prompt.root; echo /usr/ports/*/*lsof* /usr/ports/sysutils/lsof Note that this will return any matched files downloaded into the - /usr/ports/distfiles + /usr/ports/distfiles directory. Yet another way to find a particular port is by using the Ports Collection's built-in search mechanism. To use the search feature, you will need to be in the /usr/ports directory. Once in that directory, run make search name=program-name where program-name is the name of the program you want to find. For example, if you were looking for lsof: &prompt.root; cd /usr/ports &prompt.root; make search name=lsof Port: lsof-4.56.4 Path: /usr/ports/sysutils/lsof Info: Lists information about open files (similar to fstat(1)) Maint: obrien@FreeBSD.org Index: sysutils B-deps: R-deps: The part of the output you want to pay particular attention to is the Path: line, since that tells you where to find the port. The other information provided is not needed in order to install the port, so it will not be covered here. For more in-depth searching you can also use make search key=string where string is some text to search for. This searches port names, comments, descriptions and dependencies and can be used to find ports which relate to a particular subject if you do not know the name of the program you are looking for. In both of these cases, the search string is case-insensitive. Searching for LSOF will yield the same results as searching for lsof. Chern Lee Contributed by Using the Packages System Installing a Package packages installing pkg_add You can use the &man.pkg.add.1; utility to install a FreeBSD software package from a local file or from a server on the network. Downloading a Package Manually and Installing It Locally &prompt.root; ftp -a ftp2.FreeBSD.org Connected to ftp2.FreeBSD.org. 220 ftp2.FreeBSD.org FTP server (Version 6.00LS) ready. 331 Guest login ok, send your email address as password. 230- 230- This machine is in Vienna, VA, USA, hosted by Verio. 230- Questions? E-mail freebsd@vienna.verio.net. 230- 230- 230 Guest login ok, access restrictions apply. Remote system type is UNIX. Using binary mode to transfer files. ftp> cd /pub/FreeBSD/ports/packages/sysutils/ 250 CWD command successful. ftp> get lsof-4.56.4.tgz local: lsof-4.56.4.tgz remote: lsof-4.56.4.tgz 200 PORT command successful. 150 Opening BINARY mode data connection for 'lsof-4.56.4.tgz' (92375 bytes). 100% |**************************************************| 92375 00:00 ETA 226 Transfer complete. 92375 bytes received in 5.60 seconds (16.11 KB/s) ftp> exit &prompt.root; pkg_add lsof-4.56.4.tgz If you do not have a source of local packages (such as a FreeBSD CD-ROM set) then it will probably be easier to use the option to &man.pkg.add.1;. This will cause the utility to automatically determine the correct object format and release and then fetch and install the package from an FTP site. pkg_add &prompt.root; pkg_add -r lsof The example above would download the correct package and add it without any further user intervention. If you want to specify an alternative &os; Packages Mirror, instead of the main distribution site, you have to set the PACKAGESITE environment variable accordingly, to override the default settings. &man.pkg.add.1; uses &man.fetch.3; to download the files, which honors various environment variables, including FTP_PASSIVE_MODE, FTP_PROXY, and FTP_PASSWORD. You may need to set one or more of these if you are behind a firewall, or need to use an FTP/HTTP proxy. See &man.fetch.3; for the complete list. Note that in the example above lsof is used instead of lsof-4.56.4. When the remote fetching feature is used, the version number of the package must be removed. &man.pkg.add.1; will automatically fetch the latest version of the application. &man.pkg.add.1; will download the latest version of your application if you are using &os.current; or &os.stable;. If you run a -RELEASE version, it will grab the version of the package that was built with your release. It is possible to change this behavior by overriding PACKAGESITE. For example, if you run a &os; 5.4-RELEASE system, by default &man.pkg.add.1; will try to fetch packages from ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5.4-release/Latest/. If you want to force &man.pkg.add.1; to download &os; 5-STABLE packages, set PACKAGESITE to ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-5-stable/Latest/. Package files are distributed in .tgz and .tbz formats. You can find them at , or on the FreeBSD CD-ROM distribution. Every CD on the FreeBSD 4-CD set (and the PowerPak, etc.) contains packages in the /packages directory. The layout of the packages is similar to that of the /usr/ports tree. Each category has its own directory, and every package can be found within the All directory. The directory structure of the package system matches the ports layout; they work with each other to form the entire package/port system. Managing Packages packages managing &man.pkg.info.1; is a utility that lists and describes the various packages installed. pkg_info &prompt.root; pkg_info cvsup-16.1 A general network file distribution system optimized for CV docbook-1.2 Meta-port for the different versions of the DocBook DTD ... &man.pkg.version.1; is a utility that summarizes the versions of all installed packages. It compares the package version to the current version found in the ports tree. pkg_version &prompt.root; pkg_version cvsup = docbook = ... The symbols in the second column indicate the relative age of the installed version and the version available in the local ports tree. Symbol Meaning = The version of the installed package matches the one found in the local ports tree. < The installed version is older than the one available in the ports tree. >The installed version is newer than the one found in the local ports tree. (The local ports tree is probably out of date.) ?The installed package cannot be found in the ports index. (This can happen, for instance, if an installed port is removed from the Ports Collection or renamed.) *There are multiple versions of the package. !The installed package exists in the index but for some reason, pkg_version was unable to compare the version number of the installed package with the corresponding entry in the index. Deleting a Package pkg_delete packages deleting To remove a previously installed software package, use the &man.pkg.delete.1; utility. &prompt.root; pkg_delete xchat-1.7.1 Note that &man.pkg.delete.1; requires the full package name and number; the above command would not work if xchat was given instead of xchat-1.7.1. It is, however, easy to use &man.pkg.version.1; to find the version of the installed package. You could instead simply use a wildcard: &prompt.root; pkg_delete xchat\* in this case, all packages whose names start with xchat will be deleted. Miscellaneous All package information is stored within the /var/db/pkg directory. The installed file list and descriptions of each package can be found within files in this directory. Using the Ports Collection The following sections provide basic instructions on using the Ports Collection to install or remove programs from your system. The detailed description of available make targets and environment variables is available in &man.ports.7;. Obtaining the Ports Collection Before you can install ports, you must first obtain the Ports Collection—which is essentially a set of Makefiles, patches, and description files placed in /usr/ports. When installing your FreeBSD system, sysinstall asked if you would like to install the Ports Collection. If you chose no, you can follow these instructions to obtain the ports collection: CVSup Method This is a quick method for getting and keeping your copy of the Ports Collection up to date using CVSup protocol. If you want to learn more about CVSup, see Using CVSup. The implementation of CVSup protocol included with the &os; system is called csup. It first appeared in &os; 6.2. Users of older &os; releases can install it via the net/csup port/package. - Make sure /usr/ports + Make sure /usr/ports is empty before you run csup for the first time! If you already have the Ports Collection present, obtained from another source, csup will not prune removed patch files. Run csup: &prompt.root; csup -L 2 -h cvsup.FreeBSD.org /usr/share/examples/cvsup/ports-supfile Change cvsup.FreeBSD.org to a CVSup server near you. See CVSup Mirrors () for a complete listing of mirror sites. One may want to use his own ports-supfile, for example to avoid the need of passing the CVSup server on the command line. In this case, as root, copy /usr/share/examples/cvsup/ports-supfile to a new location, such as /root or your home directory. Edit ports-supfile. Change CHANGE_THIS.FreeBSD.org to a CVSup server near you. See CVSup Mirrors () for a complete listing of mirror sites. And now to run csup, use the following: &prompt.root; csup -L 2 /root/ports-supfile Running the &man.csup.1; command later will download and apply all the recent changes to your Ports Collection, except actually rebuilding the ports for your own system. Portsnap Method Portsnap is an alternative system for distributing the Ports Collection. It was first included in &os; 6.0. On older systems, you can install it from ports-mgmt/portsnap package: &prompt.root; pkg_add -r portsnap Please refer to Using Portsnap for a detailed description of all Portsnap features. Since &os; 6.1-RELEASE and with recent versions of the Portsnap port or package, you can safely skip this step. The /usr/ports will be created + class="directory">/usr/ports will be created automatically at first use of the &man.portsnap.8; command. With previous versions of Portsnap, you will have to create an empty directory /usr/ports if it does not + class="directory">/usr/ports if it does not exists: &prompt.root; mkdir /usr/ports Download a compressed snapshot of the Ports Collection into - /var/db/portsnap. You can + /var/db/portsnap. You can disconnect from the Internet after this step, if you wish. &prompt.root; portsnap fetch If you are running Portsnap for the first time, extract the snapshot into /usr/ports: + class="directory">/usr/ports: &prompt.root; portsnap extract If you already have a populated /usr/ports and you are just updating, + class="directory">/usr/ports and you are just updating, run the following command instead: &prompt.root; portsnap update Sysinstall Method This method involves using sysinstall to install the Ports Collection from the installation media. Note that the old copy of Ports Collection from the date of the release will be installed. If you have Internet access, you should always use one of the methods mentioned above. As root, run sysinstall (/stand/sysinstall in &os; versions older than 5.2) as shown below: &prompt.root; sysinstall Scroll down and select Configure, press Enter. Scroll down and select Distributions, press Enter. Scroll down to ports, press Space. Scroll up to Exit, press Enter. Select your desired installation media, such as CDROM, FTP, and so on. Scroll up to Exit and press Enter. Press X to exit sysinstall. Installing Ports ports installing The first thing that should be explained when it comes to the Ports Collection is what is actually meant by a skeleton. In a nutshell, a port skeleton is a minimal set of files that tell your FreeBSD system how to cleanly compile and install a program. Each port skeleton includes: A Makefile. The Makefile contains various statements that specify how the application should be compiled and where it should be installed on your system. A distinfo file. This file contains information about the files that must be downloaded to build the port, and their checksums (using &man.md5.1; and &man.sha256.1;), to verify that files have not been corrupted during the download. A files directory. This directory contains patches to make the program compile and install on your FreeBSD system. Patches are basically small files that specify changes to particular files. They are in plain text format, and basically say Remove line 10 or Change line 26 to this .... Patches are also known as diffs because they are generated by the &man.diff.1; program. This directory may also contain other files used to build the port. A pkg-descr file. This is a more detailed, often multiple-line, description of the program. A pkg-plist file. This is a list of all the files that will be installed by the port. It also tells the ports system what files to remove upon deinstallation. Some ports have other files, such as pkg-message. The ports system uses these files to handle special situations. If you want more details on these files, and on ports in general, check out the FreeBSD Porter's Handbook. The port includes instructions on how to build source code, but does not include the actual source code. You can get the source code from a CD-ROM or from the Internet. Source code is distributed in whatever manner the software author desires. Frequently this is a tarred and gzipped file, but it might be compressed with some other tool or even uncompressed. The program source code, whatever form it comes in, is called a distfile. The two methods for installing a &os; port are described below. You must be logged in as root to install ports. Before installing any port, you should be sure to have an up-to-date Ports Collection and you should check for security issues related to your port. A security vulnerabilities check can be automatically done by portaudit before any new application installation. This tool can be found in the Ports Collection (ports-mgmt/portaudit). Consider running portaudit -F before installing a new port, to fetch the current vulnerabilities database. A security audit and an update of the database will be performed during the daily security system check. For more information read the &man.portaudit.1; and &man.periodic.8; manual pages. The Ports Collection makes an assumption that you have a working Internet connection. If you do not, you will need to put a copy of the distfile into /usr/ports/distfiles manually. To begin, change to the directory for the port you want to install: &prompt.root; cd /usr/ports/sysutils/lsof Once inside the lsof directory, you will see the port skeleton. The next step is to compile, or build, the port. This is done by simply typing make at the prompt. Once you have done so, you should see something like this: &prompt.root; make >> lsof_4.57D.freebsd.tar.gz doesn't seem to exist in /usr/ports/distfiles/. >> Attempting to fetch from ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/. ===> Extracting for lsof-4.57 ... [extraction output snipped] ... >> Checksum OK for lsof_4.57D.freebsd.tar.gz. ===> Patching for lsof-4.57 ===> Applying FreeBSD patches for lsof-4.57 ===> Configuring for lsof-4.57 ... [configure output snipped] ... ===> Building for lsof-4.57 ... [compilation output snipped] ... &prompt.root; Notice that once the compile is complete you are returned to your prompt. The next step is to install the port. In order to install it, you simply need to tack one word onto the make command, and that word is install: &prompt.root; make install ===> Installing for lsof-4.57 ... [installation output snipped] ... ===> Generating temporary packing list ===> Compressing manual pages for lsof-4.57 ===> Registering installation for lsof-4.57 ===> SECURITY NOTE: This port has installed the following binaries which execute with increased privileges. &prompt.root; Once you are returned to your prompt, you should be able to run the application you just installed. Since lsof is a program that runs with increased privileges, a security warning is shown. During the building and installation of ports, you should take heed of any other warnings that may appear. It is a good idea to delete the working subdirectory, which contains all the temporary files used during compilation. Not only does it consume valuable disk space, but it would also cause problems later when upgrading to the newer version of the port. &prompt.root; make clean ===> Cleaning for lsof-4.57 &prompt.root; You can save two extra steps by just running make install clean instead of make, make install and make clean as three separate steps. Some shells keep a cache of the commands that are available in the directories listed in the PATH environment variable, to speed up lookup operations for the executable file of these commands. If you are using one of these shells, you might have to use the rehash command after installing a port, before the newly installed commands can be used. This command will work for shells like tcsh. Use the hash -r command for shells like sh. Look at the documentation for your shell for more information. Some third-party DVD-ROM products such as the FreeBSD Toolkit from the FreeBSD Mall contain distfiles. They can be used with the Ports Collection. Mount the DVD-ROM on /cdrom. If you use a different mount point, set CD_MOUNTPTS make variable. The needed distfiles will be automatically used if they are present on the disk. Please be aware that the licenses of a few ports do not allow for inclusion on the CD-ROM. This could be because a registration form needs to be filled out before downloading or redistribution is not allowed, or for another reason. If you wish to install a port not included on the CD-ROM, you will need to be online in order to do so. The ports system uses &man.fetch.1; to download the files, which honors various environment variables, including FTP_PASSIVE_MODE, FTP_PROXY, and FTP_PASSWORD. You may need to set one or more of these if you are behind a firewall, or need to use an FTP/HTTP proxy. See &man.fetch.3; for the complete list. For users which cannot be connected all the time, the make fetch option is provided. Just run this command at the top level directory (/usr/ports) and the required files will be downloaded for you. This command will also work in the lower level categories, for example: /usr/ports/net. Note that if a port depends on libraries or other ports this will not fetch the distfiles of those ports too. Replace fetch with fetch-recursive if you want to fetch all the dependencies of a port too. You can build all the ports in a category or as a whole by running make in the top level directory, just like the aforementioned make fetch method. This is dangerous, however, as some ports cannot co-exist. In other cases, some ports can install two different files with the same filename. In some rare cases, users may need to acquire the tarballs from a site other than the MASTER_SITES (the location where files are downloaded from). You can override the MASTER_SITES option with the following command: &prompt.root; cd /usr/ports/directory &prompt.root; make MASTER_SITE_OVERRIDE= \ ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/ fetch In this example we change the MASTER_SITES option to ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/. Some ports allow (or even require) you to provide build options which can enable/disable parts of the application which are unneeded, certain security options, and other customizations. A few which come to mind are www/mozilla, security/gpgme, and mail/sylpheed-claws. A message will be displayed when options such as these are available. Overriding the Default Ports Directories Sometimes it is useful (or mandatory) to use a different working and target directory. The WRKDIRPREFIX and PREFIX variables can override the default directories. For example: &prompt.root; make WRKDIRPREFIX=/usr/home/example/ports install will compile the port in /usr/home/example/ports and install everything under /usr/local. &prompt.root; make PREFIX=/usr/home/example/local install will compile it in /usr/ports and install it in /usr/home/example/local. And of course, &prompt.root; make WRKDIRPREFIX=../ports PREFIX=../local install will combine the two (it is too long to completely write on this page, but it should give you the general idea). Alternatively, these variables can also be set as part of your environment. Read the manual page for your shell for instructions on doing so. Dealing with <command>imake</command> Some ports that use imake (a part of the X Window System) do not work well with PREFIX, and will insist on installing under /usr/X11R6. Similarly, some Perl ports ignore PREFIX and install in the Perl tree. Making these ports respect PREFIX is a difficult or impossible job. Reconfiguring Ports When building certain ports, you may be presented with a ncurses-based menu from which you can select certain build options. It is not uncommon for users to wish to revisit this menu to add, remove, or change these options after a port has been built. There are many ways to do this. One option is to go into the directory containing the port and type make config, which will simply present the menu again with the same options selected. Another option is to use make showconfig, which will show you all the configuration options for the port. Yet another option is to execute make rmconfig which will remove all selected options and allow you to start over. All of these options, and others, are explained in great detail in in the man page for &man.ports.7;. Removing Installed Ports ports removing Now that you know how to install ports, you are probably wondering how to remove them, just in case you install one and later on decide that you installed the wrong port. We will remove our previous example (which was lsof for those of you not paying attention). Ports are being removed exactly the same as the packages (discussed in the Packages section), using the &man.pkg.delete.1; command: &prompt.root; pkg_delete lsof-4.57 Upgrading Ports ports upgrading First, list outdated ports that have a newer version available in the Ports Collection with the &man.pkg.version.1; command: &prompt.root; pkg_version -v <filename>/usr/ports/UPDATING</filename> Once you have updated your Ports Collection, before attempting a port upgrade, you should check /usr/ports/UPDATING. This file describes various issues and additional steps users may encounter and need to perform when updating a port, including such things as file format changes, changes in locations of configuration files, or other such incompatibilities with previous versions. If UPDATING contradicts something you read here, UPDATING takes precedence. Upgrading Ports using Portupgrade portupgrade The portupgrade utility is designed to easily upgrade installed ports. It is available from the ports-mgmt/portupgrade port. Install it like any other port, using the make install clean command: &prompt.root; cd /usr/ports/ports-mgmt/portupgrade &prompt.root; make install clean Scan the list of installed ports with the pkgdb -F command and fix all the inconsistencies it reports. It is a good idea to do this regularly, before every upgrade. When you run portupgrade -a, portupgrade will begin to upgrade all the outdated ports installed on your system. Use the flag if you want to be asked for confirmation of every individual upgrade. &prompt.root; portupgrade -ai If you want to upgrade only a certain application, not all available ports, use portupgrade pkgname. Include the flag if portupgrade should first upgrade all the ports required by the given application. &prompt.root; portupgrade -R firefox To use packages instead of ports for installation, provide flag. With this option portupgrade searches the local directories listed in PKG_PATH, or fetches packages from remote site if it is not found locally. If packages can not be found locally or fetched remotely, portupgrade will use ports. To avoid using ports, specify . &prompt.root; portupgrade -PP gnome2 To just fetch distfiles (or packages, if is specified) without building or installing anything, use . For further information see &man.portupgrade.1;. Upgrading Ports using Portmanager portmanager Portmanager is another utility for easy upgrading of installed ports. It is available from the ports-mgmt/portmanager port: - &prompt.root; cd /usr/ports/ports-mgmt/portmanager + &prompt.root; cd /usr/ports/ports-mgmt/portmanager &prompt.root; make install clean All the installed ports can be upgraded using this simple command: &prompt.root; portmanager -u You can add the flag to get asked for confirmation of every step Portmanager will perform. Portmanager can also be used to install new ports on the system. Unlike the usual make install clean command, it will upgrade all the dependencies prior to building and installing the selected port. &prompt.root; portmanager x11/gnome2 If there are any problems regarding the dependencies for the selected port, you can use Portmanager to rebuild all of them in the correct order. Once finished, the problematic port will be rebuilt too. &prompt.root; portmanager graphics/gimp -f For further information see &man.portmanager.1;. Upgrading Ports using Portmaster portmaster Portmaster is another utility for upgrading installed ports. Portmaster was designed make use of the tools found in the base system (it does not depend upon other ports) and uses the information in /var/db/pkg/ to determine which ports to upgrade. It is available from the ports-mgmt/portmaster port: - &prompt.root; cd /usr/ports/ports-mgmt/portmaster + &prompt.root; cd /usr/ports/ports-mgmt/portmaster &prompt.root; make install clean Portmaster groups ports into four categories: Root ports (no dependencies, not depended on) Trunk ports (no dependencies, are depended on) Branch ports (have dependencies, are depended on) Leaf ports (have dependencies, not depended on) You can list all the installed ports and search for updates using the option: &prompt.root; portmaster -L ===>>> Root ports (No dependencies, not depended on) ===>>> ispell-3.2.06_18 ===>>> screen-4.0.3 ===>>> New version available: screen-4.0.3_1 ===>>> tcpflow-0.21_1 ===>>> 7 root ports ... ===>>> Branch ports (Have dependencies, are depended on) ===>>> apache-2.2.3 ===>>> New version available: apache-2.2.8 ... ===>>> Leaf ports (Have dependencies, not depended on) ===>>> automake-1.9.6_2 ===>>> bash-3.1.17 ===>>> New version available: bash-3.2.33 ... ===>>> 32 leaf ports ===>>> 137 total installed ports ===>>> 83 have new versions available All the installed ports can be upgraded using this simple command: &prompt.root; portmaster -a By default, Portmaster will make a backup package before deleting the existing port. If the installation of the new version is successful, Portmaster will delete the backup. Using the will instruct Portmaster not to automatically delete the backup. Adding the option will start Portmaster in interactive mode, prompting you before upgrading each port. If you encounter errors during the upgrade process, you can use the option to upgrade/rebuild all ports: &prompt.root; portmaster -af You can also use Portmaster to install new ports on the system, upgrading all dependencies before building and installing the new port: &prompt.root; portmaster shells/bash Please see &man.portmaster.8; for more information. Ports and Disk Space ports disk-space Using the Ports Collection will use up disk space over time. After building and installing software from the ports, you should always remember to clean up the temporary work directories using the make clean command. You can sweep the whole Ports Collection with the following command: &prompt.root; portsclean -C You will accumulate a lot of old source distribution files in the distfiles directory over time. You can remove them by hand, or you can use the following command to delete all the distfiles that are no longer referenced by any ports: &prompt.root; portsclean -D Or to remove all distfiles not referenced by any port currently installed on your system: &prompt.root; portsclean -DD The portsclean utility is part of the portupgrade suite. Do not forget to remove the installed ports once you no longer need them. A nice tool to help automate this task is available from the ports-mgmt/pkg_cutleaves port. Post-installation Activities After installing a new application you will normally want to read any documentation it may have included, edit any configuration files that are required, ensure that the application starts at boot time (if it is a daemon), and so on. The exact steps you need to take to configure each application will obviously be different. However, if you have just installed a new application and are wondering What now? these tips might help: Use &man.pkg.info.1; to find out which files were installed, and where. For example, if you have just installed FooPackage version 1.0.0, then this command &prompt.root; pkg_info -L foopackage-1.0.0 | less will show all the files installed by the package. Pay special attention to files in man/ directories, which will be manual pages, etc/ directories, which will be configuration files, and doc/, which will be more comprehensive documentation. If you are not sure which version of the application was just installed, a command like this &prompt.root; pkg_info | grep -i foopackage will find all the installed packages that have foopackage in the package name. Replace foopackage in your command line as necessary. Once you have identified where the application's manual pages have been installed, review them using &man.man.1;. Similarly, look over the sample configuration files, and any additional documentation that may have been provided. If the application has a web site, check it for additional documentation, frequently asked questions, and so forth. If you are not sure of the web site address it may be listed in the output from &prompt.root; pkg_info foopackage-1.0.0 A WWW: line, if present, should provide a URL for the application's web site. Ports that should start at boot (such as Internet servers) will usually install a sample script in /usr/local/etc/rc.d. You should review this script for correctness and edit or rename it if needed. See Starting Services for more information. Dealing with Broken Ports If you come across a port that does not work for you, there are a few things you can do, including: Find out if there is a fix pending for the port in the Problem Report database. If so, you may be able to use the proposed fix. Ask the maintainer of the port for help. Type make maintainer or read the Makefile to find the maintainer's email address. Remember to include the name and version of the port (send the $FreeBSD: line from the Makefile) and the output leading up to the error when you email the maintainer. Some ports are not maintained by an individual but instead by a mailing list. Many, but not all, of these addresses look like freebsd-listname@FreeBSD.org. Please take this into account when phrasing your questions. In particular, ports shown as maintained by freebsd-ports@FreeBSD.org are actually not maintained by anyone. Fixes and support, if any, come from the general community who subscribe to that mailing list. More volunteers are always needed! If you do not get a response, you can use &man.send-pr.1; to submit a bug report (see Writing FreeBSD Problem Reports). Fix it! The Porter's Handbook includes detailed information on the Ports infrastructure so that you can fix the occasional broken port or even submit your own! Grab the package from an FTP site near you. The master package collection is on ftp.FreeBSD.org in the packages directory, but be sure to check your local mirror first! These are more likely to work than trying to compile from source and are a lot faster as well. Use the &man.pkg.add.1; program to install the package on your system. diff --git a/en_US.ISO8859-1/books/handbook/security/chapter.sgml b/en_US.ISO8859-1/books/handbook/security/chapter.sgml index b55da3de6e..789f3a0fca 100644 --- a/en_US.ISO8859-1/books/handbook/security/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/security/chapter.sgml @@ -1,4419 +1,4419 @@ Matthew Dillon Much of this chapter has been taken from the security(7) manual page by Security security Synopsis This chapter will provide a basic introduction to system security concepts, some general good rules of thumb, and some advanced topics under &os;. A lot of the topics covered here can be applied to system and Internet security in general as well. The Internet is no longer a friendly place in which everyone wants to be your kind neighbor. Securing your system is imperative to protect your data, intellectual property, time, and much more from the hands of hackers and the like. &os; provides an array of utilities and mechanisms to ensure the integrity and security of your system and network. After reading this chapter, you will know: Basic system security concepts, in respect to &os;. About the various crypt mechanisms available in &os;, such as DES and MD5. How to set up one-time password authentication. How to configure TCP Wrappers for use with inetd. How to set up KerberosIV on &os; releases prior to 5.0. How to set up Kerberos5 on &os;. How to configure IPsec and create a VPN between &os;/&windows; machines. How to configure and use OpenSSH, &os;'s SSH implementation. What file system ACLs are and how to use them. How to use the Portaudit utility to audit third party software packages installed from the Ports Collection. How to utilize the &os; security advisories publications. Have an idea of what Process Accounting is and how to enable it on &os;. Before reading this chapter, you should: Understand basic &os; and Internet concepts. Additional security topics are covered throughout this book. For example, Mandatory Access Control is discussed in and Internet Firewalls are discussed in . Introduction Security is a function that begins and ends with the system administrator. While all BSD &unix; multi-user systems have some inherent security, the job of building and maintaining additional security mechanisms to keep those users honest is probably one of the single largest undertakings of the sysadmin. Machines are only as secure as you make them, and security concerns are ever competing with the human necessity for convenience. &unix; systems, in general, are capable of running a huge number of simultaneous processes and many of these processes operate as servers — meaning that external entities can connect and talk to them. As yesterday's mini-computers and mainframes become today's desktops, and as computers become networked and inter-networked, security becomes an even bigger issue. System security also pertains to dealing with various forms of attack, including attacks that attempt to crash, or otherwise make a system unusable, but do not attempt to compromise the root account (break root). Security concerns can be split up into several categories: Denial of service attacks. User account compromises. Root compromise through accessible servers. Root compromise via user accounts. Backdoor creation. DoS attacks Denial of Service (DoS) security DoS attacks Denial of Service (DoS) Denial of Service (DoS) A denial of service attack is an action that deprives the machine of needed resources. Typically, DoS attacks are brute-force mechanisms that attempt to crash or otherwise make a machine unusable by overwhelming its servers or network stack. Some DoS attacks try to take advantage of bugs in the networking stack to crash a machine with a single packet. The latter can only be fixed by applying a bug fix to the kernel. Attacks on servers can often be fixed by properly specifying options to limit the load the servers incur on the system under adverse conditions. Brute-force network attacks are harder to deal with. A spoofed-packet attack, for example, is nearly impossible to stop, short of cutting your system off from the Internet. It may not be able to take your machine down, but it can saturate your Internet connection. security account compromises A user account compromise is even more common than a DoS attack. Many sysadmins still run standard telnetd, rlogind, rshd, and ftpd servers on their machines. These servers, by default, do not operate over encrypted connections. The result is that if you have any moderate-sized user base, one or more of your users logging into your system from a remote location (which is the most common and convenient way to login to a system) will have his or her password sniffed. The attentive system admin will analyze his remote access logs looking for suspicious source addresses even for successful logins. One must always assume that once an attacker has access to a user account, the attacker can break root. However, the reality is that in a well secured and maintained system, access to a user account does not necessarily give the attacker access to root. The distinction is important because without access to root the attacker cannot generally hide his tracks and may, at best, be able to do nothing more than mess with the user's files, or crash the machine. User account compromises are very common because users tend not to take the precautions that sysadmins take. security backdoors System administrators must keep in mind that there are potentially many ways to break root on a machine. The attacker may know the root password, the attacker may find a bug in a root-run server and be able to break root over a network connection to that server, or the attacker may know of a bug in a suid-root program that allows the attacker to break root once he has broken into a user's account. If an attacker has found a way to break root on a machine, the attacker may not have a need to install a backdoor. Many of the root holes found and closed to date involve a considerable amount of work by the attacker to cleanup after himself, so most attackers install backdoors. A backdoor provides the attacker with a way to easily regain root access to the system, but it also gives the smart system administrator a convenient way to detect the intrusion. Making it impossible for an attacker to install a backdoor may actually be detrimental to your security, because it will not close off the hole the attacker found to break in the first place. Security remedies should always be implemented with a multi-layered onion peel approach and can be categorized as follows: Securing root and staff accounts. Securing root–run servers and suid/sgid binaries. Securing user accounts. Securing the password file. Securing the kernel core, raw devices, and file systems. Quick detection of inappropriate changes made to the system. Paranoia. The next section of this chapter will cover the above bullet items in greater depth. Securing &os; security securing &os; Command vs. Protocol Throughout this document, we will use bold text to refer to an application, and a monospaced font to refer to specific commands. Protocols will use a normal font. This typographical distinction is useful for instances such as ssh, since it is a protocol as well as command. The sections that follow will cover the methods of securing your &os; system that were mentioned in the last section of this chapter. Securing the <username>root</username> Account and Staff Accounts su First off, do not bother securing staff accounts if you have not secured the root account. Most systems have a password assigned to the root account. The first thing you do is assume that the password is always compromised. This does not mean that you should remove the password. The password is almost always necessary for console access to the machine. What it does mean is that you should not make it possible to use the password outside of the console or possibly even with the &man.su.1; command. For example, make sure that your ptys are specified as being insecure in the /etc/ttys file so that direct root logins via telnet or rlogin are disallowed. If using other login services such as sshd, make sure that direct root logins are disabled there as well. You can do this by editing your /etc/ssh/sshd_config file, and making sure that PermitRootLogin is set to NO. Consider every access method — services such as FTP often fall through the cracks. Direct root logins should only be allowed via the system console. wheel Of course, as a sysadmin you have to be able to get to root, so we open up a few holes. But we make sure these holes require additional password verification to operate. One way to make root accessible is to add appropriate staff accounts to the wheel group (in /etc/group). The staff members placed in the wheel group are allowed to su to root. You should never give staff members native wheel access by putting them in the wheel group in their password entry. Staff accounts should be placed in a staff group, and then added to the wheel group via the /etc/group file. Only those staff members who actually need to have root access should be placed in the wheel group. It is also possible, when using an authentication method such as Kerberos, to use Kerberos' .k5login file in the root account to allow a &man.ksu.1; to root without having to place anyone at all in the wheel group. This may be the better solution since the wheel mechanism still allows an intruder to break root if the intruder has gotten hold of your password file and can break into a staff account. While having the wheel mechanism is better than having nothing at all, it is not necessarily the safest option. To lock an account completely, the &man.pw.8; command should be used: &prompt.root;pw lock staff This will prevent the user from logging in using any mechanism, including &man.ssh.1;. Another method of blocking access to accounts would be to replace the encrypted password with a single * character. This character would never match the encrypted password and thus block user access. For example, the following staff account: foobar:R9DT/Fa1/LV9U:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh Should be changed to this: foobar:*:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh This will prevent the foobar user from logging in using conventional methods. This method for access restriction is flawed on sites using Kerberos or in situations where the user has set up keys with &man.ssh.1;. These security mechanisms also assume that you are logging in from a more restrictive server to a less restrictive server. For example, if your main box is running all sorts of servers, your workstation should not be running any. In order for your workstation to be reasonably secure you should run as few servers as possible, up to and including no servers at all, and you should run a password-protected screen blanker. Of course, given physical access to a workstation an attacker can break any sort of security you put on it. This is definitely a problem that you should consider, but you should also consider the fact that the vast majority of break-ins occur remotely, over a network, from people who do not have physical access to your workstation or servers. KerberosIV Using something like Kerberos also gives you the ability to disable or change the password for a staff account in one place, and have it immediately affect all the machines on which the staff member may have an account. If a staff member's account gets compromised, the ability to instantly change his password on all machines should not be underrated. With discrete passwords, changing a password on N machines can be a mess. You can also impose re-passwording restrictions with Kerberos: not only can a Kerberos ticket be made to timeout after a while, but the Kerberos system can require that the user choose a new password after a certain period of time (say, once a month). Securing Root-run Servers and SUID/SGID Binaries ntalk comsat finger sandboxes sshd telnetd rshd rlogind The prudent sysadmin only runs the servers he needs to, no more, no less. Be aware that third party servers are often the most bug-prone. For example, running an old version of imapd or popper is like giving a universal root ticket out to the entire world. Never run a server that you have not checked out carefully. Many servers do not need to be run as root. For example, the ntalk, comsat, and finger daemons can be run in special user sandboxes. A sandbox is not perfect, unless you go through a large amount of trouble, but the onion approach to security still stands: If someone is able to break in through a server running in a sandbox, they still have to break out of the sandbox. The more layers the attacker must break through, the lower the likelihood of his success. Root holes have historically been found in virtually every server ever run as root, including basic system servers. If you are running a machine through which people only login via sshd and never login via telnetd or rshd or rlogind, then turn off those services! &os; now defaults to running ntalkd, comsat, and finger in a sandbox. Another program which may be a candidate for running in a sandbox is &man.named.8;. /etc/defaults/rc.conf includes the arguments necessary to run named in a sandbox in a commented-out form. Depending on whether you are installing a new system or upgrading an existing system, the special user accounts used by these sandboxes may not be installed. The prudent sysadmin would research and implement sandboxes for servers whenever possible. sendmail There are a number of other servers that typically do not run in sandboxes: sendmail, popper, imapd, ftpd, and others. There are alternatives to some of these, but installing them may require more work than you are willing to perform (the convenience factor strikes again). You may have to run these servers as root and rely on other mechanisms to detect break-ins that might occur through them. The other big potential root holes in a system are the suid-root and sgid binaries installed on the system. Most of these binaries, such as rlogin, reside in /bin, /sbin, /usr/bin, or /usr/sbin. While nothing is 100% safe, the system-default suid and sgid binaries can be considered reasonably safe. Still, root holes are occasionally found in these binaries. A root hole was found in Xlib in 1998 that made xterm (which is typically suid) vulnerable. It is better to be safe than sorry and the prudent sysadmin will restrict suid binaries, that only staff should run, to a special group that only staff can access, and get rid of (chmod 000) any suid binaries that nobody uses. A server with no display generally does not need an xterm binary. Sgid binaries can be almost as dangerous. If an intruder can break an sgid-kmem binary, the intruder might be able to read /dev/kmem and thus read the encrypted password file, potentially compromising any passworded account. Alternatively an intruder who breaks group kmem can monitor keystrokes sent through ptys, including ptys used by users who login through secure methods. An intruder that breaks the tty group can write to almost any user's tty. If a user is running a terminal program or emulator with a keyboard-simulation feature, the intruder can potentially generate a data stream that causes the user's terminal to echo a command, which is then run as that user. Securing User Accounts User accounts are usually the most difficult to secure. While you can impose draconian access restrictions on your staff and star out their passwords, you may not be able to do so with any general user accounts you might have. If you do have sufficient control, then you may win out and be able to secure the user accounts properly. If not, you simply have to be more vigilant in your monitoring of those accounts. Use of ssh and Kerberos for user accounts is more problematic, due to the extra administration and technical support required, but still a very good solution compared to a encrypted password file. Securing the Password File The only sure fire way is to star out as many passwords as you can and use ssh or Kerberos for access to those accounts. Even though the encrypted password file (/etc/spwd.db) can only be read by root, it may be possible for an intruder to obtain read access to that file even if the attacker cannot obtain root-write access. Your security scripts should always check for and report changes to the password file (see the Checking file integrity section below). Securing the Kernel Core, Raw Devices, and File systems If an attacker breaks root he can do just about anything, but there are certain conveniences. For example, most modern kernels have a packet sniffing device driver built in. Under &os; it is called the bpf device. An intruder will commonly attempt to run a packet sniffer on a compromised machine. You do not need to give the intruder the capability and most systems do not have the need for the bpf device compiled in. sysctl But even if you turn off the bpf device, you still have /dev/mem and /dev/kmem to worry about. For that matter, the intruder can still write to raw disk devices. Also, there is another kernel feature called the module loader, &man.kldload.8;. An enterprising intruder can use a KLD module to install his own bpf device, or other sniffing device, on a running kernel. To avoid these problems you have to run the kernel at a higher secure level, at least securelevel 1. The securelevel can be set with a sysctl on the kern.securelevel variable. Once you have set the securelevel to 1, write access to raw devices will be denied and special chflags flags, such as schg, will be enforced. You must also ensure that the schg flag is set on critical startup binaries, directories, and script files — everything that gets run up to the point where the securelevel is set. This might be overdoing it, and upgrading the system is much more difficult when you operate at a higher secure level. You may compromise and run the system at a higher secure level but not set the schg flag for every system file and directory under the sun. Another possibility is to simply mount / and /usr read-only. It should be noted that being too draconian in what you attempt to protect may prevent the all-important detection of an intrusion. Checking File Integrity: Binaries, Configuration Files, Etc. When it comes right down to it, you can only protect your core system configuration and control files so much before the convenience factor rears its ugly head. For example, using chflags to set the schg bit on most of the files in / and /usr is probably counterproductive, because while it may protect the files, it also closes a detection window. The last layer of your security onion is perhaps the most important — detection. The rest of your security is pretty much useless (or, worse, presents you with a false sense of security) if you cannot detect potential intrusions. Half the job of the onion is to slow down the attacker, rather than stop him, in order to be able to catch him in the act. The best way to detect an intrusion is to look for modified, missing, or unexpected files. The best way to look for modified files is from another (often centralized) limited-access system. Writing your security scripts on the extra-secure limited-access system makes them mostly invisible to potential attackers, and this is important. In order to take maximum advantage you generally have to give the limited-access box significant access to the other machines in the business, usually either by doing a read-only NFS export of the other machines to the limited-access box, or by setting up ssh key-pairs to allow the limited-access box to ssh to the other machines. Except for its network traffic, NFS is the least visible method — allowing you to monitor the file systems on each client box virtually undetected. If your limited-access server is connected to the client boxes through a switch, the NFS method is often the better choice. If your limited-access server is connected to the client boxes through a hub, or through several layers of routing, the NFS method may be too insecure (network-wise) and using ssh may be the better choice even with the audit-trail tracks that ssh lays. Once you have given a limited-access box at least read access to the client systems it is supposed to monitor, you must write scripts to do the actual monitoring. Given an NFS mount, you can write scripts out of simple system utilities such as &man.find.1; and &man.md5.1;. It is best to physically md5 the client-box files at least once a day, and to test control files such as those found in /etc and /usr/local/etc even more often. When mismatches are found, relative to the base md5 information the limited-access machine knows is valid, it should scream at a sysadmin to go check it out. A good security script will also check for inappropriate suid binaries and for new or deleted files on system partitions such as / and /usr. When using ssh rather than NFS, writing the security script is much more difficult. You essentially have to scp the scripts to the client box in order to run them, making them visible, and for safety you also need to scp the binaries (such as find) that those scripts use. The ssh client on the client box may already be compromised. All in all, using ssh may be necessary when running over insecure links, but it is also a lot harder to deal with. A good security script will also check for changes to user and staff members access configuration files: .rhosts, .shosts, .ssh/authorized_keys and so forth, files that might fall outside the purview of the MD5 check. If you have a huge amount of user disk space, it may take too long to run through every file on those partitions. In this case, setting mount flags to disallow suid binaries is a good idea. The nosuid option (see &man.mount.8;) is what you want to look into. You should probably scan them anyway, at least once a week, since the object of this layer is to detect a break-in attempt, whether or not the attempt succeeds. Process accounting (see &man.accton.8;) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in has occured. Finally, security scripts should process the log files, and the logs themselves should be generated in as secure a manner as possible — remote syslog can be very useful. An intruder will try to cover his tracks, and log files are critical to the sysadmin trying to track down the time and method of the initial break-in. One way to keep a permanent record of the log files is to run the system console to a serial port and collect the information to a secure machine monitoring the consoles. Paranoia A little paranoia never hurts. As a rule, a sysadmin can add any number of security features, as long as they do not affect convenience, and can add security features that do affect convenience with some added thought. Even more importantly, a security administrator should mix it up a bit — if you use recommendations such as those given by this document verbatim, you give away your methodologies to the prospective attacker who also has access to this document. Denial of Service Attacks Denial of Service (DoS) This section covers Denial of Service attacks. A DoS attack is typically a packet attack. While there is not much you can do about modern spoofed packet attacks that saturate your network, you can generally limit the damage by ensuring that the attacks cannot take down your servers by: Limiting server forks. Limiting springboard attacks (ICMP response attacks, ping broadcast, etc.). Overloading the Kernel Route Cache. A common DoS attack scenario is attacking a forking server and making it spawning so many child processes that the host system eventually runs out of memory, file descriptors, etc. and then grinds to a halt. inetd (see &man.inetd.8;) has several options to limit this sort of attack. It should be noted that while it is possible to prevent a machine from going down, it is not generally possible to prevent a service from being disrupted by the attack. Read the inetd manual page carefully and pay specific attention to the , , and options. Note that spoofed-IP attacks will circumvent the option to inetd, so typically a combination of options must be used. Some standalone servers have self-fork-limitation parameters. Sendmail has its option, which tends to work much better than trying to use Sendmail's load limiting options due to the load lag. You should specify a MaxDaemonChildren parameter, when you start sendmail; high enough to handle your expected load, but not so high that the computer cannot handle that number of Sendmail instances without falling on its face. It is also prudent to run Sendmail in queued mode () and to run the daemon (sendmail -bd) separate from the queue-runs (sendmail -q15m). If you still want real-time delivery you can run the queue at a much lower interval, such as , but be sure to specify a reasonable MaxDaemonChildren option for that Sendmail to prevent cascade failures. Syslogd can be attacked directly and it is strongly recommended that you use the option whenever possible, and the option otherwise. You should also be fairly careful with connect-back services such as TCP Wrapper's reverse-identd, which can be attacked directly. You generally do not want to use the reverse-ident feature of TCP Wrapper for this reason. It is a very good idea to protect internal services from external access by firewalling them off at your border routers. The idea here is to prevent saturation attacks from outside your LAN, not so much to protect internal services from network-based root compromise. Always configure an exclusive firewall, i.e., firewall everything except ports A, B, C, D, and M-Z. This way you can firewall off all of your low ports except for certain specific services such as named (if you are primary for a zone), ntalkd, sendmail, and other Internet-accessible services. If you try to configure the firewall the other way — as an inclusive or permissive firewall, there is a good chance that you will forget to close a couple of services, or that you will add a new internal service and forget to update the firewall. You can still open up the high-numbered port range on the firewall, to allow permissive-like operation, without compromising your low ports. Also take note that &os; allows you to control the range of port numbers used for dynamic binding, via the various net.inet.ip.portrange sysctl's (sysctl -a | fgrep portrange), which can also ease the complexity of your firewall's configuration. For example, you might use a normal first/last range of 4000 to 5000, and a hiport range of 49152 to 65535, then block off everything under 4000 in your firewall (except for certain specific Internet-accessible ports, of course). Another common DoS attack is called a springboard attack — to attack a server in a manner that causes the server to generate responses which overloads the server, the local network, or some other machine. The most common attack of this nature is the ICMP ping broadcast attack. The attacker spoofs ping packets sent to your LAN's broadcast address with the source IP address set to the actual machine they wish to attack. If your border routers are not configured to stomp on ping packets to broadcast addresses, your LAN winds up generating sufficient responses to the spoofed source address to saturate the victim, especially when the attacker uses the same trick on several dozen broadcast addresses over several dozen different networks at once. Broadcast attacks of over a hundred and twenty megabits have been measured. A second common springboard attack is against the ICMP error reporting system. By constructing packets that generate ICMP error responses, an attacker can saturate a server's incoming network and cause the server to saturate its outgoing network with ICMP responses. This type of attack can also crash the server by running it out of memory, especially if the server cannot drain the ICMP responses it generates fast enough. Use the sysctl variable net.inet.icmp.icmplim to limit these attacks. The last major class of springboard attacks is related to certain internal inetd services such as the udp echo service. An attacker simply spoofs a UDP packet with the source address being server A's echo port, and the destination address being server B's echo port, where server A and B are both on your LAN. The two servers then bounce this one packet back and forth between each other. The attacker can overload both servers and their LANs simply by injecting a few packets in this manner. Similar problems exist with the internal chargen port. A competent sysadmin will turn off all of these inetd-internal test services. Spoofed packet attacks may also be used to overload the kernel route cache. Refer to the net.inet.ip.rtexpire, rtminexpire, and rtmaxcache sysctl parameters. A spoofed packet attack that uses a random source IP will cause the kernel to generate a temporary cached route in the route table, viewable with netstat -rna | fgrep W3. These routes typically timeout in 1600 seconds or so. If the kernel detects that the cached route table has gotten too big it will dynamically reduce the rtexpire but will never decrease it to less than rtminexpire. There are two problems: The kernel does not react quickly enough when a lightly loaded server is suddenly attacked. The rtminexpire is not low enough for the kernel to survive a sustained attack. If your servers are connected to the Internet via a T3 or better, it may be prudent to manually override both rtexpire and rtminexpire via &man.sysctl.8;. Never set either parameter to zero (unless you want to crash the machine). Setting both parameters to 2 seconds should be sufficient to protect the route table from attack. Access Issues with Kerberos and SSH ssh KerberosIV There are a few issues with both Kerberos and ssh that need to be addressed if you intend to use them. Kerberos 5 is an excellent authentication protocol, but there are bugs in the kerberized telnet and rlogin applications that make them unsuitable for dealing with binary streams. Also, by default Kerberos does not encrypt a session unless you use the option. ssh encrypts everything by default. Ssh works quite well in every respect except that it forwards encryption keys by default. What this means is that if you have a secure workstation holding keys that give you access to the rest of the system, and you ssh to an insecure machine, your keys are usable. The actual keys themselves are not exposed, but ssh installs a forwarding port for the duration of your login, and if an attacker has broken root on the insecure machine he can utilize that port to use your keys to gain access to any other machine that your keys unlock. We recommend that you use ssh in combination with Kerberos whenever possible for staff logins. Ssh can be compiled with Kerberos support. This reduces your reliance on potentially exposed ssh keys while at the same time protecting passwords via Kerberos. Ssh keys should only be used for automated tasks from secure machines (something that Kerberos is unsuited to do). We also recommend that you either turn off key-forwarding in the ssh configuration, or that you make use of the from=IP/DOMAIN option that ssh allows in its authorized_keys file to make the key only usable to entities logging in from specific machines. Bill Swingle Parts rewritten and updated by DES, Blowfish, MD5, and Crypt security crypt crypt Blowfish DES MD5 Every user on a &unix; system has a password associated with their account. It seems obvious that these passwords need to be known only to the user and the actual operating system. In order to keep these passwords secret, they are encrypted with what is known as a one-way hash, that is, they can only be easily encrypted but not decrypted. In other words, what we told you a moment ago was obvious is not even true: the operating system itself does not really know the password. It only knows the encrypted form of the password. The only way to get the plain-text password is by a brute force search of the space of possible passwords. Unfortunately the only secure way to encrypt passwords when &unix; came into being was based on DES, the Data Encryption Standard. This was not such a problem for users resident in the US, but since the source code for DES could not be exported outside the US, &os; had to find a way to both comply with US law and retain compatibility with all the other &unix; variants that still used DES. The solution was to divide up the encryption libraries so that US users could install the DES libraries and use DES but international users still had an encryption method that could be exported abroad. This is how &os; came to use MD5 as its default encryption method. MD5 is believed to be more secure than DES, so installing DES is offered primarily for compatibility reasons. Recognizing Your Crypt Mechanism Currently the library supports DES, MD5 and Blowfish hash functions. By default &os; uses MD5 to encrypt passwords. It is pretty easy to identify which encryption method &os; is set up to use. Examining the encrypted passwords in the /etc/master.passwd file is one way. Passwords encrypted with the MD5 hash are longer than those encrypted with the DES hash and also begin with the characters $1$. Passwords starting with $2a$ are encrypted with the Blowfish hash function. DES password strings do not have any particular identifying characteristics, but they are shorter than MD5 passwords, and are coded in a 64-character alphabet which does not include the $ character, so a relatively short string which does not begin with a dollar sign is very likely a DES password. The password format used for new passwords is controlled by the passwd_format login capability in /etc/login.conf, which takes values of des, md5 or blf. See the &man.login.conf.5; manual page for more information about login capabilities. One-time Passwords one-time passwords security one-time passwords By default, &os; includes support for OPIE (One-time Passwords In Everything), which uses the MD5 hash by default. There are three different sorts of passwords which we will discuss below. The first is your usual &unix; style or Kerberos password; we will call this a &unix; password. The second sort is the one-time password which is generated by the OPIE &man.opiekey.1; program and accepted by the &man.opiepasswd.1; program and the login prompt; we will call this a one-time password. The final sort of password is the secret password which you give to the opiekey program (and sometimes the opiepasswd programs) which it uses to generate one-time passwords; we will call it a secret password or just unqualified password. The secret password does not have anything to do with your &unix; password; they can be the same but this is not recommended. OPIE secret passwords are not limited to 8 characters like old &unix; passwordsUnder &os; the standard login password may be up to 128 characters in length., they can be as long as you like. Passwords of six or seven word long phrases are fairly common. For the most part, the OPIE system operates completely independently of the &unix; password system. Besides the password, there are two other pieces of data that are important to OPIE. One is what is known as the seed or key, consisting of two letters and five digits. The other is what is called the iteration count, a number between 1 and 100. OPIE creates the one-time password by concatenating the seed and the secret password, then applying the MD5 hash as many times as specified by the iteration count and turning the result into six short English words. These six English words are your one-time password. The authentication system (primarily PAM) keeps track of the last one-time password used, and the user is authenticated if the hash of the user-provided password is equal to the previous password. Because a one-way hash is used it is impossible to generate future one-time passwords if a successfully used password is captured; the iteration count is decremented after each successful login to keep the user and the login program in sync. When the iteration count gets down to 1, OPIE must be reinitialized. There are a few programs involved in each system which we will discuss below. The opiekey program accepts an iteration count, a seed, and a secret password, and generates a one-time password or a consecutive list of one-time passwords. The opiepasswd program is used to initialize OPIE, and to change passwords, iteration counts, or seeds; it takes either a secret passphrase, or an iteration count, seed, and a one-time password. The opieinfo program will examine the relevant credentials files (/etc/opiekeys) and print out the invoking user's current iteration count and seed. There are four different sorts of operations we will cover. The first is using opiepasswd over a secure connection to set up one-time-passwords for the first time, or to change your password or seed. The second operation is using opiepasswd over an insecure connection, in conjunction with opiekey over a secure connection, to do the same. The third is using opiekey to log in over an insecure connection. The fourth is using opiekey to generate a number of keys which can be written down or printed out to carry with you when going to some location without secure connections to anywhere. Secure Connection Initialization To initialize OPIE for the first time, execute the opiepasswd command: &prompt.user; opiepasswd -c [grimreaper] ~ $ opiepasswd -f -c Adding unfurl: Only use this method from the console; NEVER from remote. If you are using telnet, xterm, or a dial-in, type ^C now or exit with no password. Then run opiepasswd without the -c parameter. Using MD5 to compute responses. Enter new secret pass phrase: Again new secret pass phrase: ID unfurl OTP key is 499 to4268 MOS MALL GOAT ARM AVID COED At the Enter new secret pass phrase: or Enter secret password: prompts, you should enter a password or phrase. Remember, this is not the password that you will use to login with, this is used to generate your one-time login keys. The ID line gives the parameters of your particular instance: your login name, the iteration count, and seed. When logging in the system will remember these parameters and present them back to you so you do not have to remember them. The last line gives the particular one-time password which corresponds to those parameters and your secret password; if you were to re-login immediately, this one-time password is the one you would use. Insecure Connection Initialization To initialize or change your secret password over an insecure connection, you will need to already have a secure connection to some place where you can run opiekey; this might be in the form of a shell prompt on a machine you trust. You will also need to make up an iteration count (100 is probably a good value), and you may make up your own seed or use a randomly-generated one. Over on the insecure connection (to the machine you are initializing), use opiepasswd: &prompt.user; opiepasswd Updating unfurl: You need the response from an OTP generator. Old secret pass phrase: otp-md5 498 to4268 ext Response: GAME GAG WELT OUT DOWN CHAT New secret pass phrase: otp-md5 499 to4269 Response: LINE PAP MILK NELL BUOY TROY ID mark OTP key is 499 gr4269 LINE PAP MILK NELL BUOY TROY To accept the default seed press Return. Then before entering an access password, move over to your secure connection and give it the same parameters: &prompt.user; opiekey 498 to4268 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: GAME GAG WELT OUT DOWN CHAT Now switch back over to the insecure connection, and copy the one-time password generated over to the relevant program. Generating a Single One-time Password Once you have initialized OPIE and login, you will be presented with a prompt like this: &prompt.user; telnet example.com Trying 10.0.0.1... Connected to example.com Escape character is '^]'. FreeBSD/i386 (example.com) (ttypa) login: <username> otp-md5 498 gr4269 ext Password: As a side note, the OPIE prompts have a useful feature (not shown here): if you press Return at the password prompt, the prompter will turn echo on, so you can see what you are typing. This can be extremely useful if you are attempting to type in a password by hand, such as from a printout. MS-DOS Windows MacOS At this point you need to generate your one-time password to answer this login prompt. This must be done on a trusted system that you can run opiekey on. (There are versions of these for DOS, &windows; and &macos; as well.) They need the iteration count and the seed as command line options. You can cut-and-paste these right from the login prompt on the machine that you are logging in to. On the trusted system: &prompt.user; opiekey 498 to4268 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: GAME GAG WELT OUT DOWN CHAT Now that you have your one-time password you can continue logging in. Generating Multiple One-time Passwords Sometimes you have to go places where you do not have access to a trusted machine or secure connection. In this case, it is possible to use the opiekey command to generate a number of one-time passwords beforehand to be printed out and taken with you. For example: &prompt.user; opiekey -n 5 30 zz99999 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: <secret password> 26: JOAN BORE FOSS DES NAY QUIT 27: LATE BIAS SLAY FOLK MUCH TRIG 28: SALT TIN ANTI LOON NEAL USE 29: RIO ODIN GO BYE FURY TIC 30: GREW JIVE SAN GIRD BOIL PHI The requests five keys in sequence, the specifies what the last iteration number should be. Note that these are printed out in reverse order of eventual use. If you are really paranoid, you might want to write the results down by hand; otherwise you can cut-and-paste into lpr. Note that each line shows both the iteration count and the one-time password; you may still find it handy to scratch off passwords as you use them. Restricting Use of &unix; Passwords OPIE can restrict the use of &unix; passwords based on the IP address of a login session. The relevant file is /etc/opieaccess, which is present by default. Please check &man.opieaccess.5; for more information on this file and which security considerations you should be aware of when using it. Here is a sample opieaccess file: permit 192.168.0.0 255.255.0.0 This line allows users whose IP source address (which is vulnerable to spoofing) matches the specified value and mask, to use &unix; passwords at any time. If no rules in opieaccess are matched, the default is to deny non-OPIE logins. Tom Rhodes Written by TCP Wrappers TCP Wrappers Anyone familiar with &man.inetd.8; has probably heard of TCP Wrappers at some point. But few individuals seem to fully comprehend its usefulness in a network environment. It seems that everyone wants to install a firewall to handle network connections. While a firewall has a wide variety of uses, there are some things that a firewall not handle such as sending text back to the connection originator. The TCP Wrappers software does this and much more. In the next few sections many of the TCP Wrappers features will be discussed, and, when applicable, example configuration lines will be provided. The TCP Wrappers software extends the abilities of inetd to provide support for every server daemon under its control. Using this method it is possible to provide logging support, return messages to connections, permit a daemon to only accept internal connections, etc. While some of these features can be provided by implementing a firewall, this will add not only an extra layer of protection but go beyond the amount of control a firewall can provide. The added functionality of TCP Wrappers should not be considered a replacement for a good firewall. TCP Wrappers can be used in conjunction with a firewall or other security enhancements though and it can serve nicely as an extra layer of protection for the system. Since this is an extension to the configuration of inetd, the reader is expected have read the inetd configuration section. While programs run by &man.inetd.8; are not exactly daemons, they have traditionally been called daemons. This is the term we will use in this section too. Initial Configuration The only requirement of using TCP Wrappers in &os; is to ensure the inetd server is started from rc.conf with the option; this is the default setting. Of course, proper configuration of /etc/hosts.allow is also expected, but &man.syslogd.8; will throw messages in the system logs in these cases. Unlike other implementations of TCP Wrappers, the use of hosts.deny has been deprecated. All configuration options should be placed in /etc/hosts.allow. In the simplest configuration, daemon connection policies are set to either be permitted or blocked depending on the options in /etc/hosts.allow. The default configuration in &os; is to allow a connection to every daemon started with inetd. Changing this will be discussed only after the basic configuration is covered. Basic configuration usually takes the form of daemon : address : action. Where daemon is the daemon name which inetd started. The address can be a valid hostname, an IP address or an IPv6 address enclosed in brackets ([ ]). The action field can be either allow or deny to grant or deny access appropriately. Keep in mind that configuration works off a first rule match semantic, meaning that the configuration file is scanned in ascending order for a matching rule. When a match is found the rule is applied and the search process will halt. Several other options exist but they will be explained in a later section. A simple configuration line may easily be constructed from that information alone. For example, to allow POP3 connections via the mail/qpopper daemon, the following lines should be appended to hosts.allow: # This line is required for POP3 connections: qpopper : ALL : allow After adding this line, inetd will need to be restarted. This can be accomplished by use of the &man.kill.1; command, or with the restart parameter with /etc/rc.d/inetd. Advanced Configuration TCP Wrappers has advanced options too; they will allow for more control over the way connections are handled. In some cases it may be a good idea to return a comment to certain hosts or daemon connections. In other cases, perhaps a log file should be recorded or an email sent to the administrator. Other situations may require the use of a service for local connections only. This is all possible through the use of configuration options known as wildcards, expansion characters and external command execution. The next two sections are written to cover these situations. External Commands Suppose that a situation occurs where a connection should be denied yet a reason should be sent to the individual who attempted to establish that connection. How could it be done? That action can be made possible by using the option. When a connection attempt is made, will be called to execute a shell command or script. An example already exists in the hosts.allow file: # The rest of the daemons are protected. ALL : ALL \ : severity auth.info \ : twist /bin/echo "You are not welcome to use %d from %h." This example shows that the message, You are not allowed to use daemon from hostname. will be returned for any daemon not previously configured in the access file. This is extremely useful for sending a reply back to the connection initiator right after the established connection is dropped. Note that any message returned must be wrapped in quote " characters; there are no exceptions to this rule. It may be possible to launch a denial of service attack on the server if an attacker, or group of attackers could flood these daemons with connection requests. Another possibility is to use the option in these cases. Like , the implicitly denies the connection and may be used to run external shell commands or scripts. Unlike , will not send a reply back to the individual who established the connection. For an example, consider the following configuration line: # We do not allow connections from example.com: ALL : .example.com \ : spawn (/bin/echo %a from %h attempted to access %d >> \ /var/log/connections.log) \ : deny This will deny all connection attempts from the *.example.com domain; simultaneously logging the hostname, IP address and the daemon which they attempted to access in the /var/log/connections.log file. Aside from the already explained substitution characters above, e.g. %a, a few others exist. See the &man.hosts.access.5; manual page for the complete list. Wildcard Options Thus far the ALL example has been used continuously throughout the examples. Other options exist which could extend the functionality a bit further. For instance, ALL may be used to match every instance of either a daemon, domain or an IP address. Another wildcard available is PARANOID which may be used to match any host which provides an IP address that may be forged. In other words, paranoid may be used to define an action to be taken whenever a connection is made from an IP address that differs from its hostname. The following example may shed some more light on this discussion: # Block possibly spoofed requests to sendmail: sendmail : PARANOID : deny In that example all connection requests to sendmail which have an IP address that varies from its hostname will be denied. Using the PARANOID may severely cripple servers if the client or server has a broken DNS setup. Administrator discretion is advised. To learn more about wildcards and their associated functionality, see the &man.hosts.access.5; manual page. Before any of the specific configuration lines above will work, the first configuration line should be commented out in hosts.allow. This was noted at the beginning of this section. Mark Murray Contributed by Mark Dapoz Based on a contribution by <application>KerberosIV</application> Kerberos is a network add-on system/protocol that allows users to authenticate themselves through the services of a secure server. Services such as remote login, remote copy, secure inter-system file copying and other high-risk tasks are made considerably safer and more controllable. The following instructions can be used as a guide on how to set up Kerberos as distributed for &os;. However, you should refer to the relevant manual pages for a complete description. Installing <application>KerberosIV</application> MIT KerberosIV installing Kerberos is an optional component of &os;. The easiest way to install this software is by selecting the krb4 or krb5 distribution in sysinstall during the initial installation of &os;. This will install the eBones (KerberosIV) or Heimdal (Kerberos5) implementation of Kerberos. These implementations are included because they are developed outside the USA/Canada and were thus available to system owners outside those countries during the era of restrictive export controls on cryptographic code from the USA. Alternatively, the MIT implementation of Kerberos is available from the Ports Collection as security/krb5. Creating the Initial Database This is done on the Kerberos server only. First make sure that you do not have any old Kerberos databases around. You should change to the directory /etc/kerberosIV and check that only the following files are present: &prompt.root; cd /etc/kerberosIV &prompt.root; ls README krb.conf krb.realms If any additional files (such as principal.* or master_key) exist, then use the kdb_destroy command to destroy the old Kerberos database, or if Kerberos is not running, simply delete the extra files. You should now edit the krb.conf and krb.realms files to define your Kerberos realm. In this case the realm will be EXAMPLE.COM and the server is grunt.example.com. We edit or create the krb.conf file: &prompt.root; cat krb.conf EXAMPLE.COM EXAMPLE.COM grunt.example.com admin server CS.BERKELEY.EDU okeeffe.berkeley.edu ATHENA.MIT.EDU kerberos.mit.edu ATHENA.MIT.EDU kerberos-1.mit.edu ATHENA.MIT.EDU kerberos-2.mit.edu ATHENA.MIT.EDU kerberos-3.mit.edu LCS.MIT.EDU kerberos.lcs.mit.edu TELECOM.MIT.EDU bitsy.mit.edu ARC.NASA.GOV trident.arc.nasa.gov In this case, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to not include them for simplicity. The first line names the realm in which this system works. The other lines contain realm/host entries. The first item on a line is a realm, and the second is a host in that realm that is acting as a key distribution center. The words admin server following a host's name means that host also provides an administrative database server. For further explanation of these terms, please consult the Kerberos manual pages. Now we have to add grunt.example.com to the EXAMPLE.COM realm and also add an entry to put all hosts in the .example.com domain in the EXAMPLE.COM realm. The krb.realms file would be updated as follows: &prompt.root; cat krb.realms grunt.example.com EXAMPLE.COM .example.com EXAMPLE.COM .berkeley.edu CS.BERKELEY.EDU .MIT.EDU ATHENA.MIT.EDU .mit.edu ATHENA.MIT.EDU Again, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to remove them to simplify things. The first line puts the specific system into the named realm. The rest of the lines show how to default systems of a particular subdomain to a named realm. Now we are ready to create the database. This only needs to run on the Kerberos server (or Key Distribution Center). Issue the kdb_init command to do this: &prompt.root; kdb_init Realm name [default ATHENA.MIT.EDU ]: EXAMPLE.COM You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter Kerberos master key: Now we have to save the key so that servers on the local machine can pick it up. Use the kstash command to do this: &prompt.root; kstash Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! This saves the encrypted master password in /etc/kerberosIV/master_key. Making It All Run KerberosIV initial startup Two principals need to be added to the database for each system that will be secured with Kerberos. Their names are kpasswd and rcmd. These two principals are made for each system, with the instance being the name of the individual system. These daemons, kpasswd and rcmd allow other systems to change Kerberos passwords and run commands like &man.rcp.1;, &man.rlogin.1; and &man.rsh.1;. Now let us add these entries: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: passwd Instance: grunt <Not found>, Create [y] ? y Principal: passwd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? y Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: rcmd Instance: grunt <Not found>, Create [y] ? Principal: rcmd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Creating the Server File We now have to extract all the instances which define the services on each machine. For this we use the ext_srvtab command. This will create a file which must be copied or moved by secure means to each Kerberos client's /etc directory. This file must be present on each server and client, and is crucial to the operation of Kerberos. &prompt.root; ext_srvtab grunt Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Generating 'grunt-new-srvtab'.... Now, this command only generates a temporary file which must be renamed to srvtab so that all the servers can pick it up. Use the &man.mv.1; command to move it into place on the original system: &prompt.root; mv grunt-new-srvtab srvtab If the file is for a client system, and the network is not deemed safe, then copy the client-new-srvtab to removable media and transport it by secure physical means. Be sure to rename it to srvtab in the client's /etc directory, and make sure it is mode 600: &prompt.root; mv grumble-new-srvtab srvtab &prompt.root; chmod 600 srvtab Populating the Database We now have to add some user entries into the database. First let us create an entry for the user jane. Use the kdb_edit command to do this: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: <Not found>, Create [y] ? y Principal: jane, Instance: , kdc_key_ver: 1 New Password: <---- enter a secure password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Testing It All Out First we have to start the Kerberos daemons. Note that if you have correctly edited your /etc/rc.conf then this will happen automatically when you reboot. This is only necessary on the Kerberos server. Kerberos clients will automatically get what they need from the /etc/kerberosIV directory. &prompt.root; kerberos & Kerberos server starting Sleep forever on error Log file is /var/log/kerberos.log Current Kerberos master key version is 1. Master key entered. BEWARE! Current Kerberos master key version is 1 Local realm: EXAMPLE.COM &prompt.root; kadmind -n & KADM Server KADM0.0A initializing Please do not use 'kill -9' to kill this job, use a regular kill instead Current Kerberos master key version is 1. Master key entered. BEWARE! Now we can try using the kinit command to get a ticket for the ID jane that we created above: &prompt.user; kinit jane MIT Project Athena (grunt.example.com) Kerberos Initialization for "jane" Password: Try listing the tokens using klist to see if we really have them: &prompt.user; klist Ticket file: /tmp/tkt245 Principal: jane@EXAMPLE.COM Issued Expires Principal Apr 30 11:23:22 Apr 30 19:23:22 krbtgt.EXAMPLE.COM@EXAMPLE.COM Now try changing the password using &man.passwd.1; to check if the kpasswd daemon can get authorization to the Kerberos database: &prompt.user; passwd realm EXAMPLE.COM Old password for jane: New Password for jane: Verifying password New Password for jane: Password changed. Adding <command>su</command> Privileges Kerberos allows us to give each user who needs root privileges their own separate &man.su.1; password. We could now add an ID which is authorized to &man.su.1; to root. This is controlled by having an instance of root associated with a principal. Using kdb_edit we can create the entry jane.root in the Kerberos database: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: root <Not found>, Create [y] ? y Principal: jane, Instance: root, kdc_key_ver: 1 New Password: <---- enter a SECURE password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? 12 <--- Keep this short! Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Now try getting tokens for it to make sure it works: &prompt.root; kinit jane.root MIT Project Athena (grunt.example.com) Kerberos Initialization for "jane.root" Password: Now we need to add the user to root's .klogin file: &prompt.root; cat /root/.klogin jane.root@EXAMPLE.COM Now try doing the &man.su.1;: &prompt.user; su Password: and take a look at what tokens we have: &prompt.root; klist Ticket file: /tmp/tkt_root_245 Principal: jane.root@EXAMPLE.COM Issued Expires Principal May 2 20:43:12 May 3 04:43:12 krbtgt.EXAMPLE.COM@EXAMPLE.COM Using Other Commands In an earlier example, we created a principal called jane with an instance root. This was based on a user with the same name as the principal, and this is a Kerberos default; that a <principal>.<instance> of the form <username>.root will allow that <username> to &man.su.1; to root if the necessary entries are in the .klogin file in root's home directory: &prompt.root; cat /root/.klogin jane.root@EXAMPLE.COM Likewise, if a user has in their own home directory lines of the form: &prompt.user; cat ~/.klogin jane@EXAMPLE.COM jack@EXAMPLE.COM This allows anyone in the EXAMPLE.COM realm who has authenticated themselves as jane or jack (via kinit, see above) to access to jane's account or files on this system (grunt) via &man.rlogin.1;, &man.rsh.1; or &man.rcp.1;. For example, jane now logs into another system using Kerberos: &prompt.user; kinit MIT Project Athena (grunt.example.com) Password: &prompt.user; rlogin grunt Last login: Mon May 1 21:14:47 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 Or jack logs into jane's account on the same machine (jane having set up the .klogin file as above, and the person in charge of Kerberos having set up principal jack with a null instance): &prompt.user; kinit &prompt.user; rlogin grunt -l jane MIT Project Athena (grunt.example.com) Password: Last login: Mon May 1 21:16:55 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 Tillman Hodgson Contributed by Mark Murray Based on a contribution by <application>Kerberos5</application> Every &os; release beyond &os;-5.1 includes support only for Kerberos5. Hence Kerberos5 is the only version included, and its configuration is similar in many aspects to that of KerberosIV. The following information only applies to Kerberos5 in post &os;-5.0 releases. Users who wish to use the KerberosIV package may install the security/krb4 port. Kerberos is a network add-on system/protocol that allows users to authenticate themselves through the services of a secure server. Services such as remote login, remote copy, secure inter-system file copying and other high-risk tasks are made considerably safer and more controllable. Kerberos can be described as an identity-verifying proxy system. It can also be described as a trusted third-party authentication system. Kerberos provides only one function — the secure authentication of users on the network. It does not provide authorization functions (what users are allowed to do) or auditing functions (what those users did). After a client and server have used Kerberos to prove their identity, they can also encrypt all of their communications to assure privacy and data integrity as they go about their business. Therefore it is highly recommended that Kerberos be used with other security methods which provide authorization and audit services. The following instructions can be used as a guide on how to set up Kerberos as distributed for &os;. However, you should refer to the relevant manual pages for a complete description. For purposes of demonstrating a Kerberos installation, the various name spaces will be handled as follows: The DNS domain (zone) will be example.org. The Kerberos realm will be EXAMPLE.ORG. Please use real domain names when setting up Kerberos even if you intend to run it internally. This avoids DNS problems and assures inter-operation with other Kerberos realms. History Kerberos5 history Kerberos was created by MIT as a solution to network security problems. The Kerberos protocol uses strong cryptography so that a client can prove its identity to a server (and vice versa) across an insecure network connection. Kerberos is both the name of a network authentication protocol and an adjective to describe programs that implement the program (Kerberos telnet, for example). The current version of the protocol is version 5, described in RFC 1510. Several free implementations of this protocol are available, covering a wide range of operating systems. The Massachusetts Institute of Technology (MIT), where Kerberos was originally developed, continues to develop their Kerberos package. It is commonly used in the US as a cryptography product, as such it has historically been affected by US export regulations. The MIT Kerberos is available as a port (security/krb5). Heimdal Kerberos is another version 5 implementation, and was explicitly developed outside of the US to avoid export regulations (and is thus often included in non-commercial &unix; variants). The Heimdal Kerberos distribution is available as a port (security/heimdal), and a minimal installation of it is included in the base &os; install. In order to reach the widest audience, these instructions assume the use of the Heimdal distribution included in &os;. Setting up a Heimdal <acronym>KDC</acronym> Kerberos5 Key Distribution Center The Key Distribution Center (KDC) is the centralized authentication service that Kerberos provides — it is the computer that issues Kerberos tickets. The KDC is considered trusted by all other computers in the Kerberos realm, and thus has heightened security concerns. Note that while running the Kerberos server requires very few computing resources, a dedicated machine acting only as a KDC is recommended for security reasons. To begin setting up a KDC, ensure that your /etc/rc.conf file contains the correct settings to act as a KDC (you may need to adjust paths to reflect your own system): kerberos5_server_enable="YES" kadmind5_server_enable="YES" Next we will set up your Kerberos config file, /etc/krb5.conf: [libdefaults] default_realm = EXAMPLE.ORG [realms] EXAMPLE.ORG = { kdc = kerberos.example.org admin_server = kerberos.example.org } [domain_realm] .example.org = EXAMPLE.ORG Note that this /etc/krb5.conf file implies that your KDC will have the fully-qualified hostname of kerberos.example.org. You will need to add a CNAME (alias) entry to your zone file to accomplish this if your KDC has a different hostname. For large networks with a properly configured BIND DNS server, the above example could be trimmed to: [libdefaults] default_realm = EXAMPLE.ORG With the following lines being appended to the example.org zonefile: _kerberos._udp IN SRV 01 00 88 kerberos.example.org. _kerberos._tcp IN SRV 01 00 88 kerberos.example.org. _kpasswd._udp IN SRV 01 00 464 kerberos.example.org. _kerberos-adm._tcp IN SRV 01 00 749 kerberos.example.org. _kerberos IN TXT EXAMPLE.ORG For clients to be able to find the Kerberos services, you must have either a fully configured /etc/krb5.conf or a minimally configured /etc/krb5.conf and a properly configured DNS server. Next we will create the Kerberos database. This database contains the keys of all principals encrypted with a master password. You are not required to remember this password, it will be stored in a file (/var/heimdal/m-key). To create the master key, run kstash and enter a password. Once the master key has been created, you can initialize the database using the kadmin program with the -l option (standing for local). This option instructs kadmin to modify the database files directly rather than going through the kadmind network service. This handles the chicken-and-egg problem of trying to connect to the database before it is created. Once you have the kadmin prompt, use the init command to create your realms initial database. Lastly, while still in kadmin, create your first principal using the add command. Stick to the defaults options for the principal for now, you can always change them later with the modify command. Note that you can use the ? command at any prompt to see the available options. A sample database creation session is shown below: &prompt.root; kstash Master key: xxxxxxxx Verifying password - Master key: xxxxxxxx &prompt.root; kadmin -l kadmin> init EXAMPLE.ORG Realm max ticket life [unlimited]: kadmin> add tillman Max ticket life [unlimited]: Max renewable life [unlimited]: Attributes []: Password: xxxxxxxx Verifying password - Password: xxxxxxxx Now it is time to start up the KDC services. Run /etc/rc.d/kerberos start and /etc/rc.d/kadmind start to bring up the services. Note that you will not have any kerberized daemons running at this point but you should be able to confirm the that the KDC is functioning by obtaining and listing a ticket for the principal (user) that you just created from the command-line of the KDC itself: &prompt.user; kinit tillman tillman@EXAMPLE.ORG's Password: &prompt.user; klist Credentials cache: FILE:/tmp/krb5cc_500 Principal: tillman@EXAMPLE.ORG Issued Expires Principal Aug 27 15:37:58 Aug 28 01:37:58 krbtgt/EXAMPLE.ORG@EXAMPLE.ORG The ticket can then be revoked when you have finished: &prompt.user; k5destroy <application>Kerberos</application> enabling a server with Heimdal services Kerberos5 enabling services First, we need a copy of the Kerberos configuration file, /etc/krb5.conf. To do so, simply copy it over to the client computer from the KDC in a secure fashion (using network utilities, such as &man.scp.1;, or physically via a floppy disk). Next you need a /etc/krb5.keytab file. This is the major difference between a server providing Kerberos enabled daemons and a workstation — the server must have a keytab file. This file contains the server's host key, which allows it and the KDC to verify each others identity. It must be transmitted to the server in a secure fashion, as the security of the server can be broken if the key is made public. This explicitly means that transferring it via a clear text channel, such as FTP, is a very bad idea. Typically, you transfer to the keytab to the server using the kadmin program. This is handy because you also need to create the host principal (the KDC end of the krb5.keytab) using kadmin. Note that you must have already obtained a ticket and that this ticket must be allowed to use the kadmin interface in the kadmind.acl. See the section titled Remote administration in the Heimdal info pages (info heimdal) for details on designing access control lists. If you do not want to enable remote kadmin access, you can simply securely connect to the KDC (via local console, &man.ssh.1; or Kerberos &man.telnet.1;) and perform administration locally using kadmin -l. After installing the /etc/krb5.conf file, you can use kadmin from the Kerberos server. The add --random-key command will let you add the server's host principal, and the ext command will allow you to extract the server's host principal to its own keytab. For example: &prompt.root; kadmin kadmin> add --random-key host/myserver.example.org Max ticket life [unlimited]: Max renewable life [unlimited]: Attributes []: kadmin> ext host/myserver.example.org kadmin> exit Note that the ext command (short for extract) stores the extracted key in /etc/krb5.keytab by default. If you do not have kadmind running on the KDC (possibly for security reasons) and thus do not have access to kadmin remotely, you can add the host principal (host/myserver.EXAMPLE.ORG) directly on the KDC and then extract it to a temporary file (to avoid over-writing the /etc/krb5.keytab on the KDC) using something like this: &prompt.root; kadmin kadmin> ext --keytab=/tmp/example.keytab host/myserver.example.org kadmin> exit You can then securely copy the keytab to the server computer (using scp or a floppy, for example). Be sure to specify a non-default keytab name to avoid over-writing the keytab on the KDC. At this point your server can communicate with the KDC (due to its krb5.conf file) and it can prove its own identity (due to the krb5.keytab file). It is now ready for you to enable some Kerberos services. For this example we will enable the telnet service by putting a line like this into your /etc/inetd.conf and then restarting the &man.inetd.8; service with /etc/rc.d/inetd restart: telnet stream tcp nowait root /usr/libexec/telnetd telnetd -a user The critical bit is that the -a (for authentication) type is set to user. Consult the &man.telnetd.8; manual page for more details. <application>Kerberos</application> enabling a client with Heimdal Kerberos5 configure clients Setting up a client computer is almost trivially easy. As far as Kerberos configuration goes, you only need the Kerberos configuration file, located at /etc/krb5.conf. Simply securely copy it over to the client computer from the KDC. Test your client computer by attempting to use kinit, klist, and kdestroy from the client to obtain, show, and then delete a ticket for the principal you created above. You should also be able to use Kerberos applications to connect to Kerberos enabled servers, though if that does not work and obtaining a ticket does the problem is likely with the server and not with the client or the KDC. When testing an application like telnet, try using a packet sniffer (such as &man.tcpdump.1;) to confirm that your password is not sent in the clear. Try using telnet with the -x option, which encrypts the entire data stream (similar to ssh). Various non-core Kerberos client applications are also installed by default. This is where the minimal nature of the base Heimdal installation is felt: telnet is the only Kerberos enabled service. The Heimdal port adds some of the missing client applications: Kerberos enabled versions of ftp, rsh, rcp, rlogin, and a few other less common programs. The MIT port also contains a full suite of Kerberos client applications. User configuration files: <filename>.k5login</filename> and <filename>.k5users</filename> .k5login .k5users Users within a realm typically have their Kerberos principal (such as tillman@EXAMPLE.ORG) mapped to a local user account (such as a local account named tillman). Client applications such as telnet usually do not require a user name or a principal. Occasionally, however, you want to grant access to a local user account to someone who does not have a matching Kerberos principal. For example, tillman@EXAMPLE.ORG may need access to the local user account webdevelopers. Other principals may also need access to that local account. The .k5login and .k5users files, placed in a users home directory, can be used similar to a powerful combination of .hosts and .rhosts, solving this problem. For example, if a .k5login with the following contents: tillman@example.org jdoe@example.org Were to be placed into the home directory of the local user webdevelopers then both principals listed would have access to that account without requiring a shared password. Reading the manual pages for these commands is recommended. Note that the ksu manual page covers .k5users. <application>Kerberos</application> Tips, Tricks, and Troubleshooting Kerberos5 troubleshooting When using either the Heimdal or MIT Kerberos ports ensure that your PATH environment variable lists the Kerberos versions of the client applications before the system versions. Do all the computers in your realm have synchronized time settings? If not, authentication may fail. describes how to synchronize clocks using NTP. MIT and Heimdal inter-operate nicely. Except for kadmin, the protocol for which is not standardized. If you change your hostname, you also need to change your host/ principal and update your keytab. This also applies to special keytab entries like the www/ principal used for Apache's www/mod_auth_kerb. All hosts in your realm must be resolvable (both forwards and reverse) in DNS (or /etc/hosts as a minimum). CNAMEs will work, but the A and PTR records must be correct and in place. The error message is not very intuitive: Kerberos5 refuses authentication because Read req failed: Key table entry not found. Some operating systems that may being acting as clients to your KDC do not set the permissions for ksu to be setuid root. This means that ksu does not work, which is a good security idea but annoying. This is not a KDC error. With MIT Kerberos, if you want to allow a principal to have a ticket life longer than the default ten hours, you must use modify_principal in kadmin to change the maxlife of both the principal in question and the krbtgt principal. Then the principal can use the -l option with kinit to request a ticket with a longer lifetime. If you run a packet sniffer on your KDC to add in troubleshooting and then run kinit from a workstation, you will notice that your TGT is sent immediately upon running kinit — even before you type your password! The explanation is that the Kerberos server freely transmits a TGT (Ticket Granting Ticket) to any unauthorized request; however, every TGT is encrypted in a key derived from the user's password. Therefore, when a user types their password it is not being sent to the KDC, it is being used to decrypt the TGT that kinit already obtained. If the decryption process results in a valid ticket with a valid time stamp, the user has valid Kerberos credentials. These credentials include a session key for establishing secure communications with the Kerberos server in the future, as well as the actual ticket-granting ticket, which is actually encrypted with the Kerberos server's own key. This second layer of encryption is unknown to the user, but it is what allows the Kerberos server to verify the authenticity of each TGT. If you want to use long ticket lifetimes (a week, for example) and you are using OpenSSH to connect to the machine where your ticket is stored, make sure that Kerberos is set to no in your sshd_config or else your tickets will be deleted when you log out. Remember that host principals can have a longer ticket lifetime as well. If your user principal has a lifetime of a week but the host you are connecting to has a lifetime of nine hours, you will have an expired host principal in your cache and the ticket cache will not work as expected. When setting up a krb5.dict file to prevent specific bad passwords from being used (the manual page for kadmind covers this briefly), remember that it only applies to principals that have a password policy assigned to them. The krb5.dict files format is simple: one string per line. Creating a symbolic link to /usr/share/dict/words might be useful. Differences with the <acronym>MIT</acronym> port The major difference between the MIT and Heimdal installs relates to the kadmin program which has a different (but equivalent) set of commands and uses a different protocol. This has a large implications if your KDC is MIT as you will not be able to use the Heimdal kadmin program to administer your KDC remotely (or vice versa, for that matter). The client applications may also take slightly different command line options to accomplish the same tasks. Following the instructions on the MIT Kerberos web site () is recommended. Be careful of path issues: the MIT port installs into /usr/local/ by default, and the normal system applications may be run instead of MIT if your PATH environment variable lists the system directories first. With the MIT security/krb5 port that is provided by &os;, be sure to read the /usr/local/share/doc/krb5/README.FreeBSD file installed by the port if you want to understand why logins via telnetd and klogind behave somewhat oddly. Most importantly, correcting the incorrect permissions on cache file behavior requires that the login.krb5 binary be used for authentication so that it can properly change ownership for the forwarded credentials. The rc.conf must also be modified to contain the following configuration: kerberos5_server="/usr/local/sbin/krb5kdc" kadmind5_server="/usr/local/sbin/kadmind" kerberos5_server_enable="YES" kadmind5_server_enable="YES" This is done because the applications for MIT kerberos installs binaries in the - /usr/local + /usr/local hierarchy. Mitigating limitations found in <application>Kerberos</application> Kerberos5 limitations and shortcomings <application>Kerberos</application> is an all-or-nothing approach Every service enabled on the network must be modified to work with Kerberos (or be otherwise secured against network attacks) or else the users credentials could be stolen and re-used. An example of this would be Kerberos enabling all remote shells (via rsh and telnet, for example) but not converting the POP3 mail server which sends passwords in plain text. <application>Kerberos</application> is intended for single-user workstations In a multi-user environment, Kerberos is less secure. This is because it stores the tickets in the /tmp directory, which is readable by all users. If a user is sharing a computer with several other people simultaneously (i.e. multi-user), it is possible that the user's tickets can be stolen (copied) by another user. This can be overcome with the -c filename command-line option or (preferably) the KRB5CCNAME environment variable, but this is rarely done. In principal, storing the ticket in the users home directory and using simple file permissions can mitigate this problem. The KDC is a single point of failure By design, the KDC must be as secure as the master password database is contained on it. The KDC should have absolutely no other services running on it and should be physically secured. The danger is high because Kerberos stores all passwords encrypted with the same key (the master key), which in turn is stored as a file on the KDC. As a side note, a compromised master key is not quite as bad as one might normally fear. The master key is only used to encrypt the Kerberos database and as a seed for the random number generator. As long as access to your KDC is secure, an attacker cannot do much with the master key. Additionally, if the KDC is unavailable (perhaps due to a denial of service attack or network problems) the network services are unusable as authentication can not be performed, a recipe for a denial-of-service attack. This can alleviated with multiple KDCs (a single master and one or more slaves) and with careful implementation of secondary or fall-back authentication (PAM is excellent for this). <application>Kerberos</application> Shortcomings Kerberos allows users, hosts and services to authenticate between themselves. It does not have a mechanism to authenticate the KDC to the users, hosts or services. This means that a trojanned kinit (for example) could record all user names and passwords. Something like security/tripwire or other file system integrity checking tools can alleviate this. Resources and further information Kerberos5 external resources The Kerberos FAQ Designing an Authentication System: a Dialog in Four Scenes RFC 1510, The Kerberos Network Authentication Service (V5) MIT Kerberos home page Heimdal Kerberos home page Tom Rhodes Written by OpenSSL security OpenSSL One feature that many users overlook is the OpenSSL toolkit included in &os;. OpenSSL provides an encryption transport layer on top of the normal communications layer; thus allowing it to be intertwined with many network applications and services. Some uses of OpenSSL may include encrypted authentication of mail clients, web based transactions such as credit card payments and more. Many ports such as www/apache13-ssl, and mail/sylpheed-claws will offer compilation support for building with OpenSSL. In most cases the Ports Collection will attempt to build the security/openssl port unless the WITH_OPENSSL_BASE make variable is explicitly set to yes. The version of OpenSSL included in &os; supports Secure Sockets Layer v2/v3 (SSLv2/SSLv3), Transport Layer Security v1 (TLSv1) network security protocols and can be used as a general cryptographic library. While OpenSSL supports the IDEA algorithm, it is disabled by default due to United States patents. To use it, the license should be reviewed and, if the restrictions are acceptable, the MAKE_IDEA variable must be set in make.conf. One of the most common uses of OpenSSL is to provide certificates for use with software applications. These certificates ensure that the credentials of the company or individual are valid and not fraudulent. If the certificate in question has not been verified by one of the several Certificate Authorities, or CAs, a warning is usually produced. A Certificate Authority is a company, such as VeriSign, which will sign certificates in order to validate credentials of individuals or companies. This process has a cost associated with it and is definitely not a requirement for using certificates; however, it can put some of the more paranoid users at ease. Generating Certificates OpenSSL certificate generation To generate a certificate, the following command is available: &prompt.root; openssl req -new -nodes -out req.pem -keyout cert.pem Generating a 1024 bit RSA private key ................++++++ .......................................++++++ writing new private key to 'cert.pem' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:PA Locality Name (eg, city) []:Pittsburgh Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Company Organizational Unit Name (eg, section) []:Systems Administrator Common Name (eg, YOUR name) []:localhost.example.org Email Address []:trhodes@FreeBSD.org Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []:SOME PASSWORD An optional company name []:Another Name Notice the response directly after the Common Name prompt shows a domain name. This prompt requires a server name to be entered for verification purposes; placing anything but a domain name would yield a useless certificate. Other options, for instance expire time, alternate encryption algorithms, etc. are available. A complete list may be obtained by viewing the &man.openssl.1; manual page. Two files should now exist in the directory in which the aforementioned command was issued. The certificate request, req.pem, may be sent to a certificate authority who will validate the credentials that you entered, sign the request and return the certificate to you. The second file created will be named cert.pem and is the private key for the certificate and should be protected at all costs; if this falls in the hands of others it can be used to impersonate you (or your server). In cases where a signature from a CA is not required, a self signed certificate can be created. First, generate the RSA key: &prompt.root; openssl dsaparam -rand -genkey -out myRSA.key 1024 Next, generate the CA key: &prompt.root; openssl gendsa -des3 -out myca.key myRSA.key Use this key to create the certificate: &prompt.root; openssl req -new -x509 -days 365 -key myca.key -out new.crt Two new files should appear in the directory: a certificate authority signature file, myca.key and the certificate itself, new.crt. These should be placed in a directory, preferably under /etc, which is readable only by root. Permissions of 0700 should be fine for this and they can be set with the chmod utility. Using Certificates, an Example So what can these files do? A good use would be to encrypt connections to the Sendmail MTA. This would dissolve the use of clear text authentication for users who send mail via the local MTA. This is not the best use in the world as some MUAs will present the user with an error if they have not installed the certificate locally. Refer to the documentation included with the software for more information on certificate installation. The following lines should be placed inside the local .mc file: dnl SSL Options define(`confCACERT_PATH',`/etc/certs')dnl define(`confCACERT',`/etc/certs/new.crt')dnl define(`confSERVER_CERT',`/etc/certs/new.crt')dnl define(`confSERVER_KEY',`/etc/certs/myca.key')dnl define(`confTLS_SRV_OPTIONS', `V')dnl Where /etc/certs/ is the directory to be used for storing the certificate and key files locally. The last few requirements are a rebuild of the local .cf file. This is easily achieved by typing make install within the /etc/mail directory. Follow that up with make restart which should start the Sendmail daemon. If all went well there will be no error messages in the /var/log/maillog file and Sendmail will show up in the process list. For a simple test, simply connect to the mail server using the &man.telnet.1; utility: &prompt.root; telnet example.com 25 Trying 192.0.34.166... Connected to example.com. Escape character is '^]'. 220 example.com ESMTP Sendmail 8.12.10/8.12.10; Tue, 31 Aug 2004 03:41:22 -0400 (EDT) ehlo example.com 250-example.com Hello example.com [192.0.34.166], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-ETRN 250-AUTH LOGIN PLAIN 250-STARTTLS 250-DELIVERBY 250 HELP quit 221 2.0.0 example.com closing connection Connection closed by foreign host. If the STARTTLS line appears in the output then everything is working correctly. Nik Clayton
nik@FreeBSD.org
 Written by IPsec VPN over IPsec Creating a VPN between two networks, separated by the Internet, using FreeBSD gateways. Hiten M. Pandya
hmp@FreeBSD.org
 Written by Understanding IPsec This section will guide you through the process of setting up IPsec. In order to set up IPsec, it is necessary that you are familiar with the concepts of building a custom kernel (see ). IPsec is a protocol which sits on top of the Internet Protocol (IP) layer. It allows two or more hosts to communicate in a secure manner (hence the name). The FreeBSD IPsec network stack is based on the KAME implementation, which has support for both protocol families, IPv4 and IPv6. IPsec ESP IPsec AH IPsec consists of two sub-protocols: Encapsulated Security Payload (ESP), protects the IP packet data from third party interference, by encrypting the contents using symmetric cryptography algorithms (like Blowfish, 3DES). Authentication Header (AH), protects the IP packet header from third party interference and spoofing, by computing a cryptographic checksum and hashing the IP packet header fields with a secure hashing function. This is then followed by an additional header that contains the hash, to allow the information in the packet to be authenticated. ESP and AH can either be used together or separately, depending on the environment. VPN virtual private network VPN IPsec can either be used to directly encrypt the traffic between two hosts (known as Transport Mode); or to build virtual tunnels between two subnets, which could be used for secure communication between two corporate networks (known as Tunnel Mode). The latter is more commonly known as a Virtual Private Network (VPN). The &man.ipsec.4; manual page should be consulted for detailed information on the IPsec subsystem in FreeBSD. To add IPsec support to your kernel, add the following options to your kernel configuration file: kernel options IPSEC options IPSEC #IP security device crypto kernel options IPSEC_DEBUG If IPsec debugging support is desired, the following kernel option should also be added: options IPSEC_DEBUG #debug for IP security The Problem There is no standard for what constitutes a VPN. VPNs can be implemented using a number of different technologies, each of which have their own strengths and weaknesses. This section presents a scenario, and the strategies used for implementing a VPN for this scenario. The Scenario: Two networks, one home based and one corporate based. Both are connected to the Internet, and expected, via this <acronym>VPN</acronym> to behave as one. VPN creating The premise is as follows: You have at least two sites Both sites are using IP internally Both sites are connected to the Internet, through a gateway that is running FreeBSD. The gateway on each network has at least one public IP address. The internal addresses of the two networks can be public or private IP addresses, it does not matter. They just may not collide; e.g.: may not both use 192.168.1.x. Tom Rhodes
trhodes@FreeBSD.org
 Written by Configuring IPsec on &os; To begin, the security/ipsec-tools must be installed from the Ports Collection. This third party software package provides a number of applications which will help support the configuration. The next requirement is to create two &man.gif.4; pseudo-devices which will be used to tunnel packets and allow both networks to communicate properly. As root, run the following commands, replacing the internal and external items with the real internal and external gateways: &prompt.root; ifconfig gif0 create &prompt.root; ifconfig gif0 internal1 internal2 &prompt.root; ifconfig gif0 tunnel external1 external2 For example, the corporate LAN's public IP is 172.16.5.4 having a private IP of 10.246.38.1. The home LAN's public IP is 192.168.1.12 with an internal private IP of 10.0.0.5. This may seem confusing, so review the following example output from the &man.ifconfig.8; command: Gateway 1: gif0: flags=8051 mtu 1280 tunnel inet 172.16.5.4 --> 192.168.1.12 inet6 fe80::2e0:81ff:fe02:5881%gif0 prefixlen 64 scopeid 0x6 inet 10.246.38.1 --> 10.0.0.5 netmask 0xffffff00 Gateway 2: gif0: flags=8051 mtu 1280 tunnel inet 192.168.1.12 --> 172.16.5.4 inet 10.0.0.5 --> 10.246.38.1 netmask 0xffffff00 inet6 fe80::250:bfff:fe3a:c1f%gif0 prefixlen 64 scopeid 0x4 Once complete, both private IPs should be reachable using the &man.ping.8; command like the following output suggests: priv-net# ping 10.0.0.5 PING 10.0.0.5 (10.0.0.5): 56 data bytes 64 bytes from 10.0.0.5: icmp_seq=0 ttl=64 time=42.786 ms 64 bytes from 10.0.0.5: icmp_seq=1 ttl=64 time=19.255 ms 64 bytes from 10.0.0.5: icmp_seq=2 ttl=64 time=20.440 ms 64 bytes from 10.0.0.5: icmp_seq=3 ttl=64 time=21.036 ms --- 10.0.0.5 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max/stddev = 19.255/25.879/42.786/9.782 ms corp-net# ping 10.246.38.1 PING 10.246.38.1 (10.246.38.1): 56 data bytes 64 bytes from 10.246.38.1: icmp_seq=0 ttl=64 time=28.106 ms 64 bytes from 10.246.38.1: icmp_seq=1 ttl=64 time=42.917 ms 64 bytes from 10.246.38.1: icmp_seq=2 ttl=64 time=127.525 ms 64 bytes from 10.246.38.1: icmp_seq=3 ttl=64 time=119.896 ms 64 bytes from 10.246.38.1: icmp_seq=4 ttl=64 time=154.524 ms --- 10.246.38.1 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 28.106/94.594/154.524/49.814 ms As expected, both sides have the ability to send and receive ICMP packets from the privately configured addresses. Next, both gateways must be told how to route packets in order to correctly send traffic from either network. The following command will achieve this goal: &prompt.root; corp-net# route add 10.0.0.0 10.0.0.5 255.255.255.0 &prompt.root; corp-net# route add net 10.0.0.0: gateway 10.0.0.5 &prompt.root; priv-net# route add 10.246.38.0 10.246.38.1 255.255.255.0 &prompt.root; priv-net# route add host 10.246.38.0: gateway 10.246.38.1 At this point, internal machines should be reachable from each gateway as well as from machines behind the gateways. This is easily determined from the following example: corp-net# ping 10.0.0.8 PING 10.0.0.8 (10.0.0.8): 56 data bytes 64 bytes from 10.0.0.8: icmp_seq=0 ttl=63 time=92.391 ms 64 bytes from 10.0.0.8: icmp_seq=1 ttl=63 time=21.870 ms 64 bytes from 10.0.0.8: icmp_seq=2 ttl=63 time=198.022 ms 64 bytes from 10.0.0.8: icmp_seq=3 ttl=63 time=22.241 ms 64 bytes from 10.0.0.8: icmp_seq=4 ttl=63 time=174.705 ms --- 10.0.0.8 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 21.870/101.846/198.022/74.001 ms priv-net# ping 10.246.38.107 PING 10.246.38.1 (10.246.38.107): 56 data bytes 64 bytes from 10.246.38.107: icmp_seq=0 ttl=64 time=53.491 ms 64 bytes from 10.246.38.107: icmp_seq=1 ttl=64 time=23.395 ms 64 bytes from 10.246.38.107: icmp_seq=2 ttl=64 time=23.865 ms 64 bytes from 10.246.38.107: icmp_seq=3 ttl=64 time=21.145 ms 64 bytes from 10.246.38.107: icmp_seq=4 ttl=64 time=36.708 ms --- 10.246.38.107 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 21.145/31.721/53.491/12.179 ms Setting up the tunnels is the easy part. Configuring a secure link is a much more in depth process. The following configuration uses pre-shared (PSK) RSA keys. Aside from the IP addresses, both /usr/local/etc/racoon/racoon.conf files will be identical and look similar to path pre_shared_key "/usr/local/etc/racoon/psk.txt"; #location of pre-shared key file log debug; #log verbosity setting: set to 'notify' when testing and debugging is complete padding # options are not to be changed { maximum_length 20; randomize off; strict_check off; exclusive_tail off; } timer # timing options. change as needed { counter 5; interval 20 sec; persend 1; # natt_keepalive 15 sec; phase1 30 sec; phase2 15 sec; } listen # address [port] that racoon will listening on { isakmp 172.16.5.4 [500]; isakmp_natt 172.16.5.4 [4500]; } remote 192.168.1.12 [500] { exchange_mode main,aggressive; doi ipsec_doi; situation identity_only; my_identifier address 172.16.5.4; peers_identifier address 192.168.1.12; lifetime time 8 hour; passive off; proposal_check obey; # nat_traversal off; generate_policy off; proposal { encryption_algorithm blowfish; hash_algorithm md5; authentication_method pre_shared_key; lifetime time 30 sec; dh_group 1; } } sainfo (address 10.246.38.0/24 any address 10.0.0.0/24 any) # address $network/$netmask $type address $network/$netmask $type ( $type being any or esp) { # $network must be the two internal networks you are joining. pfs_group 1; lifetime time 36000 sec; encryption_algorithm blowfish,3des,des; authentication_algorithm hmac_md5,hmac_sha1; compression_algorithm deflate; } Explaining every available option, along with those listed in these examples is beyond the scope of this document. There is plenty of relevant information in the racoon configuration manual page. The SPD policies need to be configured so &os; and racoon is able to encrypt and decrypt network traffic between hosts. This task may be undertaken with a simple shell script similar to the following which is on the corporate gateway. This file will be used during system initialization and should be saved as /usr/local/etc/racoon/setkey.conf. flush; spdflush; # To the home network spdadd 10.246.38.0/24 10.0.0.0/24 any -P out ipsec esp/tunnel/172.16.5.4-192.168.1.12/use; spdadd 10.0.0.0/24 10.246.38.0/24 any -P in ipsec esp/tunnel/192.168.1.12-172.16.5.4/use; Once in place, racoon may be started on both gateways using the following command: &prompt.root; /usr/local/sbin/racoon -F -f /usr/local/etc/racoon/racoon.conf -l /var/log/racoon.log The output should be similar to the following: corp-net# /usr/local/sbin/racoon -F -f /usr/local/etc/racoon/racoon.conf Foreground mode. 2006-01-30 01:35:47: INFO: begin Identity Protection mode. 2006-01-30 01:35:48: INFO: received Vendor ID: KAME/racoon 2006-01-30 01:35:55: INFO: received Vendor ID: KAME/racoon 2006-01-30 01:36:04: INFO: ISAKMP-SA established 172.16.5.4[500]-192.168.1.12[500] spi:623b9b3bd2492452:7deab82d54ff704a 2006-01-30 01:36:05: INFO: initiate new phase 2 negotiation: 172.16.5.4[0]192.168.1.12[0] 2006-01-30 01:36:09: INFO: IPsec-SA established: ESP/Tunnel 192.168.1.12[0]->172.16.5.4[0] spi=28496098(0x1b2d0e2) 2006-01-30 01:36:09: INFO: IPsec-SA established: ESP/Tunnel 172.16.5.4[0]->192.168.1.12[0] spi=47784998(0x2d92426) 2006-01-30 01:36:13: INFO: respond new phase 2 negotiation: 172.16.5.4[0]192.168.1.12[0] 2006-01-30 01:36:18: INFO: IPsec-SA established: ESP/Tunnel 192.168.1.12[0]->172.16.5.4[0] spi=124397467(0x76a279b) 2006-01-30 01:36:18: INFO: IPsec-SA established: ESP/Tunnel 172.16.5.4[0]->192.168.1.12[0] spi=175852902(0xa7b4d66) To ensure the tunnel is working properly, switch to another console and use &man.tcpdump.1; to view network traffic using the following command. Replace em0 with the network interface card as required. &prompt.root; tcpdump -i em0 host 172.16.5.4 and dst 192.168.1.12 Data similar to the following should appear on the console. If not, there is an issue, and debugging the returned data will be required. 01:47:32.021683 IP corporatenetwork.com > 192.168.1.12.privatenetwork.com: ESP(spi=0x02acbf9f,seq=0xa) 01:47:33.022442 IP corporatenetwork.com > 192.168.1.12.privatenetwork.com: ESP(spi=0x02acbf9f,seq=0xb) 01:47:34.024218 IP corporatenetwork.com > 192.168.1.12.privatenetwork.com: ESP(spi=0x02acbf9f,seq=0xc) At this point, both networks should be available and seem to be part of the same network. Most likely both networks are protected by a firewall, as they should be. To allow traffic to flow between them, rules need to be added to pass packets back and forth. For the &man.ipfw.8; firewall, add the following lines to the firewall configuration file: ipfw add 00201 allow log esp from any to any ipfw add 00202 allow log ah from any to any ipfw add 00203 allow log ipencap from any to any ipfw add 00204 allow log usp from any 500 to any The rule numbers may need to be altered depending on the current host configuration. For users of &man.pf.4; or &man.ipf.8;, the following rules should do the trick: pass in quick proto esp from any to any pass in quick proto ah from any to any pass in quick proto ipencap from any to any pass in quick proto udp from any port = 500 to any port = 500 pass in quick on gif0 from any to any pass out quick proto esp from any to any pass out quick proto ah from any to any pass out quick proto ipencap from any to any pass out quick proto udp from any port = 500 to any port = 500 pass out quick on gif0 from any to any Finally, to allow the machine to start support for the VPN during system initialization, add the following lines to /etc/rc.conf: ipsec_enable="YES" ipsec_program="/usr/local/sbin/setkey" ipsec_file="/usr/local/etc/racoon/setkey.conf" # allows setting up spd policies on boot racoon_enable="yes" Chern Lee Contributed by OpenSSH OpenSSH security OpenSSH OpenSSH is a set of network connectivity tools used to access remote machines securely. It can be used as a direct replacement for rlogin, rsh, rcp, and telnet. Additionally, TCP/IP connections can be tunneled/forwarded securely through SSH. OpenSSH encrypts all traffic to effectively eliminate eavesdropping, connection hijacking, and other network-level attacks. OpenSSH is maintained by the OpenBSD project, and is based upon SSH v1.2.12 with all the recent bug fixes and updates. It is compatible with both SSH protocols 1 and 2. Advantages of Using OpenSSH Normally, when using &man.telnet.1; or &man.rlogin.1;, data is sent over the network in a clear, un-encrypted form. Network sniffers anywhere in between the client and server can steal your user/password information or data transferred in your session. OpenSSH offers a variety of authentication and encryption methods to prevent this from happening. Enabling sshd OpenSSH enabling The sshd is an option presented during a Standard install of &os;. To see if sshd is enabled, check the rc.conf file for: sshd_enable="YES" This will load &man.sshd.8;, the daemon program for OpenSSH, the next time your system initializes. Alternatively, it is possible to use /etc/rc.d/sshd &man.rc.8; script to start OpenSSH: /etc/rc.d/sshd start SSH Client OpenSSH client The &man.ssh.1; utility works similarly to &man.rlogin.1;. &prompt.root; ssh user@example.com Host key not found from the list of known hosts. Are you sure you want to continue connecting (yes/no)? yes Host 'example.com' added to the list of known hosts. user@example.com's password: ******* The login will continue just as it would have if a session was created using rlogin or telnet. SSH utilizes a key fingerprint system for verifying the authenticity of the server when the client connects. The user is prompted to enter yes only when connecting for the first time. Future attempts to login are all verified against the saved fingerprint key. The SSH client will alert you if the saved fingerprint differs from the received fingerprint on future login attempts. The fingerprints are saved in ~/.ssh/known_hosts, or ~/.ssh/known_hosts2 for SSH v2 fingerprints. By default, recent versions of the OpenSSH servers only accept SSH v2 connections. The client will use version 2 if possible and will fall back to version 1. The client can also be forced to use one or the other by passing it the or for version 1 or version 2, respectively. The version 1 compatibility is maintained in the client for backwards compatibility with older versions. Secure Copy OpenSSH secure copy scp The &man.scp.1; command works similarly to &man.rcp.1;; it copies a file to or from a remote machine, except in a secure fashion. &prompt.root; scp user@example.com:/COPYRIGHT COPYRIGHT user@example.com's password: ******* COPYRIGHT 100% |*****************************| 4735 00:00 &prompt.root; Since the fingerprint was already saved for this host in the previous example, it is verified when using &man.scp.1; here. The arguments passed to &man.scp.1; are similar to &man.cp.1;, with the file or files in the first argument, and the destination in the second. Since the file is fetched over the network, through SSH, one or more of the file arguments takes on the form . Configuration OpenSSH configuration The system-wide configuration files for both the OpenSSH daemon and client reside within the /etc/ssh directory. ssh_config configures the client settings, while sshd_config configures the daemon. Additionally, the (/usr/sbin/sshd by default), and rc.conf options can provide more levels of configuration. ssh-keygen Instead of using passwords, &man.ssh-keygen.1; can be used to generate DSA or RSA keys to authenticate a user: &prompt.user; ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/home/user/.ssh/id_dsa): Created directory '/home/user/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/user/.ssh/id_dsa. Your public key has been saved in /home/user/.ssh/id_dsa.pub. The key fingerprint is: bb:48:db:f2:93:57:80:b6:aa:bc:f5:d5:ba:8f:79:17 user@host.example.com &man.ssh-keygen.1; will create a public and private key pair for use in authentication. The private key is stored in ~/.ssh/id_dsa or ~/.ssh/id_rsa, whereas the public key is stored in ~/.ssh/id_dsa.pub or ~/.ssh/id_rsa.pub, respectively for DSA and RSA key types. The public key must be placed in the ~/.ssh/authorized_keys file of the remote machine for both RSA or DSA keys in order for the setup to work. This will allow connection to the remote machine based upon SSH keys instead of passwords. If a passphrase is used in &man.ssh-keygen.1;, the user will be prompted for a password each time in order to use the private key. &man.ssh-agent.1; can alleviate the strain of repeatedly entering long passphrases, and is explored in the section below. The various options and files can be different according to the OpenSSH version you have on your system; to avoid problems you should consult the &man.ssh-keygen.1; manual page. ssh-agent and ssh-add The &man.ssh-agent.1; and &man.ssh-add.1; utilities provide methods for SSH keys to be loaded into memory for use, without needing to type the passphrase each time. The &man.ssh-agent.1; utility will handle the authentication using the private key(s) that are loaded into it. &man.ssh-agent.1; should be used to launch another application. At the most basic level, it could spawn a shell or at a more advanced level, a window manager. To use &man.ssh-agent.1; in a shell, first it will need to be spawned with a shell as an argument. Secondly, the identity needs to be added by running &man.ssh-add.1; and providing it the passphrase for the private key. Once these steps have been completed the user will be able to &man.ssh.1; to any host that has the corresponding public key installed. For example: &prompt.user; ssh-agent csh &prompt.user; ssh-add Enter passphrase for /home/user/.ssh/id_dsa: Identity added: /home/user/.ssh/id_dsa (/home/user/.ssh/id_dsa) &prompt.user; To use &man.ssh-agent.1; in X11, a call to &man.ssh-agent.1; will need to be placed in ~/.xinitrc. This will provide the &man.ssh-agent.1; services to all programs launched in X11. An example ~/.xinitrc file might look like this: exec ssh-agent startxfce4 This would launch &man.ssh-agent.1;, which would in turn launch XFCE, every time X11 starts. Then once that is done and X11 has been restarted so that the changes can take effect, simply run &man.ssh-add.1; to load all of your SSH keys. SSH Tunneling OpenSSH tunneling OpenSSH has the ability to create a tunnel to encapsulate another protocol in an encrypted session. The following command tells &man.ssh.1; to create a tunnel for telnet: &prompt.user; ssh -2 -N -f -L 5023:localhost:23 user@foo.example.com &prompt.user; The ssh command is used with the following options: Forces ssh to use version 2 of the protocol. (Do not use if you are working with older SSH servers) Indicates no command, or tunnel only. If omitted, ssh would initiate a normal session. Forces ssh to run in the background. Indicates a local tunnel in localport:remotehost:remoteport fashion. The remote SSH server. An SSH tunnel works by creating a listen socket on localhost on the specified port. It then forwards any connection received on the local host/port via the SSH connection to the specified remote host and port. In the example, port 5023 on localhost is being forwarded to port 23 on localhost of the remote machine. Since 23 is telnet, this would create a secure telnet session through an SSH tunnel. This can be used to wrap any number of insecure TCP protocols such as SMTP, POP3, FTP, etc. Using SSH to Create a Secure Tunnel for SMTP &prompt.user; ssh -2 -N -f -L 5025:localhost:25 user@mailserver.example.com user@mailserver.example.com's password: ***** &prompt.user; telnet localhost 5025 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mailserver.example.com ESMTP This can be used in conjunction with an &man.ssh-keygen.1; and additional user accounts to create a more seamless/hassle-free SSH tunneling environment. Keys can be used in place of typing a password, and the tunnels can be run as a separate user. Practical SSH Tunneling Examples Secure Access of a POP3 Server At work, there is an SSH server that accepts connections from the outside. On the same office network resides a mail server running a POP3 server. The network, or network path between your home and office may or may not be completely trustable. Because of this, you need to check your e-mail in a secure manner. The solution is to create an SSH connection to your office's SSH server, and tunnel through to the mail server. &prompt.user; ssh -2 -N -f -L 2110:mail.example.com:110 user@ssh-server.example.com user@ssh-server.example.com's password: ****** When the tunnel is up and running, you can point your mail client to send POP3 requests to localhost port 2110. A connection here will be forwarded securely across the tunnel to mail.example.com. Bypassing a Draconian Firewall Some network administrators impose extremely draconian firewall rules, filtering not only incoming connections, but outgoing connections. You may be only given access to contact remote machines on ports 22 and 80 for SSH and web surfing. You may wish to access another (perhaps non-work related) service, such as an Ogg Vorbis server to stream music. If this Ogg Vorbis server is streaming on some other port than 22 or 80, you will not be able to access it. The solution is to create an SSH connection to a machine outside of your network's firewall, and use it to tunnel to the Ogg Vorbis server. &prompt.user; ssh -2 -N -f -L 8888:music.example.com:8000 user@unfirewalled-system.example.org user@unfirewalled-system.example.org's password: ******* Your streaming client can now be pointed to localhost port 8888, which will be forwarded over to music.example.com port 8000, successfully evading the firewall. The <varname>AllowUsers</varname> Users Option It is often a good idea to limit which users can log in and from where. The AllowUsers option is a good way to accomplish this. For example, to only allow the root user to log in from 192.168.1.32, something like this would be appropriate in the /etc/ssh/sshd_config file: AllowUsers root@192.168.1.32 To allow the user admin to log in from anywhere, just list the username by itself: AllowUsers admin Multiple users should be listed on the same line, like so: AllowUsers root@192.168.1.32 admin It is important that you list each user that needs to log in to this machine; otherwise they will be locked out. After making changes to /etc/ssh/sshd_config you must tell &man.sshd.8; to reload its config files, by running: &prompt.root; /etc/rc.d/sshd reload Further Reading OpenSSH &man.ssh.1; &man.scp.1; &man.ssh-keygen.1; &man.ssh-agent.1; &man.ssh-add.1; &man.ssh.config.5; &man.sshd.8; &man.sftp-server.8; &man.sshd.config.5; Tom Rhodes Contributed by ACL File System Access Control Lists In conjunction with file system enhancements like snapshots, FreeBSD 5.0 and later offers the security of File System Access Control Lists (ACLs). Access Control Lists extend the standard &unix; permission model in a highly compatible (&posix;.1e) way. This feature permits an administrator to make use of and take advantage of a more sophisticated security model. To enable ACL support for UFS file systems, the following: options UFS_ACL must be compiled into the kernel. If this option has not been compiled in, a warning message will be displayed when attempting to mount a file system supporting ACLs. This option is included in the GENERIC kernel. ACLs rely on extended attributes being enabled on the file system. Extended attributes are natively supported in the next generation &unix; file system, UFS2. A higher level of administrative overhead is required to configure extended attributes on UFS1 than on UFS2. The performance of extended attributes on UFS2 is also substantially higher. As a result, UFS2 is generally recommended in preference to UFS1 for use with access control lists. ACLs are enabled by the mount-time administrative flag, , which may be added to /etc/fstab. The mount-time flag can also be automatically set in a persistent manner using &man.tunefs.8; to modify a superblock ACLs flag in the file system header. In general, it is preferred to use the superblock flag for several reasons: The mount-time ACLs flag cannot be changed by a remount (&man.mount.8; ), only by means of a complete &man.umount.8; and fresh &man.mount.8;. This means that ACLs cannot be enabled on the root file system after boot. It also means that you cannot change the disposition of a file system once it is in use. Setting the superblock flag will cause the file system to always be mounted with ACLs enabled even if there is not an fstab entry or if the devices re-order. This prevents accidental mounting of the file system without ACLs enabled, which can result in ACLs being improperly enforced, and hence security problems. We may change the ACLs behavior to allow the flag to be enabled without a complete fresh &man.mount.8;, but we consider it desirable to discourage accidental mounting without ACLs enabled, because you can shoot your feet quite nastily if you enable ACLs, then disable them, then re-enable them without flushing the extended attributes. In general, once you have enabled ACLs on a file system, they should not be disabled, as the resulting file protections may not be compatible with those intended by the users of the system, and re-enabling ACLs may re-attach the previous ACLs to files that have since had their permissions changed, resulting in other unpredictable behavior. File systems with ACLs enabled will show a + (plus) sign in their permission settings when viewed. For example: drwx------ 2 robert robert 512 Dec 27 11:54 private drwxrwx---+ 2 robert robert 512 Dec 23 10:57 directory1 drwxrwx---+ 2 robert robert 512 Dec 22 10:20 directory2 drwxrwx---+ 2 robert robert 512 Dec 27 11:57 directory3 drwxr-xr-x 2 robert robert 512 Nov 10 11:54 public_html Here we see that the directory1, directory2, and directory3 directories are all taking advantage of ACLs. The public_html directory is not. Making Use of <acronym>ACL</acronym>s The file system ACLs can be viewed by the &man.getfacl.1; utility. For instance, to view the ACL settings on the test file, one would use the command: &prompt.user; getfacl test #file:test #owner:1001 #group:1001 user::rw- group::r-- other::r-- To change the ACL settings on this file, invoke the &man.setfacl.1; utility. Observe: &prompt.user; setfacl -k test The flag will remove all of the currently defined ACLs from a file or file system. The more preferable method would be to use as it leaves the basic fields required for ACLs to work. &prompt.user; setfacl -m u:trhodes:rwx,group:web:r--,o::--- test In the aforementioned command, the option was used to modify the default ACL entries. Since there were no pre-defined entries, as they were removed by the previous command, this will restore the default options and assign the options listed. Take care to notice that if you add a user or group which does not exist on the system, an Invalid argument error will be printed to stdout. Tom Rhodes Contributed by Portaudit Monitoring Third Party Security Issues In recent years, the security world has made many improvements to how vulnerability assessment is handled. The threat of system intrusion increases as third party utilities are installed and configured for virtually any operating system available today. Vulnerability assessment is a key factor in security, and while &os; releases advisories for the base system, doing so for every third party utility is beyond the &os; Project's capability. There is a way to mitigate third party vulnerabilities and warn administrators of known security issues. A &os; add on utility known as Portaudit exists solely for this purpose. The ports-mgmt/portaudit port polls a database, updated and maintained by the &os; Security Team and ports developers, for known security issues. To begin using Portaudit, one must install it from the Ports Collection: &prompt.root; cd /usr/ports/ports-mgmt/portaudit && make install clean During the install process, the configuration files for &man.periodic.8; will be updated, permitting Portaudit output in the daily security runs. Ensure the daily security run emails, which are sent to root's email account, are being read. No more configuration will be required here. After installation, an administrator can update the database and view known vulnerabilities in installed packages by invoking the following command: &prompt.root; portaudit -Fda The database will automatically be updated during the &man.periodic.8; run; thus, the previous command is completely optional. It is only required for the following examples. To audit the third party utilities installed as part of the Ports Collection at anytime, an administrator need only run the following command: &prompt.root; portaudit -a Portaudit will produce something like this for vulnerable packages: Affected package: cups-base-1.1.22.0_1 Type of problem: cups-base -- HPGL buffer overflow vulnerability. Reference: <http://www.FreeBSD.org/ports/portaudit/40a3bca2-6809-11d9-a9e7-0001020eed82.html> 1 problem(s) in your installed packages found. You are advised to update or deinstall the affected package(s) immediately. By pointing a web browser to the URL shown, an administrator may obtain more information about the vulnerability in question. This will include versions affected, by &os; Port version, along with other web sites which may contain security advisories. In short, Portaudit is a powerful utility and extremely useful when coupled with the Portupgrade port. Tom Rhodes Contributed by FreeBSD Security Advisories &os; Security Advisories Like many production quality operating systems, &os; publishes Security Advisories. These advisories are usually mailed to the security lists and noted in the Errata only after the appropriate releases have been patched. This section will work to explain what an advisory is, how to understand it, and what measures to take in order to patch a system. What does an advisory look like? The &os; security advisories look similar to the one below, taken from the &a.security-notifications.name; mailing list. ============================================================================= &os;-SA-XX:XX.UTIL Security Advisory The &os; Project Topic: denial of service due to some problem Category: core Module: sys Announced: 2003-09-23 Credits: Person@EMAIL-ADDRESS Affects: All releases of &os; &os; 4-STABLE prior to the correction date Corrected: 2003-09-23 16:42:59 UTC (RELENG_4, 4.9-PRERELEASE) 2003-09-23 20:08:42 UTC (RELENG_5_1, 5.1-RELEASE-p6) 2003-09-23 20:07:06 UTC (RELENG_5_0, 5.0-RELEASE-p15) 2003-09-23 16:44:58 UTC (RELENG_4_8, 4.8-RELEASE-p8) 2003-09-23 16:47:34 UTC (RELENG_4_7, 4.7-RELEASE-p18) 2003-09-23 16:49:46 UTC (RELENG_4_6, 4.6-RELEASE-p21) 2003-09-23 16:51:24 UTC (RELENG_4_5, 4.5-RELEASE-p33) 2003-09-23 16:52:45 UTC (RELENG_4_4, 4.4-RELEASE-p43) 2003-09-23 16:54:39 UTC (RELENG_4_3, 4.3-RELEASE-p39) CVE Name: CVE-XXXX-XXXX For general information regarding FreeBSD Security Advisories, including descriptions of the fields above, security branches, and the following sections, please visit http://www.FreeBSD.org/security/. I. Background II. Problem Description III. Impact IV. Workaround V. Solution VI. Correction details VII. References The Topic field indicates exactly what the problem is. It is basically an introduction to the current security advisory and notes the utility with the vulnerability. The Category refers to the affected part of the system which may be one of core, contrib, or ports. The core category means that the vulnerability affects a core component of the &os; operating system. The contrib category means that the vulnerability affects software contributed to the &os; Project, such as sendmail. Finally the ports category indicates that the vulnerability affects add on software available as part of the Ports Collection. The Module field refers to the component location, for instance sys. In this example, we see that the module, sys, is affected; therefore, this vulnerability affects a component used within the kernel. The Announced field reflects the date said security advisory was published, or announced to the world. This means that the security team has verified that the problem does exist and that a patch has been committed to the &os; source code repository. The Credits field gives credit to the individual or organization who noticed the vulnerability and reported it. The Affects field explains which releases of &os; are affected by this vulnerability. For the kernel, a quick look over the output from ident on the affected files will help in determining the revision. For ports, the version number is listed after the port name in /var/db/pkg. If the system does not sync with the &os; CVS repository and rebuild daily, chances are that it is affected. The Corrected field indicates the date, time, time offset, and release that was corrected. Reserved for the identification information used to look up vulnerabilities in the Common Vulnerabilities Database system. The Background field gives information on exactly what the affected utility is. Most of the time this is why the utility exists in &os;, what it is used for, and a bit of information on how the utility came to be. The Problem Description field explains the security hole in depth. This can include information on flawed code, or even how the utility could be maliciously used to open a security hole. The Impact field describes what type of impact the problem could have on a system. For example, this could be anything from a denial of service attack, to extra privileges available to users, or even giving the attacker superuser access. The Workaround field offers a feasible workaround to system administrators who may be incapable of upgrading the system. This may be due to time constraints, network availability, or a slew of other reasons. Regardless, security should not be taken lightly, and an affected system should either be patched or the security hole workaround should be implemented. The Solution field offers instructions on patching the affected system. This is a step by step tested and verified method for getting a system patched and working securely. The Correction Details field displays the CVS branch or release name with the periods changed to underscore characters. It also shows the revision number of the affected files within each branch. The References field usually offers sources of other information. This can include web URLs, books, mailing lists, and newsgroups. Tom Rhodes Contributed by Process Accounting Process Accounting Process accounting is a security method in which an administrator may keep track of system resources used, their allocation among users, provide for system monitoring, and minimally track a user's commands. This indeed has its own positive and negative points. One of the positives is that an intrusion may be narrowed down to the point of entry. A negative is the amount of logs generated by process accounting, and the disk space they may require. This section will walk an administrator through the basics of process accounting. Enable and Utilizing Process Accounting Before making use of process accounting, it must be enabled. To do this, execute the following commands: &prompt.root; touch /var/account/acct &prompt.root; accton /var/account/acct &prompt.root; echo 'accounting_enable="YES"' >> /etc/rc.conf Once enabled, accounting will begin to track CPU stats, commands, etc. All accounting logs are in a non-human readable format and may be viewed using the &man.sa.8; utility. If issued without any options, sa will print information relating to the number of per user calls, the total elapsed time in minutes, total CPU and user time in minutes, average number of I/O operations, etc. To view information about commands being issued, one would use the &man.lastcomm.1; utility. The lastcomm command may be used to print out commands issued by users on specific &man.ttys.5;, for example: &prompt.root; lastcomm ls trhodes ttyp1 Would print out all known usage of the ls by trhodes on the ttyp1 terminal. Many other useful options exist and are explained in the &man.lastcomm.1;, &man.acct.5; and &man.sa.8; manual pages. diff --git a/en_US.ISO8859-1/books/pmake/gods/chapter.sgml b/en_US.ISO8859-1/books/pmake/gods/chapter.sgml index f5963d47e4..e675c4c0f2 100644 --- a/en_US.ISO8859-1/books/pmake/gods/chapter.sgml +++ b/en_US.ISO8859-1/books/pmake/gods/chapter.sgml @@ -1,953 +1,953 @@ PMake for Gods This chapter is devoted to those facilities in PMake that allow you to do a great deal in a makefile with very little work, as well as do some things you could not do in Make without a great deal of work (and perhaps the use of other programs). The problem with these features, is they must be handled with care, or you will end up with a mess. Once more, I assume a greater familiarity with &unix; or Sprite than I did in the previous two chapters.
Search Paths PMake supports the dispersal of files into multiple directories by allowing you to specify places to look for sources with .PATH targets in the makefile. The directories you give as sources for these targets make up a search path. Only those files used exclusively as sources are actually sought on a search path, the assumption being that anything listed as a target in the makefile can be created by the makefile and thus should be in the current directory. There are two types of search paths in PMake: one is used for all types of files (including included makefiles) and is specified with a plain .PATH target (e.g. .PATH : RCS), while the other is specific to a certain type of file, as indicated by the file's suffix. A specific search path is indicated by immediately following the .PATH with the suffix of the file. For instance: .PATH.h : /sprite/lib/include /sprite/att/lib/include would tell PMake to look in the - directories /sprite/lib/include and - /sprite/att/lib/include for any + directories /sprite/lib/include and + /sprite/att/lib/include for any files whose suffix is .h. The current directory is always consulted first to see if a file exists. Only if it cannot be found there are the directories in the specific search path, followed by those in the general search path, consulted. A search path is also used when expanding wildcard characters. If the pattern has a recognizable suffix on it, the path for that suffix will be used for the expansion. Otherwise the default search path is employed. When a file is found in some directory other than the current one, all local variables that would have contained the target's name (.ALLSRC, and .IMPSRC) will instead contain the path to the file, as found by PMake. Thus if you have a file ../lib/mumble.c and a makefile like this: .PATH.c : ../lib mumble : mumble.c $(CC) -o $(.TARGET) $(.ALLSRC) the command executed to create mumble would be cc -o mumble ../lib/mumble.c. (as an aside, the command in this case is not strictly necessary, since it will be found using transformation rules if it is not given. This is because .out is the null suffix by default and a transformation exists from .c to .out. Just thought I would throw that in). If a file exists in two directories on the same search path, the file in the first directory on the path will be the one PMake uses. So if you have a large system spread over many directories, it would behoove you to follow a naming convention that avoids such conflicts. Something you should know about the way search paths are implemented is that each directory is read, and its contents cached, exactly once – when it is first encountered – so any changes to the directories while PMake is running will not be noted when searching for implicit sources, nor will they be found when PMake attempts to discover when the file was last modified, unless the file was created in the current directory. While people have suggested that PMake should read the directories each time, my experience suggests that the caching seldom causes problems. In addition, not caching the directories slows things down enormously because of PMake's attempts to apply transformation rules through non-existent files – the number of extra file-system searches is truly staggering, especially if many files without suffixes are used and the null suffix is not changed from .out.
Archives and Libraries &unix; and Sprite allow you to merge files into an archive using the ar command. Further, if the files are relocatable object files, you can run ranlib on the archive and get yourself a library that you can link into any program you want. The main problem with archives is they double the space you need to store the archived files, since there is one copy in the archive and one copy out by itself. The problem with libraries is you usually think of them as rather than /usr/lib/libm.a and the linker thinks they are out-of-date if you so much as look at them. PMake solves the problem with archives by allowing you to tell it to examine the files in the archives (so you can remove the individual files without having to regenerate them later). To handle the problem with libraries, PMake adds an additional way of deciding if a library is out-of-date: if the table of contents is older than the library, or is missing, the library is out-of-date. A library is any target that looks like or that ends in a suffix that was marked as a library using the .LIBS target. .a is so marked in the system makefile. Members of an archive are specified as archive(member[member...]). Thus libdix.a(window.o) specifies the file window.o in the archive libdix.a. You may also use wildcards to specify the members of the archive. Just remember that most the wildcard characters will only find existing files. A file that is a member of an archive is treated specially. If the file does not exist, but it is in the archive, the modification time recorded in the archive is used for the file when determining if the file is out-of-date. When figuring out how to make an archived member target (not the file itself, but the file in the archive – the archive(member) target), special care is taken with the transformation rules, as follows: archive(member) is made to depend on member. The transformation from the member's suffix to the archive's suffix is applied to the archive(member) target. The archive(member)'s .TARGET variable is set to the name of the member if member is actually a target, or the path to the member file if member is only a source. The .ARCHIVE variable for the archive(member) target is set to the name of the archive. The .MEMBER variable is set to the actual string inside the parentheses. In most cases, this will be the same as the .TARGET variable. The archive(member)'s place in the local variables of the targets that depend on it is taken by the value of its .TARGET variable. Thus, a program library could be created with the following makefile: .o.a : ... rm -f $(.TARGET:T) OBJS = obj1.o obj2.o obj3.o libprog.a : libprog.a($(OBJS)) ar cru $(.TARGET) $(.OODATE) ranlib $(.TARGET) This will cause the three object files to be compiled (if the corresponding source files were modified after the object file or, if that does not exist, the archived object file), the out-of-date ones archived in libprog.a, a table of contents placed in the archive and the newly-archived object files to be removed. All this is used in the makelib.mk system makefile to create a single library with ease. This makefile looks like this: # # Rules for making libraries. The object files that make up the library # are removed once they are archived. # # To make several libraries in parallel, you should define the variable # "many_libraries". This will serialize the invocations of ranlib. # # To use, do something like this: # # OBJECTS = <files in the library> # # fish.a: fish.a($(OBJECTS)) MAKELIB # # #ifndef _MAKELIB_MK _MAKELIB_MK = #include <po.mk> .po.a .o.a : ... rm -f $(.MEMBER) ARFLAGS ?= crl # # Re-archive the out-of-date members and recreate the library's table of # contents using ranlib. If many_libraries is defined, put the ranlib # off til the end so many libraries can be made at once. # MAKELIB : .USE .PRECIOUS ar $(ARFLAGS) $(.TARGET) $(.OODATE) #ifndef no_ranlib # ifdef many_libraries ... # endif many_libraries ranlib $(.TARGET) #endif no_ranlib #endif _MAKELIB_MK
On the Condition... Like the C compiler before it, PMake allows you to configure the makefile, based on the current environment, using conditional statements. A conditional looks like this: #if boolean expression lines #elif another boolean expression more lines #else still more lines #endif They may be nested to a maximum depth of 30 and may occur anywhere (except in a comment, of course). The # must the very first character on the line. Each boolean expression is made up of terms that look like function calls, the standard C boolean operators &&, ||, and !, and the standard relational operators ==, !=, >, >=, <, and <=, with == and != being overloaded to allow string comparisons as well. && represents logical AND; || is logical OR and ! is logical NOT. The arithmetic and string operators take precedence over all three of these operators, while NOT takes precedence over AND, which takes precedence over OR. This precedence may be overridden with parentheses, and an expression may be parenthesized to your heart's content. Each term looks like a call on one of four functions: make The syntax is make(target) where target is a target in the makefile. This is true if the given target was specified on the command line, or as the source for a .MAIN target (note that the sources for .MAIN are only used if no targets were given on the command line). defined The syntax is defined(variable) and is true if variable is defined. Certain variables are defined in the system makefile that identify the system on which PMake is being run. exists The syntax is exists(file) and is true if the file can be found on the global search path (i.e. that defined by .PATH targets, not by .PATHsuffix targets). empty This syntax is much like the others, except the string inside the parentheses is of the same form as you would put between parentheses when expanding a variable, complete with modifiers and everything. The function returns true if the resulting string is empty. An undefined variable in this context will cause at the very least a warning message about a malformed conditional, and at the worst will cause the process to stop once it has read the makefile. If you want to check for a variable being defined or empty, use the expression: !defined(var) || empty(var) as the definition of || will prevent the empty() from being evaluated and causing an error, if the variable is undefined. This can be used to see if a variable contains a given word, for example: #if !empty(var:Mword) The arithmetic and string operators may only be used to test the value of a variable. The lefthand side must contain the variable expansion, while the righthand side contains either a string, enclosed in double-quotes, or a number. The standard C numeric conventions (except for specifying an octal number) apply to both sides. E.g.: #if $(OS) == 4.3 #if $(MACHINE) == "sun3" #if $(LOAD_ADDR) > 0xc000 are all valid conditionals. In addition, the numeric value of a variable can be tested as a boolean as follows: #if $(LOAD) would see if LOAD contains a non-zero value and: #if !$(LOAD) would test if LOAD contains a zero value. In addition to the bare #if, there are other forms that apply one of the first two functions to each term. They are as follows: ifdef defined ifndef !defined ifmake make ifnmake !make There are also the else if forms: elif, elifdef, elifndef, elifmake, and elifnmake. For instance, if you wish to create two versions of a program, one of which is optimized (the production version) and the other of which is for debugging (has symbols for dbx), you have two choices: you can create two makefiles, one of which uses the flag for the compilation, while the other uses the flag, or you can use another target (call it debug) to create the debug version. The construct below will take care of this for you. I have also made it so defining the variable DEBUG (say with pmake -D DEBUG) will also cause the debug version to be made. #if defined(DEBUG) || make(debug) CFLAGS += -g #else CFLAGS += -O #endif There are, of course, problems with this approach. The most glaring annoyance is that if you want to go from making a debug version to making a production version, you have to remove all the object files, or you will get some optimized and some debug versions in the same program. Another annoyance is you have to be careful not to make two targets that conflict because of some conditionals in the makefile. For instance: #if make(print) FORMATTER = ditroff -Plaser_printer #endif #if make(draft) FORMATTER = nroff -Pdot_matrix_printer #endif would wreak havoc if you tried pmake draft print since you would use the same formatter for each target. As I said, this all gets somewhat complicated.
A Shell is a Shell is a Shell In normal operation, the Bourne Shell (better known as sh) is used to execute the commands to re-create targets. PMake also allows you to specify a different shell for it to use when executing these commands. There are several things PMake must know about the shell you wish to use. These things are specified as the sources for the .SHELL target by keyword, as follows: path=path PMake needs to know where the shell actually resides, so it can execute it. If you specify this and nothing else, PMake will use the last component of the path and look in its table of the shells it knows and use the specification it finds, if any. Use this if you just want to use a different version of the Bourne or C Shell (yes, PMake knows how to use the C Shell too). name=name This is the name by which the shell is to be known. It is a single word and, if no other keywords are specified (other than path), it is the name by which PMake attempts to find a specification for it (as mentioned above). You can use this if you would just rather use the C Shell than the Bourne Shell (.SHELL: name=csh will do it). quiet=echo-off command As mentioned before, PMake actually controls whether commands are printed by introducing commands into the shell's input stream. This keyword, and the next two, control what those commands are. The quiet keyword is the command used to turn echoing off. Once it is turned off, echoing is expected to remain off until the echo-on command is given. echo=echo-on command The command PMake should give to turn echoing back on again. filter=printed echo-off command Many shells will echo the echo-off command when it is given. This keyword tells PMake in what format the shell actually prints the echo-off command. Wherever PMake sees this string in the shell's output, it will delete it and any following whitespace, up to and including the next newline. See the example at the end of this section for more details. echoFlag=flag to turn echoing on Unless a target has been marked .SILENT, PMake wants to start the shell running with echoing on. To do this, it passes this flag to the shell as one of its arguments. If either this or the next flag begins with a -, the flags will be passed to the shell as separate arguments. Otherwise, the two will be concatenated (if they are used at the same time, of course). errFlag=flag to turn error checking on Likewise, unless a target is marked .IGNORE, PMake wishes error-checking to be on from the very start. To this end, it will pass this flag to the shell as an argument. The same rules for an initial - apply as for the echoFlag. check=command to turn error checking on Just as for echo-control, error-control is achieved by inserting commands into the shell's input stream. This is the command to make the shell check for errors. It also serves another purpose if the shell does not have error-control as commands, but I will get into that in a minute. Again, once error checking has been turned on, it is expected to remain on until it is turned off again. ignore=commandto turn error checking off This is the command PMake uses to turn error checking off. It has another use if the shell does not do errorcontrol, but I will tell you about that...now. hasErrCtl=yes or no This takes a value that is either yes or no. Now you might think that the existence of the check and ignore keywords would be enough to tell PMake if the shell can do error-control, but you would be wrong. If hasErrCtl is yes, PMake uses the check and ignore commands in a straight-forward manner. If this is no, however, their use is rather different. In this case, the check command is used as a template, in which the string %s is replaced by the command that is about to be executed, to produce a command for the shell that will echo the command to be executed. The ignore command is also used as a template, again with %s replaced by the command to be executed, to produce a command that will execute the command to be executed and ignore any error it returns. When these strings are used as templates, you must provide newline(s) (\n) in the appropriate place(s). The strings that follow these keywords may be enclosed in single or double quotes (the quotes will be stripped off) and may contain the usual C backslash-characters (\n is newline, \r is return, \b is backspace, \' escapes a single-quote inside single-quotes, \" escapes a double-quote inside double-quotes). Now for an example. This is actually the contents of the <shx.mk> system makefile, and causes PMake to use the Bourne Shell in such a way that each command is printed as it is executed. That is, if more than one command is given on a line, each will be printed separately. Similarly, each time the body of a loop is executed, the commands within that loop will be printed, etc. The specification runs like this: # # This is a shell specification to have the Bourne shell echo # the commands just before executing them, rather than when it reads # them. Useful if you want to see how variables are being expanded, etc. # .SHELL : path=/bin/sh \ quiet="set -" \ echo="set -x" \ filter="+ set - " \ echoFlag=x \ errFlag=e \ hasErrCtl=yes \ check="set -e" \ ignore="set +e" It tells PMake the following: The shell is located in the file /bin/sh. It need not tell PMake that the name of the shell is sh as PMake can figure that out for itself (it is the last component of the path). The command to stop echoing is set -. The command to start echoing is set . When the echo off command is executed, the shell will print + set - (The + comes from using the flag (rather than the flag PMake usually uses)). PMake will remove all occurrences of this string from the output, so you do not notice extra commands you did not put there. The flag the Bourne Shell will take to start echoing in this way is the flag. The Bourne Shell will only take its flag arguments concatenated as its first argument, so neither this nor the errFlag specification begins with a -. The flag to use to turn error-checking on from the start is . The shell can turn error-checking on and off, and the commands to do so are set +e and set -e, respectively. I should note that this specification is for Bourne Shells that are not part of Berkeley &unix;, as shells from Berkeley do not do error control. You can get a similar effect, however, by changing the last three lines to be: hasErrCtl=no \ check="echo \"+ %s\"\n" \ ignore="sh -c '%s || exit 0\n" This will cause PMake to execute the two commands: echo "+ cmd" sh -c 'cmd || true' for each command for which errors are to be ignored. (In case you are wondering, the thing for ignore tells the shell to execute another shell without error checking on and always exit 0, since the || causes the exit 0 to be executed only if the first command exited non-zero, and if the first command exited zero, the shell will also exit zero, since that is the last command it executed).
Compatibility There are three (well, 3 1/2) levels of backwards-compatibility built into PMake. Most makefiles will need none at all. Some may need a little bit of work to operate correctly when run in parallel. Each level encompasses the previous levels (e.g. (one shell per command) implies ). The three levels are described in the following three sections.
DEFCON 3 – Variable Expansion As noted before, PMake will not expand a variable unless it knows of a value for it. This can cause problems for makefiles that expect to leave variables undefined except in special circumstances (e.g. if more flags need to be passed to the C compiler or the output from a text processor should be sent to a different printer). If the variables are enclosed in curly braces (${PRINTER}), the shell will let them pass. If they are enclosed in parentheses, however, the shell will declare a syntax error and the make will come to a grinding halt. You have two choices: change the makefile to define the variables (their values can be overridden on the command line, since that is where they would have been set if you used Make, anyway) or always give the flag (this can be done with the .MAKEFLAGS target, if you want).
DEFCON 2 – The Number of the Beast Then there are the makefiles that expect certain commands, such as changing to a different directory, to not affect other commands in a target's creation script. You can solve this is either by going back to executing one shell per command (which is what the flag forces PMake to do), which slows the process down a good bit and requires you to use semicolons and escaped newlines for shell constructs, or by changing the makefile to execute the offending command(s) in a subshell (by placing the line inside parentheses), like so: install :: .MAKE (cd src; $(.PMAKE) install) (cd lib; $(.PMAKE) install) (cd man; $(.PMAKE) install) This will always execute the three makes (even if the flag was given) because of the combination of the :: operator and the .MAKE attribute. Each command will change to the proper directory to perform the install, leaving the main shell in the directory in which it started.
DEFCON 1 – Imitation is the Not the Highest Form of Flattery The final category of makefile is the one where every command requires input, the dependencies are incompletely specified, or you simply cannot create more than one target at a time, as mentioned earlier. In addition, you may not have the time or desire to upgrade the makefile to run smoothly with PMake. If you are the conservative sort, this is the compatibility mode for you. It is entered either by giving PMake the flag (for Make), or by executing PMake as make. In either case, PMake performs things exactly like Make (while still supporting most of the nice new features PMake provides). This includes: No parallel execution. Targets are made in the exact order specified by the makefile. The sources for each target are made in strict left-to-right order, etc. A single Bourne shell is used to execute each command, thus the shell's $$ variable is useless, changing directories does not work across command lines, etc. If no special characters exist in a command line, PMake will break the command into words itself and execute the command directly, without executing a shell first. The characters that cause PMake to execute a shell are: #, =, |, ^, (, ), {, }, ;, &, >, <, *, ?, [, ], :, $, `, and \. You should notice that these are all the characters that are given special meaning by the shell (except ' and , which PMake deals with all by its lonesome). The use of the null suffix is turned off.
The Way Things Work When PMake reads the makefile, it parses sources and targets into nodes in a graph. The graph is directed only in the sense that PMake knows which way is up. Each node contains not only links to all its parents and children (the nodes that depend on it and those on which it depends, respectively), but also a count of the number of its children that have already been processed. The most important thing to know about how PMake uses this graph is that the traversal is breadth-first and occurs in two passes. After PMake has parsed the makefile, it begins with the nodes the user has told it to make (either on the command line, or via a .MAIN target, or by the target being the first in the file not labeled with the .NOTMAIN attribute) placed in a queue. It continues to take the node off the front of the queue, mark it as something that needs to be made, pass the node to Suff_FindDeps (mentioned earlier) to find any implicit sources for the node, and place all the node's children that have yet to be marked at the end of the queue. If any of the children is a .USE rule, its attributes are applied to the parent, then its commands are appended to the parent's list of commands and its children are linked to its parent. The parent's unmade children counter is then decremented (since the .USE node has been processed). You will note that this allows a .USE node to have children that are .USE nodes and the rules will be applied in sequence. If the node has no children, it is placed at the end of another queue to be examined in the second pass. This process continues until the first queue is empty. At this point, all the leaves of the graph are in the examination queue. PMake removes the node at the head of the queue and sees if it is out-of-date. If it is, it is passed to a function that will execute the commands for the node asynchronously. When the commands have completed, all the node's parents have their unmade children counter decremented and, if the counter is then 0, they are placed on the examination queue. Likewise, if the node is up-to-date. Only those parents that were marked on the downward pass are processed in this way. Thus PMake traverses the graph back up to the nodes the user instructed it to create. When the examination queue is empty and no shells are running to create a target, PMake is finished. Once all targets have been processed, PMake executes the commands attached to the .END target, either explicitly or through the use of an ellipsis in a shell script. If there were no errors during the entire process but there are still some targets unmade (PMake keeps a running count of how many targets are left to be made), there is a cycle in the graph. PMake does a depth-first traversal of the graph to find all the targets that were not made and prints them out one by one.